How can a profile be completely unavailable?

How to profile my code?

  • I want to know how to profile my code. I have gone through the docs, but as there were no examples given I could not get anything from it. I have a large code and it is taking so much time, hence I want to profile and increase its speed. I havent written my code in method, there are few in between but not completely. I don't have any main in my code. I want to know how to use profiling. I'm looking for some example or sample code of about how to profile. I tried psyco, i.e just addded two lines at the top of my code: import psyco psyco.full() Is this right? It did not show any improvement. Any other way of speeding up, please suggest.

  • Answer:

    The standard answer to this question is to use http://docs.python.org/library/profile.html#instant-user-s-manual. You'll find though that without having your code separated out into methods that cProfile won't give you particularly rich information. Instead, you might like to try what another poster here calls Monte Carlo Profiling. To quote from http://stackoverflow.com/questions/375913/what-can-i-use-to-profile-c-code-in-linux/378024#378024: If you're in a hurry and you can manually interrupt your program under the debugger while it's being subjectively slow, there's a simple way to find performance problems. Just halt it several times, and each time look at the call stack. If there is some code that is wasting some percentage of the time, 20% or 50% or whatever, that is the probability that you will catch it in the act on each sample. So that is roughly the percentage of samples on which you will see it. There is no educated guesswork required. If you do have a guess as to what the problem is, this will prove or disprove it. You may have multiple performance problems of different sizes. If you clean out any one of them, the remaining ones will take a larger percentage, and be easier to spot, on subsequent passes. Caveat: programmers tend to be skeptical of this technique unless they've used it themselves. They will say that profilers give you this information, but that is only true if they sample the entire call stack. Call graphs don't give you the same information, because 1) they don't summarize at the instruction level, and 2) they give confusing summaries in the presence of recursion. They will also say it only works on toy programs, when actually it works on any program, and it seems to work better on bigger programs, because they tend to have more problems to find [emphasis added]. It's not orthodox, but I've used it very successfully in a project where profiling using cProfile was not giving me useful output. The best thing about it is that this is dead easy to do in Python. Simply run your Python script in the interpreter, press [Control-C], note the traceback and repeat a number of times.

kaki at Stack Overflow Visit the source

Was this solution helpful to you?

Other answers

Use http://docs.python.org/library/profile.html#instant-user-s-manual. You can use it from the command line and pass in your module as a parameter, so you don't need a main method.

Johannes Charra

Here's a post I wrote some time ago on profiling with cProfile with some graphical aid. Profiling with cProfile cProfile is one of the most used python profilers out there, and although very powerful, the standard text output is somewhat lackluster. Here I'll show you how to use cProfile on your application in an easier way. There are two common ways to use cProfile, you can use it as a command in prompt to profile a given module, or you can use it inside your code, to profile specific snippets of code. Profiling a module To use cProfile to profile an entire module, simply use the following command in your prompt: python -m cProfile -o output_filename.pstats path/to/script arg1 arg2 This will run your module with the given arguments (they are optional) and dump the output in output_filename.pstats. There are http://docs.python.org/library/profile.html to read the data on that output file, but for the purpose of this post, let's not worry about those and focus on getting that graphical visualization. Profiling from inside Sometimes you don't want to profile an entire module, just a few lines of it. To do so, you are gonna have to add some code to your module. First of all: import cProfile And then, you can replace any segment of code with the following: cProfile.runctx('Your code here', globals(), locals(), 'output_file') For example, here is a test before and after profiling: import unittest class Test(unittest.TestCase): def testSomething(self): self.DoSomethingIDontCareAbout() param = 'whatever' self.RunFunctionIThinkIsSlow(param) self.AssertSomeStuff() # This is after all, a test After: import unittest import cProfile class Test(unittest.TestCase): def testSomething(self): self.DoSomethingIDontCareAbout() param = 'whatever' cProfile.runctx( 'self.RunFunctionIThinkIsSlow(param)', globals(), locals(), 'myProfilingFile.pstats' ) self.AssertSomeStuff() # This is after all, a test Converting a pstats file to a graph To convert your profiling file to a graph, you will need a couple of things: http://code.google.com/p/jrfonseca/wiki/Gprof2Dot: This module will convert your output into a dot file, a standard file format for graph descriptions. http://www.graphviz.org/Download.php: It turns your dot file into an image. After you have downloaded gprof2dot and installed GraphViz, run this command in your prompt: python gprof2dot -f pstats myProfileFile | dot -Tpng -o image_output.png You might have to use a complete path for gprof2dot and/or dot, or you could add them to your PATH env variable. After all of this, you should have an image that looks kinda like this: Hotter colors (red, orange, yellow) indicate functions that take up more of the total runtime than colder colors (green, blue) On each node, you can see what percentage of the total runtime that function used and how many times it was called. Arrows between nodes indicate which function called other functions, and such arrows also have a caption indicating what percentage of the runtime came through there. Note: percentages won't always add up to 100%, especially on code sections that reference C++ code, which won't be profiled. cProfile also won't be able to determine what's called from inside an "eval" statement, so you might see some jumps in your graph.

campos.ddc

Related Q & A:

Just Added Q & A:

Find solution

For every problem there is a solution! Proved by Solucija.

  • Got an issue and looking for advice?

  • Ask Solucija to search every corner of the Web for help.

  • Get workable solutions and helpful tips in a moment.

Just ask Solucija about an issue you face and immediately get a list of ready solutions, answers and tips from other Internet users. We always provide the most suitable and complete answer to your question at the top, along with a few good alternatives below.