[Twisted-Python] Decorator @inlinecallbacks prohibiting memory profiling

[Vikas]

Hi,

I am trying to instrument method: run() for memory profiling(I am using
memory_profiler & guppy tools for this). All I need to do is decorate
the run() with "profile" decorator and mention *start* and *end* of
memory profiling.

@profile
def run():
d = {}
l = []
hp = hpy()
before = hp.heap() => *start* memory profiling from here

d["k1"] = 'val1'
d["k2"] = 10
count = 0
while (count < 9):
l.append(count)
print 'The count is:', count
count = count + 1
print "Good bye!"
after = hp.heap() => *end* memory profiling here
leftover = after - before

print leftover
del hp


So far works fine. But moment I introduce @inlineCallbacks before
@profile(see below), I don't see memory getting profiled. Am I doing
something wrong here?
@inlinecallbacks
@profile
def run():
....

Regards
Vikas

[Vikas]

Ok this experiment works fine because it doesn't have any decorator &
generator.
Here is the actual method (with @inlineCallback and generator) where I
see method(my_daemon_fun) being profiled with @profile decorator is not
being profiled.
I am doubting that *generator* or *inlineCallback* decorator is
prohibiting the correct code from getting profiled.

To profile the code all I need to do is:
STEP#1 -> decorate the _monitor_daemons() with "profile" decorator #1
STEP#2 -> start memory profiling
STEP#3 ->* *end memory profiling

Something like this:

@profile => STEP#1
@inlineCallbacks
def my_daemon_fun(self):
"""
Monitor my daemons
"""
if self.running == True:
try:
hp = hpy() => STEP#2
before = hp.heap()
yield self._lock.acquire()
.....
if ((x_running == False) or (b_running == False)):
log.error('Daemons are not running, restarting...')
yield self._restart_daemons()
except:
log.exception('Exception restarting daemons')
finally:
yield self._lock.release()
reactor.callLater(MY_DAEMON_MONITOR_INTERVAL,
self.my_daemon_fun)
after = hp.heap() => STEP#3
leftover = after - before


Now we run the script to call method: my_daemon_fun(). After running the
script though we see memory profiling happening but the function being
profiled is *not* the one we decorated with @profile:

Line # Mem usage Increment Line Contents
1176 42.3 MiB 0.0 MiB def unwindGenerator(*args, **kwargs):
1177 42.3 MiB 0.0 MiB try:
1178 42.3 MiB 0.0 MiB gen = f(*args, **kwargs)
1183 42.3 MiB 0.0 MiB if not isinstance(gen,
types.GeneratorType):
1187 42.3 MiB 0.0 MiB return _inlineCallbacks(None,
gen, Deferred())


I am investigating why the actual function(i.e. my_daemon_fun()) was not
profiled? Meanwhile if you can help us here that would be great.

Regards
Vikas

Post by Vikas
Hi,
I am trying to instrument method: run() for memory profiling(I am
using memory_profiler & guppy tools for this). All I need to do is
decorate the run() with "profile" decorator and mention *start* and
*end* of memory profiling.
@profile
d = {}
l = []
hp = hpy()
before = hp.heap() => *start* memory profiling from here
d["k1"] = 'val1'
d["k2"] = 10
count = 0
l.append(count)
print 'The count is:', count
count = count + 1
print "Good bye!"
after = hp.heap() => *end* memory profiling here
leftover = after - before
print leftover
del hp
@profile(see below), I don't see memory getting profiled. Am I doing
something wrong here?
@inlinecallbacks
@profile
....
Regards
Vikas
_______________________________________________
Twisted-Python mailing list
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python