| View previous topic :: View next topic |
| Author |
Message |
quinox
Joined: 28 Sep 2009
Posts: 5
|
 Posted: Mon Sep 28, 2009 12:54 pm Post subject: Let python module interact with GIL to prevent freezing |
 |
|
| <Post intentionally left blank> |
|
| Back to top |
|
 |
quinox
Joined: 28 Sep 2009
Posts: 5
|
| Posted: Mon Sep 28, 2009 12:55 pm Post subject: |
|
|
First of all I'd like to express my compliments, the MiniUPNP project is a real boon 
Now my feature request:
I'd like to use MiniUPnP from a pure-python p2p application. The python module as distributed with 1.3/20090921 works like a charm, except that it doesn't seem to interact with the python GIL. The GIL is the Global Interpreter Lock, python's way of managing what threads are allowed to run (interact with objects etc.) This is a problem for me since without releasing the GIL Python will not allow any other thread to run, causing the GUI (PyGTK) to freeze up. I modified the testupnpigd.py file to include a separate thread with a stopwatch in it. This thread will try to print the time since startup once every second or so. With the python module from version 20090921 this is the output:
| Code: |
Using real miniupnpc
Discovering... delay=2000ms
--------- Hello ---------
--------- 10.3383970261 seconds elapsed ---------
34 device(s) detected
connect: Operation now in progress
connect: Operation now in progress
recv: Connection reset by peer
connect: Operation now in progress
connect: Operation now in progress
connect: Operation now in progress
connect: Operation now in progress
recv: Connection reset by peer
connect: Network is unreachable
connect: Network is unreachable
connect: Operation now in progress
local ip address :
129.125.101.92
Exception : Miniupnpc Unknown Error
--------- 36.8716480732 seconds elapsed ---------
--------- Byebye (total duration: 37.8722250462 seconds) ---------
|
(output slightly modified, the "34 devices" was really split over two lines due to the timer thread)
With a probe of 2 seconds the module will take the GIL hostage for 10 seconds with the discover() phase, then another 26 seconds while it's in the selectigd() phase. When I use a fake stub module with precisely the same code the output is as follows:
| Code: |
Using fake miniupnpc
Discovering... delay=2000ms
--------- Hello ---------
--------- 0.000127077102661 seconds elapsed ---------
--------- 1.0012421608 seconds elapsed ---------
42 device(s) detected
connect: Operation now in progress
--------- 2.00238704681 seconds elapsed ---------
--------- 3.0035071373 seconds elapsed ---------
connect: Operation now in progress
--------- 4.00465202332 seconds elapsed ---------
recv: Connection reset by peer
--------- 5.00579404831 seconds elapsed ---------
connect: Operation now in progress
--------- 6.0068731308 seconds elapsed ---------
--------- 7.00715899467 seconds elapsed ---------
connect: Operation now in progress
--------- 8.00809597969 seconds elapsed ---------
connect: Operation now in progress
--------- 9.00917696953 seconds elapsed ---------
connect: Operation now in progress
--------- 10.0102601051 seconds elapsed ---------
--------- 11.0113451481 seconds elapsed ---------
recv: Connection reset by peer
--------- 12.0118570328 seconds elapsed ---------
connect: Network is unreachable
--------- 13.0129461288 seconds elapsed ---------
connect: Network is unreachable
--------- 14.0140321255 seconds elapsed ---------
--------- 15.0151259899 seconds elapsed ---------
connect: Operation now in progress
--------- 16.0162169933 seconds elapsed ---------
local ip address :
::1
Exception : Dummy doesn't do external
--------- Byebye (total duration: 17.0173101425 seconds) ---------
|
This is the behaviour that I hope to get from the python module. I've looked at the miniupnpcmodule.c code but I couldn't figure out where to start hacking. Documents that might useful for this feature requests are:
* PEP311, Simplified Global Interpreter Lock Acquisition for Extensions http://www.python.org/dev/peps/pep-0311/
* Python/C API Reference Manual http://docs.python.org/c-api/index.html
The modified testupnpigd.py file I used to get the given output can be found at http://129.125.101.92/tmp/testupnpigd.py (stub module at http://129.125.101.92/tmp/fakeminiupnpc.py)
PS1: The 34 UPnP devices on my network are real. While this is probably not a normal situation it shouldn't freeze up Python if possible
PS2: The "Network is unreachable" messages come from the fact that many devices return a LOCATION value with an IP address belonging to the network behind the device, outside of my reach. |
|
| Back to top |
|
|
miniupnp
Site Admin
Joined: 14 Apr 2007
Posts: 1589
|
| Posted: Sat Oct 10, 2009 7:18 pm Post subject: |
|
|
I would have a look at this.
But I'm not a python developper so...
_________________
Main miniUPnP author.
https://miniupnp.tuxfamily.org/ |
|
| Back to top |
|
|
quinox
Joined: 28 Sep 2009
Posts: 5
|
| Posted: Thu Oct 15, 2009 6:54 pm Post subject: |
|
|
thanks for trying - if you need someone to test it let me know
would/will?
I'm no C-programmer alas. There is no python code involved in this feature request though - If I understand it correctly you have to call certain functions when you know for sure your module is not going to interact with python objects, and then python does the rest |
|
| Back to top |
|
|
miniupnp
Site Admin
Joined: 14 Apr 2007
Posts: 1589
|
|
| Back to top |
|
|
quinox
Joined: 28 Sep 2009
Posts: 5
|
| Posted: Fri Oct 16, 2009 9:18 am Post subject: |
|
|
Just ran it with my test script and it looks great. The discovery phase no longer freezes up other python threads, so that part is perfect. Later today I'll do more thorough testing when all of the UPnP devices are online, but I think you nailed it. Thanks! |
|
| Back to top |
|
|
quinox
Joined: 28 Sep 2009
Posts: 5
|
| Posted: Fri Oct 16, 2009 3:53 pm Post subject: |
|
|
| Tested it more extensively and it seems to work perfectly, thanks a bunch! |
|
| Back to top |
|
|
kans
Joined: 06 Jan 2010
Posts: 1
|
| Posted: Wed Jan 06, 2010 3:21 pm Post subject: Crashes in windows 7/64 with python 2.5 when using threads |
|
|
| I compiled the python extension module which seems to run fine apart from crashing the python process when I call discover inside a thread- the other methods seem to be OK otherwise- threading or not. Does anyone have any thoughts on what could be going on? |
|
| Back to top |
|
|
miniupnp
Site Admin
Joined: 14 Apr 2007
Posts: 1589
|
| Posted: Thu Jan 14, 2010 3:29 am Post subject: Re: Crashes in windows 7/64 with python 2.5 when using threa |
|
|
| kans wrote: | | I compiled the python extension module which seems to run fine apart from crashing the python process when I call discover inside a thread- the other methods seem to be OK otherwise- threading or not. Does anyone have any thoughts on what could be going on? |
Could you post a code sample that trigger the crash ?
_________________
Main miniUPnP author.
https://miniupnp.tuxfamily.org/ |
|
| Back to top |
|
|
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
