Well after many weeks we found the source of the race conditions on multicore CPUs; they were not in PySoy at all but in the Pyrex compiler.
Throughout the generated .c we found INCREF/DECREF where it didn't belong, namely in every cdef method for __pyx_v_self. In other words, even though the calling function must already hold a reference to instance in order to access it's C methods, thus preventing the instance from being garbage collected during method execution, the Pyrex.Compiler inserts redundant INCREF/DECREF calls.
These would normally just waste CPU cycles, but since our 3d engine has non-GIL background threads these refcount adjusting calls are accessed in a thread-unsafe manner and will inevitably cause either a segfault or a memory leak.
The solution is to not consider "self" a normal Python argument, which would also allow these C methods to be called with nogil: set, provided checking is done to ensure no Python methods, properties, or variables of self are accessed.
We'll be fixing this problem with our new source compiler, to be announced soon.
Subscribe to:
Post Comments (Atom)
3 comments:
Hey Arc, this is Rochelle. I'm trying to mount a device on my laptop and I can't without the root password so that I can put in the command su. I've tried the password you gave me to log into my computer and it won't work. Is there another password that is the root password? please email me at
r a y l w o r t h @c o x . n e t
There's actually a reason for those incref/decrefs. Because arguments are also local variables, they can be assigned to, and that would be really difficult to manage without being sure that the name being assigned to always holds an owned reference.
The simplest solution is probably to disallow assigning to self. That shouldn't cause too much hardship, as it's a pretty rare thing to want to do, and you can always use another local variable instead.
I'll see what I can do about this.
Seriously man, it's always good to fix bugs, but I think it's too late for us.
Not to mention that every version you release introduces new bugs for us; Pyrex 0.9.6.4 needed patching for Windows, 0.9.7 has some issue with dicts..
We can't keep up with this. We would obviously much rather not have to implement our own solution but the amount of time we've spent working around Pyrex and dealing with related bugs is overwhelming. We're all tired of it.
If it wasn't for Piet's constant help on IRC over the last two years we would have dropped Pyrex a long time ago.
Post a Comment