Friday, May 16, 2008

some real progress toward beta-3

With Pyrex 0.9.8 released yesterday, including a number of key changes we needed to move forward, we're back in action for PySoy belated Beta-3 release.

The new forward declarations are a bit kludgy (example), and I don't think it'll be very maintainable in the long term. I think changing a class's inheritance should require editing one file, not two (the .pxd) or more (every .pxd that uses it).

It works for now, until we can wipe out the entire include/ directory, and to be fair it's only so cumbersome due to PySoy's size and how the classes are interconnected.

The nogil allowance for cdef methods is a massive step toward stomping out the multicore race conditions. One of the developers testing just the fixes to soy.bodies noticed the examples often running longer than before, though there's a lot more cleanup to be done before this is package-wide.

One of the roadblocks I hit is the nogil fix can't be used with our Children class, which stores pointers to Python objects of the same type and uses those pointers (through a typecast) to access their cdef methods in a for loop.

While our usage is GIL-safe, the current version of Pyrex still wants to throw in an INCREF just to be extra-cautious, which makes it not. I've let Greg know about this, we'll see if it can't be fixed soon.

In the meantime, I've called for an IRC sprint this weekend for the pending cleanup, refactoring, documentation, and testing that needs doing before beta-3. Join #PySoy on if you want to pitch-in!


greg said...

It probably isn't necesary to require the base classes to be listed in all the places they currently are. I'll see if I can do something about that.

Arc Riley said...

As I said, I think that's caused by the size and structure of PySoy, not Pyrex.

Most Pyrex apps are fairly small, or at least their components don't all end up linking into each other in the same manner as ours.

Rearranging the classes would help, but would make the API less logical/intuitive.

I think the best solution is to not use .pxd's at all, but instead parse the source files directly.