great cheese frustration

I overall like Gnome, but the guys working on Cheese appear to be making some horrible design choices.

Such as, they've removed any possible way to set the video input device, either via gstreamer's properties, or command line, or within the cheese app itself.

Now I'm hunting for another app that does video capture from a video4linux device, or possibly writing one using PySoy.

Beta3, new features on Ohloh

So we're trying to get Beta3 out the door. No date yet, there's still a lot of cleanup work to do.

For smaller updates, Ohloh has a new "Journal" feature that resembles Twitter. What's really cool is once you associate your XMPP id with your Ohloh account you can use XMPP (via your normal IM client) to post to your journal.

Check PySoy's journal for smaller, more frequent updates on what we're doing.

bike accident

I got side-swiped by a car Tuesday. Only injuries are some scraped skin and sprained wrist (I'm either really lucky or all too experienced in surviving bike accidents), but the latter has dramatically slowed by dev pace due to having to type with only one hand.

The sprain is mending well and i should be able to do light stuff (like typing) with it this weekend, provided I keep it immobile.

In other news, bump mapping is working again, but our blocks.py lacks proper normals, texcoords, and tangents. If anyone is willing to spend an hour or two working out the calcs for these please get in touch.

hacking at 3am, seeing rainbows

There are some real advantages to hacking into the wee hours of the morning, this gem of a screenshot is one of the best of late.

Thing is, I'm sitting here after refactoring several hundred lines of code and fixing two typos that was causing our normalisation cubemaps to be bugged, staring at this window, and unable to understand why.

The rainbow effect is obviously from the normalisation cubemap, a data construct that maps tangent-space bumpmaps to object-space on the video card (without shaders). I just can't determine at this late hour why it's showing up instead of rendering the bumpmap as it used to.

I'm sure it'll make much more sense after 8 hours sleep. In the meantime, enjoy the screenshot, or compile PySoy from subversion and run TexBlocks.py yourself!

Mesh code refactoring

I spent most of the day fixing, shifting, folding, documenting, and otherwise refactoring soy.models.Mesh and it's related datatypes and atoms. I'll likely spend most of tomorrow finishing this up.

In short, the render process we used was crude, lacked some really easy features, looped through 6 different classes for each Mesh (Mesh, MaterialList, Material, Texture, FaceList, VertexList) and had data management code in these plus the two related atom classes (Face and Vertex).

In short, not only was the code difficult to grok (even for me and I wrote much of it) but was slower that need be and increasingly difficult to expand. Game engines need both rendering speed (fps) and easy expandability, so this is a high priority ticket.

The new structure moves all of the actual Mesh data into the Mesh objects themselves; MaterialList is gone (to be replaced by MaterialDict later), FaceList and VertexList will now only store a reference to their parent Mesh to draw data from.

I'm going to be centralizing data management functions in Mesh as well, so the atoms can be much simpler and we can actually talk about optimizing the VBO layout. We'll also be doing dynamic passes, based on how many textures/pass a GL implementation supports, and supporting more textures targets (ie, specularity, emissive)

Again, sprint via IRC this weekend complete with remote peer programming via Gobby. Join us in #PySoy on irc.freenode.net

why can't XMPP keep their website up?

Several times this month I've found www.xmpp.org/, the organization responsible for the Jabber protocol and it's many extensions, non-responsive. Their site has been down for at least 7 hours tonight, the longest yet (to my knowledge).

For our purposes I'll be mirroring their XEP's on pysoy.org as soon as they're back up. It's seriously frustrating to be working on this when the documents you need are often unavailable.

If anyone knows of an existing mirror we could use, please reply to this posting.

some real progress toward beta-3

With Pyrex 0.9.8 released yesterday, including a number of key changes we needed to move forward, we're back in action for PySoy belated Beta-3 release.

The new forward declarations are a bit kludgy (example), and I don't think it'll be very maintainable in the long term. I think changing a class's inheritance should require editing one file, not two (the .pxd) or more (every .pxd that uses it).

It works for now, until we can wipe out the entire include/ directory, and to be fair it's only so cumbersome due to PySoy's size and how the classes are interconnected.

The nogil allowance for cdef methods is a massive step toward stomping out the multicore race conditions. One of the developers testing just the fixes to soy.bodies noticed the examples often running longer than before, though there's a lot more cleanup to be done before this is package-wide.

One of the roadblocks I hit is the nogil fix can't be used with our Children class, which stores pointers to Python objects of the same type and uses those pointers (through a typecast) to access their cdef methods in a for loop.

While our usage is GIL-safe, the current version of Pyrex still wants to throw in an INCREF just to be extra-cautious, which makes it not. I've let Greg know about this, we'll see if it can't be fixed soon.

In the meantime, I've called for an IRC sprint this weekend for the pending cleanup, refactoring, documentation, and testing that needs doing before beta-3. Join #PySoy on irc.freenode.net if you want to pitch-in!

Building a better Pyrex - part 1

With our recent decision to drop the Pyrex codebase and start from scratch, we find ourselves repeating a process from 2 years ago;

When PySoy first forked off of Soya, we had a small list of things that needed to be fixed based on our negative experiences working with that project. The API needed to be cleaned up, pydocs added, Shape needed to be mutable and implemented with vertex arrays or VBOs rather than display lists, etc.

As we worked on it, that list grew longer, including rewriting large parts of the codebase, until we found ourselves at a point where starting over would be more expedient. Of course now there's little doubt that was a good move, learning from both Soya's mistakes and our own failed early attempts we now have a pretty rocking architecture.

The first and most foundational change we're making from Pyrex, as previously stated, is in the lexical scanner and parser. Both Pyrex and Cython are based on Plex, which was Greg's answer to processing a Python-like language.

In contrast, we'll be using Python's own tokenize module for our lexical scanner and an ASDL-based parser akin to the parser used in Python 3.0. We're extending Python.asdl with cdef, ctypedef, cimport, cinclude, etc.

This way we have less code to debug and a solid foundation to start with. With lexing and parsing almost "for free" we'll be able to focus on the important bits and get the platform able to support the kind of features we need quickly.

We've also reached a rough consensus on the following syntax changes from Pyrex:

• unicode, generators, decorators, etc will be supported
• all custom for syntaxes dropped in favor of Python's standard
• with nogil: replaced with full with support
• special methods will always have self as first argument (ie, both __sub__ and __rsub__ used)
• cinclude added for direct C header parsing

There's more to come, which I'll post in additional installments as work progresses on this project.

setup-pyrex.py works with 0.9.7.2

Greg fixed the latest bug that caused PySoy to raise a dict assignment error on import with 0.9.7.

Pyrex 0.9.7.2 was released a few hours ago with this fix, and I've updated setup-pyrex.py to work with either 0.9.6.4 or >=0.9.7.2.

Beta-3 is still blocked by the multicore race condition caused by redundant incref/decref calls on Python arguments to class methods in this latest version of Pyrex.

setup-pyrex.py now requires EXACTLY Pyrex 0.9.6.4

16:02 < maacl> import soy
16:02 < maacl> File "LoopThread.pym", line 51, in 
               soy._internals.LoopThread.__cinit__
16:02 < maacl> TypeError: 'dict' object does not support item assignment
16:03 < Arc>     from threading import _active, _active_limbo_lock
16:03 < Arc>     self._active = _active
16:03 < Arc> [...] and line 51:
16:03 < Arc>     self._active[self._threadID] = self

I'm not going to even look into this further. Pyrex 0.9.7, for a reason I'm not going to even contemplate, raises an error about dict item assignment on the last line. None of us use it due to it's barfing of depreciated lines for all our for statements, but this brave tested did and reported the problem today.

I just committed an update to setup-pyrex.py that fails when any Pyrex version besides 0.9.6.4 is used.