Saturday, May 27, 2006

Exit xml, Enter .soy

I did a number of tests last night, and even in pure Python (where XML is highly optimized) it was far, far too slow. It was also several orders of magnitude larger than it should be, even gzipped, which is bad when transfering over the network in realtime.

Another problem, too, was the Mesh API. If the Blender exporter was to have the ability to send pre-optimized data (from Blender itself) into a soya.Mesh, it dictated a very complex API which would allow both editing and exporting to be done at reasonable speeds.

So it's all ripped up now (literally, I took the newsprint draft pages and ripped them up) and I started work on an extensible binary block format comperable to AIFF, RIFF, TIFF, PNG, etc. This will allow new features to be added while both newer files compatable with older software and newer software compatable with older files. IE, full backward and forward compatability. The draft spec can be found at

The format is being designed with full optimizations already made; not only the various data structures already seperated, indexed, and linked, but ordered. That is, it can take advantage of vertex buffer optimization (with no specific software/format support for it).

It's structure overhead is very low, I'm guessing real world uses will pit it lower than 1% for size. With support for it implemented in C/Pyrex loading and saving should be simple. It also allows a much more relational data layout tree than Soya currently uses; it'll be possible to include multiple data objects (UNIX cat command can do it, just like Ogg) in a single file and reference them with path/filename#name just as you do in HTML.

While still in heavy development (and unfinished), comments and feedback are requested.

No comments: