GCI Low Hanging Fruit

I have to admit I'm pretty surprised that at halfway through Google Code-In 2011 only about a quarter of the tasks we posted for the first half have been completed or in-progress. Further, most of the tasks thus far haven't been coding tasks.

I'm going to publish here a list of what we consider "low hanging fruit"; coding tasks which are fairly easy to get started with. If you're a student age 13-17 these would be an easy way to earn a Google tshirt and some cash. All of these tasks deal with OpenGL rendering, usually just arrays for points and lines/triangles which connect the points. There's a lot of examples in the code already for this and numerous tutorials on the web (such as NeHe).

Simple Rendering

We have 2 new tasks listed for rendering simple models; Camera and Light. Camera is very simple, while Light can be as simple or complex as you want to make it.

Shapes Rendering

We have 3 tasks for rendering shapes, Box, Room, and Sphere. Any of these could be knocked out in a few hours even without prior OpenGL knowledge.

Joints Rendering

We have 6 new tasks up for rendering joints; Ball, Fixed, Hinge, Piston, Slider, and Universal. Joints (soy.joints) connect two bodies such that they can only move in respect to each other in a certain way, such as a door hinge or piston. These are all documented with graphic depictions. This is slightly more complex than the simple rendering tasks (above) in that there's two pieces to each joint and they can be rendered as either wireframe or solid (with provided materials).

As always, we're on IRC if a student wants to discuss these or other tasks.

PyTTY 0.3

Continuing my annual end-of-year coding sprint, I just released PyTTY 0.3.

PyTTY is a Python serial communication package I started last year after a friend said he couldn't use Python 3 yet because pyserial wasn't ported. The point of writing this was to show him that he didn't need an ancient, bloated, poorly-maintained package to do something as simple as serial communication.

What I wrote over an afternoon turned out to be a little over 100 lines of fairly useful code which I've since used in quite a few microcontroller projects (eg, Arduino). Its by no means complete, the only setting is baud rate and there's no Windows support, but its done everything I've needed it to over the last year. The only problem that's been reported is poor documentation which this release aims to fix. It includes a short code example ("pydoc pytty.TTY") which runs on both legacy Python and Python 3.

PyTTY 0.3 is under 135 lines of pure Python and relies only on the Python standard library. If there's a feature you need which this doesn't have either email me a patch or a feature request so I can add it. The Mercurial repository is http://hg.pytty.org/pytty.

NodeTree 0.2 Released

I just shipped NodeTree 0.2.

This version will not parse an XML stream. All it contains are some basic types representing XML nodes such as Comment, Document, and Element. As promised, text is also handled as a node but uses standard Python strings (UTF-8 strings/bytes and unicode). These should all be fairly intuitive to use.

The magic is the XML data is being managed in C using a libxml2 DOM tree but accessed through a Pythonic object-oriented API. For example, in DOM each node may have exactly one parent - in NodeTree a node may be added to any number of parents with a separate DOM node and context for each.

I started this project because the existing XML packages for Python proved too difficult to use with XMPP. Fritzy's SleekXMPP uses lxml but had to jump through several hoops to get stream parsing to work, looking over his work I certainly didn't want to repeat it with Concordance-XMPP.

Beyond this the leading XML API for Python, ElementTree, includes several unfortunate design decisions that make it frustrating to use in the best cases and unusable in others. A full list of why can be left for another time, but the difference to NodeTree can be described in their names - ElementTree is a tree of XML Element nodes with other kinds of nodes either silently dropped, mangled, or made available in bizarre ways (eg, .text and .tail). In contrast, NodeTree provides XML data as a tree of nodes starting with the Document node and includes comment and text nodes in its tree. I plan to provide 100% XML 1.0 support in a future release while maintaining a clean, simple, and intuitive API.

Storing XML data in libxml2 DOM format gives us a few advantages over other XML libraries. First, we'll have XPath, XInclude, and XSLT available without having to convert the data between formats. Second, Python objects only need to be created for nodes Python wants a reference to so when we get to parsing data this will happen much faster and with less memory.

At version 0.2 NodeTree is still in its infancy but some of its API can be demonstrated. Here's a short example:

Python 3.2.2 (default, Oct  3 2011, 00:20:58) 
[GCC 4.5.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import nodetree
>>> doc = nodetree.Document()
>>> doc.append(nodetree.Comment(' Start '))
>>> doc.append(nodetree.Element('data'))
>>> doc.append(nodetree.Comment(' Fini '))
>>> doc[1].attributes['thing'] = 'normal'
>>> doc[1].append(nodetree.Element('record'))
>>> doc[1][0].append('First') 
>>> doc[1].append(nodetree.Element('record'))
>>> doc[1][1].append('Second')
>>> doc
<?xml version="1.0"?>
<!-- Start -->
<data thing="normal">
  <record>First</record>
  <record>Second</record>
</data>
<!-- Fini -->

NodeTree 0.2 is tested to work with Python 2.6, 2.7, 3.1, 3.2, and 3.3-pre. The next release is intended to support basic file and stream parsing.

XMPP on the web

A short thread on G+ has prompted this longer sharing of my vision for XMPP on the web.

For XMPP use on a website we currently have BOSH and, in an extreme-alpha state, XMPP over websockets. The advantage of websockets is obvious, BOSH is a high overhead protocol that we'd all rather not have to use, however both have the same problem: you either must share your login credentials (and thus access to your account) with every website you use a single account with, or must create a new account (JID) for every XMPP-based website you use.

Oauth2 for XMPP might be a partial solution to this by requiring your authorization through a central identity site and using the resulting token for logging in, however, you're still opening yourself up to the 3rd party website accessing your roster, sending spam messages on your behalf, and potentially worse. All this really gives you is the ability to later disable access to websites who misuse your account.

This is the crux of the issue: when using an Javascript library provided by a website and using a proxy provided by that website, whether BOSH, websockets, or otherwise, you're giving that website unlimited access to your account. I have not seen a workable proposal to solve this and until this is solved XMPP cannot see widespread use on the web.

I'm proposing that we solve this by putting XMPP in the browser, either directly or through a plugin. Expose a standard javascript API for allowing websites to use an XMPP connection along with a security model which gives users control as to what a website is allowed to use their connection for. Ie, if a script on a website wants access to their roster the user will be prompted for it, if they want to join a MUC room display a standard prompt for that. Browsers can have multiple XMPP sessions at once and allow the user to select which account they'd like to use with an XMPP-enabled site.

This is just some early ideas, I'm nowhere near implementing this though I think the conversation would be useful to get started.

OpenGL ES support complete

The experimental branch, where the OpenGL ES migration was being done, has just been closed and merged into the default branch of PySoy.

Thanks to Steve Anton, one of our Google Code-In 2011 students, for some of the last bits of work to complete the merge. We are now one step closer to mobile support! If you're a student ages 13-17 and would like to earn a Google tshirt and some cash by helping us with Android support, sign up for Google Code-In and claim this task.

Google Code-In 2011 is Open

Google Code-In has officially begun!

From today through January 16th students age 13-17 can earn up to $500 working on small tasks for software projects such as MoinMoin, SymPy, and PySoy!

Tasks include coding, documentation, graphic design, video production, testing, translation, research, public speaking, and many other kinds of challenges of varying difficulty.

Completing just one task earns a student a Google tshirt. Every 3 tasks they complete earns them $100, and the 10 top students worldwide will earn an all-expense paid trip to Mountain View, CA to receive an award at Google.

PySoy has over 75 tasks offered for the first half of the program and another 75-100 will be made available December 16th. Our mentors are on Freenode channel #PySoy ready to help students start earning their tshirt and cash today.

Sign up today and get started!

Rugby season nearly over, getting back to work

Wow its been a long time.

We wrapped up Google's Summer of Code 2011 in August. The Python Software Foundation did wonderfully overall, for PySoy 6 of our 7 students passed. A great year overall - thanks to all the mentors and students!

Rugby has been a life changer for me. My first game was in September, after floating in and out of practice for years and training pretty heavily since April. No serious injuries, but no shortage of pain; I've frequently needed to sleep in a reclining chair to keep blood from pooling in my shoulders and nurse bruised ribs, dislocated fingers and toes, shin splints, and pulled muscles everywhere. All so worth it.

These guys are like family to me. I know it sounds sappy, but I've come to trust the men in my pack with my life - in a way we all do every time we bind onto each other a scrum. Its not that big of an adjustment culturally though due to the large number of programmers, lawyers, and IT professionals on the team. When you work behind a desk all day its nice to balance it out with a physically intensive training in the evening and games on Saturday.

The climax of the season was Hellfest October 29th in Dallas, TX. Washington Renegades brought our B-side to compete and returned with the 1st place trophy. My teammate Jimbo has more pics on his blog of the tournament, I was wearing #23 as tighthead prop.

We have two more games this season before we settle in for the Winter and indoor off-season training at the gym. A group of us plan to do a 8-week program run by a professional rugby player this Winter to get ready for the Spring season and the Bingham Cup 2012 in Manchester UK next June.

Today the PySoy project was accepted to Google Code-In. We've got a number of student tasks lined up, with many more being worked on for the first batch set to release in less than two weeks. Interested students should hop on Freenode (#PySoy) and get oriented before the program starts so they're ready to jump right into their first task!

Transcoding FLAC to Ogg Vorbis

Last night I hit a dilemma; a very old CD I ripped to FLAC and now can't find was playable only on certain players, not my Android phone (despite FLAC support) and behaving strange on many desktop players.

Usually I just use something like oggenc -q 4 *.flac since vorbis-tools supports FLAC as a source format (and preserves metadata like artist, title, etc). Strangely, oggenc didn't recognize the files in this album.

GStreamer to the rescue; for file in *.flac; do gst-launch-0.10 filesrc location="$file" ! decodebin ! audioconvert ! vorbisenc quality=0.4 ! oggmux ! filesink location="$file.ogg"; done;

Even though its much slower and more complicated, this should have done the trick. It didn't, and ogginfo showed that the Ogg muxer in GStreamer has some issues;

WARNING: granulepos in stream 1 decreases from 218558 to 205632
WARNING: granulepos in stream 1 decreases from 666174 to 655680
WARNING: granulepos in stream 1 decreases from 3150206 to 3142208
WARNING: granulepos in stream 1 decreases from 3605054 to 3597376
WARNING: granulepos in stream 1 decreases from 3828670 to 3822400
WARNING: granulepos in stream 1 decreases from 4741630 to 4733312
WARNING: granulepos in stream 1 decreases from 5636158 to 5630784
WARNING: granulepos in stream 1 decreases from 5858494 to 5854208
WARNING: granulepos in stream 1 decreases from 6096126 to 6090560
WARNING: granulepos in stream 1 decreases from 6317758 to 6314048
WARNING: granulepos in stream 1 decreases from 7668798 to 7665088
WARNING: granulepos in stream 1 decreases from 7902718 to 7898240

So with GStreamer not an option, I looked at the original FLAC files and found that whatever encoder I used added ID3 tags (which are not part of the FLAC spec). A quick ID3 removal command stripped these out so oggenc would recognize the files and work;

find . -name "*.flac" -exec id3v2 --delete-all {} \;
oggenc -q 4 *.flac

Done.

Blowing up the silos (hypothermia)

Its a common belief that fat is fat, whether we eat it or whether its stored in our bodies. Of course this is naive, our bodies are far more complex than skin balloons packed with nutrients. There are two kinds of fat tissue in our body;

The "fuel silos" are called White Adipose Tissue (WAT) made of simple cells; their membrane has a few receptors on it that trigger the cell to store or release energy, the nucleus is small and pressed against the cell membrane, and inside one big lipid droplet that shrinks or swells as needed. When dieting your pancreas releases Glucagon to trigger your fat to release their reserves. When your body releases insulin (which it does after almost every meal to some degree) your WAT cells swell back up. This is why its so hard to keep weight off.

Then there's the factory fat cells, Brown Adipose Tissue (BAT) which are much more complex; they contain many smaller lipid droplets and the enzimes needed to burn the energy right in them. They're especially important in infants because they're too small to shiver to get warm, so their "brown fat" burns their fat immediately to keep their body temperature regulated. Adults maintain a good amount of it in our upper back and neck, though strangely lean adults have more BAT than obese adults.

When our bodies become hypothermic, say by taking an ice water bath, BAT thermogenesis is activated depleting those cells of stored energy to produce body heat. Shivering also releases more energy to raise heat. While the sound of a ice water bath may sound horrifying, its actually not that bad after you've been running, decrease water temp slowly, and can even be meditative.

The goal is to destroy WAT (White Adipose Tissue) - not just slowly deplete it but leave the cells in a state ready to swell back up every time you eat, but actually reducing the number of white fat cells in your body for permanent weight loss.

Blowing up the silos (morning)

An awesome holiday gift last December was a book, "The 4-Hour Body", a 570 page hardcover hacking guide to the human body. On Dec 20 I started the "Rapid Fat-Loss" program within with great success - in less than three months I lost nearly 5% body fat, with just with a few changes to my diet and no additional exercise.

I got looser with the diet as the weather got warmer and I started exercising more, then partially tore my archilles tendon between two different accidents confining me to a reclining chair for two months. Of course I gained all the fat back; the fuel silos (fat cells) were still there in the same number, just emptied a bit, and all of them sending out messages to adjust insulin and metabolism so that they could refill.

You can see on my bodybuilding.com profile the raise starting in mid-March. However, the experience wasn't a lost cause - I have a better idea of what works and learned a number of new recipes for making it work easier, since the book only mentions veganism and doesn't provide much in the way of meal ideas (I'll share one below).

I started training June 2nd for Fall Rugby season - something I'm using as a focus point to stay motivated. I'm calling this program "Blowing up the Silos" - not just slowly using the fuel within, but mushroom clouds on the horizon. As I sit here at 9am writing this my 640 calorie breakfast burns like napalm through my veins, barely able to sit still long enough to type. When I finish with this I'm headed out for wind sprints just to burn off the excess energy.

Upon waking I immediately downed 500mg Phosphatidylserine, 500mg L-Carnitine, 300mg Alpha-Lipoic Acid, 500mg Maca root powder with a tall glass of water with 1tbsp lemon juice in it, then green tea made from 8g whole leaf supplemented with 3 grams of powdered Cissus Quadrangularis.

From there I have 30 minutes to cook breakfast while drinking the tea and chewing on a handful (24g) of cashews. Breakfast was 1 tbsp (14g) refined coconut oil in a frying pan and a batter made from 1.5tsp (4 grams) ground cinnimon and 92g gram (chic pea) flour soaked in an airtight container with 1 cup of water the night before. Makes something not even remotely akin to a pancake, but edible. Net carbs 50 grams, 26 grams protein, and a whopping 640 calories.

Along with breakfast was another handful of pills; daily cal-mag-zinc supplement (needed to replenish whats lost from green tea causing you to urinate all day), 500mg chromium, 60mg CoQ 10, and a multivitamin.

The magic in this combination is how these various elements cascade each other. I would explain in detail here how each of these do amazing things on their own, but I can't sit any longer. I'll post more on other meals and adjustments I'm making later.