A Screencasting Toolchain, and Bad Code

Inspired by
Roumen’s screencasting, I decided to see if I could build a (free) toolchain for constructing screencasts on Solaris systems; specifically,
I wanted to be able to screencast (that is, turn a desktop session into a flash animation, complete with sound)
from my living room, on my Solaris x86 laptop. A couple of late nights yielded a reasonable success.
Audio record and playback just works on this laptop, which is a good start. On the
software side, the linchpin of this is vnc2swf, and edit_vnc2swf both written by Yusuke Shinyama.
Simply put, this allows you to use VNC to create flash animations.

If you’re familiar with VNC, the first part of this is pretty much a snap.
You set up a VNC server, attach vnc2swf to the server, and it helpfully records whatever happens in the VNC
screen. It’s sufficiently interactive that you can start to record something, then pause, restart, etc.
vnc2swf has only one dependency, ming-0.2a, which is simple to
download and build.

But screencasting also needs audio, and that’s where edit_vnc2swf— and my trouble– starts. edit_vnc2swf‘s job is to allow you to edit and attach audio to the animations which vnc2swf outputs. edit_vnc2swf is written in python, a language with which I have little familiarity. It depends
on either the PyGame library or on the PIL library. Both are python extensions which
must be compiled. I have nothing against python, but at this point I was wishing for a java jar file
with the needed functionality!

PyGame is in turn built upon libSDL, SDL_image and SDL_ttf. As I mentioned above, the other
option is to forgo the PyGame/SDL/SDLttf/SDLimage combo, and instead use PIL; the drawback is that
the resultant program has less functionality. If you just want to use the basic functionality,
you can safely skip the rather harrowing procedure of getting PyGame to work.

Getting PyGame Running

Now, your mileage may vary, but this stuff is frankly a pain to compile properly; a bug in the version
of GCC we ship with S10 (due to be fixed soon) makes the build fail in strange ways (some of the component
.a’s wind up empty). And unfortunately, the python which comes with S10 is not well configured to allow
one to easily add python extensions (which is, I think, a bug. I’ll file it). You could work around this
by getting the python available from blastwave.org.
The real trick is getting PyGame to work properly. The problem I encountered was that colors were not
rendering properly; after some consultation with Yusuke, we determined that we had an endianness problem:
An attempt to render a blue square resulted in a red square! (presumably because RGB values were being
read as BGR).
The trouble is that PyGame consumes libSDL’s header files, and for whatever reason its build process
doesn’t set the right #defines; as a result, PyGame trips over the following clause in SDL_byteorder.h:

/\* Pardon the mess, I'm trying to determine the endianness of this host.
I'm doing it by preprocessor defines rather than some sort of configure
script so that application code can use this too.  The "right" way would
be to dynamically generate this file on install, but that's a lot of work.
#if  defined(__i386__) || defined(__ia64__) || defined(WIN32) || \\
(defined(__alpha__) || defined(__alpha)) || \\
defined(__arm__) || \\
(defined(__mips__) && defined(__MIPSEL__)) || \\
defined(__SYMBIAN32__) || \\

A lot of work indeed. It was a lot of work to debug this, dammit. This is a very badly
constructed piece of code, because a failure to identify the platform leads to a default of
big endian! (never mind that powerpc platforms may run in either big or little endian mode). It
would have been trivial for the author to code this as:

#if defined(... little endian list ...)
#elif defined(... big endian list ...)
#error Could not determine endianness!

Putting it All Together

At this point I had the graphics processing in hand, and just needed to work out the audio. After
some experimentation I settled on the following:

$ /usr/bin/audiorecord -s 44.1k out.au
$ sox out.au out.mp3

(At some point, it would be nice to be able to use Audacity
on Solaris for this stuff. I wonder if it works?) So now, I can use something like this:

$ /usr/sfw/bin/python edit_vnc2swf -H -a out.mp3 -o screencast.swf raw.swf

To compile the final screencast.

As a final note– In the process of building this toolchain,
I built my own libSDL, libSDL_image, and libSDL_ttf. However, I’d recommend that you simply get
the following packages from blastwave.org. As I mentioned above,
another route you could take is to try using PIL (the python imaging library):

  • From blastwave, fetch:
    # pkg-get -i vncserver -i vncviewer
  • Download and compile libming.
  • Download and compile vnc2swf and edit_vnc2swf. Setup a vnc server, and test vnc2swf!
  • From blastwave, fetch:
    # pkg-get -i python -i pil -i sox
  • If using PyGame, fetch from blastwave the following, then compile PyGame:
    # pkg-get -i sdl -i sdlimage -i sdlttf -i sox

And then just make sure to use the /opt/csw/bin/python binary which blastwave installs.
Coming soon… an actual screencast!

So here’s a challenge– I’ll post a $300 bounty (let’s say expiring at the end of 2005) for an open-source
pure Java solution to this problem (record from VNC to SWF, then delete specific frames, and attach audio
in mp3 or wav format with some “attach at frame #x” controls). Please be able to deliver it all in a single
jar archive. I presume this wouldn’t actually be to hard given the fact the java-based VNC clients and Java-based
SWF processing tools are all available already.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s