Wednesday, March 29, 2006

OSCON 2006

Well, it looks like I'll be making my yearly pilgrimage to Portland again in July. I was lucky enough to get a number of my proposals accepted for this year's OSCON. In fact, I really wasn't expecting quite that many of them to be picked, so I had to turn one down

In the end, it looks like I'll be doing a 3 hour tutorial (my first) on "Adding Version Control To Your Application With Subversion", and two 45 minute talks, one on "Writing Portable C Code With APR" and another on "Extending Ruby With C".

I'm not nervous about the two talks, I can talk for 45 minutes about APR in my sleep, and I've written enough about Ruby extensions to be sure that I won't have any trouble coming up with enough interesting things to fill the time.

The tutorial is another story alltogether though. I'm pretty much terrified about that one. In theory I know I've got enough content to go around, but at the same time it's still THREE FREAKING HOURS of time to fill. I'm sure I'll be able to come up with enough stuff to talk about, but I'm definately nervous about getting up in front of a bunch of people and talking for that long, I'm just not sure I'll be able to stay interesting, even to myself.

Anyway, I suppose I've got some presentations to work on...

Tuesday, March 21, 2006


I've been reading through the Amazon S3 developer documentation tonight, and I've gotta say I'm quite impressed. This is a perfect application for a REST API, and they sure seem to have nailed it. I fully expect that as the available APIs built on top of it mature it'll start getting built into all sorts of interesting applications.

I'd love to see a library for using this from Mozilla plugins. Seems tailor made for persistantly storing data that needs to be accessible from multiple machines, If-Modified-Since and ETags would let you avoid wasting bandwidth downloading stuff you've already seen, and Mozilla's already got all the infrastructure you'd need to do the http-fu.

The other main application I'm seeing for this is encrypted offsite backups for people who are way too lazy to maintain their own storage. Fortunately, it looks like Brackup has that use case covered.

Monday, March 20, 2006

Blast from the Past

This year there are a number of friends of mine who are turning 30. As a result, we decided a while back that there needed to be a party. There were all sorts of grandiose plans of what to do, but in the end we finally just decided to go back to Troy and hang out for a weekend.

So on thursday night I flew out to Boston (it's actually cheaper to fly to Boston from San Jose than to fly to Albany from San Jose), crashed at my parent's place for a while, and then hooked up with James and Rob and drove up to Troy.

It was a wonderful trip back down memory lane. A few of the more memorable moments:

Not 5 minutes out the door of James and Rochelle's house Rob's car blows a tire, then when trying to remove it James manages to bend the wrench. It was seriously right out of one of our fun filled cross country road trips, although this time Rochelle was able to drive down and pick us up, which is quite a bit nicer than waiting for triple A.

Emily decided that in honor of St. Patrick's day it was necessary to paint the dining room of the house green. Very green.

A trip to crossgates on saturday for no really good reason, other than the lack of anything particularly exciting to do at the house.

We confirmed, once again, that Minh really shouldn't be allowed near the goldschlager, at least not that early in the evening anyway. I hope he got his clothes out of the freezer before he went home...

A number of remarkable appearances were put in by people who aren't usually around, and they were all in rare form. Greg R. making out various people he probably shouldn't be. Maloney proving that he still remembers all the words to the Humpty Dance. Jessica leading the crowd in doing the Electric Slide to NIN's Closer. James P. sleeping in the closet, which was probably the most comfortable place available in retrospect. Rob collecting a small crowd of antisocial people out on the front porch, despite the fact that it was incredibly cold out.

As Cara said, it was a remarkably accurate recreation of her freshman year, sans drama and generally a bit more fun for everyone.

So if you didn't make it out, you missed a great time. You can see some of the better pictures I took in this flickr set, although I did remove some of the less tasteful ones ;-)

Ready For My Closeup

So one of the things I've been doing at $WORK is helping some NetBeans engineers at Sun with their work on a NetBeans Subversion plugin. It's been going on for a while now, with my participation being mostly limited to answering questions about Subversion, looking at their prototypes from the point of view of an experienced Subversion user, and just generally trying to help them make their Subversion integration just as cool as their CVS integration is now.

Anyway, a few weeks ago I stopped by the Sun Santa Clara campus and they interviewed me for the Sun Developer Channel. You can see the video here, my part starts about 20 minutes in. The whole experience was a lot of fun (we did 4 takes, it's hard to have the same conversation 4 times in a row!), and I'd just like to thank the Sun Developer Network crew from inviting me in.

Wednesday, March 15, 2006

DynamicForward, Where Have You Been All My Life?

So I just recently discovered the DynamicForward option for openssh, and man, I've gotta tell you, it's just about the coolest thing ever. I've been using ssh to forward specific ports practically forever, but this option starts up a full fledged SOCKS proxy, so you can point Firefox at it and it'll let you do arbitrary HTTP requests through your tunnel, not to mention all the other applications that work with SOCKS proxies.

Just run something like ssh -o 'DynamicForward 1080' and you'll get your SOCKS proxy listening on port 1080 on your local machine.

Now that's cool, but combine it with Firefox's PAC support for dynamically configuring proxies and you're really cooking. You see, the regular proxy settings in Firefox just let you set your proxy and exclude a list of URLs if you're so inclined. That's nice, but what I really want is for a specific set of URLs (say, the internal servers on a remote network accessible only via ssh) to be proxied but everything else should use my direct connection to the internet.

First, we write a PAC file, which is just a text file that contains a single Javascript function:

function FindProxyForURL(url, host)
if (dnsDomainIs(host, "")) {
return "SOCKS localhost:1080; DIRECT";
} else {
return "DIRECT";

Then you stick it up on a web server someplace and set the URL as your Automatic Proxy Configuration URL in your Firefox Connection Settings. From then on every time you try to connect to a host in * it'll go through the proxy (or try to go directly via the internet if the proxy doesn't work), and everything else just uses your local internet connection.

There are a bunch of other interesting and useful functions like dnsDomainIs() that you can make use of in your implementation of FindProxyForURL(), just take a look at the docs.

If you want to confirm that the things you want to go through the proxy are doing so, just start up the ssh tunnel with the -v option and you'll get debug output every time a new proxy connection is established.

I seriously cannot believe that all this functionality was just sitting there in tools I literally use every day and I never knew about it until now. No more screwing around with /etc/hosts and port forwarding for me!

Interesting Developments

Looks like Theo is looking for people to help with his latest project, a Group Communication System, like Spread, but with all the annoying bits fixed.

His list of goals looks encouraging, practically a laundry list of everything I hated about Tib/Rv back when I had to deal with it. Plus, Theo's one of those scary-smart people who's disturbingly good at most things, so if he's working on it I give it pretty good odds of kicking some serious ass.

Here's to hoping that his post pulls some contributors out of the woodwork. It would make total sense for people to work on this, it's the kind of tool that can be applied to practically any system, and the existing ones out there do have serious problems, either in functionality, reliability, licensing/cost, or most often all of the above.

Sunday, March 12, 2006

Keyboard Memory

It's disturbing how often I find myself hitting the "y" key when I'm done looking at something these days, six months of reading the majority of my email via gmail has gotten me totally trained to hit that key when I'm finished with something. It doesn't even have to be email anymore, I just caught myself doing it on craigslist for crying out loud...

Saturday, March 11, 2006

Peeking Under The Abstractions

One of the things I've been doing at $WORK lately is hacking away on atomic renames for Subversion. You see, at the moment the Subversion filesystem doesn't have an actual "rename" operation, when you run 'svn move' you actually get the equivalent of having done a 'svn copy' followed by a 'svn delete'. Back in the day this apparently seemed like a good idea, but in retrospect it turns out that it's actually important to be able to tell the difference between "I moved this thing" and "I copied this thing and I happen to have deleted the original in the same revision".

Unfortunately, it's nowhere near as simple as one might think to make the Subversion filesystem support renames. You run into all sorts of sneaky little problems, and it's not entirely clear how to resolve them. We've got a plan, but all the ramifications of the various ideas still need to be explored and fleshed out.

On top of that, it turns out that there's a pretty limited number of people who can do this sort of work. It requires a bunch of low level knowledge of how the Subversion filesystem works, and while there are a number of people who know enough to do it, most of them are off busy with other things.

As a result I've been digging into it myself, and I'll be the first to admit that I don't have anywhere near the background necessary, at least not yet. I mean I know in theory how the filesystem works, but once you get down and actually start messing with the code that implements the thing you're diving through several layers of abstraction and getting far enough into the implementation details that it's often difficult to tell what's going on at the higher levels by looking at the lower level code, it's all berkeley db tables filled with serialized objects that at first glance don't have a while lot to do with the higher level concepts they represent.

Adapting to coding in this new environment doesn't really play nicely with the fact that it's not really totally clear what the new code should be doing, let along how it should be doing it...

For the first few days I didn't make a whole lot of progress. The actual low level svn_fs_move operation had already been implemented, but there were (and are, for that matter) still a number of places in the code that don't work correctly in the face of moves. For my first task I was working on making the history tracing functions work properly. I spent several hours on two consecutive days tracing through the code, adding printfs here and there, and generally getting awfully confused about why my changes were not fixing things. Then I spent several hours staring at documentation, looking at berkeley db dumps from filesystems created during the tests I'd written, and finally after all that I was able to go back and write some code that almost worked. It's not perfect, and I've had to revise it twice already to correct problems that became clear only later, but it's definately progress, and that's something.

It's as if I spent the first couple of days walking around in the dark, bumping into things. Finally, after a few bumps and bruises I've been able to basically figure out where all the furniture is, at least enough to make my way through it without tripping over things too often.

The whole thing has definately been an eye opening experience though, pulling back those levels of abstraction that separate you from the details of how things work and actually having to tweak the parts of the system that you'd previously been able to just take for granted. I mean don't get me wrong, I'm enjoying every minute of it, but let's just say I've got a whole new respect for the people who made this stuff work, cause it's not trivial stuff, not even close to it.

Typing Lessons

So I've made my semi-annual switch from vi to emacs, and as a result I've been making way more than the usual number of typos all week long. I find it worthwhile to switch back and forth every now and then, just to keep my skills sharp, and as much as I like the simplicity of vi there is something to be said for the power of emacs.

On top of that, I've also switched my desktop setup at home around a bit. In an attempt to keep from staring at the computer ALL the time, I've hooked the laptop up to the LCD display, giving me a hell of a lot of screen real estate, but also serving to keep things so entrenched in wires that I'm unlikely to get motivated enough to take it into the living room where it can distract me while I'm consuming the latest episode of BSG or Lost or whatever...

Oddly enough, the only keyboard I had lying around to use was a Das Keyboard I ordered on a whim a few months ago. It turns out to be a very nice keyboard (feels a lot like those old IBM keyboards that sound like a machine gun when you get a decent touch typist on them, but a bit quieter), although the lack of any markings on the keys combined with the switch to emacs has certainly been screwing with my productivity a little bit.

On the bright side though, it is giving me a hell of a refresher course in typing without looking at the keys. It's remarkable how many things I never really learned to touch type (numbers and special characters come to mind) that I use constantly, but necessity is a great motivation, so I am learning again...