Saturday, March 11, 2006

Peeking Under The Abstractions

One of the things I've been doing at $WORK lately is hacking away on atomic renames for Subversion. You see, at the moment the Subversion filesystem doesn't have an actual "rename" operation, when you run 'svn move' you actually get the equivalent of having done a 'svn copy' followed by a 'svn delete'. Back in the day this apparently seemed like a good idea, but in retrospect it turns out that it's actually important to be able to tell the difference between "I moved this thing" and "I copied this thing and I happen to have deleted the original in the same revision".

Unfortunately, it's nowhere near as simple as one might think to make the Subversion filesystem support renames. You run into all sorts of sneaky little problems, and it's not entirely clear how to resolve them. We've got a plan, but all the ramifications of the various ideas still need to be explored and fleshed out.

On top of that, it turns out that there's a pretty limited number of people who can do this sort of work. It requires a bunch of low level knowledge of how the Subversion filesystem works, and while there are a number of people who know enough to do it, most of them are off busy with other things.

As a result I've been digging into it myself, and I'll be the first to admit that I don't have anywhere near the background necessary, at least not yet. I mean I know in theory how the filesystem works, but once you get down and actually start messing with the code that implements the thing you're diving through several layers of abstraction and getting far enough into the implementation details that it's often difficult to tell what's going on at the higher levels by looking at the lower level code, it's all berkeley db tables filled with serialized objects that at first glance don't have a while lot to do with the higher level concepts they represent.

Adapting to coding in this new environment doesn't really play nicely with the fact that it's not really totally clear what the new code should be doing, let along how it should be doing it...

For the first few days I didn't make a whole lot of progress. The actual low level svn_fs_move operation had already been implemented, but there were (and are, for that matter) still a number of places in the code that don't work correctly in the face of moves. For my first task I was working on making the history tracing functions work properly. I spent several hours on two consecutive days tracing through the code, adding printfs here and there, and generally getting awfully confused about why my changes were not fixing things. Then I spent several hours staring at documentation, looking at berkeley db dumps from filesystems created during the tests I'd written, and finally after all that I was able to go back and write some code that almost worked. It's not perfect, and I've had to revise it twice already to correct problems that became clear only later, but it's definately progress, and that's something.

It's as if I spent the first couple of days walking around in the dark, bumping into things. Finally, after a few bumps and bruises I've been able to basically figure out where all the furniture is, at least enough to make my way through it without tripping over things too often.

The whole thing has definately been an eye opening experience though, pulling back those levels of abstraction that separate you from the details of how things work and actually having to tweak the parts of the system that you'd previously been able to just take for granted. I mean don't get me wrong, I'm enjoying every minute of it, but let's just say I've got a whole new respect for the people who made this stuff work, cause it's not trivial stuff, not even close to it.