The Problem With The West Coast

So I'll be the first to admit, there are a lot of great things about living on the west coast. I've got a cool job, this place is filled with incredibly cool computer geek stuff to do, it doesn't snow, and the list goes on. But none of that changes the fact that I just got an evite to Rob and Jessica's annual pre-thanksgiving get together, and I won't be able to go.

If/when I bail and head back to the east coast, it'll be because of things like this. I mean really, how can I be asked to pass up on like 10 different kinds of mashed potatoes for multiple years in a row?

Oh well, at least I'll be home for the week between giftmas and new years, that's something.

Movies

So I stopped at Fry's on the way home from work tonight, with the intention of picking up a copy of Batman Begins, which just came out on DVD last week.

But of course, they were out of Batman Begins. This happens to me every time I try to get a new movie at Fry's, you'd think I'd eventually learn...

Instead, I picked up the new From the Earth to the Moon boxed set. Man this is a fantastic series. I mean I loved Apollo 13, and this is like watching the 12 hour extended directors cut of that movie.

If you have any interest in all in space, you've probably already seen it, but even if you have you should still pick up the DVDs and watch it again, it's that good.

Week One

So I just finished my first week at CollabNet, and man has it been cool.

I keep having these moments where I'll just have hit send on an email to the Subversion dev list, and I think to myself "ok, now I need to get some real work done", and then it occurs to me that sending that email was part of my real work ;-)

In more mundane news, I've managed to come to an understanding with whatever deity is in charge of the Bay Area commute, and now my morning and afternoon drive time has settled down to about 45 minutes each way. That isn't really all that bad, especially when you consider that it's 45 minutes of actual driving, as opposed to an hour of stop and go hell, which is what I was dealing with at the beginning of the week before I figured out the magical series of roads that would let me avoid taking 101 all the way up the penninsula.

Anyway, I'm off to the east coast for the weekend. Tomorrow is Psi U's 3-3, and everyone knows I just can't pass up an opportunity to spend 12 hours at a party after 6 or 7 hours on a plane. Also notable about this trip is that I'm actually leaving my laptop at home. I can't recall the last time I flew without one, but for once I'm actually not in danger of getting paged at 3 in the morning and having to drop everything to fix some problem. It's a nice feeling.

A Trip To Windows Land

I'm involved, in one way or another, with several projects that build and run on Windows systems. Now most of the developers tend to be Unix people, so the Win32 side of the fence rarely gets as much love as it should. Part of the reason, frankly, is that it's really easy to get a development version of your average piece of open source software up and running on a Unix machine, and it's really hard to do the same thing on Windows.

Why is it so hard on Windows? Well, there are two major reasons.

First, the tools tend to be harder to get. Your average Unix machine either comes with all the compilers and libraries you're likely to need, or you can get them really easily. The same tools don't tend to come in the box with your standard Windows install, so you have to go out and get them. This means spending money, or at least it did until very recently when Microsoft released their Visual Studio 2005 Express Edition, basically just a cut-down version of Visual Studio that you can download for free. Even with the appearance of a free-as-in-beer win32 development environment (I don't consider cygwin or mingw real alternatives, if I want to use Unix tools I'll do it on a Unix system) it's still pretty hard to get things up and running. Installing the Microsoft Platform SDK and getting Visual Studio to recognize it took me most of this morning, for example. That said, this is a Beta version of the IDE, so hopefully it'll be easier when the real thing ships.

The second problem is sort of a catch-22. It's hard to build most projects on Win32 systems because not too many people do it. This means that the ones who do build them tend to be able to deal with the "Pain in the Ass" nature of the system, so the process doesn't tend to get better. In Unix land there are enough people going through the pain that sooner or later they fix the problems. Thus, fewer people work on the win32 versions because the barrier to entry is so high, and the cycle repeats itself.

Anyway, this weekend I got a bit motivated, and actually downloaded and installed Visual C++ 2005 Express Edition, got the Microsoft Platform SDK working (for the record, the instructions they post don't seem to work for me, I had to use the trick mentioned here where you copy the headers and libraries into the Visual Studio install directory), and I'm inching towards getting Subversion and APR to actually build.

I'm hoping that with the advent of some decent free-as-in-beer tools it'll be easier for us Unix people to actually confirm that things work in Windows land, and the first step seems to be going through the process and figuring out how to streamline it a bit.

Producing Open Source Software

If you've ever wondered exactly what makes an Open Source project successful, and I mean really successful, as in building a sustainable community, not just having one guy madly hacking away in his spare time and giving the results away to the world, then you should absolutely take a look at Karl Fogel's new book, Producing Open Source Software. Karl's one of the reasons that the Subversion community has taken off the way it has, and while reading over the online version of the book this weekend I was continually struck by how on-target his observations were. I mean I was actually there for many of the situations he writes about, and I still didn't see all the angles that he covers.

Seriously, go read the book, and buy a dead-tree version when it hits the stores, you won't regret it.

I was particularly interested in chapter 5, Money.

Why? Well, it's all about how to keep things from getting all weird when people start getting paid to work on the project, either as full time sponsored developers who work for a company with a vested interest, or as contractors paid to fix a specific problem one of their clients needs fixed, or whatever. You see, I'm going to have to deal with that issue personally real soon now. I just accepted a job offer from CollabNet working on their Subversion team, so for the first time my day job will actually be working on some of the stuff I do in my spare time. As you might expect, I'm looking forward to this particular application of the stuff Karl writes about in chapter 5.

This is where I insert the "I'm going to miss the people at Bloglines" and "I'm really looking forward to this new opportunity" sections, but everything I write sounds totally lame. So just assume that I wrote something to that effect, cause I really will miss working on Bloglines, and at the same time I'm really psyched about getting paid to hack on Subversion.

When Should I Have Learned This?

A few days ago I read a blog entry by chromatic about the kind of things that people learn in Computer Science programs these days. He was talking about the difference between theoretical computer science and the kind of skills you need to make it as a professional programmer, two things that are somewhat related but nowhere near identical. That dichotomy is interesting, but it's not what I want to talk about today. I want to talk about the million and one bits of information that have come out of the CS world but that for one reason or another you don't really learn in a CS degree.

Every CS major back at RPI had to write an implementation of strstr at some point, but I don't remember ever learning about better ways to do it than the naive algorithm. At some point between then and now I heard the term "Boyer-Moore Algorithm", but until today I never actually read the paper and saw how it worked. If I hadn't been poking around in the mod_include source code a few weeks ago I'd never have heard of a BNDM search algorithm. Both of these are really cool ways to do a faster string search, but even with my CS degree I'd never heard of them. Fortunately, I tend to hang out in the kind of places you hear about such things, but do we really want people to learn about these sort of things by accident?

How about techniques for multithreaded programming? If I hadn't read the documentation on the Boost threads library I wouldn't have learned the trick about figuring out lock ordering based on the address of the two mutexes, and if I hadn't been following the writing of Herb Sutter I wouldn't have realized exactly how many ways that solution can break down. If I hadn't read the FreeBSD mailing lists religiously for years I wouldn't have learned about the importance of lock ordering at all, despite the fact that I recall at least two classes back in school that taught the basics of multithreading. This isn't like the string searching stuff, where we're just talking about getting the answer faster, this is about how to write multithreaded programs that actually work at all.

If there's this much more to the topics that I did learn in school, what about the topics I never learned there? That's the thing that really worries me. It seems like it's way too easy to think you're doing things the right way and to be completely and totally wrong. Where should programmers be expected to learn this sort of thing? The thing that really worries me is that most of them just aren't learning them, and as a result are going to spend most of their careers screwing up stuff that collectively we've already figured out, but nobody got around to telling them about it.

Thoughts on Threading

So this week I've been thinking a lot about threading.

Specifically I've been thinking about large, complex multithreaded servers that are difficult to write, to test, and most importantly to modify later on, especially if you are not the person who actually designed and implemented them in the first place.

Among the better papers I've seen on the subject is "Reasoning about SMP in FreeBSD", by Jeffrey Hsu, who did a lot of work on locking the FreeBSD network stack as part of it's SMP project. Interestingly enough, Hsu is now working on DragonFlyBSD, which has taken a rather different approach to its multithreading than FreeBSD did. Makes you wonder if the more traditional techniques (fine grained mutexing, a-la the Solaris kernel) used in FreeBSD are really the right way to go...

Go read the paper, it's good.

Anyway, the big idea of that paper is that to effectively think about a multithreaded system you really need to understand what it's doing. You can't effectively add locking to one part of the system without totally understanding all the interactions it has with the rest of the system. You really need to see the big picture in order to know how to make the individual decisions.

I've been having trouble seeing the big picture, that's the problem.

Planning Holiday Cheer

So I know I've already complained about how much it sucks to have to buy plane tickets and fly across the country for all those family gatherings and social events I'm expected to be at back on the east coast...

But I haven't had to do the "travel over the holidays" thing yet, so I'm sure that all my complaining is for naught, because I haven't experienced real pain yet...

Don't worry though! It's coming! I just booked my flight home for the holidays. It turns out that $COMPANY has a bunch of "Shutdown Day" days off between XMas and New Years, so I'll be home from December 23rd through January 2nd. Judging from the price Jet Blue charges, and the fact that every reasonably priced rental car is taken already, I'm sure there'll be a terrifyingly large number of people trying to get someplace other than where they live, just like me. It should be loads of fun. Now I know why Rob J doesn't travel during the holidays...

Anyway, I guess that means I'll have to figure out where the cool kids back home are going for a new year's party...

Fun with Email

So earlier this week I ran across Matt Sergeant's O'Reillynet article about Qpsmtpd.

Now I've known about Qpsmtpd for a while now, I saw a great talk about it at OSCON a few years back, and I know the ASF uses it for their mail, so I'm not exactly new to the concepts here, but one thing in the article struck me as just too damn cool. You see, while there are special plugins for Qpsmtpd that let it deliver mail to Qmail or Postfix, the simplest way to integrate it with your existing system is just to run your normal SMTP server on an alternate port, and have Qpsmtpd do all the fancy filtering and spam blocking, then just forward it on via, you guessed it, SMTP.

Back in the Apache world, some of the HTTPD developers have been working on mod_smtpd, which lets you make the Apache HTTPD speak SMTP.

In the open source world good engineers borrow ideas and great engineers steal them, so much of the mod_smtpd design is based on Qpsmtpd. Ironically enough, the Qpsmtpd design was inspired by the modular design of the Apache HTTPD server, so now the cycle is complete...

Now mod_smtpd is just getting to the point where it's able to do useful things, but it still doesn't have a queue plugin that lets you forward mail via SMTP.

So I wrote one this morning.

It's pretty bare bones, with a really basic SMTP implementation under the hood, but it is enough for my mod_smtpd server to be able to forward mail to my exisitng Postfix server, and that's the whole point. Grab the patch here, if you're interested, although honestly, if you're interested enough in mod_smtpd at this point that you'd actually want the code, you're probably on the mailing list already ;-)

APR-Template 0.0.1

I just cut the first release of APR-Template, my Apache Portable Runtime based templating system. Since I already wrote a perfectly good README file that explains what it is, I'll just quote it here:

APR-Template is a minimalist template library implemented on top of APR and APR-Util. It's designed to provide a lightweight templating solution that fits easily into an Apache HTTPD module or other APR based program.

The template language looks like this:

<html>
<head>
<title>[% print title %]</title>
</head>
<body>
[% if add_header %]<hr>[% end %]

<h1>[% print title %]</h1>

<ul>
[% for e in elements %]
<li>[% print e %]</li>
[% end %]
</ul>

[% if add_footer %]<hr>[% end %]
</html>

Which produces about what you'd expect. If you've used Template-Toolkit or EZT you'll find APR-Template to be similar, although the feature set is quite stripped down, so if you find yourself trying to do a whole lot in your template it's a sign that you either need to put that logic into your C code or find a different template engine.

For examples of how to use APR-Template in a program please refer to the tmpl command line tool (in src/cmdline/tmpl.c) and the mod_template_example apache module (in examples/mod_template_example).

There's a bunch of stuff I still want to do with it (see the TODO file in the tarball for details), but the current version has just enough functionality to be useful, so I decided it was a good point to cut a release. Get it while it's hot: http://electricjellyfish.net/garrett/apr-template/apr-template-0.0.1.tar.gz