Lucene4c 0.01: Inching Towards Usefulness

When I picked up lucene4c again a week or two back I told myself that I'd release the first version as soon as I had something that could print out all the documents in a segment that contain a particular term.

This means I need to be able to parse the segments file to figure out what the name of the segments are, the terminfos file and it's index to find out info about the term you asked for, the frequencies file to give the list of documents that contain that term, and finally the field data file and its index so we can print out the field for each document that contains the identifier for the doc.

I had sort of figured it would take me longer, but I made it to that milestone tonight, so the first release, version 0.01, is now available at http://electricjellyfish.net/garrett/lucene4c/lucene4c-0.01.tar.gz.

After building the source you should have a little program named 'lcn' in the src/cmdline directory that will allow you to play around with an index. Specifically, you can do something like this:

$ ./src/cmdline/lcn termdocs test/data/index _c7 jsp contents path
/Users/rooneg/Hacking/lucene4c/jakarta-lucene/src/CVS/Entries
/Users/rooneg/Hacking/lucene4c/jakarta-lucene/src/jsp/CVS/Repository
/Users/rooneg/Hacking/lucene4c/jakarta-lucene/src/jsp/results.jsp
/Users/rooneg/Hacking/lucene4c/jakarta-lucene/src/jsp/WEB-INF/CVS/Repository
$

This just searched the test index for the term 'jsp' in the field 'contents' and for each result printed out the field 'path'. You should also be able to use it on other lucene indices, as long as they don't use the compound segment format, since I haven't even started work on the compound stuff yet.

If you're interested in the project please feel free to download it and kick the tires, I'd appreciate any feedback you might have.

A Great Xmas Present

I think of all the presents I got this year, my favorite is the Amazon book review Ben Collins-Sussman wrote for Practical Subversion.

When I originally started work on the book my intention was to take a different approach from the existing Subversion documentation, targetting a more experienced user as opposed to the newbie. I figured that since the existing documentation was already there filling the newbie role, there would be room for a book that didn't spend a ton of time teaching the basics of version control, and just jumped right to the stuff you need to know to really get going with Subversion

According to Ben's review I appear to have succeeded.

I've seen a number of reviews of the book so far, and I've enjoyed reading all of them, but I think this one is my favorite, both because it tells me that I accomplished my original goal and because it's always great to hear that people you respect like your work.

Thanks Ben!

Giving to Open Source

Most of the time, when I "contribute" to an open source project it's in the form of a bug report, or a patch, or helping answer user questions on a mailing list or IRC. These are all important contributions, and I'm happy that I have the time and ability to do so, but that's not the kind of contribution I'm going to talk about today.

In this day and age it takes more than bug reports, source code, and user support to keep a project going, it takes money. Money for bandwidth, hardware, for (as much as we wish it wasn't so) lawyers, and for a hundred other things that are needed to keep the wheels going round. Below a certain size these sort of things can be provided by the developers, or donated by a generous employer or user, but eventually you hit the point where need exceeds availability.

This doesn't even consider the fact that there are jobs in any large open source project that don't get done because the people who are able to do them can't afford to take the time off from their paying job to do them.

Fortunately, there are a number of organizations out there who are dedicated to providing financial support to these kind of projects. Personally I have experience with three of them, the FreeBSD Foundation, the Perl Foundation, and the Apache Software Foundation. At various points over the past few years I've given money to each of these groups, and I expect I will continue to do so in the future.

What do I get in return for these contributions? Well, the FreeBSD foundation funds development on my operating system of choice, allowing developers access to hardware and in some cases direct funding for work that would not otherwise get done. The Perl Foundation provides support for various projects related to the Perl programming language, including a wide variety of development grants that fund interesting new work. The ASF provides legal and technical infrastructure for a huge number of software projects.

I do admit, it would be nice if some of these groups were a bit more vocal about exactly what they're doing, but honestly I think they're doing a decent job at that, and in the end they are funding good work that needs to be done, so I'm happy to contribute, and if you're able to do the same I hope you do as well.

Lucene4c Progress

So one of my "work on it when I'm bored with other things" projects is lucene4c, a port of the Lucene search engine from Java to C, using the Apache Portable Runtime to make writing in C more sane and portable.

Why would someone want to do such a thing? It seems like an awful lot of work, for not much gain, since the Java version of Lucene works damn well already.

Well, personally I want to be able to write an Apache module that lets you search over a Lucene index, and it's a lot simpler to do that in C than it is with Java. I'm also interested in the internals of search engines, and implementing the guts of one seems like a good way to learn. Plus, it's a neat way to kill some time on a lazy weekend at home.

Anyway, I had stopped working on it a few months ago because I got fed up with trying to track down a particular bug. This weekend I started playing around with it again, just for fun, and figured out the problem (allocating a file in the wrong pool, when I later tried to read from it the memory had already been reused) that had stopped me before. With that obstacle out of the way I'm back in business, making slow but steady progress.

My first goal is to be able to read a Lucene index, created by the Java version of Lucene. I figure a read-only version of lucene4c is at least sort of useful, and it's a lot easier than implementing the indexing side of things.

At this point I'm getting within sight of functionality. I can parse the segments file, the fieldinfos file, and the terminfos index file, although I need some tests for the tii stuff because it's likely that some of that isn't working correctly.

My next step is code to read the terminfos file (which shouldn't be that bad, since I've already got code to do much of that work from parsing the terminfos index file) and then the frequency file. With those two steps out of the way I'll in theory be able to answer questions like "what documents did the word lucene appear in", which is nice since it would at least approximate some sembalance of searching functionality.

A side issue I've found along the way is that the fileformats documentation, while decent, is not entirely up to date. I'm making a concerted effort to note places where reality seems to differ from the documentation, and at some point I hope to send some patches upstream to fix the problems.

Instrumenting the low level input stream classes has turned out to be my best way of figuring out exactly what's going on, it's one thing to trace through the Java code by hand, but actually having something say "every 10 terminfos we read this extra VInt" is a fantastic way to get pointed right to the spot you misread the Java code.

Anyway, back to parsing the terminfos file...

PDF Books

I'm really starting to like the recent trend towards publishers making books available in both PDF and Dead Tree form. It's a refreshing change of pace when compared to the regular DRM ladden crap we get from the entertainment industry, I'm glad that at least the technical book publishers appear to be a reasonable bunch with regard to such things.

Specifically, I've recently bought both Programming Ruby and Lucene In Action in both PDF and Dead Tree form, and I've been quite pleased with the results.

The way it's worked is simple, you order the book from the publisher like you normally would, and for either slightly more or the same amount you would normally spend in a bookstore you get the hard copy shipped to you and a personalized PDF (with some kind of "this pdf prepared for <your name here>" thing on each page) prepared so you can download it immediately.

For technical books this is fantastic. If I order a tech book online I'm probably interested in it right now, so the PDF means I can get some instant gratification, which is a big plus. Then, later on I get the text version which, lets face it, is much nicer to read. In the long run though, I'll always have the PDF on my laptop so I can look something up if/when I don't happen to have the hard copy on me.

As for the DRM issue, I like to think that the personalization of the PDF is a nice way to keep people from passing them around on file sharing systems, although it's possible that's just me being overly naive. Time will tell I suppose.

In any event, I'm pleased with the few books I've purchased in this manner so far, and I plan on doing so again in the future if I'm given the opportunity.

Now back to reading about Lucene...

rooneg at apache.org

It took a little while for the paperwork to go through, but today my apache.org account was set up, and I committed my first changes to the APR.

Its kind of cool being a part of something like that, I've thought the project was an important one for some time, and I'm happy to have the chance to contribute directly to it. Now I'm just hoping to not let down the people who decided I was worth giving commit access to. I figure the first "oh my $DIETY, you committed what!?!?! there are so many things wrong with that commit that I can't even begin to count them" email should show up any time now...

Plus, I must admit, the apache.org email address is pretty cool ;-)

Dependencies Suck

You know, I think it's a bit absurd that I had to install 15 separate perl modules in order to convince the Perl Email Project to successfully send an email with both html and text versions of a message.

I mean I'm all for modularity, but 15 fucking modules! And this is for a project that has as a stated goal to be "minimal in their external dependencies".

I'd hate to see what they would do if they totally pulled out all the stops and depended on whatever the fuck they wanted...

Small, Attainable Goals

So I keep meaning to write up a "What I did at ApacheCon" type entry, but my tendency towards procrastination has kept that from happening...

In the meantime though, I wanted to mention that my latest latest article just went up on O'Reillynet. It's about writing extensions for Ruby in C, and it's already gotten more of a response than my last articles up there ever got.

The last articles led to my book, so does this mean my next book needs to be about Ruby?

Just kidding ;-)

In related news though, I did get to see my book in hardcopy for the first time this week, it's damn cool. Go out and buy a copy, or two, or ten. They make great stocking stuffers.

My First Book Review!

It appears that my publisher has started sending out PDF copies of my book to people they feel would like to review it, and today I got an email from one of them pointing me to his review.

For the impatient, he seems to have enjoyed it, although he makes some valid points about one or two things I could have done better.

It's nice, after all that work, to see that Chapter 8, which was among the hardest to write, is "worth the price of admission," at least in his opinion.

New Laptop

A few weeks ago I got a new laptop.

It may suprise some of you, but it's not a PowerBook this time. Instead, I'm typing this on an IBM Thinkpad t42p. There are a number of reasons I went with an x86 machine this time, including, but not limited to, the following.

• The particular t42p I've got has a truly stunning display, it's 15 inches and 1600x1200 glorious pixels. I love it.
• This thing is damn fast. Building Subversion from scratch is so much faster than it was on my PowerBook that I haven't even bothered to go back and time it on the old machine.
• It's been some time since I worked on a real Unix machine, and I felt the need to "return to my roots" so to speak. At the moment I'm running Ubuntu Linux on it, but now that FreeBSD 5.3 is out I'll probably move to that once I get back from ApacheCon and find some free time.
• IBM has a fantastic friends and family program, which resulted in a huge discount. If you're shopping for a new laptop and you happen to know someone who works at IBM make sure to ask them about it.

It's been a while since I worked on a Linux (soon to be FreeBSD) box on a daily basis, but so far I'm quite impressed. Everything has worked out of the box, with the exception of suspend, which required me to fiddle around a bit to tell the machine to use APM instead of ACPI. Other than that it's just worked.

I will restrain myself from going off on a rant about how back in the old days it would have taken weeks and weeks of pain to make this stuff work, how I would have had to download patches, manually tweak them, email obscure mailing lists begging for help, et ceterra.

So far though, I'm happy with it. There are some aspects of Mac OS X that I miss, but it's also quite nice to be in an environment where the vast majority of Unix software doesn't actually require any manual tweaking to make it do what I want. If you're in the market for a new laptop I can absolutely recommend the IBM t42p, it is a fantastic machine.

You ever feel...

Like you just don't understand the majority of the people in the country?

mod_speedyfeed 0.05 - now with unit tests and etag support

I just finished up version 0.05 of mod_speedyfeed.

New in this release is an Apache::Test based test suite and support for ETags and If-None-Match.

I manufacture an ETag header based on the crc32 checksums of the entries, and drop from the body of the response any that match checksums seen in the client's If-None-Match header. This is similar to the RFC3229 feed support implemented in Sam Ruby's weblog, but I do it dynamically as part of an Apache filter.

For the curious, here's the relevent entries in the CHANGES file.

* Add support for ETags and If-None-Match. Our ETags are crc32 checksums of the entries, concatenated together. If we see a fragment that matches an entry's crc32 in the client's If-None-Match we drop it.
* Account for the fact that Content-Type headers can have arguments.
* Genericified some stuff so it will be easier to add support for other feed formats in the future.
* Added an Apache::Test based test suite.

Get it while it's hot: http://electricjellyfish.net/garrett/mod_speedyfeed/mod_speedyfeed-0.05.tar.gz

As usual, I welcome any comments.

It's done.

Yesterday I sent the examples from Practical Subversion off to my publisher, which was the last remaining task I had before I could call the book officially done. Sometime yesterday (or perhaps today, I'm not entirely sure) the final PDFs were sent to the printer, and I'm told that in about a month we should have actual dead tree versions of the book. It's taken far longer than I'd originally thought it would, and a lot more work than I had anticipated, but I'm glad I did it. I've learned a lot, improved as a writer, and at some point soon I'll be able to walk into a brick and mortar book store and see something with my name on it sitting up on the shelf. That's so much cooler than seeing things I wrote up on a website, and I'm glad I have been given the chance to do it.

Before you ask, no, I'm not planning on writing another book anytime soon. Instead I think I'm going to stick to actually writing code, which I've missed over the past year or so, and while I will continue to write it'll probably be limited to much smaller pieces. This whole process has truly made me appreciate the value of a quick return on investment. Taking an entire year worth of hard work before you actually see the result in a bookstore is not nearly as much fun as a few hours over the course of a week or two before you can see an article up on a website somewhere ;-)

In any event, for everyone out there who is tired of hearing me pull out the "I've got to work on the book" excuse way too often, I'm no longer going to be able to use that one, so I'll have to work on an all new set of excuses for not being all that much fun...

Another Milestone

I just sent the final post-copy-edited chapter of the book back to my copy editor.

My copy editor is great, and the book is a hundred times better because of her, but I'm so glad that I'm done with this part of the process.

Getting a copy of a chapter back with a thousand little notes on it about things you need to fix is just so disheartening.

I recently saw a description of the process of writing a technical book as being similar to "Having your teeth pulled out with pliers, slowly, over the course of a year". It's been a bit more than a year for me, but other than that it's about right...

Apache::Test++

In case I wasn't clear on it in the past, the Apache::Test perl module is just the coolest thing.

The docs on how to get it up and running if you're not building your standard everyday mod_perl module aren't exactly the best (so far I can't find a way to set things up wthout using either Apache::TestMM or Apache::TestMB, neither of which I really care about since I'm using configure + make instead of ExtUtils::MakeMaker or Module::Build, but it was easy enough to fake up an Apache::TestMB one liner to do what I need), but once you're over that part it's just so damn cool.

I've now got a test suite that automatically builds the module, configures and starts Apache, runs automated versions of the tests I was doing manually, and shuts everything down, and all I have to do is type make check.

Should have done this much earlier in the process...

mod_speedyfeed 0.04 - Works With Stock Apache2

So version 0.04 of mod_speedyfeed just hit the net.

Here's the appropriate entry from the CHANGES file:

* Fake up a status line so we can work with versions of Apache that do not have support in the core for the 226 status code.
* Work around the weird way APR-Util's XML code treats xml:lang attributes.
* Compare namespaces as well as element names when looking for the atom:entry and atom:modified elements.
* Added a configure script and makefile.

The big deal is that first bullet point. Justin Erenkrantz was nice enough to point out that even if Apache doesn't have support for the "226 IM Used" status code you can fake it up by manually setting the status_line in the request. This means you don't need to apply a patch to Apache to make use of this module.

I've also got a version of this up and running somewhere people can access it, so feel free to play around with sending various combinations of If-Modified-Since and A-IM to http://quicksort.net/index.atom and let me know what you think.

Next on the TODO list is investigating adding support for more of rfc3229, either some of the other IM values (probably range) or the ETags/If-None-Match parts of the feed IM value.

(I am, of course, an idiot, and neglected to post the link to the tarball... You can grab the new version here.)

mod_speedyfeed 0.03

Just rolled the release tarball for mod_speedyfeed 0.03.

Quoting from the CHANGES file:

* Fix bug in parsing of A-IM header that would cause us to not work in cases where more than one IM was specified by the client.
* Switch to GenX instead of the APR-Util XML code for outputting the document.
* Various code cleanups.

The big reason for the release was the first bullet point, that was a reasonably embarassing bug. The move to GenX is part of my long term plan to stop using the APR-Util XML code, since it's got a lot of weird quirks and I'd rather have something more general anyway.

Grab the new tarball here if you're interested.

As always, comments of any sort are welcome.

mod_speedyfeed 0.02

Ok, so like I said earlier, it only took a few hours of playing around to get the first version of mod_speedyfeed working and out the door.

Get it while it's hot: http://electricjellyfish.net/garrett/mod_speedyfeed/mod_speedyfeed-0.02.tar.gz

You'll also need a patched version of Apache2 to get support for the new 226 status code, grab the patch from the Apache Bugzilla.

What does it do?

In short, it allows you to only send new entries in your Atom feeds down to the clients. The client program adds a few HTTP headers (a If-Modified-Since to tell you what the last time they got was and an A-IM that indicates you support the 'feed' IM) and things just magically work.

Best of all, the content that's sent down, while smaller, remains valid Atom XML, so no real change is needed on the client side other than sending the new headers.

All you need to do on the server side is compile and install the module, it works as a filter so any content that's served up with an application/atom+xml content type is automagically effected.

(The first release is 0.02, not 0.01, since I naturally found a show-stoppingly bad bug at the last second...)

mod_speedyfeed - RFC3229 Feed Support For Apache

So in an attempt to blow of some steam and relax a bit last night I started hacking on mod_speedyfeed, an Apache filter that implements some of the stuff Bob Wyman has been talking about for using a new "feed" instance manipulation method for rfc3229, specialized for serving up Atom feeds.

Yeah, I'm a weird guy, I code Apache modules to blow of steam, I'm willing to admit it.

The idea is that it's a filter that detects responses of an Atom content type, looks for the rfc3229 headers, and if they're there parses the XML and drops the unnecessary bits an pieces out of the feed.

So far it's not quite done, but it's showing signs of usefulness. I need to figure out a few issues regarding content length, date parsing, and then write some code to rip unnecessary entry elements out of the parsed XML, but that shouldn't take more than a few more hours over the next day or so.

And of course, by posting this I've ensured that I have to finish it or I'll sound like an idiot by talking about it to the whole internet and then not following through ;-)

New Thunderbird...

And it does RSS and Atom!

Cool!

Grab it while it's hot.

P.S. Yes, I'm just posting this to see how a new entry looks when it shows up, why do you ask? ;-)

It only took 6 release candidates...

After dragging it's feet for far too long APR 1.0 has been released.

An Eventful Week

Lots of stuff going on, and I don't have time to go into all of it, so I'll just leave you with the highlights.

Moved into the new building at work, so now I have a 5 minute drive in the morning instead of a 30-45 minute drive, which is nice enough in and of itself, but on top of that I actually get an office. It's quite the change from being in a cubicle surrounded by the rest of the group I work with, but I think I'll adjust ;-)

Now I just need to finish learning my way around the new office. The place is quite nice, but it's also something of a maze. All the hallways look the same, and I'm constantly getting lost. I'm sure that will get better with time though.

In other news, my "Using the Apache Portable Runtime in a Non-HTTPD Application" talk was just accepted for ApacheCon, in Vegas in November. If you haven't guessed, I'm REALLY looking forward to this, the Apache guys were a lot of fun in Portland at OSCON, I can't imagine what they'll be like in Vegas ;-)

svn log -r 334

r334 | rooneg | 2004-08-21 20:40:15 -0400 (Sat, 21 Aug 2004) | 7 lines
Changed paths:
M /lucene4c/trunk/src/index/terminfos.c

Sometimes I'm really just too stupid to be allowed near a C compiler.

* src/index/terminfos.c
 (lcn_terminfos_idx_read): allocate the correct amount of memory for the
  index object.  this fixes the odd behavior i was seeing where additional
  allocation would stomp all over the fields in this object.

Why bother writing...

When others sum up the weekend so nicely.

Oh, and regardless of what Rob says, Alien versus Predator was still better than synchronized diving, which is what was on TV when we fled to the movies.

Damn

So it turns out that it's way too easy to do a "find -name \*.c -delete" when you intended to do a "find -name \*.o -delete".

Well, ok, there's no reason it should be easy to make that particular mistake, except that I'm a moron.

In related news, my "when do I check it in to version control" threshold has now officially been lowered to "there is any chance at all I might want this stuff later", since I clearly cannot be trusted to not accidentally delete several hours of work at any given time.

OSCON 2004 Day 6

The final day of OSCON was basically just the flight back.

I hung around the hotel in the morning, ran into a few people at breakfast, but basically I was just waiting to leave.

On the way to the airport I rode with one of the Postgres guys, although I never got his name. Had a fairly interesting conversation though, until he got caught in a conversation with someone who called his cell. I hate cell phones.

While waiting for the flight back I discovered that Casey West and Ricardo Signes were on the same flight. We talked about various OSCON related things while waiting for the flight to leave, and it seems that everyone had a great time at the conference. Casey showed me a new perl module he's working on for threading email, with a new algorithm that should be faster than the standard JWZ threading algorithm (it's 1 pass instead of 2 like JWZ's).

The movie on the plane was Jersey Girl, which kind of surprised me. Never thought I'd see a Kevin Smith movie on a plane, but I guess this one is more kid appropriate than the rest.

After the movie Casey and I took advantage of our airport cards and SubEthaEdit to pair program, working on some of the code for his new module. It worked quite well, despite the fact that we were sitting in separate parts of the plane. RJBS was too far back though, and kept getting cut off when he tried to connect to the ad-hoc network we had going.

I just barely made it to my next flight in chicago (15 whole minutes between flights!), and finally got home around 2AM.

It was a heck of a good conference, but I'm glad to be home.

OSCON 2004 Day 5

Started out this day totally stressed about the talk, so I skipped most of the first keynote and did a dry run in front of the mirror.

Wandered in most of the way through the first keynote, but I caught the end, which looked pretty cool. Others have already blogged it though, so I'll leave it to them.

Next was a talk from on of the Novell guys, which was theoretically interesting (it's neat to see a huge company like that making the switch to Linux on the desktop), but in practice felt like a sales pitch.

I decided to check out Paul's talk on Constructor Dependency Injection, since I'd been listening to him talk for much of the week, and he's a pretty neat guy. The talk was pretty cool, although the concepts were pretty simple when you come right down to it. Compared to the rest of the 'Enterprise Java' world it does seem like a good way to do things though, another of those things I'll have to keep in mind in case I'm ever in such an environment, like Groovy.

Now was my second talk, Subversion Best Practices. I think it went pretty well, although some people obviously were looking for something more like Greg's talk, an intro to Subversion rather than ideas on how to use it once you know how. I did have a number of people thank me afterwards though, and nobody told me it sucked (at least not to my face), so I guess it went about as well as I could expect.

Next up was the final keynote, with the head tech guy from Weta, the company that did the special effects for LOTR. They have an interesting set of problems (HUGE AMOUNTS OF DATA), and are being hit rather hard by RedHat's switch to per-machine fees. They need to use RedHat because it's one of the few versions of Linux that third party companies actually certify their apps on, but sooner or later they'll have to upgrade from the old RedHat version they're currently on, and the migration path looks like RedHat Enterprise Linux, but that has per-seat fees that would be a big problem for thousands of machines. On the other hand, he did have good things to say about Open Source in general, as they make use of a lot of it in house. My favorite quote was something along the lines of "Sometimes I wish I had an army of developers. Then I realize, I do have an army of developers". It's easy to forget that open source is just that, an army of developers, if you know how to use it.

After the keynote John Peacock, Justin, another Apache guy, and I went out to lunch at a seafood place near the hotel. It was good, both the conversation and the food. Then we wandered off to Powells to do the standard 'drool over the massive collection of techie books', and scattered off to our respective hotels.

I spent the rest of the afternoon taking pictures of the Red Bull Flugtog, that crazy thing where they launch home made man powered flying machines into the river. The event itself was saturday (after I was leaving), but I did get some good shots of the machines themselves as they were setting them up. I'll post them once I get a chance to get them off the camera.

Ironically this was the only time during the week I actually took any pictures. I love my camera, but it's just too big to comfortably cary around during the conference, at least if you're going to also carry a backpack and a laptop... I see one of those sweet little Canon Powershot cameras in my future, so I'll have both ends of the spectrum covered with regard to digital cameras.

In any event, I went to bed fairly early, since it had been one hell of a long week at that point.

OSCON 2004 Day 4

Ok, so I got distracted at the end of OSCON and didn't get around to actually blogging any of it in real time.

Here's thursday...

The keynotes were the Dysons (one of whom was actually the guy who came up with the Dyson Sphere, they actually showed a clip from that episode of Star Trek at the beginning of the talk), and Bdale Garbee, of HP and Debian. The Dysons were ok (I thought George Dyson's talk last year was better), but Bdale was quite good. He interspersed the standard open source stuff with bits about his experiences building amature radio satellites. It's nice to know that there are people who are realistically talking about putting a satellite in orbit around mars, without government funding.

Saw a talk on qpsmtpd, the Perl SMTPD. Cool app. Must look into it closer.

Andy Lester and Bill Odom gave a fantastic talk on Advice for Open Source Job Seekers. Not that I'm actively looking for a job, but things to keep in mind when one is looking are always good to know. One thing they mentioned which I should really do is to keep your resume up to date even if you're not looking, so you don't have to scramble if some kick ass job opportunity comes up and you need a resume. They also stressed the idea of building up your online reputation by doing cool stuff, so people are more likely to have a clue who you are when you call them up and ask for a job.

The talk on Wackamole, an open source high availability package, was quite informative. I will have to look at this closer. I was particularly amused by the program you use to control your Wackamole installation, wackatrl (pronounced, of course, "Whack a Troll").

Nicholas Clarck gave us a view inside the Perl 5 Porters with Perl 5.8.5 Was Boring, and in the process also gave a lot of useful tips on how to keep the stable releases of a software package actually stable and useful. I also managed to run into Peter Prymmer (the other $COMPANY person at OSCON) in this talk.

Next I went to the What Book Sales Tell Us About the State of the Tech Industry talk, by Tim O'Reilly and some of his hackers. It was interesting enough, but it strikes me that this info will be much more useful next year once they've done more with it. Key things (like linking in geographic data) are still missing.

Finally was the python lightning talks, which were supposed to feature Dan Sugalski being hit in the face with a pie for losing his bet over the speed at which Parrot would run Python code. Unfortunately Guido decided not to hold Dan to the bet. Amusingly it was the Perl people in the room who were most disappointed about this. The remaining talks were not overly memorable, I think the Python community needs to learn a bit more from the Perl community on how to do lightning talks. It's just not the same without the gong.

I've totally forgotten what I did for dinner on thursday, but eventually we wandered back to the hotel for the Novell "Free as in Beer" party. I skipped out after one drink, as I knew my talk was in the morning and I didn't want to be overly dead on my feet.

Unfortunately it turns out that the Perl Foundation auction was also thursday night. If I'd known I totally would have gone to it, but they had like no publicity so I missed out. It's too bad, I would have been willing to pay way more than 300 bucks for an hour of Pair Programming with Ward Cunningham ;-)

OSCON 2004 Day 3

The first day of the actual conference sessions went fantastically.

We started off with the opening keynote, from Tim O'Reilly, where he talked about what was on O'Reilly's radar. In addition to the "Internet Scale Applications" being the "Open Source Killer Apps" (like e-bay, google, amazon, etc), which he's been talking about for a while, he touched on a lot of interesting little bits. I particularly liked the part where he showed some of the data they mine from bookstores like Amazon that indicates what programming languages are currently selling the most books. The ones with declining market shares were in red, rising in green, and size of the language's square showed the size of the market share. Most stuff was either slightly green or red (Tim said it was the summer and all the hackers were on vacation), but there was this tiny little sliver of BRIGHT GREEN off in the corner, and that was Ruby. That was a breath of fresh air. He also mentioned zeroconf as being something to keep an eye on, which is rather close to my heart, and went into his standard thing about how what O'Reilly does is find the new and interesting things people are doing and put them into text form so people can read about them. Conferences like this one are where they find the new and interesting things.

Next up was The Semasiology of Open Source, by Robert Lefkowitz. He ran out of time part way through, but the talk itself was a fascinating look at the origins of some of the words that the open source world uses constantly. Quite interesting, with some nicely placed "The Princess Bride" clips for good measure, but if I try to explain it I just won't do it justice. All I wish is that he'd had time to finish up.

After the keynotes finished up I gave my first talk of the conference, "Using the Apache Portable Runtime in a Non-httpd Application", where I talked about the basics of APR and some of the issues we came across while using it to develop Subversion. I think the talk went well, and I was pleased to see Stas and Geoff of mod_perl fame sitting in the audience. These two guys have more APR experience than I'll ever have. I was able to raise at least one question in their minds though, and we've been hashing it out on the APR and HTTPD dev lists ever since. Overall I think the talk went well, and several people have since come up to me and thanked me, so I guess my first impression was right.

Next I stuck around in that same room for James Duncan's talk on "Enterprise Perl". He had a lot to say about how to make your code easier to understand and how that can help you build systems that are more reliable and debuggable. I was particularly amused by his definition of "Enterprise Code". He says it's "Code that doesn't make money itself, but that helps the company make money indirectly". At the end of his talk he noted that it was brought to you by 10 years of code that doesn't make money. I don't know if I agree with all his statements, but they all seem worth considering at the very least, and that's what's really important here.

Next up I hit Miguel de Icaza's talk on Mono 1.0. I probably could have skipped this one, as I knew most of what he was saying already since I follow the project reasonably closely. Unfortunately, he got cut off just as he was getting to the interesting technical bits at the end about where they're hoping to go in the future. Oh well, maybe next time...

After that I went to listen to Justin talk about deploying the Apache HTTPD server within the Apache Software Foundation. These guys get a scary amount of traffic, and they handle it all with minimal hardware and a volunteer staff, and it turns out that the load on their boxes is always ridiculously low, so they've figured some stuff out and are worth listening to. He also covered some of the details of how the Apache infrastructure team works with the version controls systems they provide and the email infrastructure they provide, since those are the other two main services the Apache projects rely on.

Up next was the Perl lightning talks, which were interesting, as expected. Despite technical difficulties (to be expected any time you have that many people and that many laptops) they got through everything and I learned a bit. I'm sure others have already blogged about the specifics, so I'll just leave it at that.

The last session of the day was about Groovy, the new Agile (perl/python/ruby like) programming language James Strachan (who went out drinking with us on monday) is building for the Java virtual machine. It's fascinating stuff, and is definately good to see, if I did any kind of real Java development I'd be all over this stuff.

Once the sessions wrapped up Greg Stein, Justin, and I hung around waiting for Fitz to finish his book signing. As usual when you hang around Greg you get this constant stream of interesting people who show up. Included in the stream this time was James Duncan Davidson (the original Ant guy), Brian Behelendorf (I must have spelled that wrong, but one of the head Collabnet guys and an original Apache hacker), Jason Hunter (Java Servlets guy), and a cluster of Gentoo Linux guys. Interesting conversation all around.

When Fitz finished up we headed out in search of food, and after much wandering around ended up in a not-so-great mexican place, with another large crowd of interesting people. Along with many of those already mentioned we had Dick Hardt (formerly of ActiveState) and John Viega (a security hacker now, who apparently was also one of the original authors of mailman, one of the big time mailing list managers).

After dinner some of us hopped in Dick's Hummer (yes, some of these people have way too much money) and drove over to the Stonehenge party. "Free as in Beer" was the order of the evening, and a good time was had by all. I spent some time chatting with Brian Behelendorf and Paul Hammant (who I met on monday) about version control type things, and I was quite glad that I got a chance to thank Brian for everything he's done to make Subversion a success. While he's not contributing code he is the man behind the fact that Collabnet is willing to pay people to hack on it, and for that I'll be eternally grateful. While the external community has contributed a TON to Subversion over the years without those core paid hackers we wouldn't have made it nearly as far or as fast as we have.

Around midnight Paul and I wandered back towards the hotel, talking about version control, why people use clearcase, the chances for perforce to survive in the next few years and his background and how he ended up at ThoughtWorks. All in all an evening well spent.

OSCON 2004 Day 2

Well, most of this day turned out to be spent recovering from last night's fun.

I got up about 10 minutes before the first tutorial session was about to start and ran downstairs because I didn't want to miss MJD's "Welcome to my bin" talk, but once I got there it was pretty clear that I was far too hung over to get anything out of it. I hung in until the break and then retreated to the room and fell asleep. I also didn't make it to the afternoon session, spent most of that time either sleeping or going over the slides for my first presentation.

The evening was less of a waste. I went out to dinner with the SVN/Apache guys (Joined by Justin Erenkrantz this time, who had just gotten in to town, Ken Coar, who's weblog I've read forever, but who I had never encountered in real life, Steve Mallet, the sysadmin for opensource.org, and some friends of Rich who's names escaped me).

After dinner I headed down to the evening events just in time for Larry Wall's talk, which managed to turn xscreensaver into a heartfelt talk about the last year in his life and how he feels about the community around him. I could certainly empathise, as many of the medical problems Larry's gone through hit far too close to home for me.

Next (how much does it have to suck to follow up after Larry Wall!) was Paul Graham, of "Hackers and Painters" fame, who I was quite impressed with. Much of what he had to say was very similar to stuff from his book, but it covered a lot of interesting ideas about how to find those elusive hackers who are truly gifted and get them to work for you. A particularly insightful statement of his was the fact that becuase the only way to really know how good a hacker someone is you have to work with them, thus if there's a Michael Jordan of hacking, nobody will know it, not even him. Overall I was quite impressed with his talk.

Finally, Damian Conway gave his "Life, the Universe, and Everything" talk, where he melded together "Conway's Game of Life", Maxwell's Demon, and the Klingon language, with a simulation of Maxwell's Demon within in the Game of LIfe, implemented in the Klingon dialect of Perl. You really had to be there to understand how weirdly cool this was. Truly mind blowing, as I have come to expect from Damian's talks.

Anyway, now the conference opening keynote is about to start, so I should start paying attention and stop typing...

OSCON 2004 Day 1

So the first actual day of the conference was fantastic.

I spent the morning in Damian Conway's "Best Practice Perl" tutorial, which was quite worthwhile. His major point was that you should make a concerted effort to write code such that it's easy to understand. There may be more than one way to do things in Perl, but that doesn't mean you have to use some of the odder ones in your day to day coding. He also had a number of tips on specific problems to avoid and useful idioms that you should be aware of. All in all, a worthwhile talk. Best of all, he's apparently working on a book about the same topic, for O'Reilly.

Then I went to grab lunch with Rich Bowen, which is always enjoyable. We talked mostly about writing technical books and the various problems we've both had going through various parts of the process. It's odd that so many authors seem to have the same problems, yet they still don't seem to have found good ways to avoid them.

The afternoon was spent in the "Cross Platform Rendezvous" tutorial, which was given by Stuart Cheshire from Apple, the head of the zeroconf working group and the creator of most of this technology. Much of what he had to say was about the philosophy behind the creation of the zeroconf technology, and why he thinks it has succeeded in a space where a huge number of other systems have failed in the past. Perhaps the most remarkable thing is how simple the underlying technology is. It's quite impressive how much functionality they've managed to get while still fitting pretty seamlessly into the existing infrastructure. He also talked about some of the plans for extending zeroconf techniques outside the local network and onto a company wide WAN scale network.

I'm itching to start coding on some changes to Subversion to allow it to advertise repositories via zeroconf so that you can easily find out the available repositories on a network. The idea's been in the back of my mind for a while, but it's always seemed like something of a toy project since it's not overly useful on just a local network. WAN scale zeroconf would make it quite useful though, so hopefully I'll have time to make it happen. I also talked with Stuart a bit about some of the particular issues I saw in using zeroconf and Subversion and he seemed quite receptive to the ideas I had for extensions to the current functionality to make it more useful for Subversion.

After the tutorials finished up the real fun began. I finally got a chance to meet Brian Fitzpatrick, who I've known online for years via the Subversion project. After giving Greg Stein enough time to extract himself from the traditional pile of people asking questions at the end of a talk the Subversion and Apache crowd went out to dinner at a nearbye chinese resturaunt, which was a lot of fun.

After dinner a number of us retreated to VQ, a nearbye bar, and proceeded to spend the rest of the night drinking and talking with some of the ThoughtStream guys, who brought along Ward Cunningham, of Wiki, TDD, and OO fame. I must admit that it's nights like this that make coming to these kind of conferences truly remarkable. You simply can't find a group of people like that in other environments.

The bar finally closed and kicked us out, so I made it back to my hotel room around 3AM, already fearing the hangover that was waiting for me.

OSCON 2004 Day 0

Ok, so I'm actually posting this on monday morning, but anyway...

Got up at the crack of dawn and made my way to the airport to catch my 7:45 flight. Was running a bit late, as usual.

Got there to find that the long term parking lot was full, so I had to make my way to another lot. Ok, now I'm later.

Went to the wrong terminal. Ok, now I'm later.

Finally got to the ticket counter and it appears that the "i don't want to interact with a human being" kiosks are busted, and there's this giant line. Why is there a giant line? Because my flight has been delayed 3 hours, and all the people from the flight are in line waiting to work out where and when they'll be flying out.

So I spend like an hour waiting in line and finally a very nice woman moves me to a different flight leaving at around 11:30 and heading to Denver, where I can get a connecting flight to Portland.

Spend the next 2-3 hours sitting by the gate getting progressively more annoyed at LGA's wireless network, which refused to allow me to log on. Drained my laptop's battery in the attempt and didn't get much work done.

Flew to Denver.

Spent 1-2 hours waiting for my flight in Denver, which also had a flaky wireless network, although this time I was able to actually connect to the network long enough to check my mail. Ran into a cluster of OSCON people, all of who's names I have since forgotten because I am a bad person.

Got on the plane to Portland and who do I see as I wander by but Casey West. Oddly enough, Casey also happened to be on the plane I took into Portland for OSCON last year.

Actually managed to get some work done on the plane to Portland, since I was able to charge the batteries in Denver.

Got to Portland, got my stuff, and took the MAX light rail into the city, spent most of the time on the train talking to Casey about various and sundry things. He is an interesting individual.

Got to the hotel, checked in, went up to the speaker registration and registered, hung out on IRC and found that all the other Subversion people I was hoping to hang out with are either missing in action or not getting into town for a day or two. Grabbed some quick dinner at the hotel bar and decided the best thing to do was just to crash early so I'm not jet-lagged for the tutorials tomorrow.

And there we are. Now I'm writing this entry because it's really pretty early Portland time and I'm wide awake.

OSCON 2004 Day -1

So I'm leaving for OSCON at some ungodly early hour tomorrow morning.

As a result, I have completely stopped being productive. It's not that I don't have plenty of things to do, it's just that I'm not doing any of them...

Instead, I'm alternating between stressing about my two talks and being excited about what amounts to a week long vacation where I get to totally geek out with a lot of really cool people.

Anyway, I figured if I'm going to be blogging the whole 'OSCON experience' again this year I might as well start with the pre-conference jitters part of it...

While I'm on the topic of conferences...

Rich is right on target here. Personally, I fall between the "Conference as a Classroom" and "Conference as a Social Gathering" parts of the spectrum. I do learn a lot at these things, but I also find that many of the talks are rehashing things you could get from the FAQ, and that a lot of the value in going to a conference like this is in the people that you meet.

Oh, and for those of you who are thinking to yourself "if you don't learn anything you can't find in the FAQ and you're really just going to hang out with your friends, why is work paying for this again?" let me refer you to what Casey has to say about it. I pretty much think he's exactly right on that one.

GenX4r 0.03 Release

As part of my "avoid working on things I should really be doing" campaign I released version 0.03 of GenX4r this weekend.

Grab it here if you're into that sort of thing.

Significant changes in this version are:

1. Uses the << operator to write to the object passed into the document method, so we can work on plain old Strings in addition to StringIO and IO objects.
2. Addition of begin_document, end_document, begin_element, and end_element methods for GenX::Writer, for times when using the block based methods is inconvenient.
3. Rewrite of the build system to use setup.rb instead of manually using extconf.rb for build/install and rake for tests.

The first two changes were suggested by Christian Neukirchen, the first person other than me (that I know of anyway) who's actually taken a close look at this stuff.

Photoshop Can Make Anyone's Pictures Look Good

Even mine, apparently ;-)

So I finally got around to going through all the pictures I took from the 4th of July, picked out the ones I like the most, and tweaked them a bit in Photoshop so that they look decent.

Get them while they're hot, either in a typepad photo gallery here if you like that sort of thing, or if you want the big-ass non-shrunk down versions or the raw pre-photoshop versions you can grab them from here.

Overally I think they came out pretty well, at least once they're subjected to liberal cropping and fiddling with the color levels and brightness and whatnot in photoshop.

OSCON Presentation Progress

So I did a practice run for one of my OSCON presentations last night with some guys from work as my test audience. I think it went pretty well. I was pretty much on target for time, and that should improve with a few more practice runs.

I need to work on moving smoothly from one slide to the next as opposed to pausing, giving a summary of exactly what I just said, and then having to glance at the next slide to see what the hell I'm supposed to be talking about next. As I said, practice will help with that.

I also got some good pointers on what parts of the talk worked, what parts didn't, and what parts needed to be fleshed out more. I had a good mix of people who had been exposed to some of the subject matter before and people who had not, and everyone seemed ok with it, so I imagine it should work out well for the OSCON audience.

Now I just need to finish the other talk and give that a run through to knock out any bugs before sending the slides off to O'Reilly later this week.

Random Developments

So I was actually rather productive this weekend.

I accomplished at least 3 separate things:

1. At the prompting of Richard Clamp factored the log retrieval code from SVN::Log::Index out into a new module, SVN::Log. Still waiting for his feedback on the final SVN::Log interface before I let it lose on CPAN though.
2. Finished reviewing PDFs of the first 5 chapters of the book, and made copious comments on the stuff that was unexplainably broken. If I haven't said it before, I hate Framemaker, it mangles DocBook in ways previously unknown to man.
3. Wrote GenX4r, a Ruby wrapper for Tim Bray's genx library. It's my first real work with Ruby, but it seems to work, and was quite a bit of fun to write.

Plus, I did laundry, since we now have a washer and dryer.

Oh, and Chris hooked me up with a GMail account, (rooneg@, for the curious), and it's damn cool. I'm using it for FreeBSD mailing lists just to get a feel for it. So far I'm quite impressed.

Fun with Ruby

So I've been playing around with Ruby over the past few days, and as usual when I play with Ruby I continue to be amazed at how cool it is. Now I just need to come up with a real project to use it in...

A few of the neater things I've been playing aroun with:

• Rails - A web application framework for Ruby. Actually it's made up of several subprojects, only one of which (ActiveRecord, the Object-Relational mapping package) is available at the moment, but still, it's quite impressive. When the rest of it is ready for prime time I'm going to have to throw together some toy web apps to get a better feel for it.
• RubyGems - A packaging framework for Ruby code. While it does fall into the totally lame trap of including the name of the language in it's name it still manages to be quite cool. I hope more Ruby projects start using it in the future.
• Rake - A 'make' like build tool written in Ruby. Actually, the 'rakefile' you write for your project turns out to be a complete Ruby program, so you have all the power of the language right there for you to access. Plus, with Ruby's nice readable syntax the whole thing turns out to be much easier to understand than a Makefile. How cool is that!
• Ruby 1.8.1's standard library has all sorts of neat stuff in it. Full YAML support, a unit testing framework, a framework for building custom little http servers, XML parsing, the list goes on and on...

Anyway, I'm still in the 'playing around' stage, but all in all I continue to be more and more impressed with this stuff...

Some Random Stuff

Well, I'm finished moving into the new apartment, and I don't have some need-to-get-it-done-right-damn-now thing to do on the book, so I figure I might as well post something here.

No pictures of the new apartment yet, since I'm still quite a ways from actually unpacking all my stuff, but soon, I promise.

This article by chromatic struck a nerve. I'd kill to have an editor (and other associated people at the publisher) who wanted to work like that, as opposed to the refugees from microsoft-land I currently deal with.

Anyway, I'll have to try to buy him a beer at OSCON for preaching the gospel of using reasonable tools for producing technical books.

I'm positive there's something else I meant to mention once I got around to posting again, but it's totally slipped my mind, so until next time...

Almost, but not quite, Disaster

So when upgrading a FreeBSD install via make world you really should read UPDATING first.

Especially when upgrading to a version of the OS that adds a new system call.

You do not want to install a set of userland tools that require said system call without first installing a kernel that supports it.

Figuring out how to install the kernel without working make and ls commands was fun.

At least I had already built the new kernel, so I didn't have to scramble to find a bootable rescue CD...

This is totally why I am not a sysadmin.

Realization

Only now, as I sit in my new apartment, surrounded by a significant percentage of my stuff, noticing that it's gotten rather dark, do I realize that I left all of my lamps in the other apartment.

Needless to say, since I spent all day moving that significant percentage of my stuff into this apartment, I am far too tired to do anything about this tonight.

I'll just have to add it to the list of things to grab on the next trip to Stamford.

Other than the distinct lack of light in several of the rooms, things in the new apartment are looking great. All my major bits of furniture have made their way here, and over the next week or so the remaining chunk of my clothing and consumer electronics will make the trip. Now that there's actual furniture in the rooms, they're actually looking quite a bit better than I'd thought they would. I had been a bit worried about some of the rooms being a bit small, but now that there's some stuff in them to give the whole thing some sense of scale, they actually look more than roomy enough.

Yes, I promise we'll be having some sort of apartment warming party in the reasonably near future, once we've settled in a bit.

Update

Just a few random notes from my life over the past week or so...

• Framemaker is considerably pickier than xsltproc and the Docbook XSL stylesheets. This means it's considerably more of a pain in the ass to import my book into Framemaker than one would think, considering that the book is written in Docbook and Framemaker supports Docbook...
• Coldstone Creamery rules. Any ice crean store where the employees will sing for tips is just damn cool.
• Apparently the Emacs people fixed whatever bug was screwing up syntax highlighting after using nxml-mode.
• The readdir implementation on Mac OS X can bite me.
• My parent's beach house looks totally different (and damn nice) with it's new deck.
• In related news, my Dad's friends who actually built the deck are just too damn good at estimating, both in the amount of time it would take to build and in how much raw material would be required. If I could estimate half as well at my job I'd be a god.

Page 23, Sentence 5

Well, lots of other people have been doing this...

"Our attempts to define comics are an on-going process which won't end anytime soon."

(from Scott McCloud, Understanding Comics)

The idea is to take the closest book to you, open it to page 23, find the 5th sentence, and post it to your weblog, along with these instructions.

The Moving Process

So my friend Craig and I spent much of the last week scrambling around searching for apartments in Norwalk.

Since $COMPANY is moving there later this summer, and we're both tired of paying insane amounts of rent in our one bedroom apartments, we figured now would be a good time to find someplace close to the new office with enough room for two geeks and a truly absurd collection of computers.

After a lot of procrastination followed by a few days of actually looking, we found our new apartment.

It's half of a duplex on a dead end street between railroad tracks and the highway, directly adjacent to a cemetery!

How can you go wrong with a place like that? ;-)

Actually, the place is pretty nice. Other than the lack of AC it's better than the other places we looked at. There seems to be sufficient room for all our crap, it's within walking distance of the new office, my half of the rent will come out to only slightly more than half of what I'm currently paying, and the neighborhood seems decent.

Our lease starts mid-may, so if you're in the area and feel like helping us move in return for free pizza and beer drop me an email or something.

The Dark Side of TiVo

You know, TiVo has a dark side.

When you stop watching the stuff TiVo has recorded for you, say for a little more than a week, perhaps because you're alternating between watching season 5 of Babylon 5 and working on a chapter of a book, you end up with a huge backlog of good TV that you really want to watch, but you can never find the time to get through it all.

Curse you TiVo!

Simon on Maypole

There's a pretty good article up on perl.com about Maypole, Simon Cozens' MVC web application framework for Perl.

All in all, Maypole looks pretty damn cool, and this seems like a nice intro.

It's good to see the money from the perl foundation is being put to good use.

Nice Use Of Plucene

So Casey seems to have a heck of an idea on his hands...

Perhaps I'll get motivated and try to put together such a thing myself. It doesn't seem all that difficult...

(If you haven't guessed from SVN::Log::Index, I'm a big fan of applying search technology to random stuff.)

I Want One

Apparently Brookstone sells Arcade Games.

Anyone got 3 thousand dollars?

I could really use some Ms. Pacman/Galaga Action right in the comfort of my own home...

Contractual Obligations

Well, since it's in the contract and all, I suppose I should link to OSCON's web page and encourage you all to register ;-)

(No seriously, it is in the contract, when you sign up as a speaker the contract includes a clause where you agree to 'Co-promote' the conference, which includes placing a link on your web page if you have one.)

Faith In Humanity

I like it when I go to see a movie that comes highly recommended by several friends and I actually like the movie. It gives me some faith in humanity (well, in my friends anyway, the most important part of humanity), to see that people's taste in movies make some kind of sense to me.

When I go to see a movie my friends liked and I end up hating it, it always leaves me wondering, just a little bit.

For the curious, this was prompted by Eternal Sunshine of the Spotless Mind, which I liked, and Lost in Translation and Punch Drunk Love, which I did not enjoy at all.

Playing Catch Up

So I spent the last week working out of our San Mateo office, and I haven't had time to post anything.

Fortunately, due to the magic of Stickies.app I've got a collection of neat things I've been meaning to mention, and here they are.

• Apparently there's a port of Valgrind available for FreeBSD. Seems to work pretty well.
• It looks like things are progressing on libsvn_fs_fs, the first non-berkeley-db Subversion filesystem.
• I got my first set of changes back from my copy editor, and I must say I'm impressed. It's amazing what someone who knows what they're doing can do with my writing, cause I don't think it sounded this good when I wrote it.
• I miss Wonderfalls already. I mean I suppose it shouldn't surprise me that FOX decided to just axe it, rather than letting it have a chance to build an audience, but despite the almost forgone conclusion, I'm still a little surprised. I mean honestly, do networks ever intend to start pushing anything other than reality-tv crap? If I want reality I've got it right here, I don't need a TV for that...
• OSCON has their registration page up. Sign up now and you can go see me talk.
• Fray and Eastern Standard Tribe are both incredible. Go read them.

Anyway, that's about all I had saved up.

Outsourcing: It's not that simple.

Just ran across an interesting article about how outsourcing and overseas development, as it has affected CollabNet.

Interesting stuff.

I'm not sure what I think about it.

I know there are reasons to move some stuff overseas, and it may have saved the jobs of many developers here in the US, and I know from personal experience that they're a good group of people and are trying to do the right thing...

That said, I don't know what I think about the fact that they don't hire entry level people here in the US because it just doesn't make sense on a cost basis, and I'm interested in all the questions about the culture differences making it difficult for the overseas developers to come up to speed and contribute as equals. I mean if you can't hire local people right out of school, how will they ever become the senior developers you are willing to hire? And if the overseas talent you do hire right out of school can't ever overcome the cultural issues to the point where they aren't afraid to challenge their higher ups, something that's needed in any senior engineer, where exactly does the next generation of leaders come from?

It seems like if everyone did this we'd be setting ourselves up for a big problem ten years down the road.

OSCON details

So the details of my OSCON talks showed up today.

- Conference Session
Session ID: 4890
Title: Using the Apache Portable Runtime in a Non-httpd Application
Date: 07/28/2004
Time: 10:45am to 11:30am
Location: Salon G

- Conference Session
Session ID: 4891
Title: Subversion Best Practices
Date: 07/30/2004
Time: 11:35am to 12:20pm
Location: Salon F

See you there ;-)

A Good Weekend

So you haven't lived until you've seen four drunken Irishmen (ok, technically it was three drunken Irishmen and one drunken Englishman, but whatever) wearing kilts and singing Bon Jovi songs, complete with air guitar.

It was absolutely the perfect way to end the party after Saavik's wedding ;-)

Procrastination

So I finally broke down and released version 0.20 of SVN::Log::Index.

The primary changes in this release are some code in Build.PL to check for needed executables and a fallback mode that lets the module use the svn command line client if it can't find the Subversion perl bindings.

I'd hoped to get some of the more interesting ideas I've had into this release, but I haven't had time.

Oh well, at least it leaves something for the next version ;-)

Damn

I accidentally turned my alarm on last night, so now I'm awake.

How the hell did I manage to do that?

I have no idea... I don't actually remember turning it on, but sure enough, 7AM comes around and off goes the alarm...

Not a good way to start a sunday.

Wow, Now That's Depressing

How I Lost The Big One, by Lawrence Lessig.

Just For The Record

Jessica's lab is way cooler than my office.

Bio people just have cooler stuff lying around than programmer types. All the cool stuff I get to play with is sitting in a data center hundreds of miles away from my office.

So, if you're looking for a job where you get to have all sorts of cool toys around the office, 'bio grad student' seems to have 'programmer' beat.

Of course, if you want to actually get paid a reasonable amount of money, 'programmer' still comes out on top ;-)

Scared Out Of My Mind

Well, all the cool kids are talking about their OSCON talks, so I suppose I should as well.

I submitted two proposals this year, and to my surprise both were accepted, so barring some kind of catastrophe I'll be speaking about "Using the Apache Portable Runtime in a non-httpd Application" and "Subversion Best Practices" in Portland this summer.

Both talks will be based largely on my experience from watching and participating in Subversion development, on one hand covering the issues we've dealt with regarding using APR in a large cross platform application, and on the other in how to use Subversion itself most effectively. In both cases I think we've learned a lot over the years, and hopefully this will let others benefit from our experience.

It goes without saying that I'm both excited and terrified, especially since I now have two talks to prepare when I had sort of planned on one at the most, but I'm sure I'll find time to prepare for both, and judging from last year it should be a lot of fun.

So anyway, if you're not doing anything July 26-30, feel free to join me in Portland!

I've Never Liked PowerPoint...

Perhaps this will be an alternative...

Best Screensaver Ever

I meant to post this a while ago, but got sidetracked...

In case it wasn't clear yet, the new release of xscreensaver proves once again that it is the king of all screensavers.

I mean come on, how can you not love a screensaver that simulates an old school monitor and actually has a vt100 emulator built in so you can use it as a terminal. That's just too cool for words in my book.

What A Great Line

While I've never had any use for formats in perl 5, I must admit that the exegesis about them in perl 6 is quite interesting.

My favorite part is the section on what happens when form is called in void context:

When called in a void context, form waxes lyrical about human frailty, betrayal of trust, and the pointlessness of calling out when nobody's there to heed the reply, before dying in a highly theatrical manner.

Waiting

I'm currently waiting for a number of things to happen.

I hate waiting.

A lot.

Slashdot Jumps The Gun

Just once, I'd like Slashdot to wait until a software product is ACTUALLY RELEASED before posting an announcement about it having been released.

But apparently that is too much to ask.

For the record, the real 1.0 release is supposed to go out tomorrow.

High Performance Multithreaded Servers

So at work one of the things I work on is a multithreaded server application.

I've written about bugs I've tracked down in it before, and I bitch about it a lot because multithreading in general is a huge pain in the ass. If I could possibly come up with a way to avoid it in this particular application, I would, but I haven't been able to find a feasable way to do that yet, for a number of reasons, some good and some bad.

Despite this hard earned dislike of multithreading, when I read Chris Brumme's weblog entry about some of the issues involved in hosting the .NET CLR inside a high performance multithreaded application (SQL Server specifically), I was just floored by some of the ideas he mentioned. It's stuff like this that makes you want to say "yes damn it, those kind of performance requirements are a damn good reason to use threads".

Specifically, they go to massive lengths in SQL Server to keep exactly the right number of threads executing useful code as they can, which gives huge bennefits for a number of reasons.

It's really neat stuff, and I'm now struggling to determine if any of it is directly applicable to what I'm currently doing. I suspect it'll be difficult to implement, and I'm not sure if it'll be practical due to the constraints of my application (third party code sucks in these sorts of cases), but damn it's an interesting thought experiment if nothing else.

In any event, if you're doing anything with multithreading in your applications you should read his blog entry, even if you never have any intention of hosting the CLR inside an application. The stuff he talks about, in this entry and others, has implications ranging far beyond the CLR, and can be applied to all sorts of programming problems.

NNW Atom Beta

So there is now a beta of NetNewsWire availabel that includes Atom support.

Seems to work fine as far as I can tell, although I've only tested it with two feeds.

Once again, I remain impressed with what a fantastically cool product NetNewsWire is.

SVN::Log::Index 0.1

So I finally kicked version 0.1 of SVN::Log::Index out the door.

It's a perl module that lets you index Subversion log entries for full text searching via Plucene.

It's not done, by any stretch of the imagination, I've got a bunch of things I still want to do with it (thus the TODO list), but it's far enough long that it's at least debatably useful.

So check it out, either on CPAN (once it manages to work its way out to the mirrors) or at http://electricjellyfish.net/garrett/modules/SVN-Log-Index-0.1.tar.gz if you're just can't wait.

Oh, and it's my first real perl module, as well as my first perl module released on CPAN, so while I value any feedback, please be gentle ;-)

Well, I Guess It's Officially A Book Now

Apparently my book is now up on amazon.

I suppose this means I really have to finish it now...

Back to work...

Friend Of A Friend Networks...

So I've been sucked into Orkut, just like the rest of the geek world, and so far I'm impressed.

It's quite a bit faster than Friendster, and it seems to have a better interface. It'll be interesting to see if it holds up under increasing load.

I must admit that part of it's charm is the massively geek-heavy population. I keep running across people I know from one open source project or another, which is cool.

As for the whole "what constitutes a friend" question, I'm going with the someone you would buy a beer for if they were in town definition, which seems as good as any other I've heard.

Another Doctorow Book

Cory Doctorow has a new novel, Eastern Standard Tribe, out.

As usual, it's available for free online, as well as for sale in dead tree form.

I've already ordered my copy from Amazon, and while I'm waiting for it to arrive I'll be reading the free version. I suggest you all do the same.

Plucene 1.0

So Plucene hit 1.0 today.

I've been playing with prerelease versions for a while, for a perl module I'm working on (tentatively named SVN::Log::Index) that will let you index the log entries in arbitrary Subversion repositories for full text searching. It's damn cool, and now that it's out the door I'll have to polish up SVN::Log::Index and sign up for a CPAN account so I can officially release it.

Writing A Book With Template Toolkit?

Here's an interesting article by Dave Cross on O'ReillyNet that explains how the new Perl Template Toolkit book was actually written using Template Toolkit.

I'm using DocBook to write my book, which is quite nice, but it's quite impressive to hear the kind of things they were able to do with some custom TT macros. I'll definately have to pick up their book, once I have some free time again, as TT seems like a really powerful tool.

Arch vs. Subversion, a Response

So a while back Tom Lord, the author of Arch, posted a mail to the arch-users mailing list detailing everything he thought was wrong with Subversion.

Lots of people seem to have agreed with him, to one extent or another, and if you search the web for the phrase 'diagnosing SVN' you can see a number of replies.

Greg Hudson, one of the Subversion developers, has crafted a reply which is well worth reading. In my experience Greg is almost always right, and he seems pretty much right on the money with this one.

Fun With Docbook XSL

So I really wish I had found Docbook XSL: The Complete Guide when I first started messing around with Docbook XSL for the book. It's answered pretty much all the questions I had been wrestling with for quite some time. I may even buy the hard copy version.

Are You Atom Enabled?

So I just discovered that a web site I read from time to time, which has never had an RSS feed available (and believe me, that always pissed me off), actually has an Atom feed.

This is the first time that something I want to keep track of is accessable via Atom but not RSS. I hope it won't be the last.

Of course, this would be considerably cooler if NetNewsWire, my aggregator of choice, had Atom support, but I'm told it's on it's way, so I'll just have to wait patiently for the time being.

Damn Users

You know, sometimes every post to users@subversion.tigris.org feels like it ought to begin with "I haven't read the documentation, and I totally lack any common sense, so will you please solve all my problems for me?"

I am so totally not cut out for tech support that it isn't even funny.

The Morning's Amusements

Just a few things that popped up in my morning weblog reading, so I thought I'd share...

First, Tim Bray posted his first cut at the API for his Genx XML Writer C library. My first impression is that it looks simple, easy to use, and it will be quite useful when it's done. I'm already thinking about how I can wrap it up in a C++ API so I can use it at work.

Second, Justin points to an article on how to criticize computer scientists. Makes me wish I was back in school again...

Oh, and for those who were curious how my writing went this weekend, I made it through about 85% of the chapter I was meaning to finish. I got hung up at the end by the fact that I ran into a section of Subversion that I didn't understand all that well, and I had to slow down and actually think for a while before I could write anything that made sense.

Hopefully I'll be able to finish it up tonight, and then move on to the first appendix, which would put me just about on target to actually meet a deadline for a change!

Progress

So the chapter I'm currently working on just officially became the second longest in the book, and it's well on it's way to being the longest, since I'm only about 2/3 of the way in to my outline.

Of course, it's not like I've had a chance to test if any of these examples work, or even if they compile for that matter, and it's likely that none of it makes sense to anyone but me, but at the very least there seems to be a lot of it, and that's something.

Lessons Learned

I take some small bit of comfort in the fact that the next time someone asks me to write a book about something I work on in my spare time I will at the very least know enough to say no.

Not that I don't appreciate having been given this opportunity, but as chromatic has said in the past, writing a book is something that is nice to have done, as opposed to something that is actually enjoyable to do.

(I am no doubt horribly screwing up what he actually said, but the sentiment is the same.)

Since I started writing this book, it has impacted my life in (at least) the following ways:

• Kept me from spending time with people I care about.
• Kept me from actually working on the software the book is about, something I used to enjoy doing in my spare time.
• Conveniently divided my "free time" into time during which I work on the book and time during which I feel guilty about not working on the book.
• Made me question my actual writing ability on numerous occasions.
• Kept me from working on a variety of other interesting projects, most of which I have likely forgotten, and will not remember when I do actually find some free time.
• Kept me from reading anything of substance in months.

It goes without saying that I am looking forward to finishing this damn thing just a little bit...

Just In Case You Didn't Get The Memo...

Posting a patch to a development mailing list with something like

*** please do not ignore ***

at the top of the email is quite possibly the best way to get me to ignore your email.

News flash people! Open source developers review patches sent to mailing lists when they have the time. They're not going out of their way to ignore anyone, and implying that your patch is somehow special and deserves to be dealt with rightfuckingnowdamnit instead of when the developer has time is just egocentric and rude.

It's the Little Things...

I think the thing that impresses me the most about The West Wing is the way they consistently pick really good music for all the episodes.

I mean the writing is fantastic (well, not so much this year, but in previous years anyway), the topics covered are always interesting, and the acting is great, but it still continues to impress me that they find such great music for each episode.

For the record, the music that prompted this blog entry was the Aimee Mann cover of "Shed A Little Light" (a James Taylor song) at the Rock the Vote concert in season 4.

Unfortunately, that particular recording of "Shed A Little Light" does not appear to be available anywhere, but the original was available at the iTunes Music Store.

There's a Special Place in Hell...

For all the people who've ever worked on Lotus Notes...

It's truly remarkable what a piece of crap that software is. It continues to amaze me on a regular basis with how much it sucks. There are simply no words to describe it.

Hey, Who Doesn't Hate The Autotools!

So for everyone out there who has a fun-filled GNU Autotools horror story in their past (and what programmer doesn't?), you might want to take a peek at MBuild, a C# based build tool from one of the Mono hackers. It's not done yet, but what's there has a lot of promise, and it certainly sounds better than "that better begin with a tab!" or "write your makefile in XML!".

How Much Is Too Much?

So just for the record, spending approximately 10 hours watching Queer Eye because your friends who are your ride back to your car don't want to leave yet is just entirely too long.

I was told that the post new-years recovery plan was to watch the Indiana Jones Movies Jessica got for xmas, and this was pretty much as far from that as you can get...

I guess you can't get truth in advertising anywhere these days ;-)