54 Pages!

so i just ran 'make book-rtf' and built the rtf version of my book. it's
been a while since i've bothered to do that, since the rtf doesn't look all
that good in os x's text edit app, and i hate waiting for openoffice to
open, and the html looks good enough as far as i'm concerned. anyway, it
turns out that the thing has grown to 54 pages! i mean wow! that's great!

i mean it may be crap for all i know, since nobody but me has actually read
any of it at this point, but at least it's starting to look long enough to
be a real book.

Check Your Return Values!

so i spent much of this week tracking down a really irritating bug, and since it taught me an important lesson, i figured i'd share.

the bug manifested itself as one my my servers eventually, under very heavy load, ceasing to reply to requests. when we broke into the process in the debugger, everything seemed normal, and the server's internal count of the number of worker threads it had running said that it was totally maxed out, with like 300 worker threads working away, but actually listing the threads in dbx showed no worker threads at all!

so i started out by sticking debugging output all over the life cycle of the worker threads, anywhere that it could possibly be exiting where it really shouldn't, but for days, it would run with none of those outputs showing up, and yet the problem would still occur eventually. my worker threads were dying, and i couldn't see why!

finally, i started reading some more docs about the debugger, and i discovered that there was an extra argument you could pass to the debugger that would cause it to list all the threads, not just the active ones. when i got a chance to see the problem in question occuring, i tried this, and it showed tons of zombie threads sitting around. so i start reading up on what the hell a zombie thread is...

around this time, one of my coworkers was hanging around my cube, talking about this problem and others that we are trying to track down, and he said something that pointed me right at what the problem was.

it turns out that zombie threads are the left over parts of the thread that stick around after they exit, so that you can later do a pthread_join on the thread to figure out it's exit status. we were not actually doing this when our workers eventually exited, as the exit value of our threads were inconsequential to the rest of the application, so having the zombies wait around was kind of silly. worse yet, having the wait around was causing pthread_create to actually fail later on because it was out of resources, and i wasn't picking up on this because i was a moron and didn't check the return value from that function. so i'd go along blissfully unaware that i had failed to create a thread, increment our internal counter saying 'yes, we have one more worker now', when in truth we did not. thus the weirdness where we end up with no threads but think we are maxed out.

to solve the problem, you either need to join with the threads after they die, so that the thread library can stop keeping that data around and reuse it, or you can just detach the threads, by calling pthread_detach or by setting an attribute in the thread before calling pthread_create.

once again, let's be clear that if i wasn't less of a moron and had remembered to check the return value from pthread_create, this week long debugging saga would have been considerably shorter.

there's one more mistake i won't make again...

#include <bad day.h>

today was pretty much a continuous stream of annoyances.

Ichthyology and Stuff

fishing was fun. even with the 'wake up at 2:30 in the morning to drive to the cape' part. we caught something like 17 striped bass and 30 something bluefish. of course, we could only keep 12 striped bass, but still, that's a lot of fish.

now, i'm tired. my arms are killing me, and i feel completely and totally dead.

in other news, ask finally put a version of the oscon movie up for download. unless you're reasonably well versed in the various personalities involved, you probably want to read the cheat sheet that explains all the in jokes. i got most of them without resorting to it, but that just proves i am totally lame and need to get a life.

blosxom hit 2.0 today, and along with the release has a new website. damn cool, as usual, and i'll have to see about upgrading from whatever beta version i'm currently running once i find some free time.

in otherer news, i have no free time.

i have a deadline next week for the book, and i'm not even close to done. well, i'm 'close' in that i almost have draft versions of the three chapters i'm supposed to send them. unfortunately, i'm not really almost done because as i've come closer to finishing them, i've realised exactly how much the drafts suck, and how much polishing they need. there are about a million things i've realized i left out, and i really need to find ways to work them in, or the other stuff i'm writing about just won't make any sense, and that doesn't even account for how much the writing itself sucks, since i haven't even had a chance to give it a second look, let alone anyone other than me. sigh. with luck, i'll be able to finish enough of it this week that i don't look like a totally incompetent loser when i send it off to martin next weekend, but we'll just have to wait and see.

Too Many People Are Getting Married

so for anyone who is a part of my little circle of friends, you are no doubt already aware that too many people are getting married.

i was discussing this issue with my friend jessica this week, and she had the following brilliant observation on the number of single people around lately, which several people have encouraged me to share with the rest of the world:

We are a dwindling species.

Shouldn't there be some sort of federal law to protect us and help us to breed? :)

i just thought that was the funniest thing ever ;-)

Fucking Delta

how hard can it possibly be to put a bag on a plane in portland, leave it on the same plane when it stops for a few hours in cincinatti, and take it off when you get to new york? i mean they do this kind of thing every freaking day, they should have the procedure down by now.

normally, i'm paranoid about this sort of thing, and don't check bags, but this time i figured hey, what the heck, this bag is really freaking heavy, i'll check it. so murphey's law being what it is, they lost it.

they better find the damn thing...

off to call their 800 number and see if it's turned up somewhere.

OSCON Wrapup

well, i'm sitting in the airport in cincinatti, waiting for my plane to leave (gotta love 2 and a half hour layovers), and i thought i'd write a little summary of the oscon 2003 experience.

i had a fantastic time. this was far far more fun than usenix last year. not that there's anything wrong with usenix, but this seemed like more my crowd. the majority of the talks and tutorials i went to were both interesting and useful, and next time i think i'll be better able to avoid those that won't be.

most of all though, i enjoyed actually getting a chance to meet all these people who up till now were just email addresses to me. the best thing, in my opinion, about open source is the people involved in it. the vast majority of the people i got a chance to meet were just fantastically cool, friendly, and willing to sit down and talk about all sorts of things i'm interested in, and while i get some of that from work, these really are my kind of people, to a greater degree than i've found before.

just off the top of my head, i got a chance to meet and talk to:

• casey west
• mike schwern
• alisson randal
• ask bjoern hansen
• rich bowen
• justin erenkrantz
• greg stein
• john peacock
• steve knight
• chromatic
• ward cunningham
• brian ingerson
• martin streicher
• ted ts'o

(and i aplologize in advance to anyone i've left off the list, i didn't mean to, but i just met too damn many of you this week!)

at most, these were people who i'd emailed or talked with on line before, but most of them were just random people i knew only through reputation, and all of them were just too cool for words. i feel grateful to be a part of their community, and actually getting a chance to sit down and talk to them just makes me more motivated to try to contribute more back to it.

of course, there were a huge number of people i didn't get a chance to meet, or who i only met in passing, so i guess i'll just have to come back again next year and do it again until i've gotten them all! ;-)

OSCON Day Five

george dyson's keynote went back and covered some lesser known parts of the history of computing, back to the days of vacuum tubes. i mean it's amazing to think about the time when they actually didn't have the word 'bit', but he's working on preserving the history of those days, much of which had been in danger of being lost. i was amazed by a number of the things he mentioned, and i think everyone here was thankful for this view into our culture's lesser known history. in conclusion, he thanked everyone here for working to keep computers open and non-proprietary, because that's the way it started, and that's the way it should be, and upon leaving the stage, he was greeted with a standing ovation from the audience.

the next keynote was from miguel de icaza, of gnome/ximian/mono fame. i was always impressed with miguel during the short time i was contributing code to mono, and he did not dissapoint me. the demo was impressive (they ran eclipse under mono, via a java bytecode to IL translator), and him and nat friedman demoed a cool application they've been playing with that tries to integrate all of the different disparate data sources we interact with on a daily basis (IM, email, files on disk, and many others), so it can show you items related to whatever you're currently working on. it's still under development, but even so it was impressive, and it serves as a good example of the kind of thing one can only do in an open source system, since hooking in to all the applications one works with in a proprietary os would be much much harder than it is for them on linux.

i couldn't miss jeremy zawodny's talk mysql scaling pains, since i've been reading his weblog for some time, and he has always seemd like a very sharp guy. needless to say, he did not dissapoint. he covered a lot of neat things, some mysql specific, and some not. i'm sure he's got his slides online somewhere, so i'll have to forward copies of them along to the various people at work who are using sql databases, especially those using mysql.

i skipped out on jeremy's next talk so i could hear ask talk about the single sign system they built for perl.org. it's a neat system, integrating various different web sites, cvs access, bug tracking, and probably some other things i've forgotten. the whole thing is backed with a mysql database, and when they can't write their own perl code to hit their systems, or hook mysql in to the application in question directly, they dump the data out periodically to whatever format the system in question requires (this is how they do cvs authentication and authorization). oh, best of all, they talked about subversion! they're using mod_auth_mysql for authentication in apache, and mod_authz_svn for authorization. they were a little annoyed that mod_authz_svn only supports flat files for configuration, but they conceded that it's very new software, and that they'd be on the lookout for better ways to make it work in the future. after their talk i went up and talked to them for a while about how they're using mod_authz_svn and what they'd want in the future. it was pretty much what i'd expected, they don't like that there's no way to have an anonymous user that doesn't have to log in, but recognized that it's a flaw in the way apache authorization works at the moment, and they want something that's more flexable in where it gets it's data, either by calling into mysql or something else. i'm sure such things will come in time, as mod_authz_svn is currently more of a proof of concept than anything else.

ask had to run out on me because he needed to set up his laptop to display this year's perl movie i'm not going to say anything about it other than that it's damn funny (if you know who the people in it are), and you should watch it once they've got it up somewhere for download.

the last talk of the conference was one of the guys from WETA, the group that does the special effects for the lord of the rings movies. they do everything on a gigantic linux cluster, and he showed a lot of pictures and movies, and in general it was quite cool. one of the more amusing comments was that in the down time between the movies they apparently ran seti at home on the cluster, and immediately jumped near the top of the team listings ;-) unfortunately, they didn't have a preview of the return of the king for us (last year apparently there was a preview of the two towers), since it's being saved for the dvd, but he did assure us that if we thought the battle of helm's deep was cool, we should love what they've got in store for us next.

in typically geeky fashion, we spent the afternoon wandering around powell's, and i showed remarkable restraint, only buying one book. justin did not show as much self control, and ended up shipping a ton of stuff home since he would not have been able to fit it all on the plane with him. apparently a lot of the conference attendees had the same idea we did, as i kept recognizing faces as i was wandering around the store.

OSCON Day Four

the first keynote of the day was from stormy peters (what a cool name!) of HP. she's the head of the group that reviews open source technology within HP, both stuff that they want to make open, and open source projects they want to use. it was a bit 'managerish' for my tastes, but it was kind of neat to see how a really big company is viewing these issues.

next up was mitch kapor of the open source applications foundation, a nonprofit that's been doing some interesting things will open source applications. he had some amusing things to say, but honestly, i kind of zoned out during most of the talk.

the first non-keynote session of the day was '9 views of mark jason dominus', which was just as amusing as any of his other talks. it was 9 lightning talks back to back, each with something interesting to say. i particularly liked 'why lisp will never win', and 'how not to ask for help from strangers'.

then i wandered over to justin's talk on subversion. he covered some of the same stuff greg did, but a bit more in depth. it went well, and people seemed to like it. i think when we hit 1.0 we're going to have more users than we know what to do with ;-)

i decided to check out a talk about managing a domain registry in postgresql next, and it was quite worthwhile. got a few neat tips on how to tune postgres for massive machines, which should prove useful at work in the near future (hint: just because you've got 16 gigs of ram doesn't mean you should hand a huge chunk of it off to postgres as the buffer cache, anything over like 300-700 megs and it just bogs down once the buffer gets full). also heard about some neat replication stuff that is apparently going to be available in the very near future, so i'll have to look into that.

since i've been playing with kwiki lately, i felt compelled to attend brian ingerson's last minute talk about it. he's quite a character, and the talk was interesting. it's a neat piece of software, and it's always informative to hear from the actual person behind the code, so you can see where it came from and a little about where it's going. best new feature: viewable diffs!

after missing the perl lightning talks, there was no way i was going to miss the python ones. among the highlights was hearing the details of dan sugalski (of parrot fame) and guido van rossum's bet concering the speed at which parrot will be able to execute python byte code by this time next year. i'm going to do everything in my power to show up for the conclusion of that one.

i ran over to the YAML BOF next, and got to see a few neat things there. it's apparently possible to dump the entire state of the running perl interpreter with YAML.pm, and you can see all sorts of neat things that way. i need to play with this later. i also got to meet ward cunningham, the guy who invented wikis, which was damn cool. apparently him and brian ingerson (one of the guys behind YAML, in addition to kwiki) are drinking buddies.

the night ended with the dyndns.org party, which featured an auction to benefit the perl foundation. i walked away with a few books and some expensive magnetic perl poetry, which will be going on the fridge as soon as i get home, and the perl foundation surpassed it's goal of raising 4000 dollars (thanks to a last minute bid of over 1000 dollars to prevent search.cpan.org from being orange for the next year). other highlights were the auctioning off of mark jason dominus' underwear, and a number of other amusing incidents that will most likely be blogged by others who were there.

OSCON Day Three

the day started off with tim o'reilly's keynote about the issues regarding some of the new generation of internet applications (amazon, google, etc.), which are based on open source technology, but are not open in and of themselves. they are allowing others access to their data though, which opens up many of the same opportunities that traditional open source projects provide. it's an interesting topic, but i'd read much of it before on his weblog posts, so i spent most of the time chatting on irc about companies who are heavy contributers to open source projects, which is always an interesting subject to me.

the next keynote was from one of the higher ups in the ibm eclipse project. unfortunately, this seemed to be mostly a 'sales pitch' kind of thing, going over some of the features of the application. not that it isn't a cool project, but IDE's arent' really my cup of tea, and if i'm looking for an application framework, i'd probably be more likely to go with mozilla with my app writtin in c++, as opposed to eclipse with my app written in java.

next i wandered around the exhibit hall for a bit. it seemed kind of small, but there were a few interesting booths. at the powell's booth i noticed that there's a new edition of unix network programming comming out in the fall, which should be quite cool.

the rest of the morning was spent in the "what's new in perl 6" lecture, which was primarily by damian conway, with larry wall poping up with a comment here and there. overall, i'm quite impressed with everything i've heard about perl6, and if it's half as nice as they're saying when they finish it, i'll have to give it a serious shot.

slashdot has already had enough commentary about the fact that lunch was donated by microsoft, so i'll just leave it at "it was pretty good" and move on.

at this point, i was hoping to go to the perl lightning talks, because from what i've heard they're always quite amusing. unfortunately, work is paying for this trip, and there are two talks happening at the same time as the perl lightning talks that seem quite relevant to work related stuff, so instead, i went to those.

first was a talk on distributed collaberative development, which is honestly one of the hardest problems we're wrestling with at work these days. unfortunately, this talk was pretty much a waste. the speaker wasn't very good (just read directly from her notes really), and the content wasn't especially insightful. i mean most of what she had to say boiled down to 'use good development practices', with very little specifics, at least from my point of view. i had higher hopes for this one. oh well.

next up was a talk on c++ open source libraries. the talk was reasonable, but it didn't really cover much that i hadn't already heard of, and since the idea was to just give a quick overview that would give you enough info to go out and find out more later, i probably would have been better off in another talk.

oh well, maybe i should have gone to the perl lightning talks after all.

since heading for things that looked useful for work seemed like a losing battle, i forgot about the talk on swig i was planning on attending, and went to hear dan sugalski talk about the state of parrot development. personally, i was quite impressed with it. he's a good speaker, and it's amazing how much progress has been made. i mean i read the parrot mailing list, and i still didn't realize that much of it was for all intents and purposes done. there's still a lot of work remaining, but it's not as bad as one might think.

for the final talk of the day i went to greg stein's subversion talk to lend moral support. wow, was it full. they stuck him in a pretty small room, and there must have been like 80 people there. we filled the floor, had people standing all around the walls, and there were still people outside the door looking in. i guess the conference people underestimated the number of people who really dislike cvs and are looking for a replacement. the general feelings in the room seemed positive, and after the talk i ended up talking to several people for some time about some of our design decisions and the state of the project.

eventually, the crowd subsided, and we ended up with just 6 of us remaining (me, greg, justin, a python guy who's name escapes me, john peacock, and another guy who's name i didn't catch), so we headed out for dinner at a chinese resturaunt pretty close to the hotel. there was a bit of a wait, but it was worth it. the food was great, and so was the conversation.

OSCON Day Two

so day two was a bit of a blur.

the morning was spent in a tutorial about 'programming the apache lifecycle', which gave a good overview of all the various places one can hook into apache via mod_perl. i don't do a whole lot of that sort of coding, but it was quite interesting to see all the little tricks one can do if one is so inclined. it's definately some technology i'll have to investigate further, especially going forward as mod_perl2 becomes more prevalant.

then i had lunch with my editor from apress, which was nice. it's yet another good example of now being able to put a face with the email address, which is honestly one of the best things about conferences like this.

then the afternoon was spent in mark jason dominus' talk 'tricks of the wizards', which explains how some of the more magical parts of perl really work. it was quite informative, and even if i never use most of the things i learned from it, i'm certain it will make me a better perl programmer just because i'll know how more of it is working under the hood, which is always a good thing. also, he's just a fantastic speaker, so any talk he's giving is bound to be interesting.

after mjd's talk, i wandered off and had dinner with justin and rich, which was interesting. it's always cool to hear about how the various people working on open source projects actually got involved. apparently rich got involved with apache because someone asked him to write a book about it. personally, i'd think if one was looking for an author for a book about apache, it would make more sense to find someone who was already involved, but hey, it seems to have worked out in his case.

then we went to the state of the union talks, and heard various different 'leaders' of open source projects talking about what's currently going on in their world. larry wall was interesting as always, and greg stein gave a good overview of the apache project, which seems like it never really gets the recognition it deserves.

after the state of the unions, justin and i hooked up with greg (who i had never actually met in person), and went over to the activestate party. got to meet a few more people i'd only known by reputation, and had a few drinks. the party was cool, but it was a bit hot and crowded, which is par for the course when you offer an open bar to a group of programmers i imagine.

OSCON Day One

ok, so this is a bit late. i meant to post it yesterday, but got distracted.

yesterday's tutorial was Designing and Creating Great Shared Libraries, by Ted Ts'o. it was really neat. he started out by giving us a look at what one could do with elf symbol versioning, both the hows and the whys behind it, and then spent the second half of the talk going over good design practices that would keep you from needing to fall back to something as crazy as symbol versioning.

overall, i think the tutorial was quite worthwhile, if only for the 'good design' section. the symbol versioning stuff is neat from a technical standpoint, as i know we do some of that sort of thing on our VMS system at work, and it's interesting to see how one can do similar stuff on unix, but honestly i don't see myself needing to think about it all that often. there's this really small category of projects where things like that are needed (glibc was his big example), and i don't generally hack on any of them. the design portion however was damn useful. it reinforced several good things i'd already picked up from subversion hacking, and introduced a few new tricks that i'll have to keep in mind in the future. it's nice to have had this refresher now, before svn hits 1.0 and we start becoming constrained by backwards compatablity issues (well, more than we already are).

OSCON Day Zero

so i spent the morning exploring portland a bit, and i came to two conclusions.

• there are more starbucks here than any city really has a right (or need for that matter) to have.


• i could never live in portland because i would be unable to resist the urge to spend all of my money at powell's. i mean you have to love a bookstore that fills an entire city block, and the fact that they have a whole separate store just devoted to technical books is just the icing on the cake.

then this afternoon i hung out at the perl bugathon and managed to find a few old bugs that had been fixed and a few random things that managed to get into the bug db that weren't even bugs at all. i must say, the perl crowd is a lot of fun, and i'm glad i had the chance to hang out for a while and help out a bit.

Made It

so i managed, despite my best efforts at kate's 4th of july party yesterday, to get on the plane and travel across the country to portland.

upon arrival at the airport, i (along with a gaggle of other geeks, notably casey west, and dr. bacchus) was greeted by schwern and co. * they were nice enough to give me a ride to the hotel, and even though the car was frighteningly full, we made it just fine. so far, portland seems nice enough, and this conference definately seems to be getting off on the right foot.

anyway, my laptop is telling me it's 3am east coast time, so i'm in desperate need of some sleep...

* i am such a bad person, and already forgot the name of the person who actually drove the car. this is especially bad because he was the one who wanted to find out more about subversion... oh well, i'm sure i'll see him around at some point.

A New Experiment

i'm trying something new, running blosxom under mod_perl, so if you notice anything weird about my site, please let me know.

this is far and away not the usual configuration for blosxom, so it's quite possible that something is screwed up as a result...

update: damn. it looks like blosxom's plugins aren't playing nicely with mod_perl. oh well, i'll have to try later once i've figured out why...

Ah Ha!

so it appears that the secret to being able to make progress on a book is to pick up the laptop and get out of the house. for the past few days i've been taking over a table at either borders or starbucks *, turning off the wireless network connection, and writing away.

it turns out that when i don't have a TV, a game boy advance, a game cube, a gigantic pile of dvd's, and the entire internet to distract me, i can actually write! i'm not sure if it's any good mind you, but hey, it's progress!

* yeah, it's kind of lame, but we don't have any non-starbucks type coffee shops around here, at least not that i'm aware of, and it's just too expensive to go to borders every day, i have this little will power problem when it comes to books...