So, last week I managed to find enough free time to finish up my copy of Theo Schlossnagle's book Scalable Internet Architectures. It is, without a doubt, required reading for anyone thinking about working on a large scale internet facing web application today.
Theo is a regular speaker at various open source conferences, and much of the content from his talks made it into the book. That means you get a combination of just plain old good advice on how to make things scale up (worry about logging, think about caching, plan your upgrades), tips you might not have thought of (you can use cookies as a rather effective per-user caching system as long as it's for small amounts of data) and as might be expected from someone who went to Hopkins a hundred and one tricks you can play with the Spread group communication system.
Even if you don't build your application using the techniques that Theo specifically discusses you'll absolutely benefit from his advice on what sort of things you should be thinking about as you design your architecture, since if you don't think about them then you'll just have to do it when you rebuild it later for greater scalability.
Anyway, go grab a copy, it's worth a read.