We’ve hit 259 comments on the last post. That’s a lot of words. I’m making a new one here to clear the baffles.

Update: Clearly, we’re down again. John’s working on figuring out why the problem reappeared and how to fix it. Watch for updates on the home page (Abby)

The site is back up, after having being down for a day and a night. An errant script knocked down the “read” database, and when the box crashed, some of the db files were corrupted. This meant the whole db had to be restored from the master db. In addition to the db itself, some of the log files were also affected. Turns our MySQL is a lot more finicky about how log files are handled than I’d ever known, or hoped to know. After a lot of digging around, everything is back the way it belongs, and all that was lost is a night of sleep–no data was lost.

I’d like to apologize profusely to everyone who was inconvenienced. We’ll put more safeguards into place to try to minimize such outages in the future, hopefully.

We’re really sorry about the downtime, folks. We’re working hard to get the site back up and running, but it’s taking longer than we’d like. We’ll keep you updated as soon as we know more.

In the meantime, here’s something to do while you wait! I want a new and entertaining bookpile to go on the down page. The rules of this impromptu bookpile contest:

1. Post your pictures to Flickr
2. Tag them “LTdown” (feel free to post the link to your photo in the blog comments here if it doesn’t show up right away)
3. Wait for us to pick a winner.

Update 5:11 pm Eastern: We lost the main “read slave.” No data was lost. (We have five copies at all times.) But are missing a critical machine, and have to rebuild it. John is working to rebuild the machine. I suspect it will not be up tonight.
More updates on the homepage

LibraryThing’s John McGrath has debuted a new site, TagsAhoy, with a wonderfully simple idea: searcing your tags across multiple sites.

Cross-site tag searching is nothing new; sites like Technorati do it all the time. But TagsAhoy searches your tags, not someone else’s. If you tag a lot, it will come in handy. And you’ll wonder why nobody thought of it before.

So far, TagsAhoy searches LibraryThing, Del.icio.us, Flickr, Gmail, Squirl and Connotea. More will come, and John has promised tag clouds and other cool features. Pattered somewhat on the spare design of another of his sites, Wordie (“LibraryThing for words,” “Flickr without pictures,” etc.) TagsAhoy is super-simple to use.

We at least applaud the name. It’s clunky in the way “LibraryThing” is clunky. Or was. Now all the “-Thing” names are bought up and my sub-Lovecraftian joke is almost trendy. We confidently predict “-Ahoy” will be the next “-Thing”*, or even “-cio.us”, “-r,” “-ster” and “-Space.”**

John recently moved to New Jersey and will be transitioning gradually off LibraryThing work over the next few months, as we look for a new PHP programmer with systems skills (job announcement to be posted soon). With TagsAhoy and whatever else his fertile mind creates, Abby, Altay and I wish him well.

*Research suggests PornAhoy.com is an expired domain. It sounds like a site for people who enjoy watching naked people on boats very very far away. There’s a market for everything.
**John suggests a site of just Web 2.0 suffixes, ThingAhoySter.

LibraryThing members have added well over 18 million tags. Of course, they aren’t equally distributed. Popular books now sport thousands or even tens of thousands of tags. Work pages have a small tag cloud for each book, but it only shows the most popular thirty or so.

So I added a link to show all tags for a work. It shows the whole “long tail.” It’s very long indeed. It’s stunning.

Here’s Freakonomics with the standard tag cloud:

Click “show all tags” and you get around five pages of tags. Here’s a piece of that:

If you want to see the actual numbers, you can click the “show numbers” link.

“You got your chocolate in my peanut butter!”

Over in Thingology I’ve announced the first library to use LibraryThing for Libraries—The Danbury Library in Danbury Connecticut. Works, recommendations, tags—they’ve got it all.

I’ve said I wouldn’t do as much cross-posting, now that we have a combined blog feed (see over on the right). But I thought I’d mention it here, and explain a bit about what it means for LibraryThing.

First, as members of LibraryThing, you should feel proud that your data—anonymous and aggregate, as the Terms of Use say—is helping library patrons to find books. Your passions—the books on your shelves—beat statistical “paths” through books that others can follow. Your tags–the way you think about your stuff–will help people find subjects not covered by traditional subject classification.

For those concerned about development time, I want to emphasize that LibraryThing for Libraries is good for LibraryThing. On the most basic level, it’s going to help our bottom line. That means more programmers making features and fixing bugs. Conceivably, it could mean cheaper accounts.

It also deepens our relationship with libraries, and returns a favor. LibraryThing was built on library data, and we’ve been graciously invited into the library conversation. We are charging for LibraryThing for Libraries, but our prices are in an entirely different league from what libraries are accustomed to pay for their online catalog software. And as these catalogs add “social” features, LibraryThing for Libraries will exert powerful downward pressure on prices. Ultimately, the industry needs a newcomer to take a huge slice of a smaller market. We’re not going to be that company, but we can push the trend along.

LibraryThing for Libraries has also taught us a lot about library catalogs. These are some thorny, mysterious systems! Until now, we’ve relied exclusively on the simplicity of Z39.50 connections, which most libraries don’t have. But we can do more. With out new-found experience, we can start connecting to the remaining 95%. If nothing else, this should help our language reach.

At long last, the often requested quickstart guide—A very short introduction to LibraryThing.

It’s intended as a quick overview of LibraryThing’s features, to help new members get started, all the way from signing up to creating a blog widget. It’s hard to come up with the balance of enough information to help without overwhelming, so I’m looking for your feedback. What should be added, changed, deleted, clarified…?

Discussion in this talk post.

Book Expo America, ABA’s annual book industry trade convention is in New York City this year, and I (Abby) am going to be there. I’ll be speaking on Thursday, May 31st (from 1-2pm—mark your calendars!) on a panel called “Using Social Networking to Build Author Brands.”

We just found out that the our competitor, Shelfari, is also going to be at BEA this year, and is apparently using some of their Amazon funding to co-sponsor an event. Hey! Well, not only does LibraryThing appear to have sixty-five times as many book lovers as them, but we think we have a lot more to offer authors, booksellers and publishers and we’re going to prove it.*

Authors. It was at last year’s BEA that we launched the LT Author program. After Tim and I spent a day walking around trying to describe LT in a nutshell**, we realized we had been telling people, “it’s like MySpace, but for booklovers.” Well, MySpace is all about bands and musicians promoting their music. Wouldn’t LibraryThing be a good place for authors to do the same? What better place to promote your new book than a website full of avid bibliophiles?

And so was born the LT Author button, a shiny yellow badge that connects an author’s “author page” with their profile page. So far LibraryThing has snagged 395 authors. (See the complete list.)

Best of all, they’re not just authors who clicked a box. To be part of the program, you have to have a LibraryThing account and put in at least 50 books. What is your favorite author reading? Find out.

Neil Gaiman’s author photo. Members have added over 15,000 pictures and photos of authors (see recently added ones), with alibrarian and leebot leading the pack. They deserve some kudos—it’s actually a pretty intensive process, often involving writing authors, publishers, or photographers for permission, so the sheer number of photos is all the more impressive. Plus, it makes for a nice gallery.

LibraryThing members have also added over 92,000 links to author pages—links to author home pages, blogs, publisher pages, Wikipedia pages, interviews, articles, fan sites. That’s a lot of links.

Booksellers. We’d love to add more bookstores to our “bookstores that integrate“—adding availability and pricing information on every work page. We’ve got only three so far, but we’ll be adding two major “chunks” of them in the next few months—to at least 100 total. It’s a great way for people to be able to see at a glance if a book is at their local bookstore.

Publishers. So far, we’re not doing anything for publishers! But there’s a big announcement coming soon. Be on the edge of your seats!

So what can we do to make LibraryThing big at BEA this year?

Our big idea so far is a par-tay. Of course, anyone and everyone can find some time to talk to me during BEA, but I’d like to have a big meet-up. Authors, publishers, booksellers, and hey—readers. Anyone in NYC who’s around is invited, not just the book-industry professions allowed to go to BEA (they have to restrict it, because there’s so much free merchandise on offer.)

I made a BEA 2007 group, post there with ideas of where we should meet (I’m thinking maybe a restaurant near the convention center?). New Yorkers, I call on you for suggestions!

We’re also thinking about bring a bunch of CueCats, and giving them out to authors, to entice them into becoming LT Authors… What else?

*[Written by Tim] Shelfari doesn’t release any statistics. But they do release the top 20 bookshelves. The 20th bookshelf on Shelfari has 1,360 books. LibraryThing has 1,378 members with that many. Hence 20/1,378 = 68.9 times as large. You will note that we do not abuse our other competitors–just Shelfari. Some of them are quite good! There’s a good thread going about them. We want people to check them out, and come back to tell us how to improve LibraryThing!
**”This is me in a nutshell: HELP! I’m in a nutshell!”

(photo by Rick Dikeman on Wikipedia, under GNU Free Documentation License)

Everything on the web is better if it’s faster. Slow pages are a silent killer.

So we’re working to speed thing up. We’ve long done “situational” caching. But our growth is relentless—we’ll hit 200,000 registered members today—and we’ve had no good, generalized solution. We’ve recently been working on two solutions, for database and page-level caching. Together they should speed up certain cacheable pages, like works, authors and tags. The more resources we can free, the faster the uncacheable pages, like Talk, will become as well.

So far, only subject pages are being cached, eg.,

• Science Fiction
• Man-woman relationships
• Space colonies > Fiction
• Men > Employment > Texas > Austin > Fiction

Subject pages were a big problem. The worst took a minute to load. When Google’s “spider” program went at them, with one request/second, the servers would sweat. Subject pages are now cached whenever someone hits a page, and stays so for at least week.

Subjects are a test. There are some kinks to work out. (For example, changing the non-English translations doesn’t immediately clear all affected pages.) Once we get where we want, we’ll roll it out page-caching wherever we can use it. Query caching will follow.

I used Yahoo Pipes to make a combined feed for this blog and our Thingology blog. It was easy to do, and the result is pretty useful. The three feeds are as follows:

LibraryThing Blog
Thingology Blog
Combined Blog Feed

I also edited the employee list on the right, to add Altay. He is the magic behind the LibraryThing for Libraries Javascript, but almost nobody’s seen that yet, so we’re waiting for his first user feature to give him a proper introduction.