LibraryThing is Faster, part II
Join LibraryThing to post.
This topic is currently marked as "dormant"—the last message is more than 90 days old. You can revive it by posting a reply.
I'm putting this in "new features." Speed and reliability are features!
I've definitely noticed more zippiness all over the place. Job well done.
We found the bulk of it in a very, very unexpected place.
FYI, it was a common "UPDATE" command, always done by primary keys, on a tiny int field that wasn't indexed. (The field tracked whether books had been updated since the last index.) It made sense that it expired table pages from the query cache, but we still can't explain why the write/replicate action was so painful. We—I—am fanatics about MySQL query optimization. But we focus on long queries, and this was the pressure of thousands of very short updates. I caught it on a hunch I still can't explain, but at least I have a tool for ferreting similar problems out now.
Coming up with the most efficient alternative was the reason behind my StackOverflow post (http://stackoverflow.com/questions/5891789/keeping-track-of-on-offs-without-inde...). I never got around to testing more ways. The way I chose was astoundingly faster already.
One of my favorite parts of programming is building tools to make programming easier. :D
Ha. You're a "tool sharpener" are you? I'm not, but I think most good programmers are.
As a programmer groupie (I don't code but love to read about people who do), I appreciate not only the zippier site, but also your short explanations - with the graphs to back up what you're saying.
Fair enough. I think more about people who don't write a line of code until they've written the program that writes the line of code. I'm reminded of the engineer for a company I know who decided that his little project--a flash widget to track bands--required him to develop an entirely new database system. He never finished and started work on the widget.
8> Yeah, err... that happens to me more often that I'd like to admit. Sometimes there are just too many yaks to shave.
Ummmm ... Except that there's now a visible indexing of books (not sure when this began, but was definitely there yesterday), which made searching slower, and today it's hanging just a little ways in, making searching impossible ...
Please don't tell me I've reached an upper limit of books beyond which the system isn't going to work. I'm only a little over 2,300.
Ah, thank you, jbd1.
I realized a bit too late that the progress bar was probably not the slowing factor, but you'd answered my question before I could delete it from my post. I did perceive a slowing down, all the same, hence my question re: upper limit of books before things might go wonky.
At 2,330 items catalogued, I think you've got quite a way to go before that becomes a problem. ;)
See the Top 5000 personal libraries!
But at least I'm in the top 5,000 now ... That's something, anyway. ... I guess. Of course, some of those ahead of me are book dealers, so I don't feel *too* bad about being near the bottom of the list. And it gives me a level to aim for ...
... wait, hold on, what the *hell* am I *saying*???!!!
Hello, I'm Tanuki -- and I'm a biblioholic.
I found my name on that list, even without my wishlist I would be on it. My wife can never know about that. She is already worried the floor is going to collapse.
I wouldn't mind taking action to slip down that list a bit, but my wife refuses to hear of it...
Good lord, I'm in the top 1000 - and close to the top 500. Now some of that is wishlist, discarded, and inclusions (books/stories in omnibuses)...but I'd still be in the top 1000 without those. Sheesh.
Looking at your place in the list is a really, really bad habit. Among other things, it fosters annoyance at people ahead of you who pad their catalogues (as they have every right to do) with books they don't own but read once upon a time or would like to read someday or used to own or whatever - and annoyance at you by those to your rear who see that you've included electronic books or boardgames or maps (all of which I admittedly have) or perfume bottles (which I don't).
And how do some of the listers of 12,000 works tolerate the carelessness and sheer ugliness of their catalogue displays? I can only sympathize with their books, which must be an array of broken spines, bent covers, torn pages and stained flyleaves.
At the moment when I search my library from the home page the indexing bar appears (1 book to index) and then stays there. It's not just slow it's stopped.
23> That's what I'm getting, too. I put three new covers up and now can't go look at the rest of the ones that need covers - that have the tag _scanned. What mine says, specifically, is:
Books to index: Loading
(progress bar, about 1/10th green)
then the rest of the page blank. It's sat like that for 15 minutes (while I did other stuff). I'm going to bug it (if it's not already).
You know, I've noticed a lot of improvement in Talk. But I still notice quite a bit of lagginess in hitting works. Like hitting this one just now:
Took about 5-10 seconds to load. Sometimes it seems like very work I click on has the same response the first time, but that's entirely subjective and probably biased. My assumption is that this one rolled off the cache and had to be regenerated from scratch. Might be nice if you can scrounge some savings on regenerating.
I think you're right (cache had expired), because it was instant for me. Hmmm...so the solution is for you not to look at books that no-one's looked at in a while! :D
No, agreed. The average times are good, but when you hit a popular work which hasn't been hit recently, it can be painful.
Anyway, believe me, performance is big around here now.
I'm still not seeing anything faster in Talk. Moving between Your Posts and Your Groups, for instance, is still annoying with several seconds of lag. And sometimes there is a lag of up to a minute it feels like before a post shows up.
Moving between Your Posts and Your Groups, for instance, is still annoying with several seconds of lag.
I think loading Your Groups is getting slower and slower.
#32 - It was instant for me.
#30 & #31 - It's torturous. I've started opening some my groups individually while I wait for Talk to load.
One thing that seems to have suffered is the updating of review counts ... while my most recent two reviews do show up in the review count on my Profile page, they don't show up in either the catalog view (neither the "talk bubble" nor the number reflect the reviews existing) or the review count on the Works pages (although the actual review appears!).
31/33> I wonder if it's dependent on how many unread posts you have, or how many groups you watch/join.
Clamairy belongs to way more groups than I do, but DaynaRT has fewer, so maybe that's not the explanation. But I keep all mine pretty much caught up all the time - how about you two?
#35 - I haven't been caught up in at least three years. :o)
I don't even try anymore.
Edited to add: You might be on to something, though. Hot Topics only takes a fraction of a second to load.
36> As does All Topics for me. While both have a lot of unread items in them, LT shouldn't have to check quite as many tables because it doesn't care if I watch/join any of them. Your groups and Your posts on the other hand would probably have to check those tables that record that information.
I watch A LOT of groups, so you may be onto something since hot topics loads instantly.
38> Oh yeah, it didn't even occur to me that only JOINED groups would be shown in the profile, not WATCHED ones. So yeah, sounds like a possible lead. Now we just wait until Tim circles back... ;)
This group does not accept members.
This topic is not marked as primarily about any work, author or other topic.