HomeGroupsTalkZeitgeist
This site uses cookies to deliver our services, improve performance, for analytics, and (if not signed in) for advertising. By using LibraryThing you acknowledge that you have read and understand our Terms of Service and Privacy Policy. Your use of the site and services is subject to these policies and terms.
  • LibraryThing
  • Book discussions
  • Your LibraryThing
  • Join to start using.

LibraryThing is Faster, part II

New features

Join LibraryThing to post.

This topic is currently marked as "dormant"—the last message is more than 90 days old. You can revive it by posting a reply.

1timspalding
May 10, 2011, 11:22am Top

Blog post:
http://www.librarything.com/blogs/librarything/2011/05/librarything-is-faster-pa...

I'm putting this in "new features." Speed and reliability are features!

2brightcopy
May 10, 2011, 11:24am Top

I've definitely noticed more zippiness all over the place. Job well done.

3timspalding
Edited: May 10, 2011, 11:35am Top

We found the bulk of it in a very, very unexpected place.

FYI, it was a common "UPDATE" command, always done by primary keys, on a tiny int field that wasn't indexed. (The field tracked whether books had been updated since the last index.) It made sense that it expired table pages from the query cache, but we still can't explain why the write/replicate action was so painful. We—I—am fanatics about MySQL query optimization. But we focus on long queries, and this was the pressure of thousands of very short updates. I caught it on a hunch I still can't explain, but at least I have a tool for ferreting similar problems out now.

Coming up with the most efficient alternative was the reason behind my StackOverflow post (http://stackoverflow.com/questions/5891789/keeping-track-of-on-offs-without-inde...). I never got around to testing more ways. The way I chose was astoundingly faster already.

4brightcopy
May 10, 2011, 11:34am Top

One of my favorite parts of programming is building tools to make programming easier. :D

5timspalding
May 10, 2011, 11:36am Top

Ha. You're a "tool sharpener" are you? I'm not, but I think most good programmers are.

6brightcopy
May 10, 2011, 11:40am Top

5> You couldn't tell from my incessant greasemonkeying? ;)

7elenchus
May 10, 2011, 12:50pm Top

As a programmer groupie (I don't code but love to read about people who do), I appreciate not only the zippier site, but also your short explanations - with the graphs to back up what you're saying.

8timspalding
May 10, 2011, 2:29pm Top

>6 brightcopy:

Fair enough. I think more about people who don't write a line of code until they've written the program that writes the line of code. I'm reminded of the engineer for a company I know who decided that his little project--a flash widget to track bands--required him to develop an entirely new database system. He never finished and started work on the widget.

9brightcopy
May 10, 2011, 2:34pm Top

8> Yeah, err... that happens to me more often that I'd like to admit. Sometimes there are just too many yaks to shave.

10justjim
May 10, 2011, 2:40pm Top

Naked yaks or not, good job. Well done.

11Tanuki.kun
May 10, 2011, 2:46pm Top

Ummmm ... Except that there's now a visible indexing of books (not sure when this began, but was definitely there yesterday), which made searching slower, and today it's hanging just a little ways in, making searching impossible ...

Please don't tell me I've reached an upper limit of books beyond which the system isn't going to work. I'm only a little over 2,300.

12jbd1
Edited: May 10, 2011, 2:49pm Top

>11 Tanuki.kun: - The progress bar on searching is explained here; it's not really making search slower, just showing you the progress along the way (before it was just slow without explaining what it was doing).

However, it's broken at the moment; Tim's working on it right this second.

13Tanuki.kun
May 10, 2011, 2:59pm Top

Ah, thank you, jbd1.

I realized a bit too late that the progress bar was probably not the slowing factor, but you'd answered my question before I could delete it from my post. I did perceive a slowing down, all the same, hence my question re: upper limit of books before things might go wonky.

14jbd1
May 10, 2011, 3:18pm Top

Indexing should be working again now. Sorry!

15Tanuki.kun
May 10, 2011, 3:52pm Top

And it woiks! Thank you, Jeremy. :)

16justjim
Edited: May 10, 2011, 4:36pm Top

At 2,330 items catalogued, I think you've got quite a way to go before that becomes a problem. ;)

See the Top 5000 personal libraries!

17Tanuki.kun
Edited: May 12, 2011, 11:11am Top

But at least I'm in the top 5,000 now ... That's something, anyway. ... I guess. Of course, some of those ahead of me are book dealers, so I don't feel *too* bad about being near the bottom of the list. And it gives me a level to aim for ...

... wait, hold on, what the *hell* am I *saying*???!!!

Hello, I'm Tanuki -- and I'm a biblioholic.

Thanks, justjim.

18TLCrawford
May 12, 2011, 11:21am Top

I found my name on that list, even without my wishlist I would be on it. My wife can never know about that. She is already worried the floor is going to collapse.

19anglemark
May 12, 2011, 11:32am Top

I wouldn't mind taking action to slip down that list a bit, but my wife refuses to hear of it...

20jjmcgaffey
May 13, 2011, 4:48am Top

Good lord, I'm in the top 1000 - and close to the top 500. Now some of that is wishlist, discarded, and inclusions (books/stories in omnibuses)...but I'd still be in the top 1000 without those. Sheesh.

21TomVeal
May 13, 2011, 6:56pm Top

Looking at your place in the list is a really, really bad habit. Among other things, it fosters annoyance at people ahead of you who pad their catalogues (as they have every right to do) with books they don't own but read once upon a time or would like to read someday or used to own or whatever - and annoyance at you by those to your rear who see that you've included electronic books or boardgames or maps (all of which I admittedly have) or perfume bottles (which I don't).

And how do some of the listers of 12,000 works tolerate the carelessness and sheer ugliness of their catalogue displays? I can only sympathize with their books, which must be an array of broken spines, bent covers, torn pages and stained flyleaves.

22anglemark
May 14, 2011, 5:19am Top

Dammit, now I want a Like button again.

23CDVicarage
May 14, 2011, 6:07am Top

At the moment when I search my library from the home page the indexing bar appears (1 book to index) and then stays there. It's not just slow it's stopped.

24jjmcgaffey
May 14, 2011, 6:17am Top

23> That's what I'm getting, too. I put three new covers up and now can't go look at the rest of the ones that need covers - that have the tag _scanned. What mine says, specifically, is:
Indexing Catalog
Books to index: Loading
(progress bar, about 1/10th green)
1.

then the rest of the page blank. It's sat like that for 15 minutes (while I did other stuff). I'm going to bug it (if it's not already).

25timspalding
May 14, 2011, 8:56am Top

Fixed.

26brightcopy
May 17, 2011, 12:43am Top

You know, I've noticed a lot of improvement in Talk. But I still notice quite a bit of lagginess in hitting works. Like hitting this one just now:

http://www.librarything.com/work/11084551

Took about 5-10 seconds to load. Sometimes it seems like very work I click on has the same response the first time, but that's entirely subjective and probably biased. My assumption is that this one rolled off the cache and had to be regenerated from scratch. Might be nice if you can scrounge some savings on regenerating.

27jjmcgaffey
May 17, 2011, 12:46am Top

I think you're right (cache had expired), because it was instant for me. Hmmm...so the solution is for you not to look at books that no-one's looked at in a while! :D

28brightcopy
May 17, 2011, 12:52am Top

But the un-looked-at books need the most love!

29timspalding
May 17, 2011, 1:30am Top

>26 brightcopy:

No, agreed. The average times are good, but when you hit a popular work which hasn't been hit recently, it can be painful.

Anyway, believe me, performance is big around here now.

30Morphidae
May 17, 2011, 7:19am Top

I'm still not seeing anything faster in Talk. Moving between Your Posts and Your Groups, for instance, is still annoying with several seconds of lag. And sometimes there is a lag of up to a minute it feels like before a post shows up.

31DaynaRT
May 17, 2011, 8:49am Top

Moving between Your Posts and Your Groups, for instance, is still annoying with several seconds of lag.

I think loading Your Groups is getting slower and slower.

32Aerrin99
May 17, 2011, 9:01am Top

>26 brightcopy: Wow. I just tried that one. 37 seconds to load.

33clamairy
May 17, 2011, 9:09am Top

#32 - It was instant for me.

#30 & #31 - It's torturous. I've started opening some my groups individually while I wait for Talk to load.

34BTRIPP
May 17, 2011, 10:00am Top

One thing that seems to have suffered is the updating of review counts ... while my most recent two reviews do show up in the review count on my Profile page, they don't show up in either the catalog view (neither the "talk bubble" nor the number reflect the reviews existing) or the review count on the Works pages (although the actual review appears!).

 

35brightcopy
May 17, 2011, 10:57am Top

31/33> I wonder if it's dependent on how many unread posts you have, or how many groups you watch/join.

Clamairy belongs to way more groups than I do, but DaynaRT has fewer, so maybe that's not the explanation. But I keep all mine pretty much caught up all the time - how about you two?

36clamairy
Edited: May 17, 2011, 11:06am Top

#35 - I haven't been caught up in at least three years. :o)
I don't even try anymore.

Edited to add: You might be on to something, though. Hot Topics only takes a fraction of a second to load.

37brightcopy
May 17, 2011, 11:10am Top

36> As does All Topics for me. While both have a lot of unread items in them, LT shouldn't have to check quite as many tables because it doesn't care if I watch/join any of them. Your groups and Your posts on the other hand would probably have to check those tables that record that information.

38DaynaRT
May 17, 2011, 11:17am Top

I watch A LOT of groups, so you may be onto something since hot topics loads instantly.

39brightcopy
May 17, 2011, 11:25am Top

38> Oh yeah, it didn't even occur to me that only JOINED groups would be shown in the profile, not WATCHED ones. So yeah, sounds like a possible lead. Now we just wait until Tim circles back... ;)

40clamairy
May 17, 2011, 11:32am Top

Right now I think he's under pressure to pay attention to a few hundred other things first, so I'll cut him some extra slack.

Group: New features

45,201 messages

This group does not accept members.

About

This topic is not marked as primarily about any work, author or other topic.

Touchstones

No touchstones

About | Contact | Privacy/Terms | Help/FAQs | Blog | Store | APIs | TinyCat | Legacy Libraries | Early Reviewers | Common Knowledge | 134,247,565 books! | Top bar: Always visible