Tag Searches - Going Back in Time

I think that the Tag Searches are pulling back months-old data.

Around the beginning of each month, I pull back the 999 Challenge tag for the group to give an informal "Top Watever" list of books being read by the group.

Last month, there were 6,128 uses of the tag - this month I pull it and there are only 4,241. To lose nearly 2,000 uses of the tag in a month seems rather unlikely. The strange thing is that it claims there are 113 users of the tag this month were there were only 111 last month - so one of my ideas (maybe a bunch of people converted to collections?) seems unlikely.

ETA - History of the tag searches:

http://www.librarything.com/topic/57480
I realize this could all be wrong, but I wanted to check.

More info - we've got a book where we know something's up - Middlesex.

http://www.librarything.com/work/2718

If you look at all tags and numbers, you'll see 999 Challenge is (1) right now.

Here's the book in my library with the 999 Challenge tag:
http://www.librarything.com/work/2718/book/22405956

In detailmuse's:
http://www.librarything.com/work/2718/book/20083727

In jhedlund's:
http://www.librarything.com/work/2718/book/29469779

In angelrose's:
http://www.librarything.com/work/2718/book/35019387

In BookLizard's:
http://www.librarything.com/work/2718/book/47950471

So, that's at least 5 of us...and that's more than 1. So I don't think that "collections ate our tags" is the issue.

I waited to see if a day would make a difference on this - still only 4,241 tags, so no one added a 999 challenge tag in all this time (not entirely impossible, but still).

And Middlesex is still only showing 1 999 Challenge tag in the tag details section.

Books with a large number of tags and tags with large numbers of instances have their tags cached. With 52 million tags on 42 million books, it's a bitch to do live calculations on all of them all the time. They are regenerated weekly or bi-weekly. The "recently tagged" section at the bottom of the tags page is not cached. And it always shows your *own* tags uncached.

I get that - but how did we lose 2,000 tags from last month? And how did Middlesex show 6 instances of the tag on July 4th but drop to showing only 1 now?

That's not how a cache works...

Ah, I suspect the answer is that it was counting deleted tags, and now it's not. I recall fixing a bug like that a while ago.

I don't think it's deleted tags - after all, the way I found the Middlesex tags was through our group read thread in the 999 Challenge. The tags weren't up to being in the top whatever in June, but cracked in July as a result of the group read. Even in your bi-weekly cache scenario, that should be one of the most read books in the challenge.

The Book Thief has always had more than 6 - 999 tags since the beginning of the year - and yet, now it's down to "deleted tags" for the low count? That book is on a ton of lists - and is currently the second most read book. If you look at the book's discussions, you see a number of 999 threads.

I could buy a small number of lost tags in one month, but 33%? And with no loss in the number of users for the tag (July was 111, this month, it's 113), this seems like there's something really odd.

Which brings me to one last bit of bad logic - in July, the 111 users with 6,128 uses of the tag represented a little over 55 books per users - your "corrected" math would be 38 books per user.

This challenge is a minimum of 72 books - most are aiming for the full 81. At the beginning of August, having only 38 books on average tagged would mean that the average user had no hope of ever completing this challenge. The top 21 users show usages of 76-113 - I'm currently at 67. Sure, there could be a ton of low-use users pulling this number down...but discussion and progress bars really don't support this position.

>4 timspalding: Tim said, "The "recently tagged" section at the bottom of the tags page is not cached."

So does this mean that the "recently tagged" section is or is not updated live? I'm looking at one right now that is not current.

Looking at tag knitting (sorry, Tim), the most recently tagged book is "Knitted Lace Designs of ...", cataloged by obimom. When I look at obimom's catalog, I see that "Knitted Lace Designs" was entered on July 25th. She's also entered "Knitting with Beads" on July 31st, also tagged knitting. That second book does not show up as "recently tagged" on the knitting tag page.

Not sure if this is a related bug, but the Tag Watch on my home page hasn't updated all week. It's usually different every time I open LibraryThing.

>9 dulcibelle:

I think it is related. That's where I first noticed the problem, then went and examined the pages for the tags watched at the time.

I'm looking into the issue. I'm not sure what's going on quite yet so I'll wait to comment until I do.

Chris: Ping Mike too.

When I logged in yesterday, my Tag Watch had changed. This morning, it's showing the same list as yesterday. I'm watching science fiction, fantasy, and paranormal so it's not like there's nothing being added. :-)

I just fixed this problem but Mike is also working on some of the same code and he's not quite ready to push his changes. Look for a change later tonight when we push the fixes.

Hey - something got pushed because the original issue I had is back up to over 7100 tags today. :)

>11 conceptDawg: I'm glad cD is looking at this.

But "bump" anyway.

I'm stuck again - same 7,117 999 challenge tags as I had on 8/11 - and I've had the same things in tag watch for-ev-ah.

Should be fixed...again. Some code was reverted to an older version by somebody. sigh.

Sorry for the inconvenience.

Yay! I'm seeing new things in Tag Watch.

Thanks!

On the other hand, *my* tagwatch has suddenly produced the trained monkey error.

Mine is too, now. I was so excited to have a working Tag Watch again, I was trying out some of my favorite tags. Around about the fourth or fifth edit, the trained monkeys bounded in.

I'm looking into it.

Fixed? Seems to be for the ones that I've checked.

Mine looks okay again. Thank you.

All serene! Thanks, cD!

And nice to see some new faces there! :-)

My tag watch finally moved - but the tag count on the 999 challenge tag search is exactly the same as it was last month (7117). I gave it a few extra days thinking it might be a cache issue...but it should have changed by now, right?

Are you searching the group? If so, then it is probably bringing back the same results every time. I noticed yesterday that group searching is broken. It's on my immediate list, but it is a separate problem from the tagwatch issues.

No - doing the same tag search I've been doing since the beginning of the year -

http://www.librarything.com/tag/999+challenge

I've never bothered with group searches. It's still 7,117 tags as of today. So I'm ruling out caching for sure.

Definitely something up - I've added to the tag count personally and it's been enough time for caching to catch up. I finally got The Book Thief which has had 12 instances of the tag since mid-August.

In adding the tag yesterday, I should have incremented it to 13. It's still at 12. And that's assuming no one else has added the tag at all...

Tag counts still holding steady...so absolutely no caching issues.

Are we just giving up on this now? I keep going back to the tag search page - and it's gotten more frustrating because I can see that the "recently tagged" actually updates now. Which tells me we're getting more '999 challenge tags' - and yet, the tag count holds steady at 7,117 tags. And the counts are exactly the same as they were on August 11th. My addition to The Book Thief hasn't pushed up the tag count on that one at all...

It's not just this tag, either - I added a new tag to my library a week ago - listsofbests - and you can't search for that via tag search and get a list of results either:

http://www.librarything.com/tag/listsofbests

I know we've had all sorts of fun talking about the improvements to tagmash - but this sort of shows that anything tagged in the last month isn't really going to show up in those results, doesn't it?

On a closely related topic, I thought the Tag Watch on the Home Page was fixed. I've been seeing it change regularly, as I would expect.

Today I added a book to my catalog and was curious who else might be reading on the topic, so I changed my Tag Watch to focus on "polymer clay". No results, not even the book I added and tagged. No results in the few hours since I did this, and Tag Watch claims to update every 5 minutes.

The tag page http://www.librarything.com/tag/polymer+clay also says "nothing tagged recently". The count reported on that page for my use of the tag is correct (I have 6 books so tagged).

YOu don't see the books that YOU tag in tagwatch. They're filtered out.

32: Not giving up on it...just haven't had the time to look at it. We have finite resources and I'm about 3 projects behind right now. :)

Just making sure. Thanks. :)

>34 conceptDawg: You're right, my mistake.

But, my own tagging should be included on a tag page, right? And my entry still doesn't show on the mentioned tag page as a "recent".

I understand you're busy, just making sure it didn't get swept under the rug.

37: Yes, your own tagging should show up in the other sections of the tag page, just not the tagwatch (since you really don't care about watching when you tag things, I mean you should kinda' know about that. :) ).

>38 conceptDawg: (since you really don't care about watching when you tag things, I mean you should kinda' know about that. :) )

I really do, honest! Most of the time. Know about it, that is. Most of the time.

Just wanted to ask on this again - I re-ran the search today:

http://www.librarything.com/tag/999+challenge

And was a bit optimistic when I saw that it was up to 7268 tags instead of the 7117 it's been stuck at forever, but then I was looking again at The Book Thief and it's forever stuck on 12 - even though I keep finding reports of more folks reading it and my recently added tag doesn't seem to be counting. The top books have actually remained the same with exact or -1 counts - with no books just below moving up 1.

This hasn't been the behavior over the last 8 months of tracking this search. Something would have moved up at the very least by one from tag counts.

So it's like something got unstuck a little bit, but immediately re-stuck.

It looks like Tag Watch is stuck again.

At the moment, I'm watching "polymer clay" and "knitting". My home page Tag Watch hasn't changed in some days, but I can quickly find books that have been added to others' catalogs in the past few days and have at least the "knitting" tag, and they simply aren't showing.

Bump. Still stuck.

Yep. I know about it. We're trying to finish up the last bits of the new Get This Book/Amazon-you-suck page. I'll jump on the tag watch thing as soon as I can.

Excellent. Feeling reassured.

Is that the official code name for the project now? :)

Glad to hear that it's still on the radar.

> 45 Glad to hear that it's still on the radar.

Me too. Actually, I was thinking the other day that I would love to know what you guys are working on. Even tidbits like "a surprise about book data" or "big project on search" without a lot of details are nice.

Feed the addiction! (I'm baking bread these days, so I think of my LT addiction like a sourdough culture -- it needs to be fed with new features and/or updates from the PTB* on a regular basis.)

*PTB = (1) Powers That Be; (2) Programmers That Be

The new feature is bigger than it was originally planned to be, but doesn't everything we do end up being that way? I think it'll be very useful.

I'm also reworking that horrendous page where you manage the things that are in your Buy/Borrow/Swap box. That box is now going to be called Quick Links, btw....with more possibilities for things in there.

47 > oooh ... What is "The new feature"? Revamped search? Revamped Buy/Borrow/Swap-aka-Quick Links? Something else?

Hi Chris, just so you know, the link to Amazon.ca disappeared from my Buy/Borrow/Swap list on both my yagoder and kathrynnd accounts. The link was there up until recently, I used it for shopping for books two weeks ago. Amazon.com or Amazon.co.uk is not very useful for pricing books if you live in Canada. Will some sort of Quick Links for the important sites such as this be coming real soon?

49: That will be fixed in about a day or two. Big changes coming there. Some of the datasources have had to undergo some data surgery so that we could add some new features.

Thanks for the update - this must mean that the Get This Book/Amazon-you-suck page is almost done as well? :)

Yeah. It's all part of one big project.

... coming in about two weeks, I presume?

That's fortnight.

No. Sooner than that. (did I just say that?!)

Anything looking promising on this? I've been doing the searches here and there - and while there was some minor movement a while back, I've been stuck on 7200-ish for a while now.

The new page has been up and running, so I figured it was safe to bump this up again.

This is still stuck at the same number it was on the 28th...so I'm guessing nothing was moved or pushed? Has there been any effort to research this at all?

Same number as before...which is effectively no movement since that minor push in September.

Is it more serious than we thought previously?

No, it's just that I'm off working on two (well, three really but one just ended with the release of the Local Books app to the iPhone App Store) other time-constrained projects right now and the problem has the possibility of taking most of a day to track down. I'll finish those projects in the next week and a half (I must, since there is a deadline for both) and I'll have some time for sure, if not before then.

He said less than two weeks. It must be so!

Seriously, thanks cD for keeping this on your docket.

As you can see from the Local Books iPhone app, when things are quiet that means that we're especially busy. That's the way it is with me right now. The only "problem" is that I'm working on two projects that you're likely not going to see. They're both projects for LibraryThing For Libraries. One of them might make it over to the main site and the other project may make it out as a separate product, but we'll see. Both of them have to be ready to demo at a conference in a week and a half, so a firm deadline is in place.