DATA LOSS! "Number of copies" set to zero for all books
Join LibraryThing to post.
This topic is currently marked as "dormant"—the last message is more than 90 days old. You can revive it by posting a reply.
Recently (I believe the data was there when I last looked on Thursday, but do not have a download file more recent than August 28, when the data was present and correct) the "Number of copies" data for every title in my lorax_short account was set to zero. I have carefully fllled out this data for all of the ~3200 titles in that catalog (many of which I have more than one copy of) and was shocked to see it deleted.
When can I expect to see this data restored? We've always been promised, after all, that data loss is taken seriously.
Incidentally, I do have the download file (can't call it a backup, since I can't restore from it) as proof that the data was there, in case someone wants to suggest that I was only imagining entering it.
A JSON download of my entire catalog done on 9/1/19 shows a nonzero "number of copies" on many of my books (probably all of them, but I haven't checked them all yet). Looking at my catalog right now, only the books added yesterday have a nonzero "number of copies". >1 lorax: is not imagining this. This needs to be taken seriously.
I opened a random book in the middle of my catalogue, and it's also at zero copies.
On the plus side: I did another JSON download just now, and compared it to my 9/1 JSON download. Of all the data captured in the JSON format it appears that "number of copies" is the only field affected.
Okay, we're going to be restoring the data, but it's going to take some time, and it will go in stages.
I suggest you ignore the field entirely while we work this out. You'll notice it changing, but it will go to 1 before it goes to whatever you had. (In 99.9% of cases it was one—this is the least changed field—so I'm starting by setting it to 1.)
However, any changes you make now will stick and not be replaced. So go ahead and make changes if you want.
It ain't up yet. And I'm not sure about the last few days. Working it.
Oh, I can see it's not there yet - I'm just happy this got noticed and a fix initiated quickly.
They're all still at zero for me; when should we expect to see them starting to repopulate?
I am having a the same issue. We have a large catalogue with multiple copies. Looking for an admin to look into the problem.
Data still populating. It's currently populating rather old data, because spinning up the recent backup is only partway done.
I'm no longer seeing zeroes, but the numbers in some cases are incorrect - they're one smaller than they should be in those cases. This is consistent with having been restored from old data (I'm looking at lorax_short, my short fiction account; when I enter the contents of a new anthology, rather than adding duplicate copies of a short story I increment the "Number of copies" by one). Is what we have now the final version, in which case I can start to edit the data, or is it still updating?
Any updates you make will "stick."
But, indeed, the data is old. Pedro told me the recent backup is almost finished. Once it is, I'll be replacing any differences.
I'll wait another day or two until making any changes, then; not because I'm worried about things not sticking, but because I don't want to do manually what will be done automatically if I'm just patient.
>10 timspalding: So, do you have any idea what happened in the first place?
PMM Library has its number of copies column changed from 0 to 1. LibraryThing stated this was their first step in fixing the problem (all zeros in the number of copies column) and that more changes would be coming to change these to the actual number of copies recorded.
Does this mean we should not add new books to the library until this problem is fixed?
Thank you, Betty
You can add books to your library, just don't touch the "Copies" field of any of your affected books while we work on restoring that data. If you do, whatever you edit that field as will stay.
Will there be an update here when it's done? Mine (as noted above) had the second pass to update to "old, but not all ones", but hasn't yet had the "more recent backup" refresh.
The new data has been made available. I'll be starting the process to bring it online shortly, once I can make sure that nothing you do now will be overwritten by it. It will then take maybe 24h.
I still have a recently added book that doesn't combine right with the work, number of copies stuck on 1.
My copy: https://www.librarything.com/work/book/173261489 shows one copy of the work. The tags and ratings of the work are visible here.
My copy is not listed in editions: https://www.librarything.com/work/12506310/editions/173261489
I have tried recalculating to no avail. Not sure what else I can do.
So, it's now almost two weeks after we were told that the process would be brought on line "shortly" - which to me in this context is a timeline suggesting hours, maybe a day at most - and would take 24h after that. Can we get an updated timeline?
>32 lorax: I believe Tim is still working on it, I can't recall specifics (will ask him to update) but I know there were two parts to completing the data restoration, one which wouldn't take long but one which would take some time. I'll ask him to post here with an update as soon as he can. Thanks for your continued patience!
Sorry Lorax. The process should be started on Friday or Saturday. It will probably take 24h to complete. I'm out tomorrow, at the colo. But I'll update Friday.
Okay, it's finally done.
A brief report:
The base of the restore was a backup made on 8/30/19. On top of this were layered any changes made since 9/9/19.
All-told some 47,703 books were updated from where they are now.
Changes made during the gap between 8/30 and 9/9 were not recoverable. This is unfortunate. I will be sending out notices to members that added books during that period, with a search link to see only those books added them.
The problem here was ultimately in three steps:
1. A script that screwed up all the copies. (The script was part of the new power-edit code we're working on. The power edit was rather too powerful.)
2. Not being caught quick enough for our 6-12-hour replication server to have the data.
3. Server limitations giving us a backup that was 9 days old, rather than every few days. We've been struggling to make backups as fast as we need to. This was to change starting Monday, when we finally moved to a new database server we bought some time ago, but will change next week instead. The new server completes our move to a primary database cluster (one master, two primary slaves and a delay slave) with more than twice as much storage available. That will make backups and everything else a lot easier; as it was, we've been in a constant fight with space.
This was a fairly complex process I had to go through slowly, but I have backups of the data at every stage in the process. So if there were any errors, I can do a forensic analysis of where they were. Give me specific book ids if you have any concerns.
Thanks! I'm glad that if this had to happen it was on a field that isn't used much, so that you could figure things out and get processes in place that will hopefully make it smoother if something like this happens on a more significant field. I'll check to see if there are still any issues.
This topic is part of LibraryThing's in-talk bug tracking.
Join or watch Bug Collectors to get "Bug Tracking" under "The World" in Talk all the time.
Category: Your books (catalog)
Assigned to all
Reported by lorax
Oct 9, 2019, 10:47pm
357 days since last change
This topic is not marked as primarily about any work, author or other topic.