HomeGroupsTalkZeitgeist
This site uses cookies to deliver our services, improve performance, for analytics, and (if not signed in) for advertising. By using LibraryThing you acknowledge that you have read and understand our Terms of Service and Privacy Policy. Your use of the site and services is subject to these policies and terms.
  • LibraryThing
  • Book discussions
  • Your LibraryThing
  • Join to start using.

Excel export broken

Bug Collectors

Join LibraryThing to post.

This topic is currently marked as "dormant"—the last message is more than 90 days old. You can revive it by posting a reply.

1bnielsen
Edited: Aug 18, 2016, 12:47am Top

The "export all books as excel" happily reports that it has processed all my 7003 books and then it just keeps spinning the wheel. So it never gives me a link to download the export file. (Or at least it doesn't within an hour, which in my mind is the same as "never" :-)

7,003 books processed

The tsv export works fine.

(Reloading the page gives me a page where the "Export all books" button is disabled. I.e. I can't try to start from scratch. But maybe this is another bug?)

2kristilabrie
Aug 18, 2016, 12:11pm Top

Yeah, I tested this out with a larger account, and it froze at 8,500 records for me. I kept it up for at least an hour (probably more), so something's getting stuck here. Not sure when Tim will be able to look at this, but hopefully soon! I'll see if I can do any other testing on my end today that might help.

3anglemark
Aug 18, 2016, 2:31pm Top

It worked fine for me and my ~8,500 books before the server breakdown so it might well be because of that.

4bnielsen
Aug 18, 2016, 2:46pm Top

>3 anglemark: Ah, you confess that this is all your fault :-)

(Just joking to pass the time until the excel export is fixed.)

Maybe we need a spin doctor to fix this? (Sorry)

5anglemark
Aug 18, 2016, 3:25pm Top

6ChrisRiesbeck
Edited: Aug 18, 2016, 7:17pm Top

Just adding I'm getting the same thing with Excel export. Got to 7224 books and never did anything else for 45 minutes. Reload also yielded a page with a non-function Export button.

Selecting tab-delimited worked fine and was about 5 times faster.

7ATKlibrary
Aug 19, 2016, 9:29am Top

Also wanted to report mine is getting stuck at 4,601 books. I use the export at a weekly basis and its been down for two weeks now. Thanks for your help!

8kristilabrie
Aug 19, 2016, 12:04pm Top

We're a bit of a skeleton staff today, so Tim won't be able to look at Excel export until next week, likely. Please use the export as .tsv (tab-delimited text) option in the meantime! Thanks for your patience, all.

9Lisa2013
Aug 19, 2016, 6:36pm Top

Okay. Just wanted to report that I have only 242 books here and the Excel export won't work for me. Yesterday or today. Thanks.

10librisissimo
Aug 21, 2016, 3:21pm Top

Didn't we see this same bug symptom back when Export first added the Excel option? I remember getting hung up on large files then. Might not be the exact same problem, but could be related.

11BLLC
Aug 22, 2016, 3:49am Top

I have 126 in a collection that won't export. Text delimited won't work either. I am DESPERATE - have to get this working before my vacation this week!

12bnielsen
Aug 22, 2016, 4:29am Top

>11 BLLC: "won't work either" is not a good description. TSV works for me (7003 books), so I'm surprised if it doesn't work for you.

The "eternal spinning" bug with the excel export is well described (and hopefully soon to be fixed by Tim).

13timspalding
Aug 22, 2016, 9:42am Top

Looking into some of these.

14timspalding
Aug 22, 2016, 10:20am Top

Yeah, something is dying on the production servers--the real servers--that isn't dying on the test servers. In as much as the production servers changed last week when we had our bad downtime, I need our sysadmin's help figuring out where to look, etc.

So, sorry, still working on it.

15bnielsen
Aug 22, 2016, 2:00pm Top

>14 timspalding: Now, there's an easy fix. Just swap the test servers and the real servers :-)

16chibitika
Aug 24, 2016, 9:03pm Top

All 5220 of my books have exported, but the little arrows keep spinning. No link to access my exported library. Just thought I'd let you know it still is broken. Thank you so much for offering this. I'm sure this feature is quite complicated, and that you will figure this problem out.
(Umm . . . I'm not being sarcastic - upon reading this, I see how one could be misled.)

17Lisa2013
Aug 24, 2016, 9:25pm Top

Yes, It's still broken. I'd love some information about when it will be fixed,a nd would like it fixed. Thanks.

18lorax
Aug 25, 2016, 11:30am Top

>17 Lisa2013:

This thread is the appropriate place to watch for information on when it has been fixed. Usually they don't give estimates for when something will be fixed, unless it's of the form "Not until after ". Tim's said quite recently that he's looking into it, which means this thread has been noticed and not specifically stated as being deferred; that's the best-case scenario.

19Lisa2013
Aug 25, 2016, 11:40am Top

Ok. Thanks, lorax, for that information.

20CasualFriday
Aug 28, 2016, 9:57am Top

My export stalls at 723 books, so it's not just about large libraries.

21Lasitajs
Aug 29, 2016, 12:20pm Top

My Excel export attempts repeatedly stall. Only 500 books to export; "processed" quickly, then nothing. Have tried Win 7, Win 10, Mac, and various browsers, various days; all with same problem. Highly disappointing, since export worked fine before the "improvement" of adding Excel direct export.

22jjwilson61
Aug 29, 2016, 12:54pm Top

>21 Lasitajs: Highly disappointing, since export worked fine before the "improvement" of adding Excel direct export.

1. You didn't have Excel export before and now you do but it's not working, but you can't say that you're worst off than before.
2. The Excel export was working fine until there one of their front-end servers crashed a few days after it was introduced. They've since recovered most functionality but they seem to be having a problem getting this one back.

23timspalding
Edited: Aug 29, 2016, 2:21pm Top

We're working on it.

(It may be fixed. We're testing.)

1. You didn't have Excel export before and now you do but it's not working, but you can't say that you're worst off than before.

Well, we had a fake Excel one--it was TSV, really, and it was broken in a different way that affected a minority.

24lorax
Aug 29, 2016, 2:22pm Top

TSV is not a "fake Excel export", it's a very useful export openable by a large number of methods including spreadsheet programs. Please do not drop the perfectly good TSV export just because you now have an Excel export.

25lorannen
Aug 29, 2016, 2:32pm Top

>24 lorax: Don't worry, TSV export isn't going anywhere any time soon.

26lorax
Aug 29, 2016, 2:37pm Top

>25 lorannen:

I'd have worried a lot less without that "anytime soon" on the end of that sentence, but thank you. These days I'm not taking anything for granted.

27timspalding
Aug 29, 2016, 2:40pm Top

TSV is not a "fake Excel export", it's a very useful export openable by a large number of methods including spreadsheet programs. Please do not drop the perfectly good TSV export just because you now have an Excel export.

I am referring to how we forced initial characters onto the start of the CSV and set the header so that it would fool Excel into opening. It was not a fully functional CSV export, however. It broke when there were unclosed quotes anywhere in any field.

28Lisa2013
Aug 29, 2016, 2:51pm Top

Yes, it did just work for me. Thanks very much.

29lorannen
Aug 29, 2016, 3:02pm Top

>28 Lisa2013: Glad to hear it, but we're still tweaking the fix. I wouldn't recommend everyone start re-attempting their exports until we've reported back that it is definitely fixed.

30timspalding
Aug 29, 2016, 3:05pm Top

Yeah, it's still dying in the high thousands. But it works below that.

Still working it.

31readraw
Edited: Aug 29, 2016, 8:16pm Top

Could't make it work yesterday, but now it went pretty smooth! Very impressive feature. Amazing work, guys!

32Lisa2013
Aug 30, 2016, 2:03am Top

lorannen, Okay, Thanks. I'll wait until I hear. I have so few books it might not be a stress on the system though.

33MarthaJeanne
Aug 30, 2016, 3:08am Top

I'm really grateful that I was able to use this during the first days when it worked. It was a real help in Oxford on Sunday.

34bnielsen
Aug 30, 2016, 5:22am Top

>27 timspalding: unclosed quotes as in a 15" deep hole in the ground. :-)

35drasvola
Aug 30, 2016, 2:24pm Top

Now working for 6760 books. Fantastic! Thanks.

36ulmannc
Edited: Aug 30, 2016, 2:58pm Top

I just dumped all my data both as .xls and .tsv with about 7260 entries and both look fine.

37timspalding
Edited: Aug 30, 2016, 3:11pm Top

>36 ulmannc:

If it works, it works. The stall-out-and-die seems to be in the 7,000s, but quantity of reviews and so forth affect it.

Basically, the create-Excel stuff is taking up HUGE amounts of memory for large libraries, and, while we could increase it to cover every library, it would be like handing 50% or more of the power of our main servers to a user for five minutes, with everyone else fighting for the rest. So we're looking at other options.

38ulmannc
Aug 30, 2016, 3:24pm Top

>37 timspalding: For what it's worth, I have very few reviews. . . well under 200. Saves you a lookup!

39bnielsen
Edited: Aug 30, 2016, 6:22pm Top

>37 timspalding: Not sure what you are doing, but I'm running open office calc in filter mode. Just tried it for real on my exported tsv file:

$ cat lt.tsv | tr -d \" | sed -e 's/.*/"&"/g' | sed -e 's/\t/"\t"/g' > tim.tsv
$ localc --infilter=CSV:9,34,UTF8 --convert-to xls tim.tsv

seems to work like a charm for me. You'll need to look up some of the weird parameters to make it do exactly what you want but the documentation is actually quite good. And they like backwards compatibility which is why some of the input filters have weird names like (StarCalc).
9 = TAB, 34 = double quote

(And I cheated to stay out of trouble with double quotes. I just deleted them from my input file and then "quoted"TAB"each"TAB"field"...)

$ ls -la tim*
-rw-rw-r-- 1 bnielsen bnielsen 14575632 aug 30 23:04 tim.tsv
-rw-rw-r-- 1 bnielsen bnielsen 15077888 aug 30 23:04 tim.xls
$

This took like 20 seconds on my home computer.

Ah, and my export as xls seems to work again. Thanks. I'll go look at it with a microscope :-)

Oh, that's really not good:
$ ls -la lt1.xls
-rw-rw-r-- 1 bnielsen bnielsen 28127232 aug 30 23:10 lt1.xls
$
The file is way too small. Looking at it in localc shows that things go south rather quickly. All books have the Book Id column, but title is missing from row 1223 and down except for a very few ones. The same goes for most of the columns with useful information.

row 1240 is about book id: 22213315 and I get sort key (1), role: Author; Year: 1989, hardcover, page count and a bit more, but anything interesting is MIA.

So the xls export is definitely not working for me right now.

40kristilabrie
Edited: May 30, 2017, 10:28am Top

Bump. Also happening for t.sebesta w/ 2,329 records (http://www.librarything.com/topic/229053#6062440).

41ccatalfo
Oct 16, 2017, 2:01pm Top

>40 kristilabrie: I believe this should be improved as far as stall outs on large numbers of records, as noted in the related Talk post: http://www.librarything.com/topic/229053#6062440) by amending the code to allow longer process times. I was just able to export 4k books to Excel format.

42anglemark
Oct 16, 2017, 3:59pm Top

I just exported my 8,809 books to Excel and it went fine.

43lorannen
Oct 16, 2017, 4:58pm Top

>42 anglemark: Excellent! Marking as fixed.

44bnielsen
Edited: Oct 16, 2017, 5:20pm Top

>42 anglemark: Are you sure that your books exported fine? When I try to do an Excel export, everything _seems_ fine, but when I open the file most of the information in last half of the file is just missing, i.e. the broken mess I described in >39 bnielsen:.

Ah, >43 lorannen: marked it as fixed. Sorry, no! (Reopening.)

45ChrisRiesbeck
Oct 16, 2017, 5:45pm Top

I just exported 7,286 to Excel and data seems complete all the way down.

46lorannen
Oct 16, 2017, 5:53pm Top

>44 bnielsen: Sorry, I'm afraid I don't understand much of >39 bnielsen: . Are you exporting your full library, or filtering by collection, or anything? I'll try running an export of your library and see what happens.

47bnielsen
Oct 16, 2017, 6:42pm Top

>46 lorannen: I'm exporting the whole library as Excel. I'm using Open Office rather than Excel to open the file though. If nobody else has the problem, we should just mark it as fixed. And I can reopen the bug report, if I can find some better evidence.

Summa summarum: I shouldn't have reopened the bug report without better evidence. And thanks to >42 anglemark: and >45 ChrisRiesbeck: for checking!

48ccatalfo
Edited: Oct 16, 2017, 9:26pm Top

>47 bnielsen: i'll try your library and see what I get, it's entirely possible there are encoding or data-specific issues at work, of course

49lorannen
Edited: Oct 16, 2017, 10:00pm Top

>48 ccatalfo: Also possible, if a bit unlikely, that something about our exports isn't playing nice with Open Office. I used Open Office a lot a few years ago, and would occasionally run into spreadsheets that appeared as a garbled mess. Let me know if you want my help testing!

50anglemark
Oct 17, 2017, 1:31am Top

>44 bnielsen: It looks great all the way down.

51bnielsen
Oct 17, 2017, 5:11pm Top

I'm closing the bug report. Thank for your help. I'll investigate a bit further and reopen if I find out what the problem is :-)

Group: Bug Collectors

900 members

92,324 messages

Bug Tracking

This topic is part of LibraryThing's in-talk bug tracking.

Join or watch Bug Collectors to get "Bug Tracking" under "The World" in Talk all the time.

Bug (edit)

ID: 229303

Category: Exporting your books

Assigned to timspalding

Reported by bnielsen

Status: Closed by member

Oct 17, 2017, 5:11pm

2 years since last change

By bnielsen

Status log

Reported. bnielsen (Aug 18, 2016, 12:42am)

Fixed. lorannen (Oct 16, 2017, 4:58pm)

Closed by member. bnielsen (Oct 17, 2017, 5:11pm)

About

This topic is not marked as primarily about any work, author or other topic.

Touchstones

No touchstones

About | Contact | Privacy/Terms | Help/FAQs | Blog | Store | APIs | TinyCat | Legacy Libraries | Early Reviewers | Common Knowledge | 134,229,150 books! | Top bar: Always visible