HomeGroupsTalkZeitgeist
This site uses cookies to deliver our services, improve performance, for analytics, and (if not signed in) for advertising. By using LibraryThing you acknowledge that you have read and understand our Terms of Service and Privacy Policy. Your use of the site and services is subject to these policies and terms.
  • LibraryThing
  • Book discussions
  • Your LibraryThing
  • Join to start using.

MARC exports from LT issue

Bug Collectors

Join LibraryThing to post.

1MJBEF
Oct 30, 2018, 10:52am Top

Hi -- sorry, I'm a newbie and if I missed a post somewhere else on this, please accept my apologies and redirect!

On May 1, 2018, I exported the following record out of LT successfully:
00405 2200145 i 45000010010000000030007000100050017000170200015000​34040002400049092001300073100003200086245006600118​520005000184923002500234148043716MePoLT20180501104​.0 a4834000397 aMePoLTcMePoLTerda 4a398.20951 aMatsui, Tadashi 松居 直.10aももたろう (日本傑作絵本シリーズ) Momo Tarō. a () Momo Taro by Tadashi Matsui (1965/2/20) amissing titles NOV17

Today, when I export the record it shows this:
N

The record contents are working ok on our LT site, and I can do a successful Excel export. As I workaround I could export/import using csv files, but that will be messy at the import end (we're using OpenBiblio). So, if you can help with our MARC data dilemma I'd be grateful!

Thanks, Ted Delphia (Michigan Japanese Bilingual Education Foundation)

2lorannen
Oct 30, 2018, 11:15am Top

>1 MJBEF: Can you point me to the specific record in your library? Are you exporting individual records, or is this part of a larger export? I'd like to know whether you see any other records displaying this behavior.

3MJBEF
Oct 31, 2018, 9:16am Top

Hi Lorannen -- The specific book ID that I used in from our library is 148043716 . However, I used that example for convenience: actually, all exports from our library are showing the same error (a string of spaces and N characters). For example, if I try to export books entered since July 1, I get this string: "NNNNNNNNNNNNNNNNNNNNNNNNNNNNNN" (for 30 book records, set to Create LibraryThing "basic" records).

Again, if I do an excel export, I get the following (copy-paste of headings and data from first record only:
Book ID Title Sort Character Primary Author Primary Author Role Secondary Author Secondary Author Role Publication Date Review Rating Comment Private Comment Summary Media Physical Description Weight Height Thickness Length Dimensions Page Count LCCN Acquired Date Started Date Read Barcode BCID Tags Collections Languages Original Languages LC Classification ISBN ISBNs Subjects Dewey Decimal Dewey Wording Other Call Number Copies Source Entry Date From Where OCLC Work id Lending Patron Lending Status Lending Start Lending End
157931332 つるのおんがえし (ミキハウスの絵本) 2 岩崎京子 ミキハウス 1988 つるのおんがえし (ミキハウスの絵本) by 岩崎京子 (1988) Hardcover 10.16 inches 0.88 pounds 10.16 inches 0.47 inches 9.53 inches 10.16 x 9.53 x 0.47 inches temp Japanese 4895881075 4895881075, 9784895881074 1 amazon.com books 2018-07-03 22047941

So, I know the data is there and it's fine online. I just can't get the marc data to come out like it used to.

Thanks for helping,

Ted Delphia, Michigan Japanese Bilingual Education Foundation

4MJBEF
Nov 1, 2018, 5:42pm Top

So... can anyone else do a MARC export of https://www.librarything.com/work/22047941 (the book referenced above)? At least that way, I'll know there's something at my end or with my settings that's the problem. Thanks...

5lorannen
Nov 1, 2018, 5:53pm Top

>4 MJBEF: I'll need to test—didn't have time today. My best guess at the moment is that the non-English characters might be throwing things out of whack. Thanks for the details. I'll report back tomorrow with what I've found.

6MJBEF
Nov 2, 2018, 3:08pm Top

Thanks for doing the test... just FYI, we've never had this problem while using LT before for the past 4 years, so if the export doesn't work for you either then it's something related to changes initiated since May 1 2018,

7MJBEF
Nov 6, 2018, 10:08am Top

I just added a standard book ( https://www.librarything.com/work/1879281/book/162141070 ) to test, and that book data exported fine, whereas our books with unicode Japanese characters did not. So, it is related I believe to the non-English characters. This is unfortunate, since until at least May of this year they exported just fine as MARC data.

8lorannen
Nov 6, 2018, 11:53am Top

>7 MJBEF: Yeah, that's what I've found, too. Working on chatting with devs to see what we can do about it!

9MJBEF
Nov 18, 2018, 11:50am Top

Just to verify that it was working and now it isn't:
Here is an export of a book done on 11/2/17
00326 2200133 i 45000010010000000030007000100050017000170200015000​34040002400049100004400073245002100117520002000138​.0 a4805474440 aMePoLTcMePoLTerda1 a関根, 栄一 セキネ, エイイチ.10aいばらひめ. a by , (1982) aHimawari new group 01-30-201500416 2200157 i 45000010010000000030007000100050017000170200015000​34040002400049100002000073245007200093264002800165​300001100193520002000204923003400224115902730MePoL​.0 a4834004724 aMePoLTcMePoLTerda1 a正巳, 吉崎.10aざりがに (かがくのとも傑作集―どきどきしぜん).​ 1b福音館書店,c1973. a24 p. a () by (1973) aHimawari new group 01-30-201500458 2200145 i

And here is an export of the same book now:
N

Our last successful export was May 1 2018. Our next attempt that failed was when I started this thread on Oct 30 2018. What changes took place between these 2 dates to cause the problem??

10lorannen
Nov 19, 2018, 7:31am Top

>9 MJBEF: Can you tell me what those fields are/those numbers represent? I'd meant to ask in >1 MJBEF:, but, while I see the error you're referring to, I'm not sure what that information is that you're trying to export properly, which will be useful as we fix!

With the holiday later this week, this issue may have to wait until next week. I'll keep this thread updated as I know more.

11MJBEF
Nov 20, 2018, 9:49am Top

I will try! First, there were actually 2 book records in the upload, so I'm taking only the first one: using the book record from https://www.librarything.com/work/15695364/details/115902607 , here is what the fields represent in the MARC export (I think):

00326 2200133 i 45000010010000000030007000100050017000170200015000​34040002400049100004400073245002100117520002000138​.0 (sorry, I don't know how to parse this header field(s)).

a4805474440 (this is this ISBN field)

aMePoLTcMePoLTerda1 (I think this is the LOC cataloging code "MePoLT")

a関根, 栄一 セキネ, エイイチ.10 (this should be the author)

aいばらひめ. (this is the title)

a by , (1982) (this is the publication date)

aHimawari new group 01-30-201500416 2200157 (this field is a set collection identifier specific to our LT library)

i (maybe the end of this book record?)

Maybe we can compare to a book record such as that which I tried exporting earlier with no japanese characters, such as https://www.librarything.com/work/1879281/details/162141070 ?

Thanks, Ted Delphia

12MJBEF
Nov 28, 2018, 10:07pm Top

OK, here's that Winnie-The-Pooh record:

00574naa a2200193ui 45000010010000000030007000100050017000170080043000​34020001500077040002400092090001300116092001200129​10000170014124500430015826400810020130000250028252​0006400307923000900371162141070MePoLT2018112821574​.0 (the long header like above)

181128s1996

a0525457232 (this is the ISBN)

aMePoLTcMePoLTerda 4 (LOC cataloging code, with spaces in different spots)

aPZ7 .M64 4a823.9121 (LC classification)

aMilne, A. A.10 (author)

aThe Complete Tales of Winnie-The-Pooh. 1 (title)

aDutton Books for Young Readers (), Edition:b 1st Thus., 368 pages,c1996. (publisher and edition, with date)

a368 p. :c8 inches.

aThe Complete Tales of Winnie-The-Pooh by A. A. Milne (1996) (I think this is date, similar to the field in the Japanese record)

atemp (I think this is my set collection identifier specific to our LT library)

I didn't see the letter "i" at the beginning or end of this one, unlike the Japanese record in the message before this. But, I don't really know how to parse the code so... just a guess.

13MJBEF
Dec 3, 2018, 5:02pm Top

Well... I thought I'd try again to download MARC records from a bunch of of books from our library. This time, although 60 books showed up as " N", one book with Japanese characters did show up: the book is found at https://www.librarything.com/work/21175937/details/151657735 .

Here is the MARC record for that book as downloaded:
00479naa a2200169ui 45000010010000000030007000100050017000170080042000​34020001500076040002400091100002200115245009100137​26400400022830000120026852000170028092300120029715​.0181203 a4805476168 aMePoLTcMePoLTerda1 a小林 清之介.10aファーブルこんちゅう記―えほん版 (5) (チャイルド科学絵本館). 1aunknown :bunknown,cunknown. ap.ccm. a (5) () by aFEB2018

As you can see, the author, title, and publisher fields have Japanese characters.

So, I have most Japanese books that can't export as MARC records, and one that can. Is that enough to start troubleshooting this finally???

14MJBEF
Jan 6, 1:37pm Top

Any updates? MARC is still not working for us with UTF-8 Japanese characters on most of our records.

15MJBEF
Jan 18, 9:54am Top

We will work with CSV data exports, since MARC still doesn't work.

16lorannen
Edited: Jan 25, 12:23pm Top

>15 MJBEF: Sorry for the delay. I'm working on getting some developer help with this issue as soon as possible, but we've been a bit swamped lately. I hope to have an answer for you next week.

17ccatalfo
Jan 18, 1:48pm Top

>15 MJBEF: Hi - developer here!

I've identified the error, it is indeed related to the characters and encoding of the data.

I will take a look at fixing.

18ccatalfo
Jan 23, 9:07am Top

>15 MJBEF: Can you test re-exporting your catalog as MARC? I've made a small tweak and I believe it is working now. I get 701 records, currently, in the MARC file, and I can see japanese characters within them.

19bnielsen
Edited: Jan 23, 5:00pm Top

// I think I have an old bug report about something similar. (I'm not trying to hijack this thread, I just need a pointer to the old report since it has a few useful tips: https://www.librarything.com/topic/186132)

I just tried exporting one of my own books (Book ID 18282723) and the 520 and 590 fields looks as if non-ascii is just discarded while it exports nicely in the 245 and 921 fields. I wonder if ccatalfo can confirm that the program is doing something like that?

I'm using the online converter here:
http://marcpm.sourceforge.net/cgi-bin/converter.cgi

The word stræben in the 245 line is okay, but in the 520 line it says strben which is not ok.
(æ is a unicode æ displayed in the latin-8 charset)

000 01377 2200193 i 4500
001 18282723
003 MePoLT
005 20190123164555.0
040 $aMePoLT$cMePoLT$erda
090 4 $aPN6231 .M2
092 4 $a350.0002
100 1 $aParkinson, C. Northcote.
245 10 $aParkinsons lov eller stræben efter fremgang.
300 $a117 p.
520 $aParkinsons lov eller strben efter fremgang by C. Northcote Parkinson (1958)
590 $aOmslag: Osbert LancasterOmslaget viser mdet i en komite p ca tolv personer, der slet ikke ser ud til at have nogen interesse i at n et resultatIndskannet omslag - N650U - 150 dpiOversat fra engelsk "Parkinson's Law" af Eva Hemmer Hansen $aBur
920 au $cracy$aHumour$aManagement$aRecycled $aInd
921 ho $lder Parkinsons lov eller den voksende pyramide, Den rette mand eller udvælgelseslæren, Ledere og komiteer eller ineffektivitetskoefficienten, Folkets vilje eller den årlige generalforsamling, Personlighedssortering eller cocktailformlen, Højfinansen eller interessens forsvindingspunkt, Palmebladstag - Packard eller en formel for succes, Planer og komplekser eller det administrative trauma, Injalitis eller paralytisk lammelse, Pensionspunktet eller aldersgrænsenKlassikeren indenfor området med illustrationer af Osbert Lancaster.En klassisk bog selvom man godt kan mærke alderen tynge den $aYour li
923 ra $ry

20MJBEF
Jan 24, 9:18am Top

Thank you so much Lorannen & Ccatalfo! I just tried a MARC import and it looks good with Japanese characters in the UTF-8 coding showing nicely. We really appreciate that you support us and so many others... I work with so many databases that choke on anything not ASCII, so I really appreciate your help! Ted Delphia, MJBEF

21ccatalfo
Edited: Jan 25, 12:21pm Top

>20 MJBEF: Glad to hear it's working now!

>19 bnielsen: Thanks for the heads up, I'll take a look at this as well.

Group: Bug Collectors

903 members

92,493 messages

Bug Tracking

This topic is part of LibraryThing's in-talk bug tracking.

Join or watch Bug Collectors to get "Bug Tracking" under "The World" in Talk all the time.

Bug (edit)

ID: 298150

Category: Uncategorized

Assigned to all

Reported by MJBEF

Status: Fixed

Jan 25, 12:23pm

121 days since last change

By lorannen

Status log

Reported. MJBEF (Oct 30, 2018, 10:52am)

Closed by member. MJBEF (Jan 18, 9:54am)

Needs discussion. ccatalfo (Jan 23, 9:07am)

Closed by member. MJBEF (Jan 24, 9:18am)

Fixed. lorannen (Jan 25, 12:23pm)

About

This topic is not marked as primarily about any work, author or other topic.

About | Contact | Privacy/Terms | Help/FAQs | Blog | Store | APIs | TinyCat | Legacy Libraries | Early Reviewers | Common Knowledge | 134,911,202 books! | Top bar: Always visible