HomeGroupsTalkZeitgeist
  • LibraryThing
  • Book discussions
  • Your LibraryThing
  • Join to start using.

BOOM! Add Books Adds 749 Library Sources, 38 New Countries

New features

Join LibraryThing to post.

1timspalding
Edited: May 3, 2:38pm Top

Here's the blog post:
http://blog.librarything.com/main/2017/05/boom-add-books-adds-749-library-source...

And here, because it was fun to make, is the moving GIF of the changes:



As we said in the post, coding credit goes to ccatalfo, source credit to members davidgn, lesmel and bnielsen. Heroes all, and deserving of every praise and thanks staff and members can muster!

2timspalding
May 3, 2:50pm Top

Now that we've made the improvements to updating, editing, and adding new Add Books data sources, we want what other data sources would you like to see?

I think the obvious answer is bookstores in various other countries and markets.

The trick is, what's the API?

We could, I think, do Apple iTunes by API, and B&N. But Amazon.nl has no API. And I'm note sure who else does. I really don't want to get into screenscraping Polish book websites. (I feel LT's effort would be better doing something else, no?)

Any libraries you need? In countries you need?

3MarthaJeanne
Edited: May 3, 3:31pm Top

Büchereien Wien - The Vienna, Austria Public Library System

https://katalog.buechereien.wien.at/index.asp?DB=BIB_SYS

(Engerthstrasse is one of the branches although they aren't active any more. erdberg seems more active. Hernals is still active. Perhaps one of them would have more information.)

4lorax
May 3, 3:30pm Top

Hooray! So much happier about this than about the Amazons.

Maps are indeed fun. Here's another idea for one: https://www.librarything.com/topic/189257

5Taphophile13
May 3, 4:13pm Top

Looking at that map made me realize I really need to brush up on my world geography skills.

6amanda4242
May 3, 4:40pm Top

I'd love to have B&N as a source.

7Crypto-Willobie
May 3, 5:23pm Top

Thanks to all for this!

8suitable1
May 3, 6:33pm Top

>6 amanda4242:

B&N as a source would be great.

9rosalita
May 3, 9:36pm Top

A hearty thank you to everyone involved in adding these sources. That's terrific work by all!

10elenchus
May 3, 10:49pm Top

I love the outcome, though I haven't tested (by adding books) for practical results. Let's say I'm on board in principle at this point, and looking forward to reaping practical rewards in the coming months.

But just as much as the outcome, I love the route taken. Giving LTers an opportunity to help out is great for the culture of the site. I'm not a librarian so this isn't an instance I can help personally, but I commend LT for opening up the process.

My first though for new sources: specialty collections like historical societies. professional organizations, maybe even fan sites (IMdb or isfdb?). But I realise all this depends upon the information being available in proper format, so I'll have to look for specific examples.

11jjwilson61
May 3, 10:56pm Top

>10 elenchus: Why would you have to be a librarian?

12davidgn
May 3, 11:00pm Top

I'm no librarian, just an autodidact z39.50 twiddler (autodidact, that is, for those elements I didn't learn from lesmel ). But it helps to have some idea what one is doing.

13elenchus
May 3, 11:04pm Top

>11 jjwilson61:

Yer right, I meant it mostly as shorthand for someone with the specific knowledge needed. I don't have that and am not going to pursue it now. A tip of the hat to davidgn for acquiring them on his own!

14lilithcat
May 3, 11:36pm Top

>11 jjwilson61:

Well, the blog post does say, "If you’re a librarian or library professional who’d like to help with updating and adding new sources, get in touch with our developer Chris Catalfo (ccatalfo) and we’ll add you to the group Library Add Books Sources Maintenance, which opens up source editing. Because the details are so technical, and there’s some danger of messing things up, we’re making group membership by request only."

15Ennas
Edited: May 4, 12:18pm Top

Great!

The link "Cataloging sources I'd like to see" is dead, though, so I'll post my suggestion here. I would like to see www.bol.com as a source for Dutch books. They have everything. They're ~ the Dutch amazon. :-)

16timspalding
May 4, 7:59am Top

But just as much as the outcome, I love the route taken. Giving LTers an opportunity to help out is great for the culture of the site. I'm not a librarian so this isn't an instance I can help personally, but I commend LT for opening up the process.

Thanks. We wanted to do this a long time, but there were a lot of fiddly bits, and one gets a little tired of making another graphical user interface. (As opposed to giving users access to the database tables directly.) Also, we weren't sure how much help we'd get. Obviously, we were pleasantly surprised--although so much has been one member, so I don't think it we were wrong to wonder.

I'm no librarian, just an autodidact z39.50 twiddler (autodidact, that is, for those elements I didn't learn from lesmel ). But it helps to have some idea what one is doing.

Ah, I didn't know that. (Perhaps CC did, as he's been working with you much more.)

My feeling is that really opening it up would be tricky. We'd have to create all sorts of special ways to fork, sequester, review and approve sources before users got to use them. But what do you think?

17lilithcat
May 4, 8:42am Top

what other data sources would you like to see?

The Newberry Library and the Art Institute of Chicago's Ryerson and Burnham Libraries, please!

18davidgn
Edited: May 4, 9:04am Top

>18 davidgn: The Newberry is part of Consortium of Academic and Research Libraries in Illinois (CARLI), and I don't think there's any real way to query it separately via z39.50 (though I'll take a second look). You can search it via CARLI, though. As for the AIC libraries, they're in now :-) (Running Millennium, though they're due to migrate to Alma, which means that source will probably break sometime in the next year. Be sure to holler).

>3 MarthaJeanne: Took a look at Bucherei Wien, but the OPAC is not familiar, so hard to guess the correct settings (or whether it offers z39.50 connectivity at all). It's on the list of stuff that's being looked at.

19lesmel
May 4, 9:21am Top

>17 lilithcat: & >18 davidgn: I'll look at Newberry. They may have a separate database name.

>3 MarthaJeanne: & >18 davidgn: I'll do some Google-Fu to see if I can find anything. Europe is not my forte when it comes to z39.50; but I'm game for hunting!

20melannen
May 4, 9:47am Top

Yay! This is exciting.

I'd really like to see gegnir.is (the Icelandic library union catalog) added - very few Icelandic-language books seem to turn up in non-Icelandic libraries and my sister keeps buying them for me.

Wikithing says gegnir has a z39.50 connection but I have no idea how to confirm that.

21davidgn
Edited: May 4, 9:48am Top

>20 melannen: Looked at that. It seems they restrict by IP, so someone on the staff would need to write them and request to get LT's server's IP unrestricted. It's on my wishlist, too.

22timspalding
May 4, 11:45am Top

The Newberry is part of Consortium of Academic and Research Libraries in Illinois (CARLI), and I don't think there's any real way to query it separately via z39.50 (though I'll take a second look).

Do people want subsets, though? Isn't it better to query ALL the stuff in a given consortium?

I'd really like to see gegnir.is (the Icelandic library union catalog) added - very few Icelandic-language books seem to turn up in non-Icelandic libraries and my sister keeps buying them for me.

I agree. We need SOME Icelandic library too.

23lesmel
Edited: May 4, 12:10pm Top

>22 timspalding: Not necessarily. CARLI is REALLY unique. 130+ IL institutions from university/college to public to Newberry. And as an amateur genealogist, I really don't want to have to wade through the dozens (possibly) of records from IL public libraries that have genealogy collections if I'm looking for something I know the Newberry has...and probably has the cleanest record for.

This sort of falls back to the concept of library network vs consortium. MPOW is a consortium with one ILS (soon to be five) and five unique institutions in the health sciences field. CARLI, while a consortium in name, is more like a network. So is Orbis CASCADE.

But you also have something like a community college with multiple campuses that shares one ILS. They aren't really a consortium; technically they are a network -- but their libraries don't really serve unique populations. Therefore, the community college example probably won't have super unique collections between the campuses. Although, there are exceptions, of course.

ETA: CARLI members list -- https://www.carli.illinois.edu/membership/mem-libs

24melannen
May 4, 12:13pm Top

I think ANY Icelandic library basically means Gegnir - if there are any that haven't joined they almost certainly don't have an API (or possibly computer catalog) at all. I just looked up a couple of tiny local rural libraries and trade schools near where my sister is and they're all on Gegnir. It's a very small and very interconnected country.

If you can't get the IP approved for Gegnir, Eymundsson is the largest bookseller chain, so that might be worth a try? The libraries would be so much better though.

25lorannen
May 4, 12:14pm Top

>15 Ennas: Thanks! We merged the Talk threads (originally there were two), and neglected to update the blog post. I've taken care of that now.

26PhaedraB
May 4, 12:57pm Top

One of the things I so love about LT is conversations about things like "What's a good source for books in Icelandic?" I am connected to people who read Icelandic (and a zillion other languages, too, of course). It's kinda wonderful. :-)

27timspalding
May 4, 1:26pm Top

We need a source before the Jólabókaflóð!

28MarthaJeanne
May 4, 2:20pm Top

The bookstores in Iceland are amazing! The 800 or so books published each year in Icelandic aren't enough for these people, so each section also has a good selection of books on the subject in other languages.

29melannen
May 4, 2:36pm Top

ég las ekki íslensku gott :P But somehow that hasn't stopped me from accumulating books in my library, especially if they have pretty pictures.... or from going to every used bookstore in downtown Reykjavik. (Also there are a fair number of Icelandic publications in English, which helps.)

The Jólaköttur will get us if we don't get a new Icelandic source before the Jólabókaflóð!

30bnielsen
May 5, 5:41am Top

>18 davidgn: and >22 timspalding:

https://www.carli.illinois.edu/products-services/i-share/opac/z3950-names gives a hint that specifying username nby might give the Newberry database and not the whole thing?

31davidgn
Edited: May 5, 6:20am Top

Yeah, it is possible. But we'd need to alter the check that prevents duplicate connection details to make it account for differing usernames. As things stand, the system will block it.

32bnielsen
May 5, 7:52am Top

>31 davidgn:. Ah, yes of course.

I'll see if my barebones z39.50 client know how to use usernames. If so, I'll poke around a bit on the list.

33CtrSacredSciences
May 5, 2:42pm Top

"BOOM!" Wow! So terrific! I'm very excited about this. I see a lot of soon to be new favorite sources.

"Now that we've made the improvements to updating, editing, and adding new Add Books data sources, we want what other data sources would you like to see?"

yes -- Apple iTunes
and Berkeley Theological Union

34lesmel
May 5, 3:06pm Top

>33 CtrSacredSciences: Does Berkeley Theological Union = Graduate Theological Union? http://gtu.edu/library

35CtrSacredSciences
May 5, 3:15pm Top

Yes, Graduate Theological Union.

Sorry I haven't searched yet on how to do the quote back reply link.

36lorannen
May 5, 3:17pm Top

>35 CtrSacredSciences: Just type the right arrow ">" followed by the number of the post you're replying to "35" for example, with no spaces between the arrow and number. To reply to your post, I started this one with

> 35

with no space!

37lesmel
May 5, 3:24pm Top

>17 lilithcat: Newberry in and working!!

38newcrossbooks
May 7, 1:45pm Top

Thanks for all these additional sources - but what has happened to SOAS (UK)? It seems to have disappeared from the UK source listings recently.

Although next door to University College London, SOAS library doesn't use the same library system, and it is not part of University of London Research Library Services either. SOAS library moved over to using Kuali OLE in 2015.

I last used it for cataloguing a book on 5 April 2017 but it's now 'Auto detected as not working' on my Add Books page.

39davidgn
Edited: May 7, 1:53pm Top

>38 newcrossbooks: Yeah, looks like they finally shut down the Millennium catalogue. I'll see if I can get the Kuali one working.

40lorannen
Edited: May 19, 7:27pm Top

Update: As of today, we've reached 2,160 working library sources, covering 110 countries! Tim updated the map on the blog, too.



Major kudos and thanks to davidgn, lesmel, and bnielsen for all their hard work!

Edited to make map clickable (click to see full size!).

41AnnieMod
Edited: May 19, 6:41pm Top

>40 lorannen:

I will have to ask. Bulgaria gets red on the map which means that there is a source from there, right? I do not see one in the list though. What am I missing?

Never mind - I need better glasses - found them :) Although I am not sure how/what it is looking for exactly - searching by common names and words does not find anything. But that is for another conversation.

42lorannen
May 19, 6:57pm Top

>41 AnnieMod: Glad you found it! Definitely let us know if it seems to not be working.

43elenchus
Edited: May 19, 7:15pm Top

Love it. I expect the next challenge will be who can add the most obscure source and/or country.

Is there a Vatican City source? Hmmmn (trundles off to look).

ETA Ah! I was briefly fooled, since Vatican City does indeed appear in the list of countries! But alas, as yet no source.

44timspalding
May 19, 8:32pm Top

>43 elenchus:

Actually, it's a broken source--hence being on the list. You can click to show it.

45davidgn
Edited: May 19, 10:55pm Top

>38 newcrossbooks: I emailed SOAS but never heard back. If you want to try to get in touch with them and ask them whether their catalog is still accessible via the Z39.50 protocol and, if so, what the correct connection details might be, that would be very helpful.

46davidgn
Edited: May 21, 11:52am Top

Any other requests, from anywhere in the world? Approaching the point of diminishing returns with the major lists I've been going through, and right now a specific goal would be a lot more satisfying than adding another round of public libraries in nowheresville. (Please don't say Austria, though. Believe me, I've tried. There are two or three networks that cover 90%+ of the country's libraries, and if any of them offer z39.50 access, I can't tell).

47timspalding
Edited: May 21, 1:14pm Top

>46 davidgn:

My interests lead me to notice the gap in Central Asia most of all. There's a coherent, linked culture-space there that's not well represented in our sources. Africa has the most gaps, though. And, as you say, Austria and Georgia.

Our challenge is to do something with all the new libraries.

I think it we categorize them we'll see that public libraries are the biggest need. Having an academic library in a county with its own language is really only half the battle.

Some projects for LT:

We should allow members (not just you guys) to link sources with LT local venues. It would have to be a many-to-many relationship.

I'd like LT to embark on a project to get one library record for every book we know about. That's impossible, be we can get closer. In the process, I think, we can get a better idea of what libraries are good for what.

48davidgn
May 21, 1:38pm Top

>47 timspalding: Yeah, Central Asia is a gap for Marshall Breeding's site too. Something tells me he's going to be getting some new listings out of this. ;-)

49davidgn
Edited: May 26, 6:24pm Top

>3 MarthaJeanne: Well, it's not the Büchereien Wien per se, but thanks to ccatalfo, who has now implemented support for MAB, we now have the Österreichischer Bibliothekenverbund (which I believe encompasses the bulk of the libraries in Vienna and in Austria as a whole). As a bonus, we also have the Wienbibliothek im Rathaus. How's that? :-)

Let us know if there are any problems with the catalog entries. ccatalfo still wants more testing on the MAB support.

50MarthaJeanne
Edited: May 27, 12:41pm Top

>49 davidgn: Well that would be really great - if it worked. I just tried two books I borrowed from from the library yesterday and both searches failed, so either the search isn't working or doesn't include the Vienna public libraries. (Tried both on ISBN, one also on title. I know they are in the online catalogue because I used the catalogue to find them.)

Wienbibliothek checked on the book I read there because it was too expensive to buy. That worked. This will prove very useful, as they get a copy of every book published in Vienna.

Edited to add - the Verbund does not list the Büchereien Wien in their list of members.

51davidgn
Edited: May 27, 12:43pm Top

>50 MarthaJeanne: I'll take a look this afternoon. Would you mind sharing the details on one or both of those non-working titles for diagnostic purposes (either here or privately)?

52MarthaJeanne
Edited: May 27, 12:58pm Top

https://www.librarything.com/work/15520471/details/114336386

This one works in Wienbibliothek but not in ÖBV, although they are a member.

https://www.librarything.com/work/10163655/book/141799816
https://www.librarything.com/work/2342241/book/141799630
https://www.librarything.com/work/14602859/book/141799508

are my most recent library books.

BuchDas Mädchen aus dem Song
Titelzusatz: Angie, Lola, Rita, Suzanne und Maggie May - und welche Geschichte sich dahinter verbirgt
Einheitssachtitel The girl in the song: the true stories behind 50 classic pop songs
Verfasserangaben: Michael Heatley ; übers. von Madeleine Lampe und Thorsten Wortmann
1. VerfasserIn: Heatley, Michael
Personen: Lampe, Madeleine Übers.
Wortmann, Thorsten Übers.
Impressum: Berlin : Schwarzkopf & Schwarzkopf, 2010
Umfang: 247 S. : Ill.
Auflage: 2. Aufl.
ISBN: 978-3-89602-579-1
Mediennr.: 1042461

is the beginning of the catalogue entry for one of them.

Reading more on the ÖBV website, they are primarily for academic libraries, which is probably why the Büchereien aren't part of it. For diagnostic purposes the first book is therefore the most useful.

53royalhistorian
May 27, 2:45pm Top

Dutch and Belgian bookstores are needed.

Bol.com
Bruna.nl
standaardboekhandel.be
boekwinkeltjes.nl (secondhand books. Setup your own store to sell books)

54davidgn
Edited: May 27, 5:06pm Top

>53 royalhistorian: Bookstores tend to be harder than libraries since they don't normally use standard library protocols like Z39.50 for exchanging bibliographic data. It might be possible to work with some of them on a custom basis, but that's an investment of resources upon which I have no standing to comment. Of the four you mentioned, it looks like Bol.com is the only one that advertises an open API, so that's probably going to be the best candidate, but commenting on the feasibility in practice is not my wheelhouse. Speaking realistically, the others will probably be too much effort to consider unless they offer sort of API access that they don't advertise or that I missed.

55jjwilson61
May 27, 11:47pm Top

Currently, LT supports the z39.50 interface which many libraries use and the Amazon interface. To add another interface would be a pretty big effort and Tim has never in the past been interested in doing that.

Group: New features

44,084 messages

This group does not accept members.

About

This topic is not marked as primarily about any work, author or other topic.

You are using the new servers! | About | Privacy/Terms | Help/FAQs | Blog | Store | APIs | TinyCat | Legacy Libraries | Early Reviewers | Common Knowledge | 117,012,302 books! | Top bar: Always visible