HomeGroupsTalkZeitgeist
This site uses cookies to deliver our services, improve performance, for analytics, and (if not signed in) for advertising. By using LibraryThing you acknowledge that you have read and understand our Terms of Service and Privacy Policy. Your use of the site and services is subject to these policies and terms.
  • LibraryThing
  • Book discussions
  • Your LibraryThing
  • Join to start using.

Wikipedia references for LibraryThing works

New features

Join LibraryThing to post.

This topic is currently marked as "dormant"—the last message is more than 90 days old. You can revive it by posting a reply.

1timspalding
Edited: Nov 21, 2009, 2:29am Top

I've introduced a new feature, showing all the Wikipedia pages that contain citations to LibraryThing works.

Basically, I parsed Wikipedia looking for books, and then connected them to LibraryThing works. I'd done this once before, back in 2007 (see http://www.librarything.com/blog/2007/02/wikipedia-citatons-with-feed.php). Back then it covered some 90,136 pages. Now it covers some 251,911. It's a little tough to calculate the works covered, but I believe it blew up similarly. It covers some 227,852 works—a very high percentage of works that aren't singletons. All told, there were 540,000 citations found.

Some examples:

*D-Day: June 6, 1944 http://www.librarything.com/work/40642#references
*An Inquiry into the Nature and Causes of the Wealth of Nations http://www.librarything.com/work/12307#references
*The Ship of the Line http://www.librarything.com/work/963894#references

Questions:

1. So, what you think is the winner—the work cited on the most pages? It's no longer a Pokemon. I've given you number 6 (The Ship of the Line). When you hear it, you'll hit your head in recognition.

2. Where should this feature go? It's related to Get this Book—it's a link. But it's different.

3. Can you see some other cool things we could do?

2jmnlman
Edited: Nov 21, 2009, 2:37am Top

How about a "your books on Wikipedia page"?

3timspalding
Nov 21, 2009, 2:57am Top

>2 jmnlman:

Contemplated.

Also, it's fairly good, if boring, recommendation engine—what books are cited by pages that cite a given book.

4asalamon
Nov 21, 2009, 3:26am Top

Great feature.

1. Have you parsed only English Wikipedia?

2. Is it a one-time parsing, or some kind of automatism will keep it up-to-date?

Thx,
Sala

5generalising
Edited: Nov 21, 2009, 7:16am Top

Awesome!

It would be interesting to know if you could use a similar parser to generate a link to the article *about* the work, rather than the ones referencing it - it's good to be able to go from an author page to the Wikipedia article, and a link to the article on the book would be equally interesting.

It'd be a lot more work, though; you'd need to find some way of identifying an ISBN or OCLC (etc) code in the article in such a way that it was unambiguously not a reference to a different work. I'll have a think about this.

Oh, and one small bug report: it garbles links with a slash in them, so the link to Wikipedia:Reliable sources/rewrite on the Wealth of Nations page is broken, at least for me in Firefox 3.0.

6Edward
Nov 21, 2009, 7:04am Top

I love this feature. Thank you for bringing it back!

With regard to where it should go, I think of it as something like Recommendations: "If you like this, here's some more stuff you can look at." But it's probably not an important enough feature to go above Reviews or Common Knowledge by default.

7TheoClarke
Nov 21, 2009, 7:58am Top

Surely the most cited work on Wikipedia is the 1911 edition of Encyclopaedia Britannica? Or at least, it should be, given that it was imported wholesale.

This is a great feature. Thank you.

8legallypuzzled
Nov 21, 2009, 8:01am Top

1) I'm going to guess the Bible.

2) I agree with Edward; the link should probably go below "Editions" (actually, I could see a separate grouping of "Common Knowledge" and "Your Books on Wikipedia" as user-contributed knowledge).

9prosfilaes
Nov 21, 2009, 8:28am Top

The number one work is not the Bible, because it's not one work in LibraryThing, and most citations in Wikipedia aren't of the same form as book citations.

As for non-English Wikipedias, there's about 30 large ones, which could get pretty overwhelming. On the other hand, a lot of non-English books would get links if the Wikis in their languages were searched.

10bugaderes39
Nov 21, 2009, 8:45am Top

What about Wikipedia in other languages?

11jjwilson61
Nov 21, 2009, 9:40am Top

It looks like most of those Ship of the Line pages would be removed as not notable enough if someone were to report them. If that were done then LT would have dead links, so I hope the harvesting would be done on a periodic basis.

12lquilter
Nov 21, 2009, 10:57am Top

Well, I love this; and I love this kind of feature -- more intertextual links! LT could *so* become an awesome research resource of its own.

13jbd1
Nov 21, 2009, 11:00am Top

#12: Whaddya mean could? :-)

I second the guess in #7 for the most cited work, too.

14timspalding
Nov 21, 2009, 1:59pm Top

>13 jbd1:

Nope. Another reference work, but one that's frequently updated.

Well, I love this; and I love this kind of feature -- more intertextual links! LT could *so* become an awesome research resource of its own.

Keep going. Someone's going to hit on the idea I've wanted to add for years.

15caffron
Nov 21, 2009, 3:34pm Top

>14 timspalding:

Well, we can't add works cited by another work until we've added works within a work. :) (Tonight, Brain, we're going to take over the world...)

16235711
Nov 21, 2009, 3:41pm Top

1) Merriam-Webster?

17lquilter
Nov 21, 2009, 3:50pm Top

#1 > Q1: I was going to guess something like Joseph Campbell, The Hero with a Thousand Faces -- but that only shows 5 hits in Wikipedia. So clearly that's not it but I'm going to go ahead and mention it because surely the search algorithm is missing a lot of cites? Is it for instance picking up things listed in "reference" tags?

#14 > As for something you might have wanted to add for years -- the thing that leaps to my mind as the most useful in terms of research would be interlinking books with references. Book A references Book B on page 251; Book B references Book C on page 11 and 49; etc. Tied into some kind of scholarly literature database (or maybe several) and it would be *incredibly* useful as a research tool. Tie into a personal fields -- "where did you find out about this book" (Book A, recommendation by UserName, LT discussion group Y thread Z, etc.) -- and it would be incredibly useful as a personal cataloging tool, and probably as some kind of meta-research tool, too. .... So what is it? Don't leave us hanging!

18nperrin
Edited: Nov 21, 2009, 3:52pm Top

Keep going. Someone's going to hit on the idea I've wanted to add for years.

Hmm, is it the one I've wanted for years--links to books about the book? Especially, links to criticism of the book?

ETA: What lquilter said too.

19235711
Nov 21, 2009, 4:22pm Top

No. Elephant in the room. The OED.

20_Zoe_
Nov 21, 2009, 4:39pm Top

As for something you might have wanted to add for years -- the thing that leaps to my mind as the most useful in terms of research would be interlinking books with references. Book A references Book B on page 251; Book B references Book C on page 11 and 49; etc. Tied into some kind of scholarly literature database (or maybe several) and it would be *incredibly* useful as a research tool. Tie into a personal fields -- "where did you find out about this book" (Book A, recommendation by UserName, LT discussion group Y thread Z, etc.) -- and it would be incredibly useful as a personal cataloging tool, and probably as some kind of meta-research tool, too.

Yes.

I hope we can start with a basic version of this: Book A references Book B. Pages mean proper editions and probably another year of development before we could see anything.

I'm actually more excited about the thought of the personal fields than the scholarly-reference fields, though. It has the potential to make the site so much more interconnected and fun to browse.

21235711
Nov 21, 2009, 4:59pm Top

17, 20: Amazon used to do that, but they don't seem to anymore. It would be interesting to know the reason.

(Reading post #20 brought back to me how I first found a certain book via another book through Amazon's referencing system. I must have been in the dark about it for years.)

22jlelliott
Nov 21, 2009, 5:55pm Top

Very nice, I've always thought that a work level section for external links would be a great addition. Is there any chance that it would ever be possible for users to add more relevant links (like on the author pages)?

23lquilter
Nov 21, 2009, 9:23pm Top

Most popular cited reference:

It doesn't seem to be things like Will & Ariel Durant, or Edith Hamilton, or OED (not that I can easily tell which OED "work" would be the best to check), or Encyclopaedia Britannica, or Aristotle, etc., etc. Based on the example that Tim gave us that had so very many cites, it was an encyclopedia of ships that would appeal to some history buff. So I'm going to go with some encyclopedic reference about a super-popular topic. Playboy models? Pokémons?

24justjim
Nov 21, 2009, 9:37pm Top

25VisibleGhost
Nov 21, 2009, 9:54pm Top

BibliographyThing? A compendium of bibliographies?

26235711
Nov 21, 2009, 9:56pm Top

23: (not that I can easily tell which OED "work" would be the best to check)

Me neither. It may have a shot if you ignore LT works. The "what links here" page on Wikipedia itself is huge, but many of the references are talk pages, so it's difficult to get a good idea of how many actual wiki articles there are. (And my earlier guess was an answer to the wrong question.)

24: No, that one only has 11. The number is on the work page.

27timspalding
Nov 22, 2009, 12:19am Top

I think I'll eat a hotdog now. Maybe two.

28sqdancer
Nov 22, 2009, 12:34am Top

29jjwilson61
Edited: Nov 22, 2009, 12:54am Top

Not even close. Ship of the Line, which Tim said is #6 has 764 and The Baseball Encyclopedia only has 81. Just look at the work page, it will tell you the number.

30MMcM
Edited: Nov 22, 2009, 1:33am Top

There are around 2K references to the Guinness Book of Records in Wikipedia, but it's a mess on the LT end because of multiple works (as was pointed out for other titles above).

31timspalding
Nov 22, 2009, 1:55am Top

Ding ding ding!

32justjim
Nov 22, 2009, 4:27am Top

Duly slaps head as predicted!

33235711
Nov 22, 2009, 7:48am Top

*groans*

But how appropriate that the Guinness Book of Records should be the one holding the record. Does it say so anywhere in the book itself?

34TheoClarke
Nov 22, 2009, 11:23am Top

Doh!

35timspalding
Nov 23, 2009, 1:21am Top

"There are around 2K references to the Guinness Book of Records in Wikipedia, but it's a mess on the LT end because of multiple works (as was pointed out for other titles above)."

How is this known? I'm starting to worry about my data not finding everything.

36timspalding
Nov 23, 2009, 1:27am Top

I blogged it, but then moved the post behind SantaThing, because I'm questioning whether the process worked correctly.

See http://www.librarything.com/blog/2009/11/books-of-wikipedia.php

37lquilter
Edited: Nov 23, 2009, 9:25am Top

You know what it's not finding, Tim? It's not picking up things like bibliographies on the author's pages. Probably because the author's name is not side-by-side with the titles. See, e.g., Octavia E. Butler and Mind of My Mind and Wild Seed. In neither page is the author's page picked up.

This is probably (certainly?) not all the missing mass but it's one source, anyway.

38aethercowboy
Nov 23, 2009, 9:33am Top

I'm sure it's been said before, Tim, but you == theMan always returns true.

To answer 2: Links to Wikisource! or Wikibooks.

39PeterCapek
Nov 24, 2009, 11:16pm Top

This seems like an interesting exercise, but I wonder if it doesn't say more about the interests of Wikipedia contributors than about anything else.

40kantelier
Dec 14, 2009, 1:43pm Top

41skgoetz
Apr 19, 2011, 3:09pm Top

Is this feature meant to pick up books that have their own Wikipedia page, or have I misunderstood? (If it is thus meant, I've just noticed that http://en.wikipedia.org/wiki/Autobiography_of_Mark_Twain is not picked up by http://www.librarything.com/work/109448/commonknowledge/72414772, but I don't see a way to add it.)

42jjwilson61
Apr 19, 2011, 3:41pm Top

At one point a few years ago Tim ran a process that found Wikipedia pages that corresponded to LT works and added links for them. As far as I know that process hasn't been run since and there has never been a way to manually add links to Wikipedia so any Wikipedia pages added in the last few years won't be linked to.

43lampbane
Apr 19, 2011, 6:01pm Top

>>41 skgoetz:
>>42 jjwilson61:

It picks up books that have been cited on Wikipedia by ISBN or OCLC. To add the book here, you have to make sure the book is cited on Wikipedia with one of those two numbers. I've seen more recent citations appear here on LT, so the process does get run more occasionally than this thread would indicate.

44skgoetz
Apr 19, 2011, 7:33pm Top

@43 Thanks! Good to know.

45timspalding
Apr 19, 2011, 7:41pm Top

Yes, I was thinking I'd run it again. Not top priority, but potentially fun.

46brightcopy
Edited: Apr 20, 2011, 12:32am Top

45> Is it something that requires a lot of care and feeding? If not, would be cool if there was a "recalculate wikipedia reference" for books that don't have one. I'm assuming there must be some reason why a feature allowing users to add/edit wikipedia references isn't desirable.

47fugitive
Apr 20, 2011, 8:54am Top

Tim, This sounds like something you might want to give the community an advance "heads up" so that those of us who are wikipedians have a little time to check if our favorite books have the necessary ISBN and/or OCLC numbers in the wikipedia article and update them so they would be caught when you run your bot/script.

48timspalding
Apr 20, 2011, 9:20am Top

45> Is it something that requires a lot of care and feeding? If not, would be cool if there was a "recalculate wikipedia reference" for books that don't have one. I'm assuming there must be some reason why a feature allowing users to add/edit wikipedia references isn't desirable.

No, it's just a matter of getting and downloading the latest Wikipedia dump. They're ENORMOUS so I always have to check where I'm going to put it.

49Cynfelyn
Edited: Mar 14, 2012, 6:28pm Top

As a newby still finding my way around LT, and looking at the answers to message 41 above, am I right to think there is no way to edit links to Wikipedia? What I have in mind is 'Nancy Blackett : under sail with Arthur Ransome', http://www.librarything.com/work/1661183/book/83403290, where both Nancy Blackett links go to Nancy Blackett the Amazon pirate, rather than one of them heading off to Nancy Blackett the yacht, http://en.wikipedia.org/wiki/Nancy_Blackett_%28cutter%29. Thanks.

50gangleri
Mar 22, 2012, 7:41am Top

Hi Tim! Is it possible to run your scripts based on some LT chractersistics:
a) LT authors with links to http://nobelprize.org/
b) LT authors with links to http://www.nndb.com/people/
c) on some results for LC CK search
I looked at Muller, Herta Nobel Prize (Literature ∙ 2009)

51geyser
Jan 12, 2018, 1:47am Top

This may be off topic, but I found an author bio on wikipedia and I would like to add it to the reference section for that author.

http://www.librarything.com/author/burtonkatherine-1

https://en.wikipedia.org/wiki/Katherine_Burton

How could this be done? Thank you.

52Lyndatrue
Jan 12, 2018, 2:34am Top

>51 geyser: I did this for you. Here are the steps.

Up in the corner of the author page (viewer's right) is a small area called "Links" and that's the correct place to put things like the "Wikipedia author page" (which is what it is). There are multiple types of links, of course, and the sources are richer for some genres (such as Science Fiction) and for better known authors.

I'm a bit surprised that you weren't aware of this, considering how long you've been on LT, but there's so many nooks and crannies here that I'm always being surprised myself.

53geyser
Jan 15, 2018, 12:17am Top

Thank you!

Group: New features

45,201 messages

This group does not accept members.

About

This topic is not marked as primarily about any work, author or other topic.

About | Contact | Privacy/Terms | Help/FAQs | Blog | Store | APIs | TinyCat | Legacy Libraries | Early Reviewers | Common Knowledge | 134,852,569 books! | Top bar: Always visible