LT's Top Books of 2010 by Member Count and Avg. Rating (obsessive hand-sorted edition)

Somewhat dissatisfied with the partial list of top books by rating I compiled the other day, I decided to try again using the whole CK Original publication date 2010 list. Here's the result in an Excel file at Google Documents:

LT Top Books of 2010

The list is currently sorted by my Combo factor, an attempt to balance number of members and average rating. The books that are sequels in series are segregated at the bottom. You may download the file and sort however you like.

I was rather pleased by the variety in the final list. An automated version would make a great feature addition (I don't think I'll be doing this again).

Enjoy!

Notes on method-
The base data was from a CK search for 2010 in the Original publication date field. Books may be missing from the list because they have no CK entry or because I skipped a book in error. There are also potential false positives due to bad data, deleted data (that still shows in the search) or people entering dates for later editions. I tried to eliminate the latter two, but may have missed some. I would also be shocked if I didn't make any transcription errors.

In my initial selection I kept books with either 260 or more members or an average rating of 4.00 or higher with 20 raters. I later narrowed the list to 288 and 4.07. This still allows for the top 100 books in either category with series sequels excluded. The numbers in red indicate that the book was below cut on that factor and qualified on the other.

When there was info in the series CK and the value in parentheses was something other than 1 (or "book 1" or the like) I marked the book with an "x" in the ss column. Zoe pointed out in the other thread (linked at the top) that sequels often have inflated ratings and I think the list is more interesting without them. But they can still be found at the bottom.

most interesting! thanks

Interesting data. Thx. Of the top five:
1.Room
2.The Immortal Life of Henrietta Lacks
3.The Big Short
4.Matterhorn
5.The Thousand Autumns of Jacob de Zoet,
I have read 2,4,and 5. I have no quibbles with them being there.

"(I don't think I'll be doing this again)."

Damn, now I'm jonesing for a 2009 list.

This is fantastic; thank you so much!

Damn, now I'm jonesing for a 2009 list.

If there's enough interest Tim might be persuaded to provide the data; even the ability to sort CK results by rating would be a huge help.

It will be interesting to revisit this list in six months or a year and see which of the books with a low member count end up retaining their high rating. I'm not quite sold on the combo factor, at least not in its current form; I'd tend just to choose a cut-off number of ratings/owners, or possibly a cut-off zone within which a combo factor was applied (e.g., books with 100-200 raters are significantly penalized; books with 200-300 raters are slightly penalized). It will be fun to play :)

Thank you again!

I don't know how to do filtering in Excel, so I'm putting this list here for my own benefit at least: a straight-up rating sort, excluding series and books with fewer than 200 members.

1. The Warmth of Other Suns
2. Unbroken
3. The Way of Kings
4. The Sky is Everywhere
5. The Lost Books of the Odyssey: A Novel
6. Nothing to Envy
7. Cleopatra: A Life
8. Room
9. Matterhorn
10. Nightshade
11. Revolution
12. Out of My Mind
13. The Immortal Life of Henrietta Lacks
14. The Big Short
15. Matched
16. Paranormalcy
17. Will Grayson, Will Grayson
18. The Invisible Bridge
19. Dear Deer
20. Clockwork Angel
21. Christianity: The First Three Thousand Years
22. Last Call: The Rise and Fall of Prohibition
23. Under Heaven
24. Packing for Mars
25. Feed
26. The Kitchen House: A Novel
27. WAR
28. The Lotus Eaters
29. The Thousand Autumns of Jacob de Zoet
30. Before I Fall
31. Just Kids

***Big ratings jump here***

32. Girl in Translation
33. The Distant Hours
34. White Cat
35. Major Pettigrew's Last Stand
36. Saving CeeCee Honeycutt
37. The Iron King
38. First Drop of Crimson
39. Full Dark, No Stars
40. Ship Breaker
41. The Red Pyramid
42. At Home
43. Heist Society
44. Sh*t My Dad Says
45. Every Last One
46. Hitch-22
47. Hex Hall
48. Freedom
49. Fall of Giants
50. The Hundred Thousand Kingdoms

It seems that Dear Deer was actually published in 2007, and Christianity: The First Three Thousand Years was released in 2009 in the UK. I wonder how accurate CK data is in general?

The list doesn't speak too well for the intellectual stamina of LT members/readers, I'd say.

> 7: Being read in 2010 has nothing to do with being published in 2010. Deal!

> 8: I any survey like this the most commonly popular books will turn up first. It doesn't really reflect on the intellectual level here, it's just the way these things are statistically stacked/interpreted.

Being read in 2010 has nothing to do with being published in 2010. Deal!

Post #1 specified that the dates are coming from the Original Publication date CK field. What does that have to do when books were read?

# 7 said : It seems that Dear Deer was actually published in 2007, and Christianity: The First Three Thousand Years was released in 2009 in the UK. I wonder how accurate CK data is in general?

But maybe I misinterpreted (I mostly get by in English, but I'm not a native speaker).

#11: Each of those two books had 2010 in the Original Publication date CK field until recently corrected.

Right, both were showing 2010 publication dates in CK, which is where the data for this list was taken from.

>8 helmutbooks: I don't know, there are plenty of books about serious ideas. I haven't read nearly all of the books on the list, but The Immortal Life of Henrietta Lacks, for example, raises thought-provoking issues. It just happens to do so in an accessible way rather than obscuring the points behind fancy "literary" writing, which I personally think is a plus.

#8 (Assuming you're referring to Zoe's list) Or is it just that it takes a certain kind of book to have over 200 readers in the same year it was published where the majority like it. I think I only read one book that was published in 2010 this entire year and that garnered rather diverse opinions. I suspect something similar could be said for many, many other LTers. So aggregating these statistics is likely saying more about the books than about us.

>14 aulsmith: Just to be clear, it's not necessarily 200 readers; it can include wishlist items and books purchased but not yet read. Not sure whether that would impact your argument, but I figure it's worth mentioning regardless.

15: Sorry. I thought you meant it had been rated by 200 people (which I assumed came close to readers). I see now that both lists are 200+ members with the book in their library.

Then I'm not sure this says anything about us except, like the rest of the book buying public, we buy (or think about buying) a lot of popular books the year they come out and that the people who actually read them like them.

It would be interesting to find out of the people who bought the book this year, how many still have it on their TBR pile 5 years later. I'm sitting here looking at the Watergate transcripts, which I'm thinking of discarding, which I never did read ...

Very very nice, my best compliments. I work on Market Intelligence area and this is the most interesting usage of common data I saw in the year. Just two technical questions: how did you build the Combo factor? And how did you extract data from LT? I'm curious....
I don't know so much about the top ten: I'm in Italy so the authors here are slightly different. I know only Jonathan Franzen, but not the book.

Very very nice anyway, ciao

16> I think it's a pretty safe assumption that the vast majority of people aren't going to rate a book they haven't read yet. Yes, there will always be exceptions. So the point of this is that it's a list of rated books. Yes, it would probably be better if it was filtered by # of members rating, rather than sheer # of members. But I think it's still interesting as it is.

18: I was merely arguing with post #8 that this says anything about our intellectual stamina. I am in awe of this piece of data collection.

19> Right, but I think your argument still stands. #8 was just some drive-by snobbery. They only way they might be happy is if somehow popular books aren't popular. They're turning their nose up at a tautology.

I think there is some distinction between popular and high-rated. Look at the search results for James Patterson; they tend to be in the 3.5 range with hundreds of ratings.

20> "Drive-by snobbery". Great phrase!

21> Right, but the main point being that in any list of the top rated works, there's bound to be a lot of "popular" ones. I would be rather suspicious of the data if there weren't.

If you want to have some fun and actually do some statistics then rather than inventing some new statistic like a combo factor why not just run a simple 1 way ANOVA? (Who knows, if you are in school and need a math project this would be a simple and very interesting project)

It looks like you have all of the necessary information - number of star counts per book and number of individuals rating the book - the rest is just a matter of data entry and running an ANOVA and doing pairwise testing ala Tukey-Kramer.

Among the many things you could generate would be an identification of books with statistically significant different average ratings as opposed to merely numeric differences. It would also reign in the need for empty precision.

If you are interested the details can be found in A Cartoon Guide to Statistics by Gonick and Smith.

Touchstones: The Cartoon Guide to Statistics by Larry Gonick and Woollcott Smith

Sorry for the late response. The thread didn't get much notice for a while and it dropped off my radar screen.

>17 sbaldi59:
I didn't extract the data, I transcribed it. Yes, by hand.
The combo factor is just something I made up. Looking at the list I found both the plain avg. ratings rankings and the plain # of members ranking to be problematic. Niche books with relatively few catalogers often do the best in ratings. I suspect if a wider audience of people read them, the ratings would go down. But just going by cataloging popularity would leave some books with very poor ratings high on the list. So I wanted to combine the two.

First, I changed both the avg. ratings and the # of members from raw data to standard deviation from the mean. I found that just adding those stats left too much weight on # of members at the top end because the member numbers have a much greater spread. So I took the cube root of the normalized # of members and added that to the normalized avg. ratings. I liked the result, so I kept it. I admit it has no real meaning.

>24 alco261:
Learning a bit of real statistics is not a bad idea, though not at the top of my list of priorities. I've seen ANOVA mentioned in some more statistically oriented sociological papers, but I never bothered to find out what it meant.

Unfortunately, I don't actually have the data you're asking for. I only have what you see, # of members and average rating. I didn't note the number of raters except to ensure that books qualifying for the list on ratings had at least 20 raters. So to do what you suggest would require going back to the work records, not something I'm hot to do.

On the books actually published prior to 2010: I'm not surprised. I corrected some bad CK entries myself when I noticed tags indicating prior publication (e.g. read in 2009). Statistics can only be as good as the source data.

But still, I think the results are interesting and it would make far more sense as an automated process, if Tim could be convinced of the value. An automated list would be corrected and expanded as CK data is fixed and added. I also think it would be interesting to see how the list changes over time as more people get around to reading the books. What's hot in the first year may not be what is most appreciated in the long run.

I think there's hope for getting this as an automated process, since Jeremy considered it interesting enough to include in State of the Thing.

>26 eromsted: The data is available for each of the books and can be found in the rating histograms for each book. If we take what are supposed to be the top five books as listed in >3 VisibleGhost: and call them book 1-5 the data and the analysis is as follows:

Test data

Obs Book Rating Count

1 1 0.0 0
2 1 0.5 0
3 1 1.0 3
4 1 1.5 1
5 1 2.0 3
6 1 2.5 4
7 1 3.0 38
8 1 3.5 31
9 1 4.0 141
10 1 4.5 72
11 1 5.0 224
12 2 0.0 0
13 2 0.5 0
14 2 1.0 2
15 2 1.5 1
16 2 2.0 3
17 2 2.5 4
18 2 3.0 35
19 2 3.5 25
20 2 4.0 134
21 2 4.5 67
22 2 5.0 169
23 3 0.0 0
24 3 0.5 0
25 3 1.0 1
26 3 1.5 0
27 3 2.0 4
28 3 2.5 0
29 3 3.0 18
30 3 3.5 15
31 3 4.0 83
32 3 4.5 27
33 3 5.0 86
34 4 0.0 0
35 4 0.5 0
36 4 1.0 1
37 4 1.5 0
38 4 2.0 5
39 4 2.5 1
40 4 3.0 9
41 4 3.5 5
42 4 4.0 36
43 4 4.5 16
44 4 5.0 73
45 5 0.0 0
46 5 0.5 0
47 5 1.0 1
48 5 1.5 1
49 5 2.0 5
50 5 2.5 4
51 5 3.0 27
52 5 3.5 21
53 5 4.0 100
54 5 4.5 53
55 5 5 108

Most good computer packages (Minitab, Statistica, SAS, JMP, etc) have an ANOVA package that will allow the use of a table such as the one above which allows a count variable so that you don't have to enter every single score level separately. The ANOVA table below does have empty precision in that the rating scales are only to the tenths place, however, to keep everything looking like the printed averages on LT I went ahead and printed out the 100th place even though it has no meaning.

ANOVA

------Rating--------
Book N Mean Std Dev

1 517 4.35 0.73
2 440 4.30 0.72
3 234 4.26 0.72
4 146 4.37 0.82
5 320 4.23 0.75

The overall p-value is .14 indicating there are no significant differences between book mean ratings (we are assuming a critical p-value of

>28 alco261: I think eromsted knew where the data could be found, but just wasn't willing to visit every work page again after the massive effort that he'd already put in. I'm still in awe of the determination required to extract all this data manually in the first place.

My recollection of ANOVA from my stat class in college some 30 years ago is that it stands for Analysis of Variance and it tells you which factors account for how much of the variance in one variable. For example given a data set of income, age, religion, ethnicity, and gender it could tell you how much income is related to each of the other variables.

Given that description of ANOVA I'm not sure why it would be useful for determining the top books of 2010.

>30 jjwilson61: ANOVA does stand for Analysis of Variance but the variance is the variance of one population relative to another - not the within population variance. What ANOVA does is provide an answer to the following: Given that I have a group of sample means where the samples have been drawn from two or more populations is there a significant difference between these sample population means?

If the analysis is a simple one way ANOVA, as it would be for comparison of average book ratings, then the populations would correspond to the different books. If the p-value for the one way ANOVA is significant then it means that at least one of the book average ratings is significantly different from the rest.

The test I ran on the data above indicates there's no significant difference between any of the 5 average ratings - in other words, with the data provided, there is no indication that 4.23 is different from 4.35.

For the example of income as it might be related to age, religion, ethnicity, and gender the preferred method of analysis would be regression. You could run an n-way ANOVA on the data but it wouldn't tell you how changing one or more of the variables might correlate with changes in income (magnitude and direction). The other problem with trying to run ANOVA on the above variables is that age is a continuous variable and the only practical way to treat it as a class variable would be to bin people in to say 10 year categories and run the ANOVA on the categories.

>29 _Zoe_: After re-reading the post I agree - it is most likely that >26 eromsted: doesn't want to do it again.

Since the data is available and since I must admit I am a bit curious about what one might actually be able to say about the average ratings I'll go back to the list provided by >26 eromsted: and try a couple of things.

I can take the data from the numeric top 10, re-run the ANOVA, and if I still don't see anything significant and if the variances appear to be well behaved I'll compute a least significant difference which will give some indication of how low an average rating would have to be before it would be statistically significantly different from the top 10 (p

I'm sorry for this multiple posting. LT keeps chopping off the bottom of the post.

I took the rating counts for the top 10 books cited in the first post to this thread, picked the 100th book for no particular reason and then generated random numbers from 11 to 99 and took the first 4 - books (48, 15, 36, and 81) so I would have a sample of 15 - again for no particular reason.

I ran an ANOVA on the book ratings for these books. The p-value was significant so I ran pairwise comparisons on the 15 books. Out of 105 comparisons 13 had significant differences in their average scores. These significant differences ranged from .24 to .43 with a median significant difference of .31 and mean significant difference of .32.

Granted the sample is small but the the statistics and the differences appeared to be well behaved so, as a first estimate, differences between average book ratings of .3 or greater are probably statistically significant - what a .3 rating difference would mean in terms of significantly different reading experiences remains to be seen.

Thanks for the info :)

My Reading Life should now be on the list; it didn't have enough members before.

I meant to thank mersenne6 before for the statistical work. I'm actually a bit surprised that there is statistical significance at even .3 (Though wouldn't it depend on the number of raters? Books with more raters should be more differentiable than those with fewer, no?).

But really I hope no one is taking all this too seriously. In terms of formal comparisons there are all sorts of problems before we even get to p-values. First, the things being compared are not really of the same type. Second, the pool of raters is not the same in each case. Third, there is no common standard for ratings.

In the absence of greater standardization on those three factors I'm not sure ANOVA analysis has much meaning.

But these are books that some group of LT users thought would be worth reading (the members number stat) and actually liked (the rating stat). And though the strict ordering doesn't have much meaning, ranked lists are fun and the keeping them adds something to the LT community, at least in my opinion.

But seriously, it should be automated, because what I did was crazy.

I agree with #39 and don't think ANOVA is the correct test for this data. I am not sure finding there is a statistical difference between book ratings buys you anything because of all the bias mentioned above. If the question is how to separate the high quality books from those of lesser quality then it will take a bunch of data massaging (plus possibly specious assumptions and defining what exactly is high quality) to pull out anything useful.

I am not very good at automating, but probably the best approach would be to find some books or categories you consider high quality and use the referred books (and their referred books) to build a diagram of relevant and good books. That might have some value though it will have some of the circular problems of your average dictionary but more obvious. There are still the problems of different raters per book and all the other problems of voluntary surveys, but if handled in a rigorous fashion even with the potential confounding you could likely get usable information that would make LT more marketable.

If you want to send me any data I may be able to help though I doubt I could do better than what has already been done which is somewhat useful as it is.

ANOVA is a test of differences of k means where k can be 2 or greater. Any basic book on statistics will tell you this. Many of them will also tell you that the data must be normal in order to use ANOVA but in that regard they are in error. (The Design and Analysis of Industrial Experiments 2nd edition by Davies pp. 51-56 has a great discussion of this issue).

Now, if you want to hold the rating scale to what it is - that is a Likert scale - and if you want to report it as such and if you want to go with those who absolutely refuse to treat Likert scale data as anything but pure ordinal - then you would run the non-parametric equivalent of ANOVA, the Kruskal-Wallace, and use Dunn's test for pairwise comparisons.

Chances are, what you will find is pretty much what you would find using ANOVA. (Actually, if you have a block of data of this type it is very instructive to run the analysis both ways and see what you see). If it was a really serious issue then to be conservative you would want to error on the side of caution with ANOVA and report significance only at an alpha

So here we go again with Library Thing message truncation.

...only at an alpha less than .001 instead of less than .05. Of course, if you are going to take the militant approach to Likert data then you would also want to demand that Library Thing drop all references to average book scores and report only modes and medians.

As for the issue concerning the physical meaning of a statistically significant difference - I agree. My personal guess would be that in order for a statistical difference to have any physical meaning the p value would have to be much less than .001 and the plots of the rating scores would probably look bimodal.

Just so everyone understands I'm not trying to turn this discussion into one of those nightmare Pro and Con exchanges. The only reason for replying was just to have some fun and perhaps provide some understanding of statistical analysis.

Thanks for the stats.

The truncation is caused by HTML coding. the LessThan sign (and GreaterThan) are codes for LT to 'know' that the following bit is an HTML command - formatting or a link etc - and not to be displayed. It is annoying. Someone will probably be along in a bit with the unicode you need to use to get the symbols to display.

Someone will probably be along in a bit with the unicode you need to use to get the symbols to display.

Use &lt; for < and &gt; for > (less than, greater than)

:D

I was thinking about how to adjust for number of members/ratings, and I think I might just lean toward having a few separate lists. For example, just inventing some numbers:

Top-rated books with 20-49 ratings
Top-rated books with 50-199 ratings
Top-rated books with 200-499 ratings
Top-rated books with 500+ ratings

It's clear that the more popular a book is, the more people tend to stray out of their comfort zone to read it, resulting in lower ratings. But I still prefer to look at the average rating number, which is easily understood, and to see lists ranked that way rather than by some mysterious other factor. So, here's one possible different way to deal with the issue. You know, if someone wants to do this again this year, or if the LT staff ever decide to automate the process ;)