Confusion with Recommendations

Hello,

I am very new to LibraryThing, and I want very much to like it. However, I'm primarily interested in getting recommendations, and I have issues with the way I assume the site works. I've rated quite a few books in order to get into the proper recommendations page, but I'm finding quite frequently that books that I rated poorly (sometimes less than 2 stars) are being used to generate recommendations for similar books. My question is how should I best groom the algorithm to give me books that I'm actually going to enjoy? Should I just not rate books that I did not like? That seems to be counter to the point of the website.

Ratings are not part of the algorithm for recommendations.

>1 VelvetGent: You can select which collections are considered for recommendations: https://www.librarything.com/settings/collections . So you could create a collection “Good Books,” for example, and include only books that you’ve rated higher than your desired threshold and then include only the “Good Books” collection for recommendations.

Recs get better with more books entered. 100 is the minimum to get those based on your library IIRC, so presumably they'll improve as you add more books you like. In the meantime, you can check other sources of recommendations. Work page for a book you enjoyed has many you can see by clicking the Recommendations in the left sidebar, some by members, most algorithmic. Also, if you go to Home > Recommendations, there's several other options to get some based on your entire library.

>3 Charon07: I think that works only for the 'old' recs. The new algorithm is more about finding similar libraries to yours and picking from their books.

>1 VelvetGent: Hello, welcome!

Recommendations are based on the books in your library collections. @Charon07 suggestion above is a great idea to work the algorithm the way you would like it I think. I might try that one myself :)

>5 SandraArdnas: From the thread begun in 2022 about the new recommendation system, particularly https://www.librarything.com/topic/346584#8005348 and https://www.librarything.com/topic/346584#8067194, it was my understanding that the new system considers only the collections you’ve marked to include in recommendations. Is there newer info somewhere that contradicts this?

>7 Charon07: I don't know about newer info, but in practice, yes: afaict i don't have any manga in any collections marked as "use for recommendations", but I'm constantly getting manga recommendations that are unlikely based on the collections I do have marked for recs.

It's kind of frustrating. I don't want to remove these books because they're a record of what i read and enjoyed 15 years ago, but my tastes have since changed.

I'm sure the default position for all collections is 'use for recommendations' and you have to actively turn it off for each one.

I find having just one 'Use for recommendations' is the best way to go, because it only takes one very popular book to seriously distort what you get offered.

>1 VelvetGent: Tim's thought (based on netflix) is that even though you didn't like something that particular book, you thought you would and hence it's worth basing recommendations on it to find other similar books that you might enjoy more. I disagree but I don't write the code! It probably depends on how nuanced your dislike is. The recommendations aren't very nuanced at all.

>7 Charon07: Not that I know of, I was basing my assumption on 'similar libraries' being one of the main elements of the algorithm. I guess it is possible it has its own way of determining similar libraries while disregarding the collections marked not to be used for recs since the powers that be said so.

I have a dedicated "for recommendations" collection, since I also catalog picture books I read to my niece but I don't want recommendations for those. I don't think it works for the new recommendations system but I still largely use the "classic" recommendations which is based on books, not similar accounts, and it works great for those.

>8 keristars: I see that most of your books tagged manga are in your Books I Own collection. Does that collection actually have its Include in Recommendations setting turned off? I too thought that even in the new system that in deciding on what libraries are similar to yours that it it only considers the books in your "Include in Recommendations" collections to be part of your library.

>12 jjwilson61: correct! the only collections which have "use for recommendations" marked are the Girls Series and Use For Recommendations one.

However, Books I Own and Read But Unowned are marked to use for Connections, which may be what this recommendation system is using instead, if it's not using the entire catalogue.

A little update: I finally remembered to move my manga into my "no connections" collection as a test, and the latest round of automatic recommendations is the first one I can recall without any manga in it.

I'm getting some other recs that are no good, I think because I read a few Squirrel Girl comics, so I'm going to move those into the No Connections group, too, and see if that fixes it for the next round.

Good news: another round of recommendations and for the first time ever, I got zero comics!

So that experiment has proven a success: to prevent works from being used for recommendations, be sure they are not in any collections that are marked "use for connections".

(which is a pity - I'm fine with connecting to other members based on my catalogue, but I find that many of the books contaminate recommendations with too many things I don't care for, like someone who enjoyed the Tiffany Aching books but has no interest in the rest of Discworld or readalikes for the City Guard books)

There are two checkmarks on a collection, one for use for connections and another for use for recommendations. You should be able to uncheck the latter but leave the former checked.

>16 jjwilson61: Read the whole topic. It seems that the new recommendations is largely based on connections.

>16 jjwilson61: That's what I was testing. For the new recommendation system based on comparing whole catalogues, the "use for recommendations" toggle is deprecated. Instead, recommendations are based on the books in collections marked "use for connections".

Since the new recs are based on what similar libraries have that you don't, that makes some sense. "Use for connections" is the equivalent of "use to calculate similar libraries".

I'm not sure this is intended. I think you've proved that this is the way the feature is currently working, but I think this is a bug. I think >16 jjwilson61: is how the feature is intended to work.

@timspalding - see >18 keristars: and >19 GraceCollection:

Is the new recommendations engine supposed to respect the "use for recommendations" setting for collections?

And if not, can the ticky boxes get an update to indicate the recommendations one is for old recs, and the connections one is for connections and new recs?

Most of this confusion could be solved by having the algorithm take into account the rating given to books, which is the way I assumed it worked. My dream is that someone makes a website like Criticker but for books. Look at my ratings, compare it to other users with similar ratings, suggest books to me that they've rated highly and I haven't rated yet. There are way too many books out there right now, and I'm a picky reader. Sometimes it takes me months to find a novel that I enjoy and every website designed to alleviate this problem is more frustrating than helpful.

>21 VelvetGent:

Many of us don't assign ratings at all, and those who do are inconsistent in what the ratings mean, since there is no site-supplied guideline regarding the significance of ratings.

>22 paradoxosalpha:

To emphasize by example what paradoxosalpha says: I make a significant effort to try to (1) keep the arithmetic mean of my rankings close to 3 (last I looked, it was 2.9 something) and (2) to make sure I have a normal distribution of the numbers of books rated 1, 2, 3, 4, and 5.

How many others would even think of such a thing? And could they maintain it? :-p

Most people would probably have an arithmetic mean higher than 3.0, because they would only acquire and rate books that they expect to like.

So how are you going to compare those? By normalizing the rankings? One could do that, but would anyone understand it? Probably not.

I think the best answer is genuinely to have a collection that is, in effect, a "Make Recommendations Based On These Books" collection. I just wish I'd known about the need for that when I joined LibraryThing. It's a lot harder to set that up retroactively.

I think you could easily set up such a collection by using power edit and including any books you had given at least 3,5* to.

But ratings are not likely to ever be included in recommendations. Tim does not think they are useful.

>23 waltzmn: This is grossly overthinking the issue. The bottom line is that some people find the ratings useful despite any and all issues anyone might come up with. So, one more algorithm that compares libraries based on similarly rated books is all it takes to give them one more avenue to new recs. It need not be one more element in the latest iteration of recs. It can be a separate system. We have several already.

Also, I would get a headache trying to rate books based on some average and I don't see why I even should strive for it. Rating is meant to reflect my opinion of the book's inherent quality, not its ranking among the books I've read, which would really only be useful to me anyway, because no one else has read the same set of books so their arithmetic mean would be quite different we're striving for some reason to spread those rating evenly. Ugh, hahaha. Yes, most people have average ratings higher than 3. It just means with do due diligence well enough before picking them up. Even a 3 is not a book I would heartily recommend to anyone because it is OK, but kind of mediocre, so in all likelihood there's something better on a similar topic whether it's fiction or non-fiction. Anything below 3 has serious issues of some kind and it means I was not successful in picking my reads. So many books, so little time. Nobody wants to read books they'll rate 2 if they can help it.

>24 MarthaJeanne: I think you could easily set up such a collection by using power edit and including any books you had given at least 3,5* to.

An interesting thought, but not as useful as it sounds in my case. I don't think I actually rated the CRC Handbook, e.g., but if I did, I'd probably give it 4.5 stars. But I don't want recommendations based on it. One insanely thorough but extremely boring mathematical handbook is enough. :-) That's why I'd have to create the collection manually.

PowerEdit might help me get started -- I could put all the books tagged "Folk Music" in there, for instance, and then weed some out. But to truly do my history collection... I'd have to re-read a lot of them. And, in many cases, I don't want to read them again, because I read them for some research project or other that's now over. :-)

>25 SandraArdnas: This is grossly overthinking the issue. The bottom line is that some people find the ratings useful despite any and all issues anyone might come up with.

The second statement I'll agree with. I won't concede the first. However, the whole point of my example is that (almost) no one else would ever do what I do. I personally think it important, because without some such control, I am not forced to say, "Well, yes, this is a pretty good book, but it's not as good as that one, so it has to go down a tier."

But that's me. Others will have other systems. And that's fine. Precisely because rankings aren't used for recommendations. Were they so used, then we'd have to try to standardize, and either you get fuzzy thinking or you get some "overthought" system.

When there is no guideline on what ratings mean, it's impressive what a variety of meanings they can have.

Some people will use them to mean "How much did I enjoy this book?"
Others "How important is this book?"
Others "How urgent is it for me to read this book I haven't read?"
Others "The physical condition of this book in my library?"

Some will make five stars the "best," and some will make one star the "best."

Ratings on LT, as extant over decades, are for the individual user only.

This issue has been solved by the Criticker website I mentioned by converting the ratings given by users into their respective percentiles before they're compared. So the individual's personal rating system is irrelevant, all that matters is how the movies compare to each other.

>28 VelvetGent: for what it's worth, Tim's argument is that Netflix also ignores ratings. If you picked that book to read, the fact that you thought it might be interesting is more relevant for recommending other books, than whether or not you 'liked' that specific book.

I personally think he's wrong, I'm quite capable of finding other 'fantasy' books, I want to be recommended 'good' books that correspond to my personal definition of 'good' - much more granular than whether this book is broadly similar to another book. But I accept that I and many other members lost that argument a long time, and Tim isn't likely to change his mind or the base recommendation argument any time, not even "soon".

>29 reading_fox: I can see that. It's a shame though, because I know a little bit about how much work goes into the backend for algorithms like LT's. But at the end of the day, I don't need a website to tell me that if I liked Crime and Punishment I should read Brothers Karamazov. I started this thread confused because I thought I must be doing something wrong. I assumed that with the large amount of ratings required to get to the recommendations, that surely it had to be something more helpful than Goodreads or Storygraph. I'm disappointed. Hopefully someday someone will make a book recommendation site that goes beyond the 2-dimensional.

You do not need a large number of ratings to get recommendations. You need at least 100 entries.

I only pay attention to 4- and 5-star ratings. Anything less is either ambiguous or negative.

>32 krazy4katz:

But that's why ratings can't be a basis for recommendations. See my >23 waltzmn:. In my case, something rated 3 stars is close to the mean, and I have a "normal" distribution of ratings. So, for me, 3.5 stars is pretty good, 4 stars is very good, 5 stars is exceptional and extremely rare.

My system simply does not map onto a system where 3.0 is ambiguous. 3.0 is actually fairly good, because I don't deliberately buy bad books.

I'm not saying either system is better -- yours may in fact be better if you are very good at knowing which books you will enjoy. But unless the rating systems are based on similar premises, you can't judge a book by another person's rating.

When I look at ratings I look not only at the mean for the work, but at whether there are a fair number of ratings and what the distribution is. If a book has just a few and they are all 4.5 or 5, I will be very sceptical. But beyond that, some books will have a lot of threes and fours, others will have ratings all the way from 0.5 to 5.0. Generally reviews and tags will give me a good idea as to which opinion I am most likely to agree with.

>34 MarthaJeanne:

Agreed. I pay much more attention to reviews than to ratings. And, for the same reason, I put substantial effort into my reviews, trying to assess a book as I would were I reviewing it for an academic journal or other high-quality outlet.

>33 waltzmn: As long as a 5 is better than a 4 is better than a 3 then the information from combining ratings can be useful even if someone's 3 is another person's 4.

>36 jjwilson61:

Not mathematically true, unless you always sample the same people. You need to normalize. Oh, you can know that person X likes one book better than another. But you can't compare that to person Y's opinions unless you already know how person X and person Y rate things.

But I would offer a different scenario: The depressing one of having to dispose of books, perhaps because one is moving to a smaller space. One might, of course, have to dispose of certain categories. But one might decide simply to get rid of the "worst" books, or at least the ones you like less.

This is mathematically weedy, so go ahead and skip it, but there is a genuine point to this argument.

Let's take a specific example. Suppose someone is very careful to get "good" books, so that 75% of her books are 4- or 5-star. And she has to get rid of 30% of her books. That means she has to dispose of all her 1-, 2-, and 3-star books, and some of the 4-star. How is she to know which 4-star books to drop? They are indistinguishable.

Thus, for that scenario at least, one needs to use the whole distribution range. Even if one likes every book in one's library, one needs to distinguish the very-best from the not-so-very-best -- even if it means classifying a pretty dang good book as a 2-star.

So this argues for classifying 20% as 1-star, 20% as 2-star, etc. Or 10% in each category if one uses half-stars.

But this is where my use of the normal distribution comes in. Getting rid of 10% of one's books, or 90%, is relatively unlikely. Dumping just 10% isn't going to save enough space to matter much; getting rid of 90% is a fate so much worse than death that one might as well just die. :-) Odds are, when one culls, one will have to cull somewhere between 25% and 75%.

But that calls for a lot more granularity. If you distribute evenly across 1-star to 5-star, and need to get rid of, say 35%, you're somewhere in the middle of the 2-star books. But if you distribute the ratings normally, you have (in effect) added more data about the mid-range books, which are where the hard decisions have to be made.

Is this important enough to enforce a rating system with a normal distribution? Maybe not, if one finds it difficult to do such a thing. For me, it is not particularly difficult, so I do it.

>37 waltzmn: You're example is completely different than a recommendations algorithm where what is important is the relative rankings of the books not the absolute rankings.

>38 jjwilson61:

Of course it's different. But that still doesn't alleviate the need to normalize.

Let's take an example. We'll confine ourselves to two people. Call them, for the sake of pure arbitrariness, Sam and Ron. And both are expressing opinions of The Lord of the Rings and Harry Potter and the Philosopher's Stone. Both think one is good and one is bad. Sam, who balances his ratings, rates HPPS at 2 and LotR at 4. Ron, who doesn't do that, rates HPPS at 5 and LotR at 2.

Now I emphasize: Their ratings are exact inverses. If you add up their actual opinions, they are entirely neutral about the two books. But if you add up their ratings, then HPPS ends up with an average rating of 3.5 and LotR ends up at 3.0.

This even though both raters are honest by their lights and have rated reliably according to their system. In both cases, you can correctly determine which book they like better. But combining their ratings is completely non-informative. This is the point.

Is there a cure? Possibly. You can add up a person's total ratings and adjust so that everyone gets the same number of "voting points." Since both Ron and Sam have two books, they are allowed to award (say) three points per book, or six points. Sam did award six points; his ratings need no adjustment. Ron awarded seven points instead of six; therefore we multiply his ratings by six-sevenths. This "works" -- it gives Ron's and Sam's ratings the same weight -- but it is very confusing when someone awards a whole bunch of books a 5 and it is treated as a 3.2. :-p

Again, there can be no collective usage of rating unless all people rank by the same rules. You don't have to rate by my rules; that's fine, since ratings are arbitrary. But a collective rating by different scales is meaningless.

I could add massive documentation and more examples, but I suspect people would far rather I didn't. :-p

>39 waltzmn: I see the problem. You're assuming that the algorithm needs to assign "good" or "bad" to each rating in order to decide how to treat each book. It doesn't have to work that way. The algorithm could just assign the strength of the correlation for each book to the rating such that a book with a 5 rating would have a higher effect on the final answer than a rating of 1 would. If that's the algorithm then not normalizing the data would cause some loss of accuracy but the answer would still be accurate.

>36 jjwilson61: As long as a 5 is better than a 4 is better than a 3

And there is no guidance given to LibraryThing users in that respect. You might think it's obviously the case, but I think there's plenty of room for doubt.

Tim's reasoning is sound, I think. A book cataloged is a book cataloged, whether you hate it or love it. Something about it made it worth cataloging, and that something is data worth feeding into recommendations. After all, the purpose of recommendations is discovery, i.e. to draw your attention to previously-unnoticed books that you might find relevant or interesting, not to force you to read them nor to guarantee that you will enjoy them.

>40 jjwilson61:

No, I am assuming that mathematics has meaning!

My height is 23 egprijsles.

Your arm length is 65 injles.

Which one of us is older?

Those three variables correlate, to a limited extent. But only if measured by a common standard and properly calculated.

Without the data set and the common measurement standard, you can't even compare the magnitude of the two numbers.

>42 waltzmn:

It's not that mathematics doesn't have *meaning* - it's that in this case, it doesn't have *predictive value*. There's been a lot of talk about "Tim's opinion", but my recollection is that it was a judgment based on both internal and published research showing that recommender systems don't generally predict user behavior better when considering ratings than simpler algorithms which just consider ownership. What's the point of spending compute cycles and not getting a superior product?

To be fair, this is a *very* old and hazy recollection of conversations at LT and elsewhere, and the state of the research may well have moved on. It's a counter-intuitive finding that's worth continuing to take pokes at. But it is (or was) based on actual testing.

>43 hipdeep: It's not that mathematics doesn't have *meaning* - it's that in this case, it doesn't have *predictive value*.

Note that that's my point! (I have a degree in mathematics, so I believe in mathematics.) @jjwilson61 apparently thinks that predictive data -- or something -- can be extracted from ratings. I don't.

>43 hipdeep:

I'd be interested in seeing citations to that published research, if anybody remembers where it was found.

Making productive use of ratings is probably quite hard.

I mean, if I have one book about the Sokoto Caliphate that I've rated one star, then Tim's logic that obviously I was interested in the subject and might appreciate a better book about it seems sound; but if I've got ten books about it all rated one star, then more probably I hate the subject but had to read up on it for some reason, or I'm using the rating system idiosyncratically.

>46 AndreasJ: And what if you hate stars, so you give the best books the fewest stars :-)

>47 bnielsen:

I think that falls under using the rating system idiosyncratically :)

A book I recently enjoyed has been entered by 262 members. 37 ratings.
Another entered by 1886. 484 ratings.
Entered 1068. Ratings 143.

These are all books published in the past few years. It doesn't look to me as if there are enough ratings to do anything with. I usually rate books when I read them. However I have only rated about half the books I have entered. The other thousands I have perhps not read in the past 20 years. Some because they are my husband's and don't interest me. Others are more for reference. Or just haven't been weeded out yet. As I try to weed, I find my personal ratings very useful.

The algorithm has to give recommendations both to those who rate and the many more who don't.

I suppose one of the few things that could get me rating again would be changing recs to only take rated books into account ...

After I burn Tim in effigy, obviously.

>46 AndreasJ: Or you love the subject and you keep hoping to find a good book on it.