I like that it asks me for books I like, that is a good start.
The thing that I was trained to do when I was a librarian, which I never see in recommendation engines, is to ask the question "what did you like about this book?", or "why did you like this book?", or any number of related questions that get to what you really want. Understandably, an algorithmic approach usually assumes the book is a single unit that is liked (or not) by others, and can thus be linked to other things people like on a fancy graph. This is useful because you don't need a granular understanding or human-level comprehension of the books. But, it tends to make for more generic, less interesting recommendations.
Sometimes the answers people give to these questions are really interesting: "there was one scene that really reminded me of a canoe trip I took with my dad" not just "the user wants [books with canoes]". At present, you need to be human to actually understand that it's not the canoe that matters, it's something else, and the right recommendation may be a book that appears completely different on the surface.
Great point... On Sword and Laser (a book club/podcast) the idea of different types of reader is often discussed; i.e. what's the main driver behind liking a book:
- Plot
- World Building
- Character
- etc (There's a specific list mentioned, but I can't recall it)
That's the simpler classification side of things, so easier to code up... Your example of the canoe trip would be significanlty more complex; but definitely more a scenario worthy of AI over basic heuristics... and something that would need a conversation to drill down into (i.e. did the story evoke the scenery of their trip, or was it reflecting the relationship between the people, etc).
I found that it recommended the same books that I'd get from any non-AI-powered site when I say I like "Three Body Problem", "Brave New World", and "Dune". What I'd really like is for it to help me find books I wouldn't think to try reading by asking me questions about what I enjoy in life and topics I find interesting.
I'm echoing my other comment a little bit but I've been following Shepherd and thought I would comment since you're here. I like a lot of your ideas but I haven't quite managed to get recommendations that consistently sound interesting. I think it has something to do with authors and it would be interesting to know how you're thinking about how authors or writing styles are related.
For example, I like Bill Bryon and have read most everything he has written. He's good at making subjects I am not interested in captivating. For example medical history and anatomy in his book The Body.
I would love to read other books that make the uninteresting interesting but when I put The Body into Shepherd (or most recommendation engines) I mainly get other books about anatomy. This makes sense on the surface, but I've already read a book on anatomy. I'm ready for something else interesting.
I realize that might not be very actionable or useful, but when I am looking for something new to read I don't know what I am looking for. I just know the "vibes" books and authors I like, and often find that recommenders default to similar subjects only. Maybe this is difficult or impossible to capture, but it's been nagging me as something missing.
Thanks, this is super helpful! I love Bill Bryson as well :)
Yep, I totally understand what you are saying, and that is what I am working toward. It is a challenge.
Right now, I ask authors to share 5 books they love around a topic/theme/mood. So I am collecting a human grouping of books similar in some way that humans decided along with why they love it.
For example, if you like "Notes from a Small Island" or "Down Under" (two of my fav Bill Bryson books), here are books that authors have grouped with it and also loved:
So I am using human groupings to help discover books that humans associate with his work somehow. And above each book, their list title helps you understand how their mind grouped it.
For example, "Into The Wild" is associated with it based around a human grouping books that capture the spirit of Jack Kerouac's On The Road.
What do you think?
What is next to improve this?
I'll add filters to help people hone in on why they loved the book to help pull that thread out and show books along those lines. For example, if I liked Catch 22 for its absurd humor, I want to make that the common denominator on the page. Or if I like "From a Small Island" for the humor and history, picking those to try to find books with a similar book DNA.
In about 4 to 5 weeks I'll ship the first version of this on the bookshelf pages. And in maybe 12 to 16 weeks on the books-like pages, as that needs a lot more work.
And, I am hoping next year to launch Book DNA, where I pull in reader data and help line up books that are your all time favorites with others to map this out a bit further. Slowly getting there and a lot will dependon collecting more data from readers in 2024.
Nice! These "books-like" lists feel and resonate better with me. From the first list I've read Notes from a Small Island, Into the Wild, and Wild, and now some of the others on the list that I don't know, like Llama Drama, look promising. I think you're on the right track.
I like the idea of pulling as many threads out of a book to drill into why you might like something. Especially if those threads are not necessarily related to each other as that should widen the recommendation pool.
I still think there is something worth doing with author links as well. I played around with gnooks.com after someone else posted a link and within a few minutes found authors I had never heard of that had books that looked interesting.
I want to read more books like "Financial Shenanigans, Fourth Edition: How to Detect Accounting Gimmicks and Fraud in Financial Reports". However, neither the book nor the first author are listed in your database. I couldn't find any narrow topic to match either.
Yep we only have books in our system that the 8,000 authors we have done these interviews with so far loved. So we are still missing a ton of books. We are adding more every week and eventually, I bet we will find someone who loves it. I am also hoping in late 2023 and early 2024 to expand and bring in the 3 favorite books that readers read that year. So that will give us a bit more data.
I just tried it. The look and feel is really nice. The topical search was helpful. Bookshelves are a cool idea and "lists we think you will like" in the footer was helpful when I found a list that was OK but with books that were too academic for my interests in a topic.
Book Title search seemed like it could use some kind of "not found, but..." Helper, or suggestion functionality.
I searched "Moscow Calling," "Remembered Prisoners of a Forgotten War," and "Nixon at the Movies" with no title results. Huckleberry Finn worked OK.
Would use it again though, thanks for dropping a link in the discussion.
Ya, the search is really simple and more of a browsing tool. I wanted to try something unique and get it out quickly. It is not working well; I am going to redo that more in the style of Google search. And ya I will def do something better if it is not found in the future.
We don't have those books in our system yet as no author has recommended them is why nothing is showing up. We are still pretty limited as we have interviewed 8,000 authors in different areas, but that isn't a lot in the grand scheme of books. Slowly getting in more every week :)
Lots of book sites have recommendations. E.g. "Readers also enjoyed" on Goodreads and "People who bought this also bought" on Amazon.
They're anonymous-ish (in that you don't have to be logged on, though I'm sure tracking cookies gunna track), and they're direct, but they're only for one book at a time.
GPT4 is pretty good at this. Here's what I got on a first run, though you can keep prompting it to go more esoteric if you want. I honestly think that GPT4 will obsolete most existing recommendation systems; it's already better than what I previously used (Goodreads).
> Prompt: give me some books similar to "Three Body Problem", "Brave New World", and "Dune" - but dont give me the common suggestions, give me some more esoteric stuff i probably haven't heard of
> I understand that you're looking for lesser-known, yet thought-provoking science fiction novels similar to "Three Body Problem," "Brave New World," and "Dune." Here are some esoteric suggestions that you might not have heard of:
> "Blindsight" by Peter Watts - This hard sci-fi novel explores the nature of consciousness and humanity's first contact with a terrifyingly alien intelligence.
> "The Quantum Thief" by Hannu Rajaniemi - A post-human heist story set in a future Solar System, where advanced technology has given birth to an array of strange and wondrous societies.
> "The Left Hand of Darkness" by Ursula K. Le Guin - A groundbreaking work of feminist science fiction that explores gender and societal norms on a planet where inhabitants can change their sex at will.
> "The Stars My Destination" by Alfred Bester - A classic sci-fi novel that combines elements of revenge, teleportation, and corporate intrigue in a richly imagined future world.
> "The Dispossessed" by Ursula K. Le Guin - A thought-provoking tale of two societies, one anarchistic and the other capitalistic, and the physicist who attempts to bridge the divide between them.
I can vouch for the entire recommendations list (Blindsight or Quantum Thief series are my go-to answer when people say "what's your favorite book"), but I wouldn't really call them "lesser knowm" - they all won prestigious awards like Locus or Hugo.
I played around with something similar to what you are talking about - take your Goodreads bookshelf, see what you've read and what you have marked as "to-read" and then make a recommendation based on that. It is not perfect by any means, but would love feedback!
If you've got the patience to feed all your books into librarything, the recommendations are incredible - for me virtually the whole first page are all books I've already read, but don't own.
I don't think its recommendation engine is anything sophisticated, it's just working with really really rich data.
Personalized recommendation does not address that use-case. All you need to do is check reviews, lists and favorites among users on r/books or other platforms.
That is very interesting. I think most home libraries must go deep on a few genres, rather than selecting the best from many genres. I can absolutely see how it would fail for you.
It's got 2.8m members who have entered 186m books that they own. It cross-references those collections in some way (I don't know anything about the internals).
I just checked, and it says it's recommending a book to me (Singularity Sky) based on five other books I own (Engine City, Spin State, A Deepness In The Sky, Recursion, The Skinner). But as it's always "based on these five books", I have to assume it's doing a lot more matching than that.
Psss. Please don't ruin Librarything by getting more people on there. Its recommendation algorithm is pure perfection—basically magic—and I fear it's mostly because it's only used by a very particular group of people.
My point is that recommendations that are based on social networks become less and less useful as that network grows. And LT seems to have hit the perfect size for its recommendations to be useful. I'm actively dreading the day that they either get worse (i.e. the network gets too big), or that LT goes away entirely because it has no place in the modern internet anymore, multiple times a week, to the point where I'm thinking about ways to preserve its current form in some way or another.
If more people would be on there it wouldn't be as useful too anyone anymore, instead of being as useful to more people. So instead of gatekeeping it's more like preservation of something that would otherwise vanish. Librarything is basically one of the few national parks the internet has left.
Finally, and maybe most importantly, my original comment was really meant as a light-hearted comment on the magic of LT with some of the thoughts I have on the matter sprinkled in. But I should've known that HN would interpret it in the least favorable way.
I'm also a fan of LT, but most importantly I'm a fan of spreading the wonderful feeling you get from reading. If LT's recommendation algorithm is only good because of its pool size, then it's bound to fail at some point in the future. Rather than delaying this inevitability -- which effectively sweeps the issue under the rug -- the better solution would be to ensure that their recommendations stay high-caliber regardless of scale. I'd imagine the only way to perform this test would to be for them to actually grow in scale, so they should be advertised as much as possible.
I really think it's the depth, not the breadth, that makes the recommendations work. I've been a member since 2005, and SciFi was pretty good right out of the gate. Now, everything's pretty good. If it hasn't been harmed by its growth up to this point, I honestly doubt a little more will hurt it.
Infinite Jest, 1984 and Moveable Feast return Catcher in the Rye, Sun Also Rises, Gravity's Rainbow and Slaughterhause-Five. These are very basic recommendations. Not great
Searched for
1.The flute and flute-playing : in acoustical, technical, and artistic aspects,
2. An essay on the construction of flutes, giving a history and description of the most recent improvements, with an explanation of the principles of acoustics applicable to the manufacture of wind instruments
3. A treatise on the flute
It suggests
Beauty and the Beast and Other Fairy Tales
The Flower of Paradise
Songs for Bass in a Comfortable Range
Philip Steele of the Royal Northwest Mounted Police
and The Flute Book (this is the only relevant result).
The recommendations have a heavy bias towards high-popularity books. That's... not extremely helpful.
I kind of expected that on my first sample set (Ishmael/Accelerando/A Gentleman in Moscow), but even the second one (Priory of the Orange Tree/Traci Lords:Beyond it all/Civilized Man's Eight Deadly Sins) is severely tilted to "everybody is reading it!"
A last try with a decided "60's Scifi" slant - Small, still voice of trumpets/The Tomorrow People/Deathworld. And I get Hitchhiker's Guide, War of the World's, Rainbows End.
I don't look for book recommendations to learn what everybody is reading, those books are inescapable anyways. I'm looking for thematically relevant, but might surprise and delight me.
Doing this well is hard even with curated lists and experts, but it's definitely possible to do better by just weighing popularity less heavily.
Similarly, the top two suggestions for me were both sequels to books that I had selected.
Interestingly, I asked ChatGPT for recommendations based on the same list and it gave me a very similar result, including matching the top two suggestions! Coincidence? (I added a note that I was interested in finding new authors and it gave me a much better list.)
Probably a great project, but I couldn't use it as it shows me all book titles in German. Obviously it is based on my IP address. But I'm not a German speaker, so I'd like to control this behavior.
Unfortunately, it seems that some modern websites, including Google, tend to prioritize IP address over user preferences when it comes to language selection. I've ensured that my browser language, preferred language, and Linux locale are all set to English, and I make a point to change any settings I find in German to English as well. Despite this, some websites still display content in German.
To further prove my my point, I've just connected to a Singapore VPN, refreshed the page, and all book titles switched to English. When I disconnected from the VPN and refreshed again, the titles reverted to German.
Recommended me Hunger Games when I put my favorite books as (just to test it):
"The Quran"
"Do Androids Dream of Electric Sheep"
"Jupiter's Legacy Vol 1"
Pretty solid recommendations. I picked a specific topic and chose three books that I've read about it and it recommended two other books that I have on my list, one that I have heard good things about but have decided against reading it and one that I haven't heard about and will check out.
Though as always, the problem with book recommendations is not that it's hard to find books about a specific topic, the problem is finding books that are a) good and b) different enough from other books I've read about a topic so as not to be very repetitive.
If this could solve a) I'd be very happy, though I cannot see how it would accomplish that.
The best attempt at solving this for myself was just going by book recommendations from actual people, but even that is hit or miss. Many people recommend books about a topic on which they've only read that one book and so they don't actually have anything to compare their book to. A book recommendation like that doesn't contain information about how good a book is, just if the one who read it liked it.
So in my opinion this is actually a pretty difficult problem and I'm not sure throwing an llm at it (I'm assuming that's what the ai in the title refers to) will solve it.
If you want to compare books, I've got a project for that: https://findsight.ai I use it for my learning as a PM to skip over books that just copy & paste from each other.
A related project is https://shepherd.com - but it focuses on high quality human suggestions.
Love this. I put my three favourite books, and got out two recommendations that I already read and loved, and two ones that appear to be likely new favourites
I find that it just seems to pick the most well known author of the three and gives me back mostly just other books by them. Or otherwise doesn't seem to recommend based on the content or themes or historical context of the book, but broad current cultural patterns.
Like, I get that the type of guy who likes Infinite Jest would probably like the three James Joyce books recommended to me, but thats more of an Amazon recommendation than an informed AI one. Its also surely the case that an IJ fan is aware of Joyce already!
I tried to be more obscure and personalized: 2666 by Robert Bolaño, The Dissertation by RM Koster, and Notable American Women by Ben Marcus (which is a novel despite its title). I got back a David Foster Wallace book, another Bolaño book, a book of short stories that is maybe actually interesting but very different from my input, and a homemaking book about reducing waste in your home.
Kinda seems like a waste of energy hooking up AI to this, you could just scrape Goodreads and get the same quality output.
Have to considered using referral links for some of the recommended books?
I don't think that everything needs to be monetised. On the contrary - there's too much stuff existing merely to just extract value from users. But, the recommendations I've seen here look really good and I'm already ordering one of the books (the other one I've already bought).
> The problem I found with this (and many other recommendation engines) is that I've read all of the suggestions.
Immediately had the same experience: input a Theroux, Bill Bryson and Bruce Chatwin into the engine, as I've been longing for some new travel books to read, but naturally it just gave me... five or six other books by Paul Theroux and Bill Bryson.
Definitely needs more input parameters to be useful.
A couple of weeks ago, I built an ChatGPT-powered recommendation engine for everything.
You begin by entering what you want to get recommended, and some examples of what you like and don’t like. After this it lets you swipe cards with suggestions to like/dislike them and further train the program.
Love the idea. All online book recommendations I know about don't really work for me. That being said my first try with this site seems unsuccessful too.
I entered Blood Meridian, Barbarian Days and The Cider House Rules. I got
The Road -> Same Author as Blood Meridian and I guess Mccarthys most mainstream successful book?
Barbarian Days -> ???
The World according to Garp -> Same author as Cider House
The Devil in the White City and The house on Mango Street are the two recommendations I don't know which also don't seem too obvious. But on first look both books don't appeal to me.
Promising idea (and a real unsolved problem for me) but at the moment I am getting the same as kind of recommendations others are experiencing. Mostly books I already know well or would see in other recommendation lists. Maybe three books isn't enough?
FWIW I've noticed I tend to gravitate more to authors than specific subjects, and finding a new author whose catalog I can mine is somewhat rare.
The recommendations were great! I have read all except one. Which mean that it really knows what kind of book I read as well.
Can I ask if you use GPT api for this? or you used your own model.
I'm also interested in the model where it can summarize book, index it and be able to fetch it later.
I wonder if there’s an AI that could deeply look into my data, and I have hundreds of read books along with reviews and ratings, and recommend me some real non-obvious books.
I tried asking chatgpt, but it’s too much data for it to digest. And from what it can digest it gives me some real obvious and generic recommendations.
I tried it and it doesn't seem to recommend as many books as other book recommendation sites I use.
I prefer Readow.AI for instance because it recommends a lot more books, it's quicker, and the books it recommends are occasionally ones I've never heard of (and thus more valuable to me).
It gave up finding me recommendations on the basis of The Legacy of Totalitarianism in a Tundra and The Legend of the 10 Elemental Masters. Both authors have made more works, so pretty weak.
I'm interested in old game programming books. I put in three that I have, and I got a couple modern programming book recommendations. Is there a way to add consideration for age of material?
Is it like "Given preference for A, B and C, suggest more"? One would like to understand what makes (specifically to this task) such box a good engine.
Reliability of ChatGPT GPT-4 isn't consistent in my experience. It seems to respond to prompts differently, but I'm not sure yet if it's based on the number of prompts in the last 8 hours or overall server load. I would guess that the API is more consistent than the ChatGPT frontend, but can't confirm for sure.
You can tune the temperature parameter and bring it to 0 (if using the API). Although technically it's not fully deterministic, it will reply with the exact same answer >99% of the time in my experience.
(This is for GPT-3 and ChatGPT. Haven't tested GPT-4)
gave some progression fantasy recommendations (basically high octane anime flavored action fantasy) and it gave me some historical fantasy (not the same genre) some fantasy romance (even further away) and emma by jane austen (????) doesn't seem to work.
selected three books that I love, two out of three recommended books are childrens books. Weird.. My selected books obviously were not childrens books. Maybe because I selected Dutch books?
I think these will be the first books most people think of.
Your idea is great and I hope it works out. I could only get errors when trying.
For me, I don't need any more book recommendations. Once I started reading more a few years ago, each book referenced or even recommended other books, and I'm still riding that wave.
I'm not the kind of person to question down votes, but really? Am I getting down voted for suggesting the two most read books of all time to be included?
The thing that I was trained to do when I was a librarian, which I never see in recommendation engines, is to ask the question "what did you like about this book?", or "why did you like this book?", or any number of related questions that get to what you really want. Understandably, an algorithmic approach usually assumes the book is a single unit that is liked (or not) by others, and can thus be linked to other things people like on a fancy graph. This is useful because you don't need a granular understanding or human-level comprehension of the books. But, it tends to make for more generic, less interesting recommendations.
Sometimes the answers people give to these questions are really interesting: "there was one scene that really reminded me of a canoe trip I took with my dad" not just "the user wants [books with canoes]". At present, you need to be human to actually understand that it's not the canoe that matters, it's something else, and the right recommendation may be a book that appears completely different on the surface.