Omg this is totally incredible I love it - congrats OP!
To address _a_ particular negative comment in this thread.
1. I am often weary of browsing subreddits for their episode discussions because it's quite easy to see spoilers by using the search field, since it will bring up future discussions of episodes where spoilers are allowed. YMMV but this has definitely happened to me before.
2. Some subreddits conveniently have episodes on their sidebar, some only have for the most recent season, some don't have at all
If this picks up traction it would be great to see an api so that apps like track.tv and others could pull data from reddit discussions in an structured way :)
I initially searched for marvel shows presently in limelight.
Loki -> Not available
What If -> Not available
The Falcon and the Winter Soldier -> Not available
Wandavision -> Available
Marvel Shows are more discussed in marvel subreddits like /r/marvelstudios than individual subreddits like /r/wandavision
After sorting out by No of comments only i found out, author of the website has actually grouped them together under Marvel. So Wanda vision Discussions are shown under two places but not linked to each other.
Another one aspect the website fails to capture is the discussions going on in multiple subreddits created for the same TV Show.
For example game of Thrones had individual subreddits for each houses in the show, out of which /r/freefolk was very famous (1M+ Subscribers) and had in depth discussions going on when each episode aired.
I think this is brilliant because it possibly allows for a spoiler-free way to browse through reddit and find discussion of the show and episode without giving away stuff that happened after that episode. Thanks a lot, OP!
I scrolled down to my favorite show and I see you have threads for each episode. Thats exactly how I hoped you would have it organized.
An interesting direction to go with this might be to integrate it with something like Plex or Jellyfin via plugin so that you could see top reddit comments per episode or something like that.
A show I hadn't heard of. I watched it the other week after noticing it in its prominent spot on this website, and it was pretty good! Sometimes, decent shows just don't take off on Reddit.
I like Josh Gad, so I'll check out 1600 Penn, then maybe you'll be able to say that you're one of two people who liked the show.
Alpha House was pretty good, but (to me) it suffered from not being able to decide whether it was humor about things that couldn't realistically happen or humor about things that might, under the right circumstances, happen.
There's a scene where Janel Moloney (Donna of West Wing fame), playing a conservative Senator from ND, refuses to relinquish her gun inside the Capitol[1]. This was ~2013.
And this happened in reality this past January: [2].
One problem I've noticed about Reddit+popular media -- there's not a lot of room for critique unless you amass a large enough collective (eg, 'saltierthancrait'). You can't engage in a critical manner on a subreddit dedicated to worship of the material.
Parsing arbitrary titles for season/episode numbers was a mess of regexes ... 95% of threads are simple "S1E1" or "Season 1 Episode 4" patterns, but a few are hokey:
My regexes grew gnarly from occasional tweaks, so I ripped it all out and changed up my process a bit.
At some point mid-collection, I decided to default to season 1 if not mentioned (as many first-season subreddits fails to mention season number), but failed to re-run the parsing on everything.
Just reparsed everything and deployed a new version. That resulted in ~1000 changes, but a cursory glance shows that it's all good changes, so should be much better now!
How did you select the subreddits? Most seem to match. I found that The Last Airbender: The Legend Of Korra has /r/Korra with 1.7k subscribers, but there's also /r/legendofkorra with 200k subscribers and more rewatch threads.
> My regexes grew gnarly from occasional tweaks, so I ripped it all out and changed up my process a bit.
Are you still using regex? There was a thread recently about collecting book recommendations from HN comments [1]. The app creator used deep learning and fine-tuned a BERT model with their own data. The results were impressive and I think it could work with unstructured data like yours too. I don't know what's your stack, but anything with Named Entity Recognition (NER) would be a good start.
Most of the subreddit/show pairs came from here: http://tv-subreddits.wikidot.com/ . Sourcing this data is the tricky part: I can't find any other sources. Idea I just had: maybe searching /r/television posts for "/r/..." links would be a good lead?
> Are you still using regex?
Yup, just in a much better way. I won't bother explaining how it was before, but now: I have a collection of regexes that try to match a "season" and "episode" at the same time. If that fails, I fall back to matching for them separately. If "episode" is not found, I ignore the thread as a possibly related to a specific episode.
If I have further issues with the parsing, I'll consider a whole new approach like you've recommended. But I don't have any experience with that stuff so my first thought is that it'd be overkill.
Actually yeah, I suppose so. I hadn't realized it was a soap opera and assumed it was a non-standard season/episode format. It's moot in this case, because the show doesn't seem to have any structured episode discussions in their subreddit. This one instance is the only thing my process found.
Apparently they have 4 episodes/week. And there's been no success in starting regular discussion threads. And if they did, they'd probably prefer one thread per week (covering many episodes). It's quite unintuitive that communities formed around soap operas wouldn't have robust discussion threads. I guess because it'd be a full time job at the rate they release episodes...
Ah, this is great. For anyone who has ever tried to hunt down episode discussions on reddit, you know it is usually a multiple step process of searching and clicking through pages to find it depending on the show. And if you're on mobile, good luck to you with reddit's terrible mobile UX. If you could make a site like this for reddit.com/r/anime I'd be really happy. Or open source it so I can fork it.
I'll give it a shot! There's currently a gap for subreddits that contain many different shows. It was simpler to ignore those at first. Got the same issue with Marvel shows, too.
What are some of your favorite animes? I'll make sure at least those work first.
I'm mostly watching seasonal stuff so it makes it a bit more challenging. You'd need to know what comes out every season and scrape the discussion threads for that. https://anichart.net/ is an example of listing all shows each season. I wonder if it has an API somewhere. There are also rewatch discussion threads that pop up for older things that would be nice to have. Yeah, it's fairly challenging :). Could possibly scrape the top 250 shows from anilist/MAL which would cover rewatches. But seasonal would be the most useful.
I looked at Dragon Ball Z. The first two entries were for actual episodes, but of Dragon Ball Heroes, not Dragon Ball Z. The last one was for "season 5 episode 65" but was actually a thread for the most recent movie, that contained the string "5.65" in the title.
Of course, I did choose this series because I thought it would be especially likely to have problems, but YMMV.
Generally Subreddits only have the most recent episode threads pinned, while this give full lists by episode. So, saves you a search. Decently useful if you want to check out multiple episodes threads, I guess.
Generally, but not always, a subreddit will have a discussion thread hub pinned or as a wiki page, but not always. They were outright missing enough of the time that I was bothered enough to make this.
Although the main reason I made it was because I like to read discussion threads after every episode I watch. So I'd google "reddit <show I'm watching> Season .. episode .." _a lot_. And have to tweak the query sometimes to find something. So I made a website.
The show you're thinking of is That Mitchell and Webb Look. The subreddit is for the actual comedy duo and covers all their work.
The website definitely isn't set up to handle multiple shows from a single subreddit, however. The first couple shows listed as Peep Show there are just some new show that David is in.
Yea, entire process currently falls apart when multiple shows are in the same subreddit. This is a big issue with all the Marvel shows, which mostly all in /r/Defenders or /r/Marvel
Most of the subreddit/show data came from a giant, stale list of "most popular tv subreddits". The rest came after asking /r/television what I was missing. And I get a regular trickle of ~3 suggestions/week in my Twitter DMs.
Because IMDB doesn't want it there. They shut down the message boards. Sure, the reviews are still there if you want to say something in general. But they aren't good if you want to talk about a particular scene, character, or line. Or interact with something another person said.
Presumably written by an American. The first thing I searched for is "Line of Duty", not there. Next, "The Office" - only the US version is present, and it's simply titled (incorrectly) "The Office", not "The Office (US)".
Yes, data-scraping projects written as a hobby tend to not have perfect data. There's over 600+ shows here! I collected most of the show/subreddit data from an existing list of "most popular" Tv subreddits. The original The Office wasn't in that list.
Also, "You're obviously an American" comes off as weirdly hostile, which I choose to assume was unintentional.
1. You can just go to the show's subreddit and browse, or browse with filters such as top, hot, etc. And you can filter the time period.
2. You can use Google. This works great for me. Just type "the wire reddit" or "site:reddit.com the wire", and the first result will be the subreddit page, but if you scroll down, you will start to see threads from other subs as well. This way you can access more discussions outside of the dedicated subreddit.
____
In a more general note, you can find all sorts of different stuff by using Google to browse reddit this way.
I have watched quite a few really good movies by searching "your top 5 movies reddit", "your favorite movies reddit", or "best movies reddit". I just go through the past viral threads and find highly upvoted responses, or responses with some of movies I liked and some that I haven't watched.
I have discovered Clue, Primer, Hot Fuzz, Tropic Thunder, The Big Short, etc. this way.
Or, now you can go to this website and jump to the desired location quicker than ever.
I list out some motivations in a different comment.
Addressing the points: 1) This takes too long. 2) A google search usually works for me too, but not all the time, and sometimes requires multiple attempts. And I'm not trying to land on the subreddit: I want the thread for the exact episode I just watched.
To address _a_ particular negative comment in this thread.
1. I am often weary of browsing subreddits for their episode discussions because it's quite easy to see spoilers by using the search field, since it will bring up future discussions of episodes where spoilers are allowed. YMMV but this has definitely happened to me before.
2. Some subreddits conveniently have episodes on their sidebar, some only have for the most recent season, some don't have at all
3. _Sigh_ Yes you could of course do this manually: Hacker news hasn't changed a bit lol https://news.ycombinator.com/item?id=9224
If this picks up traction it would be great to see an api so that apps like track.tv and others could pull data from reddit discussions in an structured way :)