You don't need a full-blown LLM to find answers to questions on StackOverflow. You can do that today with any search engine.
> "I’ve even gotten it to walk me through things like installing WSL 2 without using the Microsoft Store after I nuked all Appx packages."
That's a search engine query you can issue already today, without involving any LLMs. It will cost a fraction of the cost of running this with an LLM, and it'll actually bring you to the "source" of the information (a thread on StackOverflow with the full context - including "wrong answers" which are just as useful) - unlike an LLM.
The problem with SO is asking questions though. It’s a horrible site for that purpose. I rather get a slightly wrong answer from ChatGPT than the insane experience of trying to formulate my question just right so it won’t get closed or deleted on SO.
I remember cross referencing with Google to see where ChatGPT was pulling from and it wasn’t immediately obvious. So yes I probably could have found a guide through a search engine with more digging. But clearly the LLM is the superior product.
Sure, you can keep throwing those hyped VC/MBA-style retorts all around, or you can acknowledge the simple reality that when it comes to the task of finding answers to questions on SO, traditional search technology outperforms LLMs on every parameter:
1. Cost: it's much easier to build and serve a traditional search index for questions/answers at scale
2. Speed
3. Quality: with search engines you don't just get one answer (you get a spectrum of answers). You have access to the "source" of those answers, which often include additional data that can help you choose the most appropriate solution to a problem.
In this specific case, the "horse" is the LLM - and the "car" is the traditional search engine with modern indexing technologies. But please, don't let simple textbook facts about technology take you off your trip.
> "I’ve even gotten it to walk me through things like installing WSL 2 without using the Microsoft Store after I nuked all Appx packages."
That's a search engine query you can issue already today, without involving any LLMs. It will cost a fraction of the cost of running this with an LLM, and it'll actually bring you to the "source" of the information (a thread on StackOverflow with the full context - including "wrong answers" which are just as useful) - unlike an LLM.