Welcome to Software Development on Codidact!
Will you help us build our independent community of developers helping developers? We're small and trying to grow. We welcome questions about all aspects of software development, from design to code to QA and more. Got questions? Got answers? Got code you'd like someone to review? Please join us.
How can Q&A sites coexist with LLMs?
New LLMs like ChatGPT are now creating competition with Q&A sites like Codidact and StackOverflow. Moreover, this is parasitic: LLMs get "boosted" by Q&A sites because they can use them for training, but Codidact does not benefit from AI models for example because AI content is not allowed here.
Do LLMs present a threat to Q&A sites like this? Or is there a way for them to coexist? What niche would the sites fill?
1 answer
My take on this: Q&A sites used to fill two distinct roles, but only one of these (the more boring one that doesn't matter anyway) is usurped by LLMs. The interesting one is not yet in danger of being supplanted.
There's always been two types of questions on StackOverflow:
- Trivial, highly specific questions that would be directly answered by documentation etc. but the asker either doesn't know there is a manual, is too lazy to read it, tried reading it but it was too confusing, or the manual is missing in parts or in whole. These tend to solve the one instance of a problem each. Even very similar other problems are often hard to recognize as such and people end up asking many duplicate questions about the same thing.
- Complex, deep questions where there's not really "one definitive answer", and it's not so much about "answering" the question but learning something from the discussion that is valuable. These usually deal with central concepts in an area that, once properly understood, can be used to independently resolve a whole class of problems.
I came to SO for the first kind, but stayed for the second. I am now here, because over the years SO moderators began to crack down on the second category. IMO the second category is the most interesting, while the first is quite boring. As an expert, I would not really want to be answering the first kind too much, the second would be the interesting one. This can be seen in how many eminent scholars choose to write comprehensive textbooks on their subject rather than practice tests.
I think LLMs currently excel at answering the first kind, because no manual is too boring for them, so they effectively "read them all". Even then, the models are generative, often hallucinate, and it's never clear what's generated and what's fact - so they tend to be unreliable with it as well. Although I expect that the reliability will improve over the next few years.
For the second kind of question, LLMs are no good at all. For one, it requires the ability to reason at a high level and synthesize laterally related information which the LLM cannot do. It can generate content if deep treatises of similar topics happened to be in its training data, but there are not always readily available analyses on every conceivable subject - not many people can write one, and it's work to do. So I think this is the real niche of a Q&A site - the ability to support thoughtful discussion of key concepts. This is probably out of reach for AI until we see a true breakthrough in the field.
1 comment thread
0 comment threads