Welcome to Software Development on Codidact!
Will you help us build our independent community of developers helping developers? We're small and trying to grow. We welcome questions about all aspects of software development, from design to code to QA and more. Got questions? Got answers? Got code you'd like someone to review? Please join us.
Comments on What’s the point of Q&A sites anymore?
Parent
What’s the point of Q&A sites anymore?
Lately, whenever I consider asking a question on a site like Software Codidact, I end up just asking an LLM chatbot instead. Just thinking of the effort of distilling a minimal working example, formatting the post correctly, and anticipating the downvotes, is enough to have an aversion to the site. So what’s the point of these Q&A communities if the friction to participate often outweighs the benefit, especially when a language model can give you an answer instantly?
In short: LLMs are trained based on what we already know. I'm oversimplifying massively, but essentially they produce a …
6mo ago
Although I was tempted to add a sarcastic comment suggesting you could just ask this Meta question of an LLM, I do think …
6mo ago
Asking the last five or so questions on SO (I am new here), I felt pretty much the same as you do. I've put a lot of eff …
2mo ago
There are always problems that LLMs can't answer, or they give trash answers. Then you check the traditional sources, do …
5mo ago
Just thinking of the effort of distilling a minimal working example, formatting the post correctly, and anticipating the …
6mo ago
Post
In short: LLMs are trained based on what we already know. I'm oversimplifying massively, but essentially they produce a string of words which is likely to follow the prompt you provide based on extensive training on samples of language, code, etc. At their core, they are statistical models, and as the saying goes— lies, damned lies, and statistics. They are not general artificial intelligence; they carry no intrinsic understanding of their input or output, and are therefore not capable of knowing that their output is correct in the same way a human is.
They're useful for some things, in the same way data and statistics are. Here is an example of a model:
The points are my training data; the dotted line is my model. My model predicts that given an input of 11, the output will also be 11. In this way, a good model can make good predictions for what comes next; LLMs work essentially the same way, but on billions of input parameters and training data points. That said, what's to say that my graph doesn't show a sequence which maintains $x=y$ from 1 to 11, but then jumps to 13, then 24, then 35, etc? It's not possible for my model to know or predict this, in the same way that it's not possible for an LLM to predict with 100% accuracy the correct answer.
Sorry, slight tangent.
What does that mean for Q&A sites? An LLM may be able to give you a correct answer a lot of the time, particularly for predictable or repetitive tasks; computer code often follows a very well-defined syntax and API so is often predictable enough for an LLM to get it right.
What the LLM doesn't know is anything that it hasn't been trained on. Anything new. If I were to create a new programming language and not share a formal syntax or API with anyone, then—quite apart from the fact that it would be useless—LLMs would be unable to accurately predict what comes next.
That goes for anything that's new. For Q&A communities like this, perhaps that means a pivot away from asking simple questions looking for help debugging code, and towards asking complex questions looking for ideas that only humans can generate. What should I consider when structuring this large and complicated project? What are the trade-offs of this niche technical decision?
Just what is a foo, anyway?
2 comment threads