I'll gladly diss LLMs in a whole bunch of ways, but "common sense"? No.
By the "common sense" definitions, LLMs have "intelligence" and "understanding", that's why they get used so much.
Not that this makes the "common sense" definitions useful for all questions. One of the worse things about LLMs, in my opinion, is that they're mostly a pile of "common sense".
Now this part:
> Add in the commercial incentives of 'Open'AI to promote usage for anything and everything and you have a toxic mix.
I agree with you on…
…with the exception of one single word: It's quite cliquish to put scare quotes around the "Open" part on a discussion about them publishing research.
More so given that people started doing this in response to them saying "let's be cautious, we don't know what the risks are yet and we can't un-publish model weights" with GPT-2, and oh look, here it is being dangerous.
After studying it extensively with real-world feedback. From everything I've seen, the statement wasn't "will never release", it was vaguer than that.
> they explicitly stated that they wouldn't release GPT-3 for marketing/financial reasons
Not seen this, can you give a link?
> it being dangerous didn't stop them from offering the service for a profit.
Please do be cynical about how honest they were being — I mean, look at the whole of Big Tech right now — but the story they gave was self-consistent:
[Paraphrased!] (a) "We do research" (they do), "This research costs a lot of money" (it does), and (b) "As software devs, we all know what 'agile' is and how that keeps product aligned with stakeholder interest." (they do) "And the world is our stakeholder, so we need to release updates for the world to give us feedback." (???)
That last bit may be wishful thinking, I don't want to give the false impression that I think they can do no wrong (I've been let down by such optimism a few other times), but it is my impression of what they were claiming.
I was confusing GPT3 with GPT4. Here's the quote from the paper (emphasis mine) [1]:
> Given both THE COMPETITIVE LANDSCAPE and the safety implications of large-scale models like GPT-4, this report
contains no further details about the architecture (including model size), hardware, training compute,
dataset construction, training method, or similar.
By the "common sense" definitions, LLMs have "intelligence" and "understanding", that's why they get used so much.
Not that this makes the "common sense" definitions useful for all questions. One of the worse things about LLMs, in my opinion, is that they're mostly a pile of "common sense".
Now this part:
> Add in the commercial incentives of 'Open'AI to promote usage for anything and everything and you have a toxic mix.
I agree with you on…
…with the exception of one single word: It's quite cliquish to put scare quotes around the "Open" part on a discussion about them publishing research.
More so given that people started doing this in response to them saying "let's be cautious, we don't know what the risks are yet and we can't un-publish model weights" with GPT-2, and oh look, here it is being dangerous.