Yes I know, as I said they are very knowledgeable and in some ways very intelligent. We just need to bear in mind their processing architecture is radically different from our. This makes our intuitions about their abilities highly error prone.
Absolutely. The shoggoth metaphor is extremely apt here.
What I was specifically responding to is the claim that they can only solve certain kinds of problems because those kinds of problems (and their solutions) were in the training set. By now there's plenty of counter-examples of unique problems that are nevertheless solved. At which point I think we do have to call it "understanding" and "reasoning", even as we acknowledge that it is a very alien form of understanding and reasoning that we just barely managed to squeeze into something that kinda sorta feels humanish.
simonh says >"We just need to bear in mind their processing architecture is radically different from ours."<
The hardware architectures are certainly different but there is a possibility that at least parts of the "software" architectures may be remarkably similar.
