It is invoked because LLM+CoT requires a polynomial amount of scratch space to represent P, which is in NP.

I didn't suggest that it was a definition of Intelligence.

The Church–Turing thesis states that any algorithmic function can be computed by a Turing machine.

That includes a human with a piece of paper.

But NP is better though of the set of decision problems verifiable by a TM in polynomial time. Any TM or equivalently lambda calculus or algorithm can solve the Entscheidungsproblem, which was used by Turing to define Halt.

PAC Learning depends on set shattering, at some point it has to 'decide' if an input is a member of a set, no matter how complicated the parts are on top of that set, it is still a binary 'decison'

We know that is not how biological neurons work exclusively. They have many features like spike trains, spike retiming, dendritic compartmentalization etc...

Those are not compatible with the fundamental limits of computation we understand today.

HALT generalizes to Rice's theorm, which says all non-trivial symantic properties of programs are undecidable.

Once again, as NP is the set of decision problems verifiable by a DTM in poly time, that is why NP is important.

Unfortunately the above is also a barrier to formal definition of the class of AI-complete.

While it may not be sufficient to prove anything about the vague concept of intelligence, understanding the limits of computation is important.

We do know enough to say that the belief that AGI being obtainable without major discoveries is blind hope.

But that due to the generalization concept, which is a fundamental limit of computation.