Some people aren't cut out for the particular style of thought needed to write algorithms in a programming language, yes. Or at least some people require disproportionately more instruction to succeed and thus are likely to always fail in a college intro programming course.
Agreed. Some people just aren't as effective at abstract symbol manipulation. I've found that people who can pick up programming languages can pick up other human languages and vice versa.
This doesn't mean that they can't learn programming. It means that they would be much better off focusing on their strengths instead. There's a difference between "can't do it" and "fails course" as you rightly suggest.