I teach CS1. A lot of this post resonated with me.
In particular, I don't think that beginners are well-served by relying on AI to complete their assignments. Later on, once they've developed some computational thinking abilities, sure. Starting out, no.
There's a real dearth of good options available to computer science educators today for teaching introductory material effectively in the face of all the new and existing ways there are for students to cheat. A lot of what people offer up as alternatives are unworkable or downright bad ideas:
* Paper exams represent an unrealistic environment, encourage terrible programming habits, are a nightmare to grade, and don't test student abilities to identify and correct their mistakes—which is maybe the most important thing we want to assess.
* Oral exams also don't scale and raise obvious equity issues.
* Beginners have to build basic skills before they are ready to work on larger open-ended projects.
We're fortunate at Illinois to have a dedicated computer-based testing facility (https://cbtf.illinois.edu/) that we can use to allow students to take computer-based assessments in a secure proctored environment. This has been a really important support for our ability to continue to teach and assess basic programming abilities in our large introductory courses. I'm not sure why this idea hasn't caught on more, but maybe AI cheating tools will help drive broader adoption. (Such facilities are broadly useful outside of just computer science courses, and ours is heavily scheduled to support courses from all across campus.) Anything would be better than people returning en masse to paper programming exams.
For CS1 you probably don't need much compute power for assignments. Schools are already well funded enough to sometimes offer freshman ipad pros. You could offer them a raspberry pi or some extremely cheap, low powered PC, make them return it after the semester if you want to save them money. You can firewall it from the open internet, and have students turn in their code from this device alone to a university server. They can still cheat, sure, but to do it would mean transcribing code from one device to another by hand, which is enough friction and a timesink where fewer students would consider it versus actually paying attention in class.
Have you taught students before? Many will spend inordinate amounts of time to not learn the material. Often times it seems there is no friction too great if it allows one to not think too hard.
Yes, and I've been one, and know that time is finite and you have more than one class that demands work on a deadline, along with all the other fun stuff college has that pulls you away from your studies, and the not so fun stuff like part time employment. If you leave the system as it is today, its easy to copy and paste code. If you do something akin to what I proposed, you've eliminated copy and paste, and made cheating into a literal chore that isn't saving you nearly as much time as it would have otherwise, and fewer students will end up cheating. You'd be surprised at how many students I knew in undergrad who would be broke and would still pay like $400 a semester on textbooks because the friction of doing hackery things like photocopying chapters of the book in the library, or googling "my math book 2nd ed. pdf" and finding the library genesis result was just too much.
Of course the death blow for this sort of cheating is the exam, which you weight quite a bit more than the homework. A student who just copy and pastes code will still fail the class, since they can't use chatgpt in the lecture hall during exam time.
IT dept can disable loading volumes that aren't the university shared drive for assignment turn in. They can also supply laptops vs desktops with wired keyboards.
You're now well past the original proposal of a Raspberry Pi. And people who need different input devices for ergonomic reasons (laptops keyboards are not a good fit for many people) are going to push back hard.
The device can make network connections, right? Someone's going to come up with a very short program you can type by hand, compile, and then pull down arbitrary other code over the network.
Honestly I appreciate the exercise pentesting my hypothetical academic computer. Lets drop full network access entirely then. Allow it to only connect to a university intranet for assignment deposits, along with plenty of logging of what happened on the device. Lets rule out the "typing verbatim" method too by checking whether the user typed in all in one go or spent time debugging and revising using our own keylogger perhaps. If they need special tooling to interface with the computer, that will probably come with a recommendation from student disability services and be totally legit.
Its a classic arms race, but at the end of the day with enough effort on the issue, IT departments will either win entirely, or make it hard enough to cheat for the vast majority of users that only the extremely small minority who do manage to cheat probably deserve a cs degree. If its hard people won't do it, just like how they keep buying textbooks because finding a free pdf online is only slightly harder, but enough effort to put off most people from forcing campus bookstores out of business.
Issuing a modified laptop running a large amount of custom security software at least seems like an IT departments bread and butter. Maybe if the issue gets big enough microsoft sells a dedicated educational laptop that's preconfigured as such (akin to google's chromebook), but since they own chatgpt they can presumably bake in even more internal controls to flag putative chatgpt content.
If the problem gets widespread, seems like there are possible responses that could and would be made. Just like how in time, schools went from letting you upload your essay and that's that, to running that essay through plagiarism software and baking the tech into their disciplinary process.
On top of the other issues discussed, I think that giving effective oral exams is hard for the same reasons that interviewing is hard. It’s a test of the subject’s ability to quickly and confidently say things that sound roughly correct. Some people stumble over their words and cannot effectively speak and think completely accurately in the time required to give a reasonable answer without an uncomfortable pause.
Now, these issues could be mitigated by asking each person the exact same questions and taking careful notes of their responses, but then you’re just back to a bad essay that can’t be revised, edited, planned, or recollected as easily as a real essay.
Exams can be anonymized when graded to reduce teacher bias. You can't do that with oral exams. In addition, you can't get a second opinion for an oral exam if you suspect you've been graded unfairly.
Language ability, manner of speaking, physical stature and presentation, reputation from previous interactions with staff, you name it—all worsened by the fact that many of these are probably going to be done by course staff. It's not clear to me that any other form of assessment has as much potential for subconscious bias.
Orchestras started using privacy screens for auditions for a reason. And I'm not familiar with an equivalent for the human voice, particularly for hiding halting, labored, or elliptical speech—possibly by a non-native speaker—that they could straighten out on the page.
But then why say "equity"? Blind grading can, of course, eliminate bias and achieve equality. But to achieve equity, you need to know the race, sex, religion, etc in order to "positively" discriminate.
That being said, the ultra dense morons who think "oral" can't be extended in a special case to simply mean "without extra time to think" (which is what is is, mostly) are typically not experienced with these exams.
The parent teaches CS1 and is not likely to have been given a formal oral examination in their likely short career (these are usually reserved for PhD qualifying and special MScs).
What are the obvious equity issues of oral exams? PhD program have oral exams, are they engaging in wrongthink?
One option is to teach some ethics and accountability and have real and immediate consequences for cheating. Make the policies very clear and enforce them. You cannot detect every possible instance of cheating but you can detect many instances, you can test students to determine if they really wrote or could write such code and when they are found, you make examples of them as has been done in previous times. People want to treat literal cheating using ChatGPT as if it is like a calculator and not something fundamentally different.
If you accept this cheating, you may as well not have the class at all nor the degree program.
In particular, I don't think that beginners are well-served by relying on AI to complete their assignments. Later on, once they've developed some computational thinking abilities, sure. Starting out, no.
There's a real dearth of good options available to computer science educators today for teaching introductory material effectively in the face of all the new and existing ways there are for students to cheat. A lot of what people offer up as alternatives are unworkable or downright bad ideas:
* Paper exams represent an unrealistic environment, encourage terrible programming habits, are a nightmare to grade, and don't test student abilities to identify and correct their mistakes—which is maybe the most important thing we want to assess.
* Oral exams also don't scale and raise obvious equity issues.
* Beginners have to build basic skills before they are ready to work on larger open-ended projects.
We're fortunate at Illinois to have a dedicated computer-based testing facility (https://cbtf.illinois.edu/) that we can use to allow students to take computer-based assessments in a secure proctored environment. This has been a really important support for our ability to continue to teach and assess basic programming abilities in our large introductory courses. I'm not sure why this idea hasn't caught on more, but maybe AI cheating tools will help drive broader adoption. (Such facilities are broadly useful outside of just computer science courses, and ours is heavily scheduled to support courses from all across campus.) Anything would be better than people returning en masse to paper programming exams.