A lazy physics teacher can make every question a math question in disguise. If a physics teacher is worried that a better calculator will make their test trivial, maybe they need to teach physics instead of testing math.
A lazy humanities teacher can make every question a writing question in disguise. If a better spell checker will make a humanities assessment trivial, then maybe they need to teach humanities instead of testing essay writing.
I'm saying this in a kinda inflammatory way, but does the quality of ones ideas really correlate well with a well written essay?
I agree that professors can probably develop ai resistant evaluations.
But there is minimal institutional support for this. Everybody is going it on their own. And the ai tools are changing rapidly. Telling a ntt faculty member teaching a 4/4 that they just need to stop being lazy and redo their entire evaluation method is not really workable.
Iteration is also slow. You do a new syllabus to try to discourage cheating. You run the course for a semester. You see the results with now just a few weeks to turn around for your spring semester courses. At best you get one iteration cycle per six months. More likely it is one per year since it is basically impossible to meaningfully digest the outcome of what you tried and to meaningfully try something different mere weeks later.
I've been urging this kind of adaptation for the past couple years, with very little success. Switching to evaluation by only on-site work was simple to do, but creating new work forms has been largely a non-starter. Young professors are focused on their publication count and most old codgers (like me) have their eyes on retirement and hoping to escape before accountability hits the fan. When I offer to help explore restructuring their teaching materials, the typical response is glazed eyes and sudden awareness of the need to be elsewhere.
What needs to happen is a return to a more Socratic method of teaching the humanities.
Instead of being about the consumption and production of texts -- texts which only the TA and maybe the prof will read ... ever -- the class should be more about the dialectical process of discussing the ideas in the texts and lectures the students consume.
An LLM will be able to easily rattle off an undergrad level paper. But it can't sit in the classroom and have an intelligent conversation with peers.
Of course none of this will happen because it doesn't scale economically nearly as well. It is far more labour intensive.
Yeah, I've been thinking about when the calculator came to bear. I was in high school physics when (rich) kids started bringing in "scientific" calculators. There was a flap then about whether they should be allowed at all.
I'm not sure they are analogous to LLMs, but a compelling argument at the time was that the students were going to use them in the field anyway. (That certainly seems to be the future for software engineering students.)
There's always an incentive to cheat: economic, reputational… people cheat in single-player video games. People cheat in sudoku puzzles. By removing all incentives, all you do is destroy the concept of "cheating", not prevent the behaviour.
I think you've listed several disparate incentives that should be looked at separately.
Economic incentives are what this thread is about. If those are removed, some instances of cheating will subside.
Reputational incentives are similar and harder to address, but they are also less strong because the cheating has a much higher chance to backfire. If you are found out, your economic benefits might continue, but your reputation is immediately damaged.
Singleplayer games are an entirely different situation. There's clearly no economic incentive here, and reputational incentive only applies if you are looking to share your singleplayer result with other people, in which case I could argue it's no longer singleplayer in the game theoretic sense. If it's not that, the only remaining motivation to cheat at something like Sudoku or a singleplayer videogame is to learn or at least satisfy an itch or curiosity, which I think is a perfectly legitimate motivation.
It’s the thinking that precedes writing that should be the bulk of your grade. Possessing what looks like an artifact of organized thought isn’t enough these days to evaluate a student.
The “thinking that precedes the writing” is not where the gold is. That kind of thinking is often confused, internally inconsistent, liable to miss critical details or nuance, and full of deductive leaps which may or may not pan out. Writing demands rigor of thought, it forces us to question premises, find evidence for a point of view, discern between the hypothetical and the factual, and try to organize these such that they cohere.
The thinking _is_ the writing. To be able to write is to be able to think, and if you are surrendering the writing to a machine that’s not ultimately what you’re surrendering—you’re giving up independent thought itself.
You misunderstand. I mean the thinking that you say “_is_ the writing”. But the final words aren’t the thoughts that you should be graded on (aka “this is not a pipe”, but your essay is not your thoughts). What a new assessment could look like is an open book debate with a human (or, hell, with a rubber duck LLM). If at the end your thoughts are organized and self consistent you’ve done well.
A lazy humanities teacher can make every question a writing question in disguise. If a better spell checker will make a humanities assessment trivial, then maybe they need to teach humanities instead of testing essay writing.
I'm saying this in a kinda inflammatory way, but does the quality of ones ideas really correlate well with a well written essay?