I agree in principle but you should still literally slap anyone that says they “could care less” when they mean they couldn’t

As it’s used, “I could care less” works as a sarcastic statement much better than “I couldn’t care less” as a literal one. Generally it’s possible for me to care less about some minute difference, but in absolute terms I don’t care enough for it to matter.

That sounds like a poor justification for simply using the wrong phrase. I've never heard it used sarcastically, just incorrectly.

You damn! I felt that slap ;) :P

'literally' is often used figuratively to mean 'figuratively', that doesn't change it's literal meaning :D