seems like the latter given "Originally I tried making the dev servers the same base OS that our Kubernetes nodes ran on, thinking this would make the development environment closer to prod"
But I thought the whole point of the container ecosystem was to abstract away the OS layer. Given that the kernel is backwards compatible to a fault, shouldn't it be enough to have a kernel that is as least as recent as the one on your k8s platform (provided that you're running with the default kernel or something close to it)?