So at what point do we start producing CPUs specifically aimed at running a kernel/userland? Why don't we have a CPU architecture where a master core is dedicated to running the kernel and a bunch of other cores run userland programs? I am genuinely curious. I understand that x86 is now the dominant platform in cloud computing. But it's not like virtualization needs to be infinitely nested, right? Why not have the host platform run a single CPU to manage virtual machines, which each get their own core or 20? Would the virtual machines care that they don't have access to all the hardware, just most of it?
> Why don't we have a CPU architecture where a master core is dedicated to running the kernel and a bunch of other cores run userland programs?
How will your "userland core" switch to other userland programs safely? A pointer-dereference can be a MMap'd file, so its actually I/O. This will cause the userland program to enter kernel-mode to interact with the hardware (yes, on code as simple as blah = (this->next)... the -> is a pointer dereference potentially in a mmap'd space to a file).
So right there, you need to switch to kernel mode to complete the file-read (across pointer-dereference). So what, you have a semaphore and linearize it to the kernel?
So now you only have one-core doing all system level functions? That's grossly inefficient. Etc. etc. I don't think your design could work.
Sounds like we would need a new paradigm for how to handle that. But it seems to me that x86 is in now way the panacea of COU design. Wouldn’t you gain some good trade offs by changing up how things are done?
MMap'd files, and... in-demand paging... are pretty much on every CPU architecture worth making an application for. ARM, POWER9, X86, SPARC, MIPS, and more.
In-demand paging is another situation where a simple pointer-dereference can suddenly turn into a filesystem (and therefore: kernel-level / hardware-level) call.
What the commenter above you was describing about mmap'ed files and dereferencing invoking the kernel implicitly -- that's true on all current CPU architectures (everything from x86 to SPARC to ARM and back again.)
At the low end, sure. At the medium-to-high end, each VM is bound to one or more physical cores of the host, or sometimes an entire host ("dedicated instances.")
I don't know enough about the IaaS market to know what the relative revenues of low-end compute vs. medium-to-high-end compute are for your average vendor, though. Is most of the profit in the low end?
I'm also curious on what the impact on margins would be if IaaS vendors decided to switch away from serving the low-end compute demand with "a few expensive high-power Intel cores per board, each multitasking many vCPUs", to serving the demand with "tons of cheap low-power ARM cores per board (per die?) with each core bound to one vCPU."
Would the kernel even need RAM access for its internal operations? It seems like today’s CPU caches are so large that a kernel could safely operating without ever leaving the chip, aside from anything that the userland asks it to work on. So in that case you wouldn’t need to ever run userland code against kernel CPU’s caches.
The pagetable itself can be very large; disk cache and network buffers can also take a huge amount of memory and are probably great targets for data exfiltration.
