The demo in the README is pretty impressive for an independent implementation, but I also noticed the repo includes server code (https://github.com/Devolutions/IronRDP/tree/master/crates/ir...). I wonder if tools like Proxmox could use this as a more efficient alternative to VNC (which is slow and weird) or SPICE (for which there are very few non-Linux tools).
> I wonder if tools like Proxmox could use this as a more efficient alternative to VNC (which is slow and weird) or SPICE (for which there are very few non-Linux tools).
Yeah, we at Proxmox are actually evaluating such things, and we hope that IronRDP and QEMU display [0] can be part of a stack that replaces SPICE in the long term, but it will need a bit more time to see how this play out and what exact role it can play in Proxmox VE.
Another experiment is to see if we can add a more modern video encoding to QEMU, as the recently released noVNC 1.6 gained support for H.264 [1]; albeit we naturally would prefer something more open like AV1.
We would be very happy if proxmox were to adopt IronRDP. I see you've already found the work from Marc-André Lureau, he's been doing an incredible job for the IronRDP server side. He's also been working on adding QOI image codec in IronRDP with incredible results. You're welcome to pop in the IronRDP matrix channel: https://matrix.to/#/!opeocvkWZVaLDouykU:matrix.org?via=matri...
Compute overhead of H.264 encoder is non-negligible for a VM host where I want all my CPU cycles to go to user VMs. Datacenter-class Intel CPUs (Xeon) don't include H.264 encoders in hardware. QuickSync circuitry is generally limited to consumer-grade CPUs. Not to mention MPEG licensing issues.
AV1 eliminates MPEG licensing issues, but encoding in hardware is even more limited. Also, AV1 is great for encode-once use cases (e.g. YouTube) since it's heavily geared towards reducing bandwidth requirements vs. encode speed. It's workable for real-time streaming in the lowest settings, but H.264 is still better overall.
Note that this is a bit of a POV thing. For one, CPU cycles handling display also go (indirectly) to your user. And if your users gets a crisper and better picture with less bandwidth due to a modern codec then it can be also seen as win in my book.
Modern CPUs more often have the building blocks included for video encoding, and getting one of those, or a dedicated GPU, probably makes sense if the Users/VMs workload depends on graphical output.
That said, you're definitively also right that it won't be a win for every use case on every hardware, so definitively something to look at more closely, and if it really is worse than the status quo on systems without dedicate GPU and where the CPU has now HW accelleration than the status quo, which I doubt, then adding an opt-out will definitivelys make sense.
Wouldn't it also be a problem that, IIUC, a CPU only has one encoding engine, so you could only have one active stream (at full speed), in a multi-tenant scenario?
Here I'm talking about our integration, which means full audit of what's possible in a simple POC, then a more involved look to see how we can integrate it nicely in the Proxmox VE stack, from low level QEMU to the REST API and its ACL system.
We started evaluation a few weeks ago in combination with the QEMU display work, which is still experimental and was initially announced mid-January this year, so we're still at the POC stage and thus this work will be a bit more time. I did nowhere say that IronRDP will need more time.
And FWIW, without an integrated approach RDP gains you almost nothing over plain VNC if you have only the reduced interfaces and access from outside the VM, so using something like IronRDP earlier on its own would not have gained us much besides an extra component to maintain, that's why we did not checked out integration earlier.
Part of what makes RDP performant compared to VNC is that the RDP server is aware of the GUI state and a lot of compositing is actually done by the client.
A hypervisor just sees the video output from the VM and has no knowledge of the underlying GUI's state, so even with RDP it won't be able to do much better than stream compressed bitmaps.
You're right, but that's why SPICE has a so-called Agent that runs inside the guest and provides a neater integration (like seamless switching from client to host, ad-hoc USB passthrough, sharing the clipboard, detecting video streams (theoretically), ...) and that would be also an option for an RDP approach.
True, but that also goes for VNC (though VNC isn't as optimised as RDP is).
Modern RDP also has the option to turn (parts of) the screen into a simple h.264 stream so the connection doesn't die when you open up a web page with an animated background.
Seems like what we need is a specialized compression algorithm. Since the domain is largely constrained it sure seems like there's some low-hanging fruit here.
Lots of tradeoffs. If you invent a new codec, it's unlikely to make it into hardware for a while (even AV1 encoders are not yet as widely supported) and therefore you will have to do encoding and even decoding in CPU, which takes away resources from the workload. h.264 is still probably the best general purpose codec for real-time desktop streaming - low bandwidth requirements, 444 support, build to lossless, low latency, moderate CPU usage if GPU is not available, and well supported in GPUs for a long time (e.g. even back to Kepler).
RDP is aimed at a different use case than VNC. Proxmox and other virtualization managers (e.g. VMWare, Nutanix) use VNC because you get a stream directly from the hypervisor (e.g. KVM, ESX) which is very useful for low-level debugging. The VNC protocol also has very low overhead (you don't really want h264 encoding CPU overhead on your VM host). VNC is not really intended for remote desktop use cases, which require higher fidelity/frame rate, etc.
So -
* VNC: Low overhead / Low fidelity
* RDP (and other remote desktop protocols, e.g. Frame Remoting Protocol, Horizon Blast, Citrix ICA/HDX): Higher overhead / High fidelity
Low CPU overhead. VNC streams screen grabs with minimal (if any) compression, which results in lower CPU overhead, high bandwidth consumption and low frame rate. This is okay for the use-case of low-level VM debugging that it's used for in context of virtualization management systems, not so great for desktop remoting.
While RDP may run okay on 56k with low color mode for some use cases (e.g. simple Windows admin), it requires significantly more bandwidth and compute overhead (either CPU or GPU) for other more advanced use-cases (e.g. video editing, CAD etc.)
That might practically be where VNC finds usage today, but when it was introduced in the 90s, remote desktops were the intended use case.
"In the virtual network computing
(VNC) system, server machines supply not only applications and data
but also an entire desktop environment that can be accessed from any
Internet-connected machine using a simple software NC." -- https://www.cl.cam.ac.uk/research/dtg/attarchive/pub/docs/at... (1998)
Given the CPU load I've witnessed on VNC servers, I don't think "low overhead" is right these days.
VNC was designed for remote desktop use. All the other streaming features came along later. I don't see why RDP would make for a worse choice here, other than that Windows VM integration would make for an better solution.
RDP used to be far inferior because it was proprietary Microsoft stuff with buggy open source clients and undocumented servers that kept changing stuff around. These days, open source RDP server software is actually quite solid. I don't know if Gnome/KDE leverage the partial update mechanism that makes RDP so useful on Windows (doesn't seem to seeing the performance I'm getting out of VMs), but I find RDP to be a lot more useful for interactive desktop streams than VNC.
> I don't know if Gnome/KDE leverage the partial update mechanism
I guess that would be something for the wayland compositor to manage. Maybe a wayland compositor that is also an RDP server? or maybe they're all like that already?
Also note that there is a critical difference in how they talk to the OS:
* VNC (and other non-RDP solutions like TeamViewer etc): fully independent application, does not change how Windows works because it's effectively just an interactive screen recorder running for your user account.
* RDP: is an actual Windows remote user session that hijacks the computer (so a local user can't see what's happening) and hooks directly into Windows with its own device bindings and login properties (e.g, you can't just click start -> shut down, instead you need to command-line your way to victory).
If you want to remote into a machine that's playing audio without interfering with that, RDP is flat out not an option. Even if you pick "leave audio on the remote", the fact that RPD forces windows to use a different audio device is enough to interfere with playback.
RDP doesn't need to tie into the OS like that. Plenty of ways to run X11 over RDP, for instance. And unlike in VNC, you can actually use the forward/back buttons on your mouse!
RDP in Windows happens to be implemented using some fancy tricks that make it a much better OS for remote work than any Linux distro, but that doesn't mean that's the only possible implementation. Whatever logic can be used to detect block updates in VNC works just as well over RDP. Audio over RDP also works fine on both Windows and Linux so I don't see what the problem would be anywhere else.
As for the shutdown thing, Linux seems to do that too. Makes sense if you use your computer as a terminal server, I guess. I don't reboot my computer over RDP enough to care, really. Still, that's just an implementation choice, nothing to do with the protocol itself.
