I’ve spent the last week playing around with running WASM apps executing WebGL.
I’m exhausted by the modern web. There are so many layers and layers and layers, and none of it works very well!
But WebGL rendering on the web offers a unique and compelling alternative. I can write my frontend and backend in Rust. I can use gui libraries like egui directly if I want or write my own UIs if I prefer.
It’s almost like going back to the bad flash days, but with open web standards.
There are a bunch of downsides (some the article mentions. Accessibility isn’t there yet, the platform is immature so a lot of basic stuff you have to write yourself (form handling, etc), and a big one for me is: text rendering! There appears to be few if any mature text rendering libraries that you can drop into a webgl project and get good crisp text, shaping, and kerning.
15 years ago I was at the same place you are. Believe it or not, I built an entire web-like UI using Tcl/Tk!
But then I hit limits and started re-implementing the wheel. So I bit the bullet and learned all those layers of web development. Never looked back.
When you get to that point, may I suggest you master HTML and submitting forms. You can do a lot without Javascript. Look at Laravel for an easy to comprehend framework (PHP). It is just perfect.
I have also been developing a Wasm UI framework first with canvas2D and Go, but the wasm was too big, and now using Zig and WebGL. But yes, fonts/text is a pain with GPU, all that power and you'd think by now you could upload a font to the GPU and call drawText(x,y). I thought about a shader for that, or custom svg renderer, but probably months of work, and now thinking to layer a Canvas2D over the WebGL and keep the font rendering on the browser side and let it do its z compositing, but probably lose a lot of the performance benefits of a pure WebGL UI... And I already found a bottleneck trying to render animated SVG onto a texture map that results in a copy back each frame, and yet 'video' can be rendered to texture directly by blit... frustrating. Almost thinking bare metal PI fb development would be more fun :)
My ideal stack is plain html, enhanced with HTMX for things like form validation.
Then for more complicated stuff I use self contained web components. That web component might be a terminal, or it might be a text editor, or it might be some kind of video calling widget.
One example, you don't need javascript for a chat interface (aside from htmx). You can use htmx to subscribe to a server-sent-event stream for receiving messages, and for posting them you just use a normal form element. It lets you write a lot more app-like functionality as standard html, and for when you can't do that a custom element is often the answer.
Don't know why web components haven't caught on more.
Having built a few GPU/HTML hybrids in the past, (e.g. https://arcentry.com/) I found that it teaches you a lot about the things you take for granted when developing for browsers and that you now have to manage yourself. This includes for example:
- figuring out when to redraw the screen or sections of it
- organizing your UI as a hierarchical tree that can be traversed
- handling events in general. what did the user click on? What Z order are things in? Can we find out using 2D boxes or do we need to do 3D ray intersection?
- combining UI hierarchies and events into event propagation
- rendering text and images crisply is surprisingly tricky
- blending layers over each other requires care.
- layout flows (margins, paddings, things pushing against other things) need managing.
With some exceptions, the Qt Graphics View Framework satisfies all these goals with very little additional code from the user.
It won't automatically render things as a tree (but the widget system does that for you) or doing layout flow (it's not a layout manager). But it does all the other things so well it's amazing it hasn't been cloned into every other graphics framework.
> figuring out when to redraw (...) sections of it
What happens if you just brute-force and re-render everything when there's a change? I'm all for optimizing but is this really required these days? Even considering laptops and mobile power requirements, rendering off-screen and flipping is lightning fast.
Layout flows are a different matter and can be expensive to compute.
> Even considering laptops and mobile power requirements, rendering off-screen and flipping is lightning fast
I believe it is not speed, but battery life. Most of the time nothing happens on the screen besides some small widget changing like once per second (e.g. clock), doing 59 full rerenders every second for that (or even more with high refresh rate screens) is significantly higher drain.
I think that's right. I started of just rendering everything on every requestAnimationFrame callback - and things rendered smoothly, especially when nothing needed copying to the GPU. But users complained that their laptop fans sprang into action and their battery life plummeted.
Your point is correct and there is another changing thing beyond the clock that's common - the blinking caret in a textbox. Firefox used to have really bad battery usage on macOS for a long time because in order to only change the caret you have to do a lot of things the Apple way.
Oh didn’t think of that! When you are more in control of the hardware then hardware planes could help with that - at least cursors in display managers are handled that way.
Does native OS frameworks have some escape hatch for these slightly changing, constantly damaging elements?
figuring out when to redraw the screen or sections of it
Yes. I did a fun side project where I wrote a client-server protocol and browser for terminal user interfaces and I had to learn all of this. It was a lot of fun though and I learned a lot about browser design and how it manages cookies.
Aren't most native UI toolboxes also GPU-backed? For example Quartz on macOS is. I assume that Windows has something similar, too. Someone else mentioned GTK4 is on Linux. If you want a fast, accessible, user-friendly GPU-back UI, just use the native one.
I keep hearing that learning 2 or 3 native UI frameworks is too much work and too costly, and the answer keeps being to learn yet another new cross-platform UI framework that creates apps that are some combination of slow, lack useful features, and don't look right to the users. At this point, wouldn't it be less costly to just learn the native UI frameworks?
> At this point, wouldn't it be less costly to just learn the native UI frameworks?
People seem to be really unwilling to do this, I don't know why. Maybe it's a language issue. Mind you, the official Microsoft story for how to write a native app is .. messy. But its XAML-based renderers do use the GPU.
> People seem to be really unwilling to do this, I don't know why.
They are usually _horrible_ to write in. Win32 is an archaic mess. Linux doesn't even _have_ a native UI framework, X and the like don't handle widgets. GTK is the least unpleasant but crafting UIs with it is very tedious. KDE is C++ based and interoperates poorly with anything that's not C++. Wrappers exist but they vary in quality. The Apple equivalent is almost pleasant to work with in comparison even though it has its warts (iOS you can deal with it directly in Objective-C no problem, OSX is more idiosyncratic)
And then you have to repeat for all supported platform and rewrite everything.
You could use something like wxWidgets, but it's not pleasant to use either. Language bindings are still problematic.
No? Basically all the GUI frameworks are horrible to write in, and if you haven't reached a point where you're pulling your hair out then your application just isn't complex enough. There is an unavoidable amount of complexity in writing GUIs and using a cross-platform framework means you get to wade through that unavoidable complexity N+1 times: once for each platform and then once for your framework!
Microsoft sticking to XAML is one of their biggest mistake ever WRT native UIs, no wonder it's a dreaded target
Then you have both Google and Apple who invested massively into new languages and framework to facilitate the developments of apps and therefore native UIs
- Compose for Google in partnership with Jetbrains for Kotlin
- SwiftUI with Swift for Apple
Flutter in the way to make things cross-platform, all of them declarative, all of them native, all of them doesn't require learning a dedicated markup language
The problem are the politics between DevDiv (owns VS and .NET) with WinDev (owns COM, WinRT, C++).
WinRT came to be, as WinDev kept redoing Longhorn ideas in COM since Vista reboot, having .NET and C++ as the re-invention of .NET had it been fully based on COM instead of CLR (Ext-VOS original idea).
The new Windows 11 widgets started out as COM and C++/WinRT (which uses a primitive ATL like tooling), because only WinDev could come up with the idea of using COM, C++ and XML inline strings, for what Apple and Goole use nice Swift/Kotlin/Java APIs. Only after some backslash did they come up with the C# version of the API.
So we have WinDev doing WinUI 3.0/WinAppSDK, porting UWP out of the sanbox into regular Win32 stack, with lesser tooling (.NET Native, C++/CX deprecated, no designer), while DevDiv is focused on MAUI and Blazor (apparently putting into Web widgets is also a good idea).
First Microsoft needs to get rid of their internal GUI civil war.
I don’t think they are GPU backed. Fonts are rendered with CPU and draw really slow. The same with graphics primitives like lines, ellipses. A QML scene graph supports only text, images, and rectangles and the text is CPU rendered. Windows still mainly uses CPU for graphics. Video with changing text overlays is a difficult problem to do efficiently.
Does Cocoa draw controls like checkboxes and what not with the GPU?
In GTK 4's case, drawing the widgets, CSS transitions, shadows, etc is all on the GPU. Video from GStreamer arrives as GL textures, same for WebKit integration and therefore can be used with all of the other high-level render-tree snapshotting features.
I’m way out if my depth here, but isn’t certain things faster to render on the CPU still? I believe it is mostly vector graphics - I have seen a few papers showing that it can be done fast(er?) on the GPU but that may not be that widespread yet.
> but isn’t certain things faster to render on the CPU still?
It really depends on the context.
If you look at it in a certain light, fonts are just "vector graphics" and there are ways to do that without using SDF (signed distance fields) and get really high quality precisely positioned fonts fully rendered on the GPU.
An example of that is the GLyphy project and we have a branch which can render fonts using that with GTK 4. It's a bit slower, but better quality and still easily able to handle 144hz full screen text scrolling. It requires some preprocessing before uploading glyph curves to a data atlas on the GPU.
For other shapes, there are options, but given GTK has so few upstream developers, they aren't quite ready for prime-time so they'll likely be part of GTK 5 in the form of "GskPath" if you feel like browsing the branches with that.
Windows 95 rendered its UI on CPU and it was fast. You need GPU for big bitmaps and useless effects. For square with border and text CPU is more than enough. Then you send this small square bitmap to GPU to add shadows and whatnot to it.
Windows 95 doesn't even support Unicode. The level of UI you can draw there is a very small subset of what you can do in say Chrome.
Indeed correct unicode text rendering is one of the difficult problems. It's the same thing that is missing from basically all games. So when people say stuff like games can run at 120fps, how hard can UI be? it's basically the same Windows 95 argument. The UI work being done is a small subset.
AFAIK modern browsers all use GPU accelerated rendering, so using HTML/CSS does give you a GPU-backed user interface.
Also desktop PWAs are IMO underrated and a really good way to distribute a desktop app without having to bundle a whole browser engine with your app. We do this for Construct (www.construct.net) - a complete game development IDE - and I think it's worked out great.
Saying HTML/CSS is "GPU-backed" gives the wrong impression.
What's the point of saying a UI is GPU-backed (which usually means a UI hand-crafted using mesh, texture atlas, shaders, minimal draw calls, etc.) if it's also grouped together with one of the most bloated and inefficient means to output pixels on the screen (HTML/CSS).
I love the web, not hating at all, but this is comparing apple and oranges.
The point is that GPU-backed is faster than not, which is why your browser (and your OS) is doing all those things except maybe hand-crafted meshes (and I don’t agree that hand-crafted meshes is the usual definition of a GPU-backed UI). The GPU-backed UI does benefit from simple things like faster scrolling, especially on high DPI displays, in addition to speeding up the rendering of new pixels for HTML/CSS/SVG/image/video elements. The point is the browsers are actively trying to fix the inefficiency you’re referring to, and it might now be further along than you think.
BTW, the GPU-backed browser UI and GPU-backed OS UI, these come with a bunch of things that someone’s hacked UI doesn’t, namely transparent software fallback when a GPU isn’t available, full compatibility with the UI spec, accessibility support, and tolerance for a wide range of GPUs. People making their own UIs directly almost never do these things (I’m guilty as charged), and as such their UIs are often some combination of more brittle, device dependent, and limited to a narrower audience.
Yes sir, I did in fact read the article. I used to work in game dev. After that I built a WebGL-based whiteboard a lot of people used. Now I help make GPUs. I think it’s great if you agree with the author.
> The browser is GPU-accelerated, it is not what anyone in the industry would consider a GPU-Backed UI.
Now this is a different argument than what you said above. I agree that browsers are what make it GPU-backed, not the HTML/CSS spec per se. But HTML/CSS is becoming effectively a GPU-backed UI in practice, by virtue of the fact that all the browsers use GPUs to render HTML & CSS. This is also true of the UIs provided by Windows and MacOS and Linux - the UI isn’t required to use the GPU, but the OSes will use GPUs for performance when possible. This is all anyone here actually means when they call HTML & CSS a GPU-backed UI.
Is there a meaningful difference once there doesn’t exist a modern browser that doesn’t use a GPU to significantly render web apps?
What I reject is the notion that “GPU-backed” is limited to more than what the words mean literally. If the implementation is using a GPU to render the UI elements, then it’s GPU backed. It doesn’t need to be hand-coded, or a one-off, or using shaders, or guaranteed by the UI spec.
> What I reject is the notion that “GPU-backed” is limited to more than what the words mean literally.
This is exactly what I'm arguing against. We name stuff to represent something. The article very explicitly mention "GPU-Backed UI" as one of the alternative to app that are made with web tech (HTML / CSS). Because it is talking about a very specific way to build UI. And the name they choose is "GPU-Backed UI", which is what anyone would use too.
But you are arguing that a Web Page / Browser / HTML / etc is "GPU Backed UI" and should be group together. But then we have the issue of how we name that type of UI that they are talking in the article which is very obviously something different than what a UI built using a browser would be.
Why do you say it’s a “name”; why are you assuming & asserting the words “GPU backed” are a term of art? Do you have any references to support this assertion? Why do you disagree that UI implemented with a GPU can be called “GPU backed”; aren’t overloaded meanings common enough to consider them possible and reasonable? What, exactly, is your definition of “GPU backed” then? Does it make room for the fact that OpenGL can run on a CPU? I would also argue that Scaleform on the Wii is GPU backed UI, for example.
This thread is now hung on a gate-keeping technicality, and has lost the actual point, which is that, regardless of what you want to call it, html & css rendering have been getting faster due to the increasing use of GPUs, which is narrowing the gap between bespoke UIs and interfaces made with more standard toolkits.
With that definition, could you give an example of something that isn't GPU-backed? As far as I'm aware the GPU is always there at the bottom of the stack, pushing bits to the monitor. Heck in many desktops the monitor plugs directly into the GPU!
Not saying I disagree HTML can be fast, in fact I think it's quite a bit faster than the vast majority of homecooked solutions. But tautological categorization isn't very helpful.
Now this is a distinction worth discussing, you’re right. I do assume that “GPU backed” means you’re interfacing with the hardware acceleration features of the machine, and not only using the frame buffer as pixel storage. This is what I mean when I say all the major browsers and OSes are “GPU backed”, and I’m not trying to sneak in a silly tautological straw man about the GPU being the only connection to the monitor. The browsers and OSes are making increasing use of GPU APIs for hardware acceleration like Vulkan, Metal, and DirectX, and that is what I think of when I read or
say “GPU backed”.
Makes sense. I was going to propose a line at "cpu writes to memory-mapped io pins directly controlling pixel data". The two definitions seem roughly equivalent.
In any case, HTML surely meets the criteria. A much more significant gain is to be had via performance engineering JS than translating it as-stands to rust, wasm, or any other "closer to the GPU" thing. And if comparing two post-optimizations, the HTML/JS will be much easier to debug, style, and make accessible, with practically identical performance.
In fact the flexibility of JS/TS may make the process of discovering a higher performing algorithm/data structure easier, resulting in a faster implementation than had the same amount of effort been dedicated to a worse algorithm in a "faster" language.
Important to note JS can get pretty close to the metal itself with glsl where it matters, while keeping the rest of the app traditional web tech.
> Trying to push the narrative that HTML / CSS is somehow a GPU-Backed UI is almost comical.
To me it rather seems like there are different definitions of what GPU-backed means.
>The browser is GPU-accelerated, it is not what anyone in the industry would consider a GPU-Backed UI.
In which industry?
As a former web dev I would consider any calculation GPU-backed if it is utilizing a GPU.
With that definition and considering html/css is used to build user interfaces, rendering in modern browsers could certainly be called GPU-Backed UI.
If this is a term that already has a different meaning to just the words it is composed of, that's fair to mention.
But it doesn't help a discussion to escalate using wording such as "Trying to push the narrative ... is almost comical"
It is stated right there in the article that "GPU-Backed UI" is one of the alternative to the mainstream way of building app nowadays that are basically just web page.
Now people argue that web page are "GPU-Backed UI".
I get the word might be misleading, but the article is talking about a specific way of building UI using the GPU, which does not include web page, so trying to fit the web into that category makes no sense.
In some loose sense every UI is "GPU-Backed", so how do we name that specific way of building UI that the article is mentioning? They choose "GPU-Backed UI" which if anyone would have told me those words I would immediately know what they are talking about and I would've never included HTML/CSS in that category (even if in some broad loose sense of the words it does fit).
That's a good point. So up to which abstraction level would you consider "GPU-Backed UI" to be the right term? Looks like it doesn't end with low level APIs like OpenGL as the article also mentions e.g. the rust framework https://github.com/emilk/egui
Drawing a line is always hard (when did in our evolution we become sentient?), but HTML / CSS is probably the furthest you can be from that line in modern UI tooling.
As long as the UI or it's underlying engine was built with GPU in mind and not as an after-thought to accelerate it, it can be safe to say it probably fit the "GPU-Backed UI" definition.
All I'm saying is that the article refer to "GPU-Backed UI" as a very specific way of building UI which doesn't include HTML / CSS.
Which is very different than saying that it doesn't use the GPU. Virtually every UI solution is "GPU-Backed", but when I hear the term "GPU-Backed UI", I know exactly what type of UI architecture they are talking about.
Not really it is a bit more subtle than that, unless you do some magic incantations with CSS, like z-ordering or transforms, it may bet that software rendering is used instead.
There are a couple of talks regarding CSS tricks to force GPU redrawing.
I think that was the case several years ago but I thought these days much more of the browser rendering pipeline was automatically GPU accelerated. It's hard to find up-to-date references (would appreciate any links) but for example going to chrome://gpu shows it using GPU acceleration for both compositing and rasterization (including "multiple raster threads"), as well as the usual suspects like video decoding, canvas and WebGL.
Do we know for sure if this is still true in modern browsers?
Browser makers are constantly updating their rendering engines and these pieces of wisdom do go out of date so it's important to verify them every few years.
> We do this for Construct (www.construct.net) - a complete game development IDE - and I think it's worked out great.
Mind sharing what you used for backend / frontend of your desktop PWA? I'm planning to work on a similarly complex UI and have been searching around for best-in-class (or best-trade-offs) technologies today
Any books, tips, tutorials, whatever you can share that inspired your architecture decisions would be ___immensely___ appreciated.
We have an in-house UI library, mainly because we started writing code in the mid-2010s and there was nothing good out there at the time. Maybe that's changed but while I haven't researched it thoroughly, I also haven't come across anything that looks like it does everything ours does. But our UI lib aims to be as simple and straightforward as possible, standard model-view-controller, no virtual DOM, making full use of HTML and CSS features. Personally I view a lot of modern CSS frameworks with suspicion as they all seem to massively overengineer this stuff. We have a whole desktop IDE running on basically old fashioned desktop app engineering style but on top of HTML, and it works great.
Apart from a few extra services on the side, the main app is all static resources so no real backend to speak of. Again, engineered like a traditional desktop app with everything running locally, which also means the UI always responds promptly (no waiting for network) and it all works offline.
Maybe I should write up a blog post about how it all works, but not sure how many people out there want to hear more!
If it's any indication, I posted a link on a related topic earlier today and it made the front page, so I think folks are definitely interested in the topic ;-)
At some point with your own renderer you have to implement a large part of the DOM, styling and rendering layers. And why should a text editor team have to do that?
I think a lot of teams would be better off by learning how to make faster web apps, where there's a subset of HTML/CSS that approaches a tree of GPU textures with layout and style on top. Do things like: use fewer elements, very simple CSS selectors, `contain: content`, `contain: strict`, or `content-visibility: auto` everywhere possible, use shadow roots, use a fast vdom-less template system, and a framework where each component is its own render root, use virtualization for large lists...
If you do that you can also ship a fast web app. And as VS Code shows by popping up in more and more sites, being an actual web app has some very nice distribution and reuse advantages.
That matters very little. By that logic, everything is GPU backed these days and rendered by a compositor, on all platforms.
> At some point with your own renderer you have to implement a large part of the DOM
No, not really. Unless you are saying "DOM" to mean any tree structure. And you might not even need that for your app, depending on its layout.
When people say "GPU rendering" in this context, think user interfaces found in games. They are directly rendered by the game code without any intermediate layers. And very often, without anything that one would even remotely recognize as a DOM (3D games have a scene graph, but that's still different).
> And why should a text editor team have to do that?
Ask Sublime Text and iTerm. They already do that to render their content (and implement native OS dialog controls for the rest).
> I think a lot of teams would be better off by learning how to make faster web apps
True, just because of the amount of effort. _Users_, on the other hand...
Being a webapp and adding GPU rendering are not orthogonal. Figma for instance famously renders it's UI in WebGL.
While I mostly agree that creators of today's web forget that there are a number of simple techniques to render fast pages, there are also a bunch of places where the HTML+CSS+JS web technologies serve us poorly.
* Content Editable is a terrible editing canvas
* Dom and CSS are pretty terrible core animation primitives
* DOM and CSS make for pretty poor interactive graphics capabilities
And in these cases WebGL offers some interesting alternatives to explore.
I remember when I first tried Flutter+Dart and was really put-off by how it rendered to <canvas>, then later versions did use the DOM but looked like the worst case of divitis I’ve ever seen, it even made React.js’s output look handwritten.
But why do people do this? I note that a big problem with the web, as-is today, is how anemic and inflexible HTML’s standard <input> elements are, which forces devs to build their own widgets (often poorly), maybe use a framework like Angular, and then eventually go all-in on web abstractions like Flutter/Blazor/etc. If HTML were better then many (but far from all) of these cases could be avoided - for instance, I don’t understand why HTML still doesn’t have absolutely basic widgets like a combobox, a single-line textbox that wraps, a date-range input, hierarchical drop-downs (optgroup is single-level), defined styling and non-text content of <select>, and so on… (though we did recently get color and range/sliders - though there’s still cross-browser quirks which hamper adoption).
I hated the days of ActiveX like everyone, but when you’re building an internal-only (I.e. IE-only) webpage it was neat that we could pull-in any of the standard Windows controls - and plenty others - into a page and it would just-work. And you got accessibility for free too, which is more than what you get with <canvas>-based frameworks still - or a multi-megabyte JS/WASM bundle download either (or an npm-based build system that breaks every time you update your dependencies…) ]. Okay yes I’m jaded.
Content Editable seems to be quite good these days, if you don't use it as is, but are willing to layer a framework on top of it. I tried Lexical, but am now going to roll my own, but still entirely based on HTML/CSS and ContentEditable. No WebGL required, although I might very well use it for a particular editing component.
> And as VS Code by popping up in more and more sites,
On Linux, VS Code can't even resize without glitching and showing unrendered portions of the window like it's 1999. Nor can it kinetic scroll. Those are two expected highlights of quality GPU-backed software.
Hopefully running in an actual browser would make this better simply because you can choose a browser that isn't Electron.
afaict, VS Code doesn't employ most of these modern techniques for faster layout and rendering. Last I checked they didn't have CSS containment or shadow roots anywhere, and they have very, very large DOM trees.
They're the classic example of jumping from unoptimized HTML/CSS to GPU to try to fix performance as they moved the terminal to be GPU-based, and it doesn't seemed to have helped because that introduced rendering bugs that would have been impossible with HTML.
I only use VS Code as an example of how many places you app can be used if it's web-based.
As others are pointing out, most UIs are hardware accelerated directly or indirectly.
Most "native" apps, are native in the sense that they use some some UI toolkit that renders using the platform specific and typically hardware accelerated ui infrastructure.
With webgl and wasm, those same toolkits can now work in a browser.
As for accessibility, that is indeed a challenge. You would need to use some ui toolkit that supports accessibility features. Those of course do exist, browsers don't have a monopoly on accessibility.
A related challenge is the integrating that with the browser properly so that the native accessibility facilities in your OS can do whatever it needs to do (screen reading, enhancing contrast, etc.). This is not a solved problem. But it sounds like it could be a solvable problem that would require some changes to wasm and webgl.
> most UIs are hardware accelerated directly or indirectly.
To widely varying degrees. Some do a lot of work on the GPU, others use only the rasterization features, some only accelerate the compositing. This unfortunately means that using simple surface level terminology like GPU accelerated doesn't sufficiently describe what is actually happening so that we could make meaningful comparisons between toolkits.
Ultimately I think what's needed is proper benchmarking of performance and measuring of battery life. A toolkit that uses GPU acceleration for a simple step might actually be slower than a full CPU implementation, because there is a cost to syncing your data between CPU and GPU. Also if the whole UI is rather simple, then even a solution that is built completely on the GPU might be slower than CPU, again because of the associated costs. If it takes 2ms to get everything set up on the GPU for a frame, then it doesn't matter if the GPU renders it in only 1ms vs 2ms on the CPU, as total time is now 3ms with the GPU vs 2ms with CPU only. It's a contrived example, but one that could easily happen without a focus on measuring.
Technically, it's a bit of a bait and switch. But one that has largely already happened. It's just that web developers have yet to notice. Javascript at this point is a legacy compilation target for stuff that hasn't been ported to WASM yet.
WASM is getting better with every browser release. Likewise for webgl. It was perfectly fine for doing 3d games many years ago. It's just that we are stuck doing the same react/js/whatever crap that we've been using for the last decade or so. It's just awful how low the standards are for web UI. But it was never good enough for game consoles or mobile.
At this point, it's probably easier to run a browser as a wasm program inside of a modern browser just to render some legacy web app from way back than it is to get that app working with modern browsers. Why even bother? Just run IE 6 in an emulated browser on an emulated windows 95. And probably it would be fine in terms of performance too.
Javascript + css has always been a bit of a cringe fest in terms of capabilities. If you develop a UI that matters for a gaming console, a desktop OS, or anything else, you wouldn't use any of that. It's why native development is a thing for IOS and Android. The phones are fast enough to do web development. It's just that the whole space is too competitive for that to be even worth considering.
So, maybe it's becomes a better browser without that?
Maybe not compared to what browsers were a decade or two ago, but I think the definition has changed. These days browsers are just sandboxes with convenient tooling for finding, downloading, and running arbitrary code. And I don't think that's a bad thing: that's what allows Photopea and Figma to exist in the browser at a moment's notice.
Once you generalize the definition of browser, both ends of the spectrum coexist: from basic HTML pages to fully WebGL rendered WebAssembly apps.
It's already that, but what I got from the OP was to add native controls, like what is necessary to (re-)implement accessability. (That builtin accessability mechanism of the browser itself won't do for various reasons.)
The thing that holds many web UI's back is the web/html DOM and the respective CSS recalculations. You get get close to 1000x performance on web apps if you make sure the DOM will never cause a UI block or re-computation on the layout. The web is great for simple apps but if you're building anything complex you have to play a bunch of tricks to get web UI to work. Going full GPU and building your own controls is certainly possible but I'm hoping there's a middle ground.
This would be way easier if everyone didn't default to using React or even worse React-Native-Web. Not only do you have to deal with handling CSS recalcs (not so bad on its own), you also have to deal with React rerendering the entire DOM when you don't want it to and then the layers of div-soup and absurd style abstractions React-Native-Web uses.
Tauri + Vue/Svelte/Solid/Etc seems like a great way to build a decent desktop and web app, but you just don't have teams picking anything but React React React React.
We could have amazing HTML/CSS performance if we had programmable "HTML to bitmap" caching. But: a) it's not secure if we can render HTML to bitmap and send over network, b) cache invalidation problem - we'd have to invent mechanisms to lock HTML against further changes or track changes and rerun "HTML to bitmap" procedure.
All this is not suitable for web, but for thing like Elector - hell yeah. HTML to bitmap extension would be really welcome. Then you could render UI in WebGL or WebGPU. Adobe Flash had something similar if I recall correctly.
GTK 4 renders with OpenGL, has native a11y built-in, styled with a comprehensive CSS engine, runs on Linux/bsd/Mac/windows, has bindings for almost every noteworthy language…
What does this mean? `a11y` isn't some standard for making your app accessible, in fact it mostly seems to be guidelines for how to make your website more accessible. Is there an `a11y` native app checklist somewhere that Gnome is compliant with (and presumably QT/kde isn't?). Am I missing some protocol Gnome is implementing that QT/kde isn't?
Legitimate question, not snark, I've looked into in the past and I really don't see what Gnome's doing to comply with this that everyone else isn't.
GTK 4 has built-in support for at-spi in it's native controls and provides GtkAccessible interfaces which you can use to make custom widgetry accessible.
> I really don't see what Gnome's doing to comply with this that everyone else isn't.
That's because Sun and GNOME created those layers two decades ago for Xorg.
All the news about GTK were about big projects dumping it in favor of Qt or other, did anything changed significantly? I only seen it used cross platform for extremely basic GUIs for Python apps. It also looked like shit==GNOME on Mac and Windows.
Well new GTK is claiming that they're making it easier to write non-gnome GTK apps. What that looks like to me is that a lot of functionality is moving into a gnome-specific library (libadwaita) and you'll need to find another provider for those gnome specific widgets and functions and stuff. Pop OS makes another add-on library like that.
Meanwhile QT continues to work great and they've largely solved the licensing issues that were a problem in the past. It's also a comparatively stable project and supports targets like android, iOS, rendering directly on to buffers (you can embed it into games). If you're starting a new project I think QT should be your default unless you have a good reason to use GTK.
I'm not qualified to assess the code quality of either project, but I have been impressed by QT for the few things I've needed to build a gui for. Things just work but if you need really specific functionality (like rendering directly to a framebuffer with no display server on embedded linux) it's not too hard to make it work. I'm honestly a bit mystified as to why anyone would choose GTK for a new project, unless they were specifically targeting the Gnome desktop and only the Gnome desktop.
> a lot of functionality is moving into a gnome-specific library (libadwaita)
Sort of?
Mostly just design patterns that are explicit to GNOME. And that is primarily a response to people being (rightfully or not) unhappy that too much GNOME was ending up in GTK itself.
True, I seen also a lot of Qt usage in proprietary Desktop apps and I was surprised. In my work Deskop apps are no longer popular, everything is a website now, SPA.
Qt also supports GPU rendering. Ultimately Qt is based around a software rasterizer that can also be implemented using hardware primitives. It handles the abstraction better than any other framework I've used.
GTK 4 allows widget to snapshot into a render-tree, which has OpenGL, Vulkan, and Cairo for software fallback rendering. It also provides diff'ing for minimal damage and all the other important bits.
I was thinking at Wireshark, LXDE and a very popular Youtube Video where Linux Torvalds explains that GTK is so bad that the GUI app he was creating for diving switched to Qt because even his RedHat friends were unable to help him to do some ncie things in GTK. I never seen some cross platform app switching from Qt to GTK because GTK is better.
I've heard that native a11y support only works on GNOME, although I don't know for sure since Orca is just another GTK application. I believe a11y support was available starting back in GTK 2.x but I don't know ATK's full history.
As I’ve said above, not all windowing systems have a concept of
global coordinates or allow direct positioning of the windowing from
an application. Wayland, for instance, does not.
The purpose of the Wayland project was clearly to impose a set of weird limitations like this one on everyone. The best explanation I can think of is that it's all part of some cult at Red Hat.
Are they wrong? SwiftUI doesn't have a concept of window positioning coordinates, but that doesn't mean that MacOS isn't a desktop OS. They just implemented their windowing stack differently, and honestly I think the Quartz method is a lot nicer.
There's a lot to complain about with Wayland, but this is one of the few changes that seems rooted in logic rather than laziness.
This is literally the dumbest argument ever. It's one extra line of code. If you have the GDK X11 backend, get the XID and call MoveWindow() yourself. If you have a macOS Window, call [NSWindow setFrame]. I don't see why you expect GTK to be your abstraction for insecure UI practices.
The compositor should be in control of this policy, not applications. Which is why that API isn't there any more.
You're right, Linux isn't a desktop OS. Neither is Darwin or NT.
To make an OS you need a system of desktop-providing tools. Windows is an operating system that provides a desktop stack to the NT kernel. MacOS is an operating system built on Darwin and Mach. Following this line of logic, there are dozens of opinionated Linux distributions that qualify for your definition of desktop OS. (RHEL, Fedora, Ubuntu, KDE Neon, et al.)
Linux systems are no different than Darwin or NT ones if you compare like systems. If you spend all day comparing kernel features to OS features, I don't think you're going to make any meaningful discoveries.
Neither of RHEL, Fedora, Ubuntu, KDE Neon have APIs that GUI developers can rely on.
To be short:
First Linux company that will decide to provide stable API similar to `WndProc(window, params)` (Windows) or `class_addMethod(window,@selector)` (Mac/iOS) will win Linux Desktop war - will make the Linux GUI for years to come.
If someone knows such company - please let us know, at least I am willing to participate.
WNDPROC is part of Win32, not the kernel. You seem to have mixed something up.
If we're talking about kernels with windowing capabilities, there are none. If you're asking about OSes with a reliable, documented graphics stack, RHEL is literally what you're looking for. It's the shitty, "Windows-ified" model of desktop development, and almost everyone ignores it for desktop use. You might argue that it doesn't qualify, but you can't argue that nobody has tried it before.
Users and UI developers don't care. It is part of OS and it is always there.
The only things I know for sure are:
1. Desktop OS is not a distribution but set of popular GUI applications that work on that OS natively including tested specifically on it.
2. Lack of stable window and graphics API leads to fractured GUI developers community. Keeping in mind that number of Linux GUI developers is in magnitude of times less than for Windows/MacOS, makes Linux GUI perspectives very grim.
In general "desktop Linux" is not fair to its most loyal users because of these. Users are forced to pay extra price for hardware just to be able to run those Electron applications - no real native options for similar functionality.
> Desktop OS is not a distribution but set of popular GUI applications that work on that OS natively including tested specifically on it.
That's what a Linux distribution is.
> Keeping in mind that number of Linux GUI developers is in magnitude of times less
It's roughly proportional to the number of people using Windows or MacOS for serious networking applications, yes.
> Users are forced to pay extra price for hardware just to be able to run those Electron applications
What are you even talking about anymore? Linux is not responsible for shitty Electron apps, Windows and MacOS is. People wanted a univeral runtime that worked across both operating systems - neither Microsoft nor Apple budged. So the browser it is - it just came with the side-effect that other browser-enabled platforms can also run Electron. Linux is not forcing you to use bad software because Apple and Microsoft can't work together to fix desktop computing.
If you don't like the Linux desktop, fine. Go use 9front or something. It still exists though, you cannot deny it's usability because you don't like the way it does something. I dislike the way MacOS handles it's desktop, but that doesn't mean it's not a desktop OS.
Currently, they are rendered using Pango (thus Cairo) to memory and then uploaded into a texture atlas. This happens once per-glyph/font/size (plus or minus some variants for x/y alignment).
You then use the "color shader" to copy from the texture atlas into the destination FBO to both create the shape and color at the same time. Since you often do this for a whole text editor, you can copy all of the glyphs and color them in a single glDrawArrays()s.
When GLyphy lands, it's a bit different. You preprocess the bezier curves into a simplified format (which doesn't have overlaps) and upload those to a texture atlas (but for general data). Then use GLyphy's shader program to render them, similarly to how the color shader was used.
For more general paths, which is much slower but also still very neat, you would do something akin to Pathfinder.
On windows 8 and newer, microsoft use new framework to create UI for apps. They are all GPU accelerated. Calculator, Settings,..
And it is up to the framework if it support all feature like accesiblity or system spellcheck or whatever.
And on Mac OS
Cocoa use GPU acceleration as well.
As if you want to show something nice and fency you will need GPU acceleration.
But still you can fall back to CPU rendering.
There is Open GL on CPU in MESA. same for DirectX.
It is shower for some fency stuff but good enought for basic UI
Disclaimer: I work at Warp (warp.dev) and wrote the initial How Warp Works blog post (https://www.warp.dev/blog/how-warp-works) that discusses why we chose to write our own UI framework.
This is a great article! Writing a custom UI framework to render on the GPU is a big cost--but it has been worth it for us. It has let us build a UI-heavy terminal while continuing to match the performance of some of the fastest terminals on the market (such as Alacritty). There is certainly an initial velocity hit for new engineers as they have to learn the semantics of our framework, but overall I wouldn't say it has a big detriment on our velocity as a consumer of the framework.
The callout to a11y is a good one--we have basic a11y support but it is by no means robust. There are additional downsides to building your own GPU-rendered UI framework other than a11y that I want to call out: you have to build most of the interactions with the platform yourself instead of getting it for free when rendering using system APIs.
One example of this is properly supporting i18n. A robust framework should support RTL text, positioning IME / the emoji picker, so forth. While implementing these yourself is doable (we have decent support for IME at Warp, for example) it requires an engineer or two to spend a few weeks to implement.
For Warp, this has definitely been worth it, but it hints that the community needs better cross-platform GPU-rendered UI frameworks so people don't need to consistently build them themselves.
I don't want to brag, but I wrote GLterm (for OSX) in 2002 or so, first ever terminal emulator using OpenGL... It was a shareware tho (it was all the rage back then)...
In fact I probably reported and had ATI and nVidia fix bugs that even made it possible to have pixel aligned textures, partial refreshes of GL surfaces etc. It was all basic stuff, but it was always overlooked because "gaming"...
I find it hilarious that the GL UIs are all trendy now, I've done many of them professionally for the last 20+ years. One of first one was Sibelius v2, a desktop software for writing (engraving) musical scores, I converted the windows GD code to a GL backend all the way back then, rendering in textures etc etc.
Interesting terminal, sadly not yet available for other platforms than Mac.
I went to your GitHub (https://github.com/warpdotdev/Warp) in order to see if I couldn't figure out how to build it for Linux, but seems that's like a "issues" repository and I cannot find the source code anywhere, nor is it linked on the main website (https://warp.dev).
I'm guessing at this point that it's on purpose? The source won't be available and it's only a closed source product?
They've talked about it being on the roadmap for a while, but I'm not aware of any recent advancement in getting it to work cross-platform (much less publishing the source).
Indeed, "at this point" they've demonstrated that cross-platform development is a low priority for them. Even if they do ship a Linux client, I feel like it will be a hard-sell when there are so many other terminals treating Linux like a first-class target.
First off: thanks for sharing! Warp is building some real innovation in a Terminal, and it’s been a blast to follow the project.
It’s interesting that you mention this now - I’ve been a Warp user for a few months but just yesterday started seeking out another option because the UI has been feeling sluggish in Warp.
I installed alacritty and kitty, and they both feel insanely fast, comparatively. iTerm is approximately the same as Warp.
I think the difference is significantly smaller than most people would intentionally target with UI paints. But when I press enter, it feels like warp just has a minuscule delay, where kitty has instantaneous printed the results within the same frame that my finger hit enter.
Do you know if there’s a specific # of ms that Warp is targeting for a UI response from user input? I’m not at all an expert in this domain, but am curious if it’s “all in my head”, or if there really is a difference here. Do you have similar stats for other terminals?
Edit:
Only asking because I know it’s hard to get the right balance between extra feature & speed. The autocomplete & syntax highlighting are really awesome features that I love about Warp, and those likely take many more CPU cycles. Warp is far more powerful than kitty in that way. But how many ms does it cost to have those features?
I know the answer isn’t the same for everyone about what the right trade-offs are between features and speed, and I haven’t made up my mind yet which is more important for me, to be honest. Warp is crazy impressive and has been a step up in functionality. Again, really love the work you’re doing!
Would be great for the rust community and also for warp's rep among developers if y'all opensourced the UI framework. The rust ecosystem just doesn't have a good answer when it comes to choosing a UI lib yet.
I was shocked to find that there wasn’t a fully fleshed out text rendering engine for WebGL.
On the one hand I think you could argue that if you have a lot of text maybe html would suit your needs better. On the other hand a lot of UIs just need some text and it’s so east for it to look like junk.
There's an entire layer of mid-level web APIs missing to better connect WebGL and WebGPU to all the other browser systems (like text and DOM rendering).
Ideally canvas with WebGL and WebGPU would sit at the bottom of the browser API layer cake, the DOM at the top and squeezed inbetween public APIs for text, shape, CSS+HTML rendering. But instead we got this weird canvas thing dangling off as another high-level DOM element.
Totally. It would be /amazing/ if I could leverage the browser's existing (and largely excellent) document and text layout engine from /within/ WebGL. Let me build up my UI as I want, and inject a DOM into that.
This is an ongoing trend that will continue until some of the new GUI-backed UIs gain some popularity. What we need right now is a real alternative to Electron based applications, and probably carrying new technologies. I know that we already have alternatives, like Qt, which is an amazing framework, but it doesn't fit all developer's needs and sometimes looks also "too big". I think that many devs are saturated of the HTML+CSS+JS ecosystem and want something new, maybe something less bloated in the sense of the tech specs. I celebrate the numerous projects that are appearing these days, to name a few: egui, iced, imgui, GuiLite, RmlUI, elements. They need more exposure, more collaborators, and more development, they need to grow.
Having said that, I'm working on an alternative GPU accelerated and retained-mode GUI called eepp (https://github.com/SpartanJ/eepp/) while I work on a new code editor similar to Zed called ecode (https://github.com/SpartanJ/ecode). The framework still needs a lot of work but presents some ideas that might be interesting to some. Retained-mode GUIs are a complex beast and it's hard to achieve simplicity (I think immediate-mode GUIs can't be beaten in that regard, but they have their own downsides). So I invite anyone interested in GPU accelerated GUIs to take a look at my projects (online demos are available), I'm looking for collaborators! (sadly C++ isn't very appealing these days, but my code-base is huge and I can't write everything from scratch in a new fancy and cool language).
Flutter is quickly shaping up to be the best way to accomplish this. The new renderers landing on all platforms including web are really nice. I've ignored Flutter until now, but it's getting too good to ignore for much longer.
I absolutely hate how 60 FPS is the goal for these canvas-based solutions. I can render DOM at 240 FPS (and much higher if my monitor had a better refresh rate) no problem, but things like Flutter barely ever even make it to the triple digits. It's an embarrassing case of contentment. 60 FPS should not be considered ideal. It should be considered the absolute bare minimum and really only applicable for resource-constrained systems. If you're not doing absurd amounts of number crunching like raytracing, then you should aiming for 1000 FPS.
Flutter web is in the process of moving to a new rendering pipeline that is C++ compiled to WASM and backed by WebGPU.
I don’t think they are going to have troubles performance wise for long.
This is actually an interesting read I came across by someone from their team recently which poses some hypothetical directions they could take things in the future depending on how the web platform evolves over time.
This is pretty much what Flutter is. Except Flutter has system-specific UI's created by Google so each app can have that native look depending on where it's running.
Impeller. New shader compilation backend. Heroic coding. Metal/Vulkan/WebGPU translators. The bar is pixel perfect across gpus which makes it so challanging. My current favorite open source project: Android & Win ARM ports ;)
> It takes a lot of work to make a usable UI out of triangles, so most applications did not go this path. One big exception was games, which were already heavy GPU users
In the past, Scaleform was close to ubiquitous, at least for large games. Scaleform is Flash-based, so it was a vector graphics engine with a JavaScript-like language. It was presumably faster and less bloated than a browser engine would be, as a game middleware from the Xbox 360 era. But from a developer experience point of view, it closer to a browser engine than to "[making] a usable UI out of triangles."
Scaleform is still pretty popular, but it has become popular to create game UIs with web technologies. You can find libCEF.dll in pretty much every game nowadays. Sometimes it's just used to render the update news or in-game store, but often the entire UI is HTML.
That's weird that the author completely ignores popular projects like ImGui [1] or lvgl, maybe the author wanted to talk about W.I.P. GPU based UIs written in Rust only? In that case, there doesn't seems to be a lot of choice
I had some notes on Dear ImGui (and the prevalence of immediate-mode across other frameworks that it inspired), but ultimately cut it for length because I wanted to focus on applications rather than frameworks/GUI toolkits.
Maybe not a constant 60+ FPS, but if I scroll, and I can tell it's rendering at 20-30 FPS max on my 144 Hz monitor, I'm going to be annoyed. I want smoothness.
> Unlike the UIs provided by the operating system, where you can say "draw a button with the label 'delete'", you have to tell OpenGL how to make a button out of triangles.
Just use Sciter, by default for rendering it uses Direct2d/DirectX (+ Vulkan||OpenGL) on Windows, Metal||OpenGL on Mac and Vulkan||OpenGL on Linux. With support of a11y by the way.
But the UIs provided by the operating system are making DirectX/Metal calls for you... they are GPU accelerated. And they have been for like 16 years now.
> And the same situation on Linux/GTK, system uses Cairo - pure CPU rasterization library.
GTK 4 renders by default using your GPU. The only part that is CPU rasterized is if we don't have an accelerated path for it (rare) and glyphs are first rendered on CPU, uploaded to a texture atlas, and then GPU colored/blit from there on.
While WPF is hardware accelerated, it is not very efficient[1]. I'm not sure if this has changed lately. Always had the impression that WPF was not Microsoft's favorite child.
He can't "Just use Sciter", as you recommended. Proprietary libraries and frameworks might be cost prohibitive or just plain incompatible with most projects.
Thats what every game does - draws a UI directly on the GPU. Game engines like Unity have in-built frameworks for creating UI and then can package the same application for Windows, Linux, playstation, etc.
Many people have (ab)used it to create cross-platform software. It's very high-performance, you just get large applications and load times.
> It takes a lot of work to make a usable UI out of triangles, so most applications did not go this path. One big exception was games, which were already heavy GPU users, and usually had minimal UIs. The 3D renderer Blender is another example.
Only briefly, but seems covered to me.
Edit: now I get it — they mention games, but not game engines as a means of creating other applications.
This is one more reason I am loving godot (nightly compile from main). I see this as a real opportunity to learn an eco system I can do more than just make games with.
> Traditionally, if you wanted to write desktop software, you would use the operating system’s APIs to create a user interface. That’s why Windows apps looked like Windows apps, and Mac apps looked like Mac apps
That is oft repeated but having lived through those times that seems just untrue, at least on Windows which was far more dominant than it is today. So many applications, big and small, were using completely custom UIs instead of native things. From WinAmp to Microsoft Office to all of Adobe/Macromedia suites to all cross-platform apps using GTK/Qt (or god forbid, Java). One particular example is the pre-electron Spotify app, which was wonderful small single-exe thing.
I find it strange how, in an industry where people will eke out minor percentage improvements on conversation rates, "we" are so happy to just intentionally slice off a portion of the potential user base to never be able to use the software.
But because conversion rates are typically low and because of how the math works out for a funnel product, there's a larger business impact depending on which 2.3% is cut out. (Ignoring the negative business impact of being non inclusive/a11y)
12% vs. 10% is a 20% decrease in performance.
100% vs. 98% is a 2% decrease in performance.
I'm not arguing against a11y. I'm just saying, that's why ADA exists. It's not in the business interest without the law. Also it's just good ethics to be accessible.
I'm not sure exactly what this 2.3% is (for "blindness"?), but significantly more than 2.3% of the population wear perscription glasses. Those people have visual impairment and use assistive technologies
Screen readers are not the only assistive technology. Especially on phones, many people use assistive tech to increase font weight or size. Ignoring these options will remove customers.
DIY accessibility is harder than it looks, even if you plan to hook into each platform’s native stack for it. For all its flaws, the web stack standardizes the structure of UI, greatly reducing the dev cost of making accessible apps.
That's something that I have always wondered. And, having built a few of these myself on my free time for personal projects (starting with a university homework), I have my $.02
It is really difficult to do this correctly. The simplest systems to build with are the ones where you have a large visualization window and you only need some knobs to control it.
The things that you think are going to be tricky (say, skeuomorphic knobs) are actually almost trivial to implement, even if compared to standard GUI toolkits(but not browsers). Things that you have taken for granted (textboxes) require months to implement correctly. And you will still find corner cases, be it keystrokes(even without taking different operating systems into account).
It is easy to make it look 'flashy'. It is surprisingly difficult to make it look 'good'. We see some beautiful UIs in games and we think our applications can look the same. They really can't, not without great graphic designers. And no matter how great it looks, it will look 'alien' compared to other applications in the system.
If you need any kind of dynamic layout (which you normally do to handle resizing), you are now looking into an entire system and months of development.
Native OS controls usually have ways to hook up to them from external apps. Not so when you are drawing yourself. So that breaks accessibility but also any chance of automation.
All those drawbacks (and more I haven't touched) aside, there are benefits.
They can be incredibly fast, faster than native OS controls. Rendering a few quads is no problem for any GPU. You can have incredible interactivity(this needs to be explored more by non-Games), since rendering is so fast, faster than the older days of blitting stuff in framebuffers(and a lot of it can be handled by the GPU).
You can also implement transitions that are impossible to do in anything else (maybe except for web browsers). You can transition between 2D and 3D seamlessly.
For color blindness you can just apply a very simple shader over the whole thing.
I think there is a lot of potential hidden by this approach. Think of some the of GUIs imagined by works of science fiction. The main issue is that there's a lot of work to be done before you have anything functional and if you use something out of the box, it will be probably difficult to extend.
That said, I've considered taking this idea to an absurd level and write, say, a text editor on something like Unreal Engine and see how far into absurdity we can extend this idea.
One of the things I liked about Blender UI implementation, eons ago, is that I could fire it up on a very modest old Linux laptop, and it started up in an instant.
What I'd think was the most demanding GUI application on my laptop also seemed the most responsive.
This article doesn't answer why. I can come up with some uses of a GPU backed interface, but you know people will start using this to render their six page web app in due time. Why not just use the already existing GUI frameworks???
I feel there is something wrong with this approach but I struggle in putting my finger on it.
Something about using a level of abstraction at the wrong layer of the stack.
I would expect GPU rendering to be used way deeper in the stack (OS level).
Off topic, but I wish more tech articles were written like this. The writing style is really easy to follow, succinct, plain-spoken, but with enough technical detail. Difficult balance to get right.
Fundamentally? No. It's really just compositing whatever buffers represent the content of the windows, but those may well be wholly or partially rendered on the CPU. Almost nobody does text rendering on the GPU, vector graphics are often CPU-based as well.
Regarding the last point, does this mean developing UIs like this is illegal in some parts of the world? I have no idea asking if someone is more knowledgable than me (e.g. europe).
I don’t think developing a UI that isn’t accessible is ever illegal? I believe the illegal thing is providing a service that doesn’t have “reasonable affordances”. For example, providing a “m.coolco.io” where that subdomain has fundamentally the same service in an accessible fashion is allowed.
Independence of screen resolution and vector graphics do not align very well with rendering via OpenGL. And what if your app makes heavy use of rendering 3D on the GPU? You would better stay away from creating too much workload for the UI.
I’m exhausted by the modern web. There are so many layers and layers and layers, and none of it works very well!
But WebGL rendering on the web offers a unique and compelling alternative. I can write my frontend and backend in Rust. I can use gui libraries like egui directly if I want or write my own UIs if I prefer.
It’s almost like going back to the bad flash days, but with open web standards.
There are a bunch of downsides (some the article mentions. Accessibility isn’t there yet, the platform is immature so a lot of basic stuff you have to write yourself (form handling, etc), and a big one for me is: text rendering! There appears to be few if any mature text rendering libraries that you can drop into a webgl project and get good crisp text, shaping, and kerning.