Android Open Source Project and operating systems based on it like GrapheneOS are Linux distributions. The kernel drivers are Linux kernel drivers. The userspace drivers are part of Android's Treble hardware abstraction layer providing forwards compatibility with future Android releases and SELinux-based sandboxing with the drivers split up into isolated processes. Most of the driver complexity is in userspace with most kernel drivers acting as shims between the isolated drivers and the hardware. It's done that way for practical reasons on Android but it's good for security.
Treble's compatibility system isn't very relevant to us right now. There's a new Android release every month: a monthly, quarterly or yearly release. The devices we currently support (Pixels) receive each of these updates officially. Most Android devices do not get the monthly or quarterly updates, only the yearly ones. Other devices rely on partial backports of security patches (Android Security Bulletins) to an initial release, which are provided for ~3-4 years after the initial yearly release. If we supported typical Android devices with that situation, then we'd at least partially rely on Treble to provide the latest OS version. Pixels are currently the only devices meeting our hardware security requirements listed at https://grapheneos.org/faq#future-devices. Having proper updates for 7 years from launch is just part of that, most of the requirements are for hardware-based security features like the secure element features, hardware memory tagging, pointer authentication, USB controller support for blocking new connections and disabling USB data, etc.
GrapheneOS uses the 6.1 and 6.6 long term support branches of the Linux kernel with 6.12 as the next one that's likely going to be used to replace 6.6 for on-device virtual machines and the emulator with Android 16.
> But they're drivers that are not upstreamed and which therefore make it hard to move to a newer kernel, right?
It's no harder than it would be dealing with them if they were upstream. Google ports all the Pixel drivers to newer LTS branches and a recent branch of the mainline kernel themselves.
With the recent Android 15 QPR2 release last month (March 2025), 6th/7th generation Pixels moved from the 5.10 branch to 6.1 and 8th generation Pixels moved from 5.15 to 6.1. 6th, 7th, 8th and 9th generation Pixels share the same kernel and kernel driver source tree now. They have them ported to 6.6 and mainline kernels too, it's just not ready to ship yet. 6.6 is used for virtual machines run on the devices and the emulator. 6.12 will be what's used for Android 16.
You can see at https://android.googlesource.com/kernel/google-modules/aoc/+... that they still port the 6th gen Pixel drivers to newer mainline kernels. They're ported to 6.13 and probably moving along to 6.14 soon. It doesn't mean they're going to ship them. Only LTS branches make sense to ship and they need long term stabilization first. The likely model is going to become ~12 months of stabilization and then ~12 months of usage since LTS kernel branches are moving back to 2 years of support. It was essentially increased to 6 years for 6th gen Pixels having 5 years of support, but they moved along to upgrading to newer LTS branches for 8th gen Pixels moving to 7 years of support. Greg KH works for and with Google on the LTS kernel maintenance / testing so it's actually quite Android centric rather than server centric. Long term support desktop/server distributions historically maintain their own LTS branches so they're not really the ones involved in that for the most part.
Drivers that are upstream don't actually get much free maintenance and testing. People making API changes blindly update the drivers without testing them. They get lots of updates and refactoring, but not much ongoing maintenance. Maintainers still need to deal with it. It's very common for upstream drivers to continuously regress. They can go a long time without actually working properly across releases due to how few people use them. Most people are using distributions with frozen package versions like Debian, not a rolling, and people using a rolling release like Arch Linux can use an LTS kernel branch to avoid a lot of this. The drivers for embedded hardware and things not used much by enthusiasts on desktops often break without it being noticed.
Android made a Generic Kernel Image system with ABI stability for drivers which does not benefit Pixels because they update the drivers to match the latest GKI kernel they ship. Similarly, Pixels don't really need the Treble HAL ABI forwards compatibility because they update all the vendor code to the latest monthly, quarterly and yearly OS releases anyway. It's helpful that drivers don't need to add all the new standard features to keep providing working support for new OS versions though. It's nice having it all nearly all neatly standardized and stable. We like Treble because of the sandboxing. The forwards compatibility benefits are largely unrealized because the vendors needing it aren't doing updates much anyway. Qualcomm is moving to 8 years of full update support for Snapdragon to partially match Pixels anyway.
First: I did momentarily forget that you're only targeting Pixel devices that are actively getting updates from Google. In light of that, so long as Google stays on top of maintaining those devices, yeah in your case that's probably fine. I'm somewhat accustomed to less responsible vendors and a lot of my views are shaped by that.
That said, I'm not wholly convinced that Google's downstream kernels are as good as running from upstream. AFAICT, for example, GrapheneOS is currently shipping a kernel for the Pixel 6 that's 3 minor versions behind. Trying to track things through the Android development process is... unintuitive... to me, so forgive me if I've missed something... If I go to https://github.com/GrapheneOS/device_google_raviole-kernels_... , grab a .ko file that shows as being committed yesterday, and run modinfo on it, I get a version of 6.1.131 which https://cdn.kernel.org/pub/linux/kernel/v6.x/ChangeLog-6.1.1... says is from March 13 and has been superseded by 6.1.134 at this writing (from checking https://www.kernel.org/ ). Contrast https://archlinux.org/packages/core/x86_64/linux-lts/ which says that Arch's LTS kernel is at 6.12.23 which is the latest of that line. EDIT: Actually, the much better comparison is that Debian 12 is shipping 6.1.133 according to https://packages.debian.org/stable/kernel/linux-image-arm64 now. So the super stable slow moving distro is in fact still ahead of Android, even slightly lagging as it is.
As to breakage/testing... Yes, someone has to test new versions. Ideally that'd be a CI farm flashing and testing devices, but I appreciate that it's not exactly a trivial problem. Nonetheless, if that results in Graphene not shipping the latest bugfixes, I feel like that's an extremely awkward position to be in.
Having the kernel that is being shipped to customers be less than 1 month behind is not that bad.
>So the super stable slow moving distro
Slow isn't referring to the speed Debian takes for hot fixes. It is referring to how long Debian stays hot fixing packages over updating to the latest version.
You can see that we're on the latest 6.1.134 and 6.6.87 now because Greg KH caught up the Android GKI LTS branch to the kernel.org LTS branch. That will be included in our upcoming OS release in the next couple days.
We ship GKI LTS updates quickly but the GKI LTS branch can sometimes lag behind by up to a few weeks as it was doing when you looked. You happened to look at the maximum point it lags behind. It should really be kept closely in sync with the kernel.org releases but it's almost entirely Greg KH dealing with it and he deals with a lot of other stuff too.
Linux 6.12 is not better for security than Linux 6.1 or Linux 6.6. It doesn't work that way. Newer kernels have substantially more attack surface and complexity. They also have tons of new bugs. Bug density is far higher in new or recently changed code. Bug density drops over time. However, backporting patches gets increasingly less complete for the older branches. There's a balance between the new and older branches. 6.12 is far too new to be a reasonable choice. Google already ported Pixels to Linux 6.12, etc. It's not what is shipped because it's full of serious bugs. Separately from that, if you believe using an LTS release and shipping the latest revisions means you avoid regularly having serious regressions, that's very wrong with the Linux kernel.
Pixels only recently started moving to new LTS releases and will likely be moving to a new LTS release each year going forward. Moving the older generations to 6.1 to match current devices was done in March 2025. They'll likely move along together to a new branch each year going forward.
Linux kernel LTS revisions are nothing like LTS revisions of most other projects. They're largely untested patches blindly applied by the LTS maintainers based on patches to mainline being marked for stable backports. If the patches apply cleanly, they ship. If they don't apply cleanly, they largely don't ship. Whether it works is an open question.
> GrapheneOS is currently shipping a kernel for the Pixel 6 that's 3 minor versions behind
That's not quite right.
We're shipping the latest Linux 6.1 and Linux 6.6 GKI LTS branch releases from Greg KH. They're currently in between 2 upstream revisions upstream, not on a specific one. The devices all use both 6.1 and 6.6, not just 6.1. They use 6.1 for bare metal and 6.6 for virtual machines. Even the Pixel 6 has been ported to 6.13 by Google but that doesn't mean that's a stable enough kernel ready to ship.
The Android kernel branches also have a bunch of backports not included in the kernel.org LTS releases, including security patches and improvements they decided were important. Google does their own backporting and fixes in the GKI branch and Greg KH merges those into the GKI LTS branch. The kernel branch we use is the combination of the Google GKI backporting/fixes with the kernel.org backporting. The kernel.org LTS releases are far messier than you realize, and combining these things is messy too.
Linux LTS kernels are not very well tested and have tons of regressions. Quickly updating to the new LTS versions is problematic and we regularly encounter major regressions, especially in certain areas like f2fs and USB. We still update right away to the new GKI LTS versions. We're currently on the latest GKI LTS releases for each branch.
You'd have to ask Greg KH why there are still delays despite Google supporting it. It still seems to be him doing most of the kernel.org LTS and also GKI LTS work by himself, without nearly as much review or help by others as you would think. This is also tied into the severe regressions regularly happening with the LTS releases. Those can be security regressions too. Immediately updating to them is not necessarily a great idea with how much goes wrong at the moment.
They unfortunately sometimes lag behind the kernel.org releases. We used to merge the latest upstream kernel.org LTS releases ourselves but Greg KH convinced us we don't need to do that anymore and should just use the GKI LTS branch instead. We're not completely happy with it since it's not fully kept in sync but we're using our resources elsewhere at the moment.
> Actually, the much better comparison is that Debian 12 is shipping 6.1.133 according to https://packages.debian.org/stable/kernel/linux-image-arm64 now. So the super stable slow moving distro is in fact still ahead of Android, even slightly lagging as it is.
Debian is usually further behind than Greg KH's GKI LTS branch. Comparing at a snapshot in time doesn't mean much. GKI LTS branch should really be kept in sync but the GKI ABI stability system makes maintenance hard and is entirely worthless for Pixels. We would prefer if the whole GKI system did not exist. For Pixels, the kernel image and all the drivers are built with the same kernel source tree for Pixels so the whole system for driver ABI compatibility is just making things more complex and worse.
> As to breakage/testing... Yes, someone has to test new versions. Ideally that'd be a CI farm flashing and testing devices, but I appreciate that it's not exactly a trivial problem. Nonetheless, if that results in Graphene not shipping the latest bugfixes, I feel like that's an extremely awkward position to be in.
We do heavily test them. Our community helps with it. We OFTEN find regressions in the new LTS kernels. It often takes months before the issues we find and work around get fixed upstream. It's worth noting that due to mistreatment we've largely stopped helping them except for firmware, hardware or things we don't want to maintain downstream for some reason. It would be better if everyone collaborated on maintaining LTS kernels but instead it's largely 1 person and a couple others doing it with support from Google for testing, etc.
Right, no, I wasn't particularly objecting to 6.1, I was pointing at the patch level on it. I would personally take [quickly checks kernel.org] 5.15.180 (latest 5.15) over 6.1.130 (not-latest 6.1), because I'm more concerned with bugfixes than feature releases at this point. If the GKI LTSs are backporting fixes, that may well cover it, although that starts to veer into making me nervous because I rather agree with
> Linux kernel LTS revisions are nothing like LTS revisions of most other projects. They're largely untested patches blindly applied by the LTS maintainers based on patches to mainline being marked for stable backports. If the patches apply cleanly, they ship. If they don't apply cleanly, they largely don't ship. Whether it works is an open question.
I'm also not super fond of frankenkernels. And... I'm confused how you feel about them? If backports suck, shouldn't you want to be chasing the very bleeding edge? I wasn't originally intending to argue that everything had to ride the very latest and greatest, but if backporting is inherently fragile and bug-prone, shouldn't you want to be on the very latest stable version (so as of this writing, 6.14.2)?
We want to be on an LTS branch, but we'd prefer to be on a newer LTS branch. We would currently want to be on 6.6 with a near future move to 6.12 once it was stabilized enough. However, we would much rather be on what Google is heavily testing than using a kernel they're not using on their CI infrastructure and production devices.
Pixels moving 6th, 7th and 8th gen devices to 6.1 happened in March 2025. It's the first time they've moved to new LTS branches. It's likely going to move to using a newer LTS release where it would currently be using 6.6 and then moving to 6.12 after the next one comes out. We expect they move to having around a year to stabilize the new LTS and then use it for around a year before moving to the next. That fits nicely into the new 2 year lifespan for LTS kernels. This is a transition period. Once the longer than 2 year LTS kernels are gone, the quality of the LTS kernels will rise because there won't be as many to maintain. There are currently too many combined with too few people working on it. Greg KH having to handle both the kernel.org LTS and GKI LTS doing a huge portion of the work is clearly a problem. We'd also like to see the end of GKI ABI stability but that's highly unlikely. Yearly moves to new LTS kernels will at least make it a lot better.
> I would personally take [quickly checks kernel.org] 5.15.180 (latest 5.15) over 6.1.130 (not-latest 6.1)
Latest 5.15 has far fewer fixes backported for the same time periods than 6.1. The missing fixes in 5.15 are far more than a couple minor revisions. Similarly, 6.6 has more than 6.1 for the same time period and 6.12 has more than 6.6.
> And... I'm confused how you feel about them? If backports suck, shouldn't you want to be chasing the very bleeding edge? I wasn't originally intending to argue that everything had to ride the very latest and greatest, but if backporting is inherently fragile and bug-prone, shouldn't you want to be on the very latest stable version (so as of this writing, 6.14.2)?
Linux kernel code quality, review and testing is quite low. The bleeding edge kernels are nearly unusable in production for this use case (users running all kinds of software, using all kinds of different Bluetooth, USB, etc. accessories and so on while caring deeply about battery life) and have a ton of newly added security bugs which aren't found and fixed yet. We think that's a much bigger issue. We happily use Arch Linux for a lot of stuff but we use the LTS kernel package which is 6.12 at the moment.
If LTS quality was increased substantially, then we'd want to be on the latest LTS branch a while after the initial release, i.e. 6.12.15+ or so. However, at the moment, some serious regressions take them so long to find that it's still too new. We have high stability requirements where we can't have niche USB functionality, Bluetooth, video recording, etc. functionality regressing. The out-of-tree drivers are an area we don't have as much pain with since they're nearly all made for Exynos / Pixels and the drivers from the vendors so changes actually get tested well. The regressions are in the upstream code. More stuff coming from upstream would make LTS updates more, not less, painful, other than the GKI ABI stability nonsense we don't want.
Ton of information here.
But was hoping to find out how mobile Linux distros (mobian, postmarket, pureos, new ones) could support newer phones, like these Pixels. I still don't know after reading this thread. :-D
I don't want to use Android, I want to use Linux and Phosh or similar. But so far, the supported hardware is junk.
GrapheneOS is a mobile Linux distribution. It's not systemd and GNOME which makes it a Linux distribution but rather the Linux kernel. There's nothing stopping people from running a traditional desktop Linux software stack on the same hardware we support. That doesn't interest us since it would be a massive privacy and security regression from the Android Open Source Project. It would also give a lot of usability, robustness and the huge mobile app ecosystem including a large number of open source mobile apps.
The Linux kernel is increasingly the elephant in the room when it comes to security and hasn't experienced anything like the massive progress made in Android's security in userspace. Piling on many more exploit mitigations to the Linux kernel won't really change this. We need to do a lot more work on it than we already do.
GrapheneOS has hardware virtualization support, which is going to be one of the ways to avoid depending so much on the Linux kernel's fragile security. The main usage for it in GrapheneOS will be running nested GrapheneOS instances for better sandboxing rather than running other operating systems. Android supports using the virtualization support to run other operating systems via the Terminal app and we have support for GUI applications, speaker, microphone and opt-in GPU acceleration backported to the Terminal app. The main use case for that app will be running desktop applications from other operating systems for the desktop mode. Windows 11 support would be a compelling addition to it and we may implement that in the next year or so.
I'd like a mobile OS where I can reuse my existing knowledge. Write software for it with Rust, Python, pipewire, systemd, Wayland, etc. Login with ssh.
No interest in Android apps or Windows (hah). (Though maybe I'll try Waydroid one of these days.)
I don't know anything about your distro, not the graphics stack or what the package manager is or even if there is one? Meanwhile my starlite tablet is awesome because it works just like my desktop Fedora or Mint, though I installed Phosh on it.
Security is nice, but not before there is even a single feasible device on the market. Librem is just barely limping along, and I mean barely with a five year old handset that was obsolete when it debuted.
If your kernel is so advanced it really should be upstreamed, so these other distros could use it and support new Pixels. Y'all working together with other mobile projects would be so much better than the current surveillance dystopia we are currently living in. Maybe it's hard, but it is incredibly important. I can help though have limits.
GrapheneOS is an Android/Linux distro, not GNU/Linux or musl+busybox/Linux; I suspect most of their security work isn't portable to the unixy Linux distros.
I read this whole thread again and it seems that the answer to the original question, is: the pixel drivers are maintained outside of the kernel tree by Google, and not these Graphene folks.
Sounds like I should complain to them instead. Yet they are known for being unreachable.
There are some huge drivers from companies like Broadcom and Qualcomm. There's still a massive amount of kernel driver code along with the massive amount of core kernel code. Android is the main driving force behind the push for Rust for Linux kernel drivers because Google wants all these SoC and device drivers to start being written in Rust for security reasons but also for stability too. Driver memory corruption bugs are a huge source of both security and stability issues. A majority of new code in Android releases since Android 13 has been in memory safe languages. The Linux kernel is increasingly the elephant in the room with the monolithic kernel driver (zero isolation / sandboxing within the kernel) combined with memory unsafety. It's largely the opposite of Android's extensive work to improve security in userspace. They've worked hard on getting exploit mitigations into the kernel but those tend to be much weaker than they are in a lot of userspace (browser renderer processes have similar issues).
For entire classes of applications, you can treat Linux as a black box. Things like syscalls, /proc & /sys, are all incredibly stable. So that's what Go does with its syscall package; it completely sidesteps libc, ld.so, and whenever it can, it just produces static builds - at least on Linux.
They tried to get away with it on OpenBSD, macOS, etc and got their hand chewed off.
There are a few reasons I avoid Go, and their syscall pacakge is on the list. It breaks a bunch of tooling that requires LD_PRELOAD or static linker interposition. (One example: Interposing on the clock to test timeout logic.)
I wish they had an option that just used libc. Maybe, someday someone will add a target architecture/port that just uses posix. I’d prefer that in Linux too.
That's not Go-specific, it's just how static linking works. As nolist_policy advised, it's more useful and precise to intercept programs at the syscall (rather than library call) boundaries. Programs are free to do all sorts of insane stuff, there was e.g. a Scheme interpreter that kept growing the stack until it caught SIGSEGV, at which point it'd run a compacting GC and reset the stack pointer. ¯\_(ツ)_/¯
Regarding LD_PRELOAD, I highly doubt that this is required by POSIX. macOS (which is a certified UNIX) uses DYLD_INSERT_LIBRARIES instead. OpenBSD (which is known for their pedantic documentation) uses LD_LIBRARY_PATH, doesn't mention any standards, and refers the reader to SunOS 4.0. If this is somehow standardised, I'd love to read the actual document.
Treble's compatibility system isn't very relevant to us right now. There's a new Android release every month: a monthly, quarterly or yearly release. The devices we currently support (Pixels) receive each of these updates officially. Most Android devices do not get the monthly or quarterly updates, only the yearly ones. Other devices rely on partial backports of security patches (Android Security Bulletins) to an initial release, which are provided for ~3-4 years after the initial yearly release. If we supported typical Android devices with that situation, then we'd at least partially rely on Treble to provide the latest OS version. Pixels are currently the only devices meeting our hardware security requirements listed at https://grapheneos.org/faq#future-devices. Having proper updates for 7 years from launch is just part of that, most of the requirements are for hardware-based security features like the secure element features, hardware memory tagging, pointer authentication, USB controller support for blocking new connections and disabling USB data, etc.
GrapheneOS uses the 6.1 and 6.6 long term support branches of the Linux kernel with 6.12 as the next one that's likely going to be used to replace 6.6 for on-device virtual machines and the emulator with Android 16.