This model is a preview of Qwen's upcoming Qwen4 architecture. It is a 125B-A6B MoE model, meaning it has 125B total parameters with 6B active at a time, but it also has a 51B parameter engram with it. The engram is basically a lookup table for tokens to my understanding. It allows the model to have access to a much larger amount of info if utilized well.
They said the model is intentionally under-trained since it is mainly for R&D purposes of proving the new architecture. Many people are excited for models in this range as they are a step above the common ~27B models, while not requiring exorbitant sums of money to run like much larger models.
I got downvoted yesterday for complaining about the name/brand confusion from all these Chinese models with similar names all claiming they are the best. While I'm an AI enthusiast, it's being hard to keep track.
It's selection bias. There's a huge number of GC language users, but those who experience problems tend to be the ones who comment.
You're right; the vast majority of people use GC just fine and go about their day. We should update little to none when we see evidence of GC hardship.
It depends a lot on the scale of the system, which often means that those fewer users are actually doing things that are more complex and more valuable - either at the lower end or at the high end.
I sometimes run into browser JavaScript GC issues for browser games specifically but on the JVM side i have not run into any pain points for a long time.
(I run a first-person shooter Minecraft server, and for this and other fast-paced gaming in general pause times under a millisecond or so is generally good.)
There are edge cases where GC issues can crop up, in particular specific "serverless" models (eg AWS lambdas) where the JVM can get "paused" between executions and GC doesn't cleanly run, causing memory to trend upwards until the next cold-start happens (especially if you're running it within a docker container yourself). Limited CPU situations that can exist in these kinds of runtime environments also limit GC in several ways, too.
What's it like running JVM inside serverless? Python gets interesting enough when it pauses waiting for another call sometimes, the JVM seems like it'd introduce its own interesting things :o
Other than some GC issues, meaning we had to give it more memory than we'd otherwise like thereby making them a bit more expensive, it's mostly been fine. The key to ephemeral java is keeping scope limited and not using it when cold starts matter (which can cause a bit of jitter in execution lag). Our use case was asynchronous, so it worked.
We wanted to investigate using the native AWS java lambdas with snapstart and/or using parallel GC, but the place I used them at a year ago was under the tyranny of product having complete control of our backlog, so we didn't get to go that far with it or even more serious tech debt for that matter.
More memory may or may not even be much of an issue depending on your workload. If you can handle the odd OOM error it would have been fine, but for our use case lambdas were so cheap that it didn't really matter, outside of us techies preferring to do it "right". Having the max execution time be less than 15m could also minimize the heap bloat at the cost of more cold starts.
Have you tried real-time audio processing for digital radio communications on a JVM that requires sub-millisecond latency on older, temperature-hardened CPUs?
Does it provide hard latency bounds like Azul does (did?), and are they lower than disk/network latencies on modern hardware?
I moved to c++/rust years ago because those languages do, and tens of milliseconds matter for network services. At the time Java could pause for 10’s of seconds, which was 1000x worse than waiting for a spinning disk to seek.
These days, disks are 100s micros to single digit millis, so I guess if Java GC is finally working 30 years after they “fixed” its performance problems, then I’d want to be able to tune ZGC to not pause the app for more than ~ 500us, max.
This article is from last year, but suggests they’re still off by an order of magnitude:
If you hammer a 100GB-1TB heap in steady state with small allocations for, say, a month at 100% CPU, does it eventually do the typical Java thing, where a major compaction takes the process down for seconds or even minutes, or does it just slow down application requests so it can keep up with load?
> If you hammer a 100GB-1TB heap in steady state with small allocations for, say, a month at 100% CPU, does it eventually do the typical Java thing, where a major compaction takes the process down for seconds or even minutes, or does it just slow down application requests so it can keep up with load?
No. Every garbage collection in Java relocates objects. Compared to malloc, memory fragmentation in long-lived processes is less of a concern. Freelists track only large segments of available memory. The allocator reserves a segment per thread and simply advances a pointer. Small short-lived objects are never visited by the collector. Instead, live siblings are relocated elsewhere before the entire segment is reclaimed.
The above holds true for all of the collectors. The difference is how they deal with concurrent changes to object pointers by the application. Generally, stopping the world uses less net CPU than the memory barriers required by G1GC and ZGC, but most applications are willing to provide more memory and CPU in exchange for shorter pauses.
> Does it provide hard latency bounds like Azul does (did?), and are they lower than disk/network latencies on modern hardware?
Yes and yes (although we need to be more precise when we talk about latencies; see next paragraph).
> These days, disks are 100s micros to single digit millis, so I guess if Java GC is finally working 30 years after they “fixed” its performance problems, then I’d want to be able to tune ZGC to not pause the app for more than ~ 500us, max.
1. You don't need to tune it. The algorithm simply doesn't collect garbage in stop-the-world pauses.
2. Hiccups are sporadic. They should not be compared to the average latency of normal operation. The relevant question is, is ZGC introducing longer hiccups than those a non-realtime kernel would, and the answer is no.
> This article is from last year, but suggests they’re still off by an order of magnitude
The article doesn't measure GC pauses when it shows latencies (it says: "With ZGC on the other hand, the longest GC pause time observed is ~50 microseconds"). It measures the response latencies of some service. Note that allocation stalls also occur with malloc, it just isn't reported conveniently.
Of course, one of the greatest advantages of moving collectors still applies: Under high allocation rates, moving collectors (but not malloc/free!) allow you to compensate for increased CPU spent on memory management by increasing the heap (i.e. if your allocation rate doubles, you can increase the heap and keep the CPU cost of memory management the same). In the past, this advantage translated to higher throughputs compared to malloc/free, but suffered from GC pauses. Those pauses are gone today.
> If you hammer a 100GB-1TB heap in steady state with small allocations for, say, a month at 100% CPU, does it eventually do the typical Java thing, where a major compaction takes the process down for seconds or even minutes, or does it just slow down application requests so it can keep up with load?
No, it does not. You could, of course, construct some pathological cases where you'd have a high allocation rate for long-lived objects which would result in high CPU utilisation by the GC, but it's easier to get into pathological malloc/free cases in C++ (or Rust) than with ZGC. Let me put it another way: no matter your memory management strategy, it's possible to overwhelm it, but the likelihood that a real, "naive" program would overwhelm a malloc/free allocator is higher than it would the JDK's GCs.
It won't bite initially, it will bite when you go to update your version of javac in the future and this becomes the default. Or when you update a library that just so happened to be compiled with a newer version of javac.
This particularly matters when you have something likes this
class Local {
private final ThirdPartyObject tpo;
}
Nope, if the deserializer is initializing that field by directly setting values both by the field and by the internals of the field, it'll be a problem. The fix is to update the deserializer to a newer version. Apache Fury recently fixed this very issue, but it still relies on internal JDK APIs in order to do it's work.
> I’m also pretty sure that cracking final fields is already disabled by default.
Nope. There's sun.misc.Unsafe apis that allow for cracking and modifying those final fields. There are new jdk.internal APIs for doing the same that you'd have to move over to in order to accomplish the same. This JEP is about making final (mostly) meaning final. At very least, it will enforce that final once observed is final with the internal APIs allowing a final field to be set once, just outside the constructor.
sun.misc.Unsafe will eventually be no longer, plenty of methods have already been removed since safer alternatives were introduced.
Java isn't alone in this, as usual due to their relationship, .NET is also clamping down some of this stuff, including changing the memory model around unsafe code blocks.
I know a lot of people like to dump on Firefox, but a number of paywalls, including the Guardian one, completely disappear when you use it. For those that don't, some of them disappear if you use reader mode.
You may be surprised to learn that there is more than one person on HN, and also that among the group of people made up of more than one person, some of them have different opinions than others.
there are a lot of sites that do that, especially in germany. and it annoys me to no end because i believe it is actually illegal. at least that's how i understand it. i might be wrong but i believe that tracking cookies must always be optional, paid or not.
reply