This sounds very like Licklider's essay on Intelligence Amplification: Man Computer Symbiosis, from 1960:

> Men will set the goals and supply the motivations, of course, at least in the early years. They will formulate hypotheses. They will ask questions. They will think of mechanisms, procedures, and models. They will remember that such-and-such a person did some possibly relevant work on a topic of interest back in 1947, or at any rate shortly after World War II, and they will have an idea in what journals it might have been published. In general, they will make approximate and fallible, but leading, contributions, and they will define criteria and serve as evaluators, judging the contributions of the equipment and guiding the general line of thought.

> In addition, men will handle the very-low-probability situations when such situations do actually arise. (In current man-machine systems, that is one of the human operator's most important functions. The sum of the probabilities of very-low-probability alternatives is often much too large to neglect. ) Men will fill in the gaps, either in the problem solution or in the computer program, when the computer has no mode or routine that is applicable in a particular circumstance.

> The information-processing equipment, for its part, will convert hypotheses into testable models and then test the models against data (which the human operator may designate roughly and identify as relevant when the computer presents them for his approval). The equipment will answer questions. It will simulate the mechanisms and models, carry out the procedures, and display the results to the operator. It will transform data, plot graphs ("cutting the cake" in whatever way the human operator specifies, or in several alternative ways if the human operator is not sure what he wants). The equipment will interpolate, extrapolate, and transform. It will convert static equations or logical statements into dynamic models so the human operator can examine their behavior. In general, it will carry out the routinizable, clerical operations that fill the intervals between decisions.

https://www.organism.earth/library/document/man-computer-sym...

Wow. fascinating insights he had.

e.g. (amongst many others) Desk-Surface Display and Control: Certainly, for effective man-computer interaction, it will be necessary for the man and the computer to draw graphs and pictures and to write notes and equations to each other on the same display surface. The man should be able to present a function to the computer, in a rough but rapid fashion, by drawing a graph. The computer should read the man's writing, perhaps on the condition that it be in clear block capitals, and it should immediately post, at the location of each hand-drawn symbol, the corresponding character as interpreted and put into precise type-face.

This is essentially what any relu based neural network approximately looks like (smoother variants have replaced the original ramp function). AI, even LLMs, essentially reduce to a bunch of code like

    let v0 = 0
    let v1 = 0.40978399*(0.616*u + 0.291*v)
    let v2 = if 0 > v1 then 0 else v1

    let v3 = 0
    let v4 = 0.377928*(0.261*u + 0.468*v)
    let v5 = if 0 > v4 then 0 else v4...

Thats a bit far. Relu does check x>0 but thats just one non-linearity in the linear/non-linear sandwich that makes up universal function approximator theorem. Its more conplex than just x>0

Multiply-accumulate, then clamp negative values to zero. Every even-numbered variable is a weighted sum plus a bias (an affine transformation), and every odd-numbered variable is the ReLU gate (max(0, x)). Layer 2 feeds on the ReLU outputs of layer 1, and the final output is a plain linear combination of the last ReLU outputs

    // inputs: u, v
    // --- hidden layer 1 (3 neurons) ---
    let v0  = 0.616*u + 0.291*v - 0.135
    let v1  = if 0 > v0 then 0 else v0
    let v2  = -0.482*u + 0.735*v + 0.044
    let v3  = if 0 > v2 then 0 else v2
    let v4  = 0.261*u - 0.553*v + 0.310
    let v5  = if 0 > v4 then 0 else v4
    // --- hidden layer 2 (2 neurons) ---
    let v6  = 0.410*v1 - 0.378*v3 + 0.528*v5 + 0.091
    let v7  = if 0 > v6 then 0 else v6
    let v8  = -0.194*v1 + 0.617*v3 - 0.291*v5 - 0.058
    let v9  = if 0 > v8 then 0 else v8
    // --- output layer (binary classification) ---
    let v10 = 0.739*v7 - 0.415*v9 + 0.022
    // sigmoid squashing v10 into the range (0, 1)
    let out = 1 / (1 + exp(-v10))

i let v0 = 0.616u + 0.291v - 0.135 let v1 = if 0 > v0 then 0 else v0

is there something 'less good' about:

    let v1  = if v0 < 0 then 0 else v0 

Am I the only one who stutter-parses "0 > value" vs my counterexample?

Is Yoda condition somehow better?

Shouldn't we write: Let v1 = max 0 v0

The relu/if-then-else is in fact centrally important as it enables computations with complex control flow (or more exactly, conditional signal flow or gating) schemes (particularly as you add more layers).

I'm not sure that's the fully right mental model to use. They're not searching randomly with unbounded compute nor selecting from arbitrary strategies in this example. They are both using LLMs and likely the same ones, so will likely uncover overlapping possible solutions. Avoiding that depends on exploring more of the tail of the highly correlated to possibly identical distributions.

It's a subtle difference from what you said in that it's not like everything has to go right in a sequence for the defensive side, defenders just have to hope they committed enough into searching such that the offensive side has a significantly lowered chance of finding solutions they did not. Both the attackers and defenders are attacking a target program and sampling the same distribution for attacks, it's just that the defender is also iterating on patching any found exploits until their budget is exhausted.

Then the point still stands, this makes things even worse given that it's adding its own hallucinations on top, instead of simply relaying the content or idealistically, identifying issues in the reporting.

Being tall doesn't automatically make you good or dominant at basketball, you can even be too tall. Wemby might just be at that threshold, but the unusual thing about him is his dexterity despite his height; such maneuverability and flexibility is trainable. I hear he also spent the summer training, likely harder than most.

No, but being short is completely disqualifying, so being tall is certainly a component of the physical traits that make you good at basketball. If you're 5'2" , it doesn't matter what other gifts you have -- you will not be a pro male basketball player today.

In tennis, being too tall is clearly net bad, but being too short is also definitely bad. 80% of male pro tennis players are 5'10" - 6'4", which is certainly not the statistics of the general population.

Absolutely it's a combination of many factors. However height is undeniably very important. Wemby at 5'5" won't be as impressive a player, no matter how much he trained.

Dennis Rodman is a famous counterexample (tall, no particular talent, became an All-Star for rebounding and shot-blocking)

It's an artifact of post-training approach. Models like kimi k2 and gpt-oss do not utter such phrases and are quite happy to start sentences with "No" or something to the tune of "Wrong".

Diffusion also won't help the way you seem to think it will (that the outputs occur in a sequence is not relevant, what's relevant is the underlying computation class backing each token output, and there, diffusion as typically done does not improve on things. The argument is subtle but the key is that output dimension and iterations in diffusion do not scale arbitrarily large as a result of problem complexity).

You are right and the idea of LLMs as lossy compression has lots of problems in general (LLMs are a statistical model, a function approximating the data generating process).

Compression artifacts (which are deterministic distortions in reconstruction) are not the same as hallucinations (plausible samples from a generative model; even when greedy, this is still sampling from the conditional distribution). A better identification is with super-resolution. If we use a generative model, the result will be clearer than a normal blotchy resize but a lot of details about the image will have changed as the model provides its best guesses at what the missing information could have been. LLMs aren't meant to reconstruct a source even though we can attempt to sample their distribution for snippets that are reasonable facsimiles from the original data.

An LLM provides a way to compute the probability of given strings. Once paired with entropy coding, on-line learning on the target data allows us to arrive at the correct MDL based lossless compression view of LLMs.

Unless you're also writing your own graphics and game engine from scratch, if you're making a truly novel and balanced game, then it should not be possible to crank out code with AI. When working in engines, the bulk of the work is usually in gameplay programming so the fact that its code is so predictable should be concerning (unless the programming is effectively in natural language). Not spending most of your time testing introduced mechanics, re-balancing and iterating should be triggering alarm bells. If you're working on an RPG, narrative design, reactivity and writing will eat up most of your time.

In the case you're working as part of team large enough to have dedicated programmers, the majority of the roles will usually be in content creation, design and QA.

Why would the proportion of high quality games increase? The number yes, but I expect not the proportion. Lowering the entry barrier means more people who have spent less time honing their skills can release something that's lacking in polish, narrative design, fun mechanics and balance. Among new entrants, they should number more than those already able to make a fun game. Not a value judgement, just an observation.

Think of the negative reputation the Unity engine gained among gamers, even though a lot of excellent games and even performant games (DSP) have been made with it.

More competitors does also raise the bar required for novelty, so it is possible that standards are also rising in parallel.

We had shovelware games 25+ years ago (and probably 40 years ago, though I suspect the lack of microcomputers limited that). There were bargain-bin selections (literally bins full of CDs) that cost a few bucks and were utterly shite. I suspect the target audience was tech-unaware relatives who would be "little Johnny likes video games, I'll get him one of these...". Most of them were bad takes on popular games of the time.

Unity + Steam just makes this process a bit easier and more streamlined. I think the new thing is that as well as the dickwads who are trying to rip people off, there are well-intentioned newbie or indie developers releasing their unpolished attempts. These folks couldn't publish their work in the old days, because making CDs costs money, while now they can.

But why isn't this merely papering over a more fundamental issue with how these models are "aligned"? LLMs are, for example, not inherently sycophantic. kimi k2 and o3 are not, and Sydney, mentioned in the blog post, was most decidedly not.

In my experience, the issue of sycophancy has been longest in the Anthropic models, so it might be most deeply rooted for them. It's only recently, perhaps with the introduction of user A/B preference tests such as by lmarena and the providers themselves has this become a major issue for most other LLMs.

Thinking that simple actions like adding an anti-evil vector to the residual stream to improve behavior sounds naively dangerous. It would not surprise me if unexpected and unwanted downstream effects resulted from this; which a future paper will address too. Not unlike what happened with tuning for user preference.