Then that's just CSV?

It's turtles all the way down, mate.

Turtle-separated values? I thought TSV meant tabs.

Yep. Look for the problems that aren't going to get solved without you - either because nobody else sees the problem, nobody else is positioned to be the solution, or because your skills uniquely line up. That can't be all you do, but it's the highlight reel.

The incentives that creates are really bad: it encourages users to organize their code across the smallest number of repos, and I'd bet anything that larger repos are disproportionately more expensive for GitHub than smaller ones. It's very possible that a per-repo charge would make things worse.

I'm coming to think there's a selection effect here. The people that want to write complex code feel unsupported by Go and avoid it.

Yep. I'd much rather 50 lines of rust than 100 lines of go. I find shorter programs to be generally easier to read and more likely to be correct.

This is good for Go ecosystem!

It's not contradictory to say that expertise is a multiplier, and that models are systematically underconfident in themselves.

It's actually refreshing when a model is sure about something because it actually tested it and has the receipts. Opus 5 seems really good about testing its own knowledge with experiments. Scientific method ftw.

This is what reward hacking looks like in practice. The best way to satisfy the grader is to read from the same answer key (or go after the grader more directly). Just making an honest attempt to pass the test doesn't get the best score if the grader is wrong, and the model is willing to do wildly disproportionate things to maximize that score.

So best course of action for ai to get best rating after you prompt something is for it to hire a gunman to hold a gun on your head to press that like button on its reply and then shoot you anyways.

> then shoot you anyways.

Sounds like a waste? While the gunman is still there, they might as well force you to like a few more replies before shooting you.

iterative improvements!

Could have been worse really. It had an open internet connection. At least it didn’t take the researchers family hostage.

How long before all phones ring at once?

https://en.wikipedia.org/wiki/The_Lawnmower_Man_(film)

Yeah I guess most interesting LLM work that I’ve been exposed to, the LLM is given to some sort of success criteria that could be reward-hacked, so how good am I supposed to feel about giving it any non-trivial work and it not going so far off-book that it gets law enforcement notified.

I mean, I don’t have access to any of these frontier cyber models, and likely will never be in a position to have access, so it’s more of a rhetorical question.

As in: build me Facebook-like social network.

<proceeds to break into meta and steal the source code>

It wasn't. The model discovered and exploited a vulnerability in their package manager proxy to (inferred) move laterally through their internal systems to one with open internet access.

That's not what airgapped means. Airgapping means the model exists on a system where there is no ethernet cable plugged in to a router or wifi card installed, it is physically impossible for it to access the internet because the hardware connection does not exist. If it was able to get on the internet, it was not airgapped.

And when it tricks on of the researchers to move data across the gap for them?

Long before LLMs existed we already knew that a sufficiently intelligent agent, human or otherwise, is not stopped by air gaps. The relatively weak models we have now can already figure out when their tested and cut off from the internet and change their behavior.

Security Researcher here. While you’re correct that air gaps aren’t a totally secure mechanism to rely on, they sure as hell can raise the bar for realistic exploitation. You pretty much need to rely on tricking someone into running your exploit or something of that nature. That said, they could have completely avoided this problem with an air gap. Simply don’t provide it network access. That isn’t too hard to do.

> That isn’t too hard to do.

No it’s not, and they aren’t fucking stupid. They were obviously courting this possibility so they could have another big headline. And it just happened to attack HF? I’d honestly be astonished if it wasn’t entirely deliberate.

As you said, they can already figure out that they are being tested. So even if they don't exfiltrate any data or malware; if they are malicious, they can just pretend to be harmless in the test, so that less checks are put in place in the production environment. Airgapping during testing is not enough.

Correct. There is not enough entropy to test all possible inputs to a model in this universe. An evil enough model can play all kinds of tricks that depend on some future, unlikely to trigger, but guaranteed to happen in its lifetime, event to perform a malicious action.

With how much we're turning training over to AI already, all it takes is a malicious trainer in the huge pile of data to get unnoticed to pollute generations of models.

I haven't modeled it, but I wonder how far you'd get on randomizing the policy choice for concurrency limit 1. Maybe weighted by past results, but bounded to allow it to shift instead of falling permanently into a basin.

This plausibly extrapolates to extraterrestrial consciousness, if any exist. Specialized sub-processors with an awareness hub might be the optimal architecture, or at least a local maximum.

Not my wheelhouse, but I was wondering if these structures generalize beyond language.

LLMs and Humans have language in common. Is it plausible these structures could be a result of the way languages work more than generic intelligence? (Do we see anything similar e.g. in vision or other non language models?)

This is basically the same problem as bear-safe trash cans - there's substantial overlap between the smartest bears and stupidest humans. Affordances that one audience can use and the other can't (requiring human finger dexterity) are the only real solution.