The more AI-generated AI bad stories we get the more likely LLMs will produce more!

LLMs are told what to produce.

"Write me a 500 word post about how AI is great" and such shit.

What such stories would change is worsen the training data, so that we get more of that style of writing (rather than angle).

I haven't had a drink in 6 months or so. Not because I wanted to stop, I just have not had the desire to drink recently.

Now I would love to tell you about all the amazing magical healthy benefits that have come with that, but unfortunately there are none. I feel no difference at all.

When I stopped drinking I almost immediately lost like 15 lbs. It just fell off without doing anything else different over a few weeks. It was just the extra calories from both the drinks themselves and the snacks I would usually eat with or after drinks. It stayed off too.

Haha! I'm ~4 years 'sober' and I can say the same!

However I have a rather large confound: Kids.

I'm sure that if I was drinking regularly still that I would feel much worse than I do today. But the effects of children are powerful enough to my system that they overrule any positive effects of this forced sobriety.

(To be clear: I'll have a drink every blue moon or so, but it's quite rare I get enough time off to enjoy anything like a buzz these days)

Based from my experience, stopping drinking didn't have noticeable effects as well. However, whenever I would drink once in a blue moon socially, it feels even worse than what I remember. I think it just felt normal before.

Yeah, I did the same for roughly 6 months and I don't think that I felt better .. I just never felt SHIT.

Sleeping better? Clearer skin? Lost fat? Biological age < calendar age?

But I bet whenever (if?) you do have a beer or two again, you’ll notice how it affects your sleep and probably just makes you feel kinda meh.

I say that as an occasional drinker. Some of my family loves to drink, nearly every evening. But I join them occasionally and just feel kinda bloated and bad, especially if it’s been a few weeks. Makes it harder to make other healthy choices

But yeah love the taste of beer so it is all about tradeoffs!

You should not be touting magical health benefits on a sample size of one either way.

There are already plenty of studies available to be found.

Doesn't matter who reads it. The point is that you will probably never learn to do "high level system design" well if you do not have enough experience writing and refactoring code yourself. It's like you wanting to become the chef of a kitchen and giving instructions without having ever prepped food.

There is indeed something useful about trying to write elegant code. Not because others read it. But because that's how you learn about the engineering tradeoffs and abstraction that exist everywhere.

I give it one to two more years before open source models have fully caught up. Products are commodities and models are commodities too. GPUs cores are still hard to get for inference at scale right now. They need a platform with lock in but unsure what that would look like and why it wouldn't be based on open source models.

What does "fully caught up" mean in the context of an ever evolving technology? I think I'm in support of open weight models (though there are safety implications), but these things aren't cheap to train and run. This fact alone gives no incentive for leading labs to release cutting edge open weight models. Why spend the money then give the product for free?

Now if "fully caught up" means today's level of intelligence is available for free in two years, by then that level of intelligence means very little

It’s never free your shifting costs from paying a company for their api use vs the power costs of running it locally.

Sure, but it’ll be orders of magnitude cheaper in a few years. The consumer industry is already moving in this direction, with Apple leading the pack

Yeah I don't understand it, it's a marathon with three companies perpetually a minute ahead, and people keep saying "I expect the stragglers to catch up".

The only thing I can see them meaning is what you said, "in a minute the stragglers will be where the leaders were a minute ago", which, yeah, sure.

By my estimation, there is a point where these models are "good enough" for the vast vast majority of all appropriate tasks, after which point further investment by the major labs will have diminishing returns. While they might stay ahead by some measure, the open models will be good enough too, and I assume significantly cheaper like they are now.

Or AGI hits and this theory collapses, but that's feeling less likely every day.

It's not a marathon, or any race. There is no a finish line. It doesn't matter that much that someone is a minute ahead.

It makes perfect sense if you think things cannot improve indefinitely

They do approximate any function... within the range they're trained on. And that range is human limited, at least today.

Also, there is a good enough point where improvements for a given use case are on heavy diminishing returns

That's fine. I can afford to wait a minute if it means I pay $10/m instead of $5k/m.

What is the transition state where people start using open source models that you imagine actually happening?

Play out a scenario. An open source model is released that is capable as Mythos. Presumably it requires hardware big enough that running it at home is unfeasible. You are imagining that individuals can run it in the cloud themselves for cheaper than api tokens would cost? Or even small companies? And that Anthropic and OpenAI won't be able to cut costs deeper than their competitors while staying profitable?

If it is fundamentally a commodity, that means "running it yourself" also isn't really interesting as a proposition. Many of the world's biggest companies sell commodities. It's a great business to be in if you can sell them cheaper than anyone else.

The value add here isn't the model, it is "having a bunch of compute and using it more efficiently than anyone else".

Why do people have such faith in "open source" models? There's nothing "open source" about them. No individuals have the ability to train such modules. They are just released by companies to commoditize the models of the competition.

If Mythos is the endgame, companies won't release open-weight equivalents, and no private individuals have the capital to train such models.

The open models cannot be taken away. Anyone with the right hardware can host these. Unlike the API/subscription services where you can be banned from, may have drastic price increases or reduction of their limits.

> There's nothing "open source" about them. No individuals have the ability to train such modules.

I expect that people on subscriptions can be asked to donate 1 query a month towards an open source distillery.

It should be good enough to distill SOTA models over time.

The result won't be perfect, but it will be close.

Think SETI@home, but it'll be model distillation instead.

Good lucking getting GPUs.

Not really, when OpenAI was formed in 2015 there were no LLMs, at least none that worked well. It was a regular AI research lab mostly doing Reinforcement Learning on game environments like Atari similar to DeepMind. Once they struck gold with LLMs (2019 or so?) and saw there is money to made everything changed, as expected when a bunch of SV types get involved.

I watched the video but I don't quite get it. I feel like I'm missing something? A nicer git workflow is not what I need because I can ask an LLM to fix my git state and branches. This feels a bit backwards. LLMs are already great at working with raw git as their primitive.

I'm curious what their long term vision they pitched investors is.

Yeah, it looks like a model issue to me. If the harness had a (semi-)deterministic bug and the model was robust to such mix-ups we'd see this behavior much more frequently. It looks like the model just starts getting confused depending on what's in the context, speakers are just tokens after all and handled in the same probabilistic way as all other tokens.

The autoregressive engine should see whenever the model starts emitting tokens under the user prompt section. In fact it should have stopped before that and waited for new input. If a harness passes assistant output as user message into the conversation prompt, it's not surprising that the model would get confused. But that would be a harness bug, or, if there is no way around it, a limitation of modern prompt formats that only account for one assistant and one user in a conversation. Still, it's very bad practice to put anything as user message that did not actually come from the user. I've seen this in many apps across companies and it always causes these problems.

Understandable, I'd probably do the same in his position. Still sucks, we've seen this pattern a thousands times before and what happens next is pretty obvious.

I was prototyping something with pi under the hood for a personal project, going to switch off it now.

Like he iterates in the blog post multiple times: It's still MIT licensed, you can fork it to your heart's content. Or keep using the mainline and merge new features to your own fork.

For me the reason to add dependencies to my projects is exactly because they are maintained upstream and I don't need to worry about maintaining them myself. If I need to fork and maintain it myself I'd rather write my own version of it that perfectly fits my use case, or use another dependency that is maintained.

The company is from the person who created Flask which has remained very open and free. It might be a little too early to be cynical about this.

> I was prototyping something with pi under the hood for a personal project, going to switch off it now.

For what it's worth, it's pretty straightforward to recreate it I found, at least it's base idea. Readline w/ nice output is a bit of a pain, but still, doable, and if you don't care about that part of it, then the overall agent loop that you'd build on top of? You could build it, I promise.

What you are suggesting might sound difficult to some people, it is possible: in the last week I co-wrote (with Antigravity with Claude as the backend) an Emacs package for agentic coding that also just uses ‘rg’ for finding relevant code for the context, call out to a model, and handle creating a diff with merging tools. I love using my own code that provides inside Emacs vibe coding and I would suggest others try building the same thing.

EDIT: here is a link of what I wrote just for my own use: https://github.com/mark-watson/coding-agent

Exactly! The actual base loop of these agents is remarkably simple.

https://github.com/girvo/girvent this is my silly one :)

looks nice!

How does DMCA apply? The license is MIT.

I have a private Gitea exactly for stuff like that and Gitea can mirror GitHub repos out of the box and keep it in sync (and it's Git so you can always revert).

Or fork it?

Are you sure they are not just refusing to solve your UI bug due to safety concerns? They may be worried you'll take over the world once your UX becomes too good.

I don't want the world, sounds like people would start bothering me all the time. I'd never get a moment to relax. But if I had the world, I'd do better with it than what we got.