How do you manage to run out of tokens so quickly? I probably run more threads every working day, usually on medium, and I'm still below the 5x limits.
Do you use the official harness? OpenAI's models are generally best in class for token efficiency. It seems to me like they push for that much more than their competitors.
I've long speculated this when I see these types of comments, because it's actually really difficult to hit usage caps with an efficient dev flow, even when running multiple threads for hours every day.
I think some combination of:
1) Using 1 thread for everything
2) Reviving old threads which are no longer in cache
3) Really broad prompts on badly vibecoded codebases, so model spends huge amount of time tracking down whatever you're trying to do.
4) Non-coding workflow which is more output than input heavy
5) (Less likely IMO) Intelligent use of many passive CI/cron-like scans. E.g. regular security, quality etc scans. Automated issue resolution/PR
Just a guess. I think 3 is likely the primary reason.
You can literally go all day every day with multiple threads with Sol on the Codex 100/month plan IME
That's my experience too. I've found OpenAI really quite generous with tokens. I sometimes wonder how some people manage to run out of them really. Do they just type prompts that much faster than me or use the highest reasoning mode for everything just because they can? Idk.
I generally agree with those reasons, although using a single thread may be less of an issue than it seems because of context compacting which should happen automatically when you're near the limit.
My use cases are iterative and sometimes require reading a lot of code or reevaluating work.
Token efficiency is near meaningless when the workload is input-heavy. It can't always just choose to read less, depending on the task.
I can have cheaper agents do the reading but it's not appropriate for all use cases because they'll misjudge and choose the wrong things to emphasize, summarize, extract for the bigger model.
The problem is also that at a lot of code bases are not designed around LLMs and their token usage.
If you have a monolete codebase, you need to clearly define in the prompt what modules are involved. And even then, your wasting tokens with the first 1 or 2 steps where it needs to located the modules.
if you have a github codebase, with a lot of your code into nice little repos, its even worse because then the model needs to pull data, and a ton of more steps.
Most people do not open their coding agents in the module directories because "it may need something out of it".
I mean, we used to program by creating utils directories to deal with repetitive code but models (a) find it and use it (but it cost steps and tokens to read), (b) do not realize it there and make their own version of whatever or (c) combination of both.
And ironically, i feel like we need to give up on this idea of reusable code, and literally keep things into single modules, with as minimum external dependencies. That in return reduces searches and thus steps/tokens burn. But very few agents / harnesses have proper implementation of groups/projects and sub-module structures. Aka they only open a single dir, so your then forced to create dozens of tabs > per dir > cli ...
2. That is also a issue if you open multiple agents. Maybe now your working in A, B but C, D, E are not doing anything. And their cache expires... Now you go back to D because A, B needed to be done, and now your paying Cache Write + Input cost.
Its hard to have a good flow to keep things cached, when to really /new and when to not have it expire (and that assumes there are no issue with the provider moving your session around and forcing new cache hits. MiMo did that a lot in the past).
Something that i also advice more and more to people. Get a microphone, download openwisper and talk (text to prompt). You tend to give more information vocally, then writing as its in our habit of programming to not be verbose. Its like people are afraid of long prompts. While just talking to the LLM tend to give it much more information to work with, often resulting it being able to skip steps.
If I take a photo and then use GIMP to edit it, then did I take a photo or just direct my camera at something and then steer my computer to make changes to it? After all, it's not me drawing each pixel manually. It's vibe-photoed!
No. It's vibe-photoed when you use AI models which are trained on countless hours of actual human work. Otherwise you apply deterministic tools at your will and according to your skill level.
lucky you, but here's the issue (note: there have been at least 3 issues before this one) https://github.com/zed-industries/zed/issues/58651 crossing fingers it's solved for real now. i only use wayland on my laptop where it seems to be fixed, my go-to machine is with Xorg where there's no copy-paste problems
I guess it does make their communication feel a bit more personal, especially since they usually don't use corporate language on personal twitter accounts.
It’s just a different form of sanitized corporate PR even if the voice is tweaked for the medium.
At least traditional bland corporate PR doesn’t try to build up a weird parasocial relationship with the product owner/dev trying to act like your buddy, and per OP also often follows a consistent process without memorizing the personal handles of the current org chart.
Truly a terrible day in AI. We shall all rise against the 5-hour limit.
Also some Sol token usage optimizations are coming, as described in the tweet. Seems like OpenAI is really trying to remain the leader in token efficiency.
I'd say it's more like going to a Ferrari dealership and they tell you they will build a car for you and bill you per gram of parts used. It might also not work. And they might also not build it ever, really — but they will bill you for any attempts to build it.
That's fine though. If that doesn't work for you, don't buy. There are all manner of situations where what one wants or needs and what they get don't match up well. You don't price out every situation - it's take it or leave it. Pricing in a way that is somehow based on cost structure at least enables the provider to work to reduce the cost and hence price and win. Costco prices at a small margin above cost, they don't price "if this meets value prop X, pay Y and if only value prop A, pay B".
Do you use the official harness? OpenAI's models are generally best in class for token efficiency. It seems to me like they push for that much more than their competitors.
reply