I can definitely see them wanting to have models that can run on Windows computers or Surface tablets locally - although their focus seems to be sticking CoPilot into absolutely anything and everything possible, but why synthetic data models? Other companies have made small parameter models, but they don't seem to keep them up to date (correct me if I'm wrong).
I've always noticed that when I'm giving advice to someone or trying to help out, it always feels their problem is easier than whatever problem I have. As someone with some anxiety around things like calling some company to get something done or asking a random stranger for some help in a store, I would gladly do it if it was to help someone else (family member or friend). But when it's for me I find it harder.
I wonder how much psychologically we can be more confident and less anxious when we're doing something for others vs ourselves..
People in the ADHD community are outspoken about a tangential concept: cleaning. Cleaning your friends place is a fun, novel, non-emotional activity. Cleaning your own space is a mental slog, boring and often painful due to having to rid yourself of mementos.
In that case, my theory is that you get to shed your learned helplessness about how things look. I suspect it’s similar with giving advice.
This is ozempic territory: a technical solution to your own shortcomings is most effective.
I have solved all my issues with doing house chores with wireless headphones, tablet, and youtube @ 2x speed. Sure, it means that I can't load my dishwasher until I find something half-decent to listen/watch but once I do find it, I have 10-50 minutes of just pure closing. Dishwasher loaded, countertops empty, new load of laundry, dry clothes in the closet, gym bag packed, trash taken out. Frankly, kinda enjoy it now.
This is me. Finally buying some bluetooth headphones 15 years ago changed my life. I finally became a person who cooks every meal, cleans everything, does chores, and exercises daily, even pushes around the house.
I like listening to debates since they are the most stimulating. So long as I can find a good one, I’m about to make dinner and unload the dishwasher.
An audiobook that’s good enough can be so captivating that I run out of things to do while listening to it.
I have pretty extreme adhd which might be related. But I’m just glad I bought those headphones back then.
"other people's same problem easier" i see, but have never seen messiness as example at least in communities w adhd comorbid with depression. personally the concept of other people cleaning my private mess, even/especially if they are close family/friends is terrifying and already overloads my head and i can only project the same sentiment (and some extrapolation of my own experience helping friends/family with cleaning... it is super hard, we are talking about intruding on what the person values as trash or not trash, and that itself can be a source of great shame i.e. my mother who lived to much worse abject poverty than the children she helped raise with a better life. sorry for being dramatic about an otherwise straightforward point but yes in my experience that "cold" reduction of the problem into something actionable would be key, though people arrive there differently i noticed, e.g. me and my "armchair courage" that any unseen sideeffect is not my problem, for my mom (okay sometimes for me as well) it is about being able to forget that she has problems just by the appearance of having the luxury to give advice
A line I always remember (from Babylon 5) is: "When I clean my place, all I've done is clean my place. But when I help you clean your place, I'm _helping you_."
tl;dr you should ask your badass partner for strategic help when the entire galaxy is under threat, even if she seems busy.
Reverend Will Dexter: "You know before I got married Emily he
used to come by sometime to help me
clean out my apartment well I asked her
how come here so he could help clean up
my place when your place is just as bad
she said because cleaning up your place
helps me to forget what a mess I made a
mind and when I sweep my floor all I've
done is sweep my floor but when I help
you clean up your place I am helping you."
My girlfriend and I both have ADHD and are medicated. I will run laps around her tidying up her place, but struggle at my own place.. its so hard to understand
What's the opposite? Where solving your problems is easy and but solving your friend's problems are very difficult because your advice are never relevant?
This is something I've noticed as well. I've talked about this with my psychiatrist and she calls this brave, reassured version of ourself the "me-mentor" (jag-mentor in Swedish). Similar to our inner child, this is a core part of our who we are.
The idea, if I understood correctly, is to build this me-mentor more and let it help us feel more safe. Let it support our insecure parts/personas.
Somewhat related, a psychologist I talked to in the 2000s said she really liked the Patronus concept in the Harry Potter books. You imagine an entity that's fueled by your positive memories and emotions, and that protects you from certain anxieties and other stressors.
Things like that seem to be used in at least some schools of psychology.
It reminds me on something my psychologist told me, when trying to find this me-mentor, it can help to take inspiration from someone I find really safe with and trust a lot. Aka someone I have good memories with / of.
When trying to examine someone else's problems, you can see the problem itself. But what you aren't seeing is a pile of all the little habits, beliefs, behaviors, impulses and assorted mind defects that prevented them from solving it in the first place.
It takes intimate familiarity to know all of those things about someone.
If you were in their shoes, the problem might genuinely be trivial, for you. Because you're not that person, and that problem isn't your own failure mode - you would instead fail at a different "trivial" problem and in an entirely different way.
Or maybe you are flawed in the same way, but don't know it yet. You never quite know. Humans aren't any good at that whole "self-awareness" thing.
> When trying to examine someone else's problems, you can see the problem itself. But what you aren't seeing is a pile of all the little habits, beliefs, behaviors, impulses and assorted mind defects that prevented them from solving it in the first place.
This is accurate. The roadblocks to solving their problem are often several small things completely unrelated to the problem itself.
The opposite conclusion is that you are more risk-taking when it comes to dictating the actions of others, because neither their gains nor their losses directly accrue to you. But human beings feel loss aversion more keenly than they desire gain, so this biases the advice you would give others (but not yourself) riskier in general.
I think this is exactly it. It's easy to see that there's a chance to improve things while ignoring the ways it could make things worse when they won't affect you.
Should you quit the job you don't like? "Of course" the friend will say. But then you might just end up with a job you hate more that pays less, or even no job.
Whether the outside perspective is helpful probably depends on how much your own perception deviates from reality. Though people do have a tendency to prefer the status quo until things change, so maybe you should always prefer the "change" option when you aren't sure.
This phenomenon is called "Solomon’s Paradox" - People think more clearly about other people’s problems than their own. When facing their own issues, they reason less rationally.
Yet, a study from 2014 showed that seeing your own problem from an outsider view removes the gap between how wisely you think about yourself and how wisely you think about others.
I imagine it has to do with vulnerability. When you are asking for something or sharing something, being turned down feels personal. When doing it for someone else, it's no big deal if they say no.
> I've always noticed that when I'm giving advice to someone or trying to help out, it always feels their problem is easier than whatever problem I have.
One mundane reason is that you've probably already solved that problem for yourself.
Almost by definition, the big problems we have are in areas where we're less competent than others.
When someone asks for advice, I often find if I pay deep attention, that advice is aimed at myself as well. Listen to the advice you give, because often times, the advice giver should follow it as well.
Probably because our desire to help and not let down a person we care about gives us courage. That courage serves as motivation to go outside our comfort zone.
The problem with your problem is you have a desired outcome.
And the other is you are not required to do the heavy lifting.
One method is to find a way to bless "future me". Future me will thank current me sometime in the future and while current me won't enjoy future me's rewards directly, he will think kindly, instead of with contempt.
It's something I've been wondering about for a long long time. Thanks for bringing up the question. Sometimes my problem-of-the-day is not even that hard but I have near zero drive to finish it, but if anybody comes with an issue, I then feel motivated (up until I realize his/her issue was hard I guess).
I see three dimensions:
- natural pleasure of helping someone
- ignorance about the problem, making it seems easier
- a saturation aspect: my problem has probably something i've been dealing with for days, my brain is full of unanswered questions about it and has no more "space" for it
>I wonder how much psychologically we can be more confident and less anxious when we're doing something for others vs ourselves
Thank you for taking the time to type this up. I would be extremely interested in any sort of research around this and may add( maybe others face the same ) that's incredibly difficult to introspect yourself and solve problems for yourself as easily as you can for others.
This is a fascinating phenomenon, isn't it? I've heard it invoked as "it's always easier to clean someone else's room." And anxiety does seem to be the key. Very often the actual blocker isn't the difficulty of a task, but how we relate to it.
I find the same thing between doing something entrepreneurial vs. doing it for work. If my boss tells me to call a customer, I will have no problem doing it. Calling for my side hustle......way more anxiety.
I'm sure there's a proper name for what you described, but I call it Rip van Winkle syndrome. He helps everyone in the town with their needs, while allowing his own property to fall to ruin.
That's why it's good to have close friends so that we don't have to be perfect ourselves in all respects in our private lives... humans are a political animal after all
Since you asked me, you are using the same concept and now I need to help you solve your problem (which seem to be the one I also have..)
I think the solution must be we're primarily responsible for ourselves, and that unless we ask others for help all the time we need to figure things out. I also lately have been thinking from the perspective of the person I'm anxious to interact with, and feel that they may actually be happy to interact with me, receive some warm greeting and help out by answering my question or doing my task.
If you could do something for others but feel anxious doing it for yourself, it must be "in our head" and logically we should be able to get over that and choose to be brave. I think in really it's often missed how we can be brave doing the action if it was for someone else, and that the bravery may actually already be inside us.
One thing is also ability to have clearer start and end and boundaries, or some sort of mental boxing for the case at hand.
If you're visiting someone else, you arrive, and you leave. The helping them clean part has at least some sort of boundaries. Even if you don't finish, you have helped them along.
When you're at home, even if you start, if you leave it halfway, it will be your problem after you stop. And tomorrow still and so on. So it feels more daunting.
Probably to help you avoid being distracted given the higher friction of rebooting if your dual booting or going to another device vs just launching another browser tab..
Is the company valued at $500 billion or is the sum of the digital assets they’ve collateralised worth $500 billion?
Because if you buy the tokens you presumably do not own the company. And if you buy the company you hopefully don’t own the tokens - nor the assets that back the tokens.
For comparison Blackstone is worth ~$180bn with ~$1 trillion AUM.
So somehow this crypto firm and its investor think it can get a better return than Blackstone with a fraction of the assets. Now, sure, developing market and all that. But really? If it scaled to Blackstone assets level of $1 trillion then you’d expect the platform valuation to scale, perhaps not in lockstep but at least somewhat. So with $1 trillion in collateralised crypto does that make Tether worth $1.5 trillion? I’d love someone to explain that.
Tether is projected to generate $15 billion in profits. So 500 billion is like a 33 times earnings multiple.
Now the main thing is how sustainable these earnings are and if they will continue to be a dominant player in stable coins and if there will continue to be demand for them.
Another difference to Blackstone is Tether takes 100% of the returns on the treasuries backing the coins, whereas Blackstone gets a small fee from AUM, and their goal is to make money for their investor clients.
If crypto wanted to really be decentralized they'd find a way to have stable coins backed by whatever assets where the returns of the assets still came to the stable coin holder, not some big centralized company.
If my mom gives me 1000 dollars for 1% of my lemonade stand, that doesn't mean my stand is worth 100k. Tether is in talks with investors to mayb raise 20b at a 500b valuation. Keep in mind also that crypto investors overvalue companies to create the hype and then lobby for better regulations etc. It doesn't mean at all that someone would be interested to buy 100% of tether for 500b. Now, if they were public is a different story, like Tesla etc
I think at least part of the 10% is if AMD stock reaches 600.
Not that I disagree that this looks weird. Why was that needed to be offered? Couldn't they just buy the AMD chips if they're good enough? Or Nvidia is it's better?
I also don't get why there commiting so much to the future, are they sure of the quality of the products and their demand that much?
>Couldn't they just buy the AMD chips if they're good enough?
OpenAI would presumably need to raise money to buy the AMD chips.
The "genius" of this deal is that AMD is "giving away" 10% of the company (at $0.01/share) to OpenAI. Then OpenAI will presumably turn around and sell those shares (or borrow against them) to raise enough money to purchase the AMD GPUs.
They sign the purchase order on 1/1/26. AMD issues invoice to be paid in 30 days, that is 2/1/26. OpenAI triggers warrant and informs AMD on 1/2/26. OpenAI receives shares on 1/4/26. On 1/5/26 OpenAI and AMD announce the GPU purchase deal. On 1/30/26 OpenAI sells its shares in AMD. From proceeds, OpenAI pays AMD on 2/1/26. Thus, AMD financed OpenAI's GPU purchase via AMD's shares.
translated, AMD buys GPU from itself and gives them to OpenAI for free. OpenAI gets GPUs for free, AMD hopes the market will reward the deal enough to increase its valuation by more than the dilution cost.
I have to ask - is this even legal? I understand it can be, but somehow it feels wrong. I guess AMD would report revenue of those GPU sale and equity issuance / dilution as part of payment terms, and OpenAI would record hardware purchase expense as well as investment income or maybe capital gain when selling those shares. What makes it legal is probably it all needs to be transparently communicated in time?
Yes, I was being somewhat flippant in my description of the transaction. But the net result of the transaction is the same. OpenAI can finance the GPU purchases by borrowing against the contractual guarantees it received from AMD to receive warrants in exchange for acquiring AMD GPUs. Whether the transaction is partially or entirely financed will depend on AMD's share price movement in the interim.
It seems to me that there is an aspect of marketing to this deal. Nvidia has the mindshare, so this would help legitimise AMD offerings. This is almost product placement/sponsorship for AMD.
Also, this would battle test AMD's platform and provide enhancements so it's also a beta-testing service.
The CEO of Tesla, Elon Musk, was sued over an extremely similar situation. So somebody will care.
That said, this is really about the principal. Sure, if I give you $10 and you give me a hamburger it's not like some illegal transaction. But to say the $10 comes from thin air is wrong. It doesn't come from thin air.
I would bet that if one day OpenAI decided to sell 10% of AMD the stock would crash from $600 to below $150. IIUC, there's 1.6B shares of AMD while only 54M shares trade daily so dumping 160M shares would tank their price [1]. If AMD gives OpenAI 10% of the company and OpenAI goes under, it's going to take AMD's share price with it.
His 56 billion pay package [1]. In order for him to receive it the stock would need to increase 13x [2] (the AMD stock increase from 150 to 600 is only 4x). Despite succeeding at doing that, he and Tesla were sued over the pay package.
If OpenAI fails then its going to have to liquidate the company. Selling 160M shares of AMD is going to tank it's price.
It's not really a similar situation. His pay package was invalidated because of how much control Elon had over the board and the shareholders not being fully informed of the likelihood that the necessary goals would be hit.
Whatever the merits of the lawsuit, it wasn't about the pay package being too diluting.
If OpenAI fails, it will be acquired and/or the shares will be sold in bulk. They're not going to log in to etrade and sell 160m shares on the open market.
> Whatever the merits of the lawsuit, it wasn't about the pay package being too diluting.
But this is exactly the point. If somebody is going to sue Elon when he 13x the share price then of course somebody is willing to sue AMD if they 4x it.
Elon's pay package was voted on by shareholders. AMD's deal with OpenAI had none of that so if anything it's more ripe for a lawsuit.
I think there's a covenant preventing OpenAI from dumping AMD shares on the open market. Obviously AMD's price will move down during the crash but at least the shares will be liquidated in an orderly fashion.
What is AMD getting that's worth giving OpenAI $100B? Sure, they're giving it from other stockholders not from their pocket, but still. It's presumably a lot of value, there has to be a good reason, no?
Is it that Sam promises to somehow make AMD increase their market cap, or help at least?
Where is this other $300B coming from? Is OpenAI paying AMD $400B or what? I looked at the article but it seems disjointed and hard to parse for me. And I don't see where it mentions some $400B coming to AMD one way or another. It's implied... how?
Sorry, this isn't sarcasm or anything like it. I just don't get it and your answer does not help.
The traditional "efficient market" theory would be: synergies. The market believes that AMDs value increased BECUASE OpenAI now owns it. That is to say, the market believes that OpenAI taking a stake in AMD increases the value of AMD.
There are a host of different hypothesis you could pose to explain that. Maybe OpenAI has some secret sauce they'll share with AMD now that they have a stake. Maybe OpenAI will be more likely to buy from AMD in the future. Maybe AMD can use the experience they get serving OpenAI to better their products. Heck, maybe OpenAI will pump the stock by having Sam Altman talk about it on some podcasts.
It's impossible to disentangle all of those theories, because different investors will have different beliefs and you only get an aggregate.
Imho AMD itself needs to have a theory, which underpins their signing of the deal. For my clueless self, that investors have various theories and we don't know what they are is ok-ish, but that AMD has a theory but keeps it secret yet it gets the result of stock rise... is fishy.
Everyone is going in circles making suppositions and estimations based on who knows what. That can't be healty, can it? There used to be requirements that publicly listed companies act with some level of transparency, and those requirements existed for a reason. I guess. I am certainly no expert in finance.
> but that AMD has a theory but keeps it secret yet it gets the result of stock rise... is fishy.
It's not secret at all. Companies announcing a deal like this usually include some PR material alongside it [1]. In this one, the quote is:
“Our partnership with OpenAI is expected to deliver tens of billions of dollars in revenue for AMD while accelerating OpenAI’s AI infrastructure buildout,” said Jean Hu, EVP, CFO and treasurer, AMD. “This agreement creates significant strategic alignment and shareholder value for both AMD and OpenAI and is expected to be highly accretive to AMD's non-GAAP earnings-per-share.”
"significant strategic alignment", "shareholder value", and "billions of dollars in revenue" are all things that should be expected to move the market cap. The "tens of billions in revenue" would generate upwards of 100 billion in market cap alone, assuming AMD's current multiple.
I think the idea is that, OpenAI using AMD GPUs will help AMD become competitive against Nvidia in the AI space. If OpenAI is able to use them for their models, other companies will see AMD as a legitimate option and might switch to AMD for GPUs as well.
This would be where AMD is to gain new money.
OpenAI also has to gain, if it means access to more GPUs allows it to compete and be the winner of the LLM race. As the winner of the race, it would make new money, but also likely need to spend even more money on AMD to buy even more GPUs for years to come.
AMD was desperate enough to sell 10% of their company to get 1 customer.
The issue here is now, that every large customer of AMD will now probably ask for equity. AMD has put itself into a pit hole with that deal.
If I were Hyperscaler CEO, I would basically ask for the a similiar deal as OpenAI or no business. Sorry Lisa Su but as a CEO giving equity to a customer is an absolute red flag because it starts a negative spirale you can't stop.
It seems that no matter the discount, OpenAI wasn't ready to make deal without equity. This tells you exactly how AMD is seen in the AI world.
OpenAI will take the compute for free and help AMD to rise stock value but it won't help AMD one bit because if AMD remains in the current position then OpenAI and Hyperscalers can get great deals with equity from AMD. The incentive isn't now to improve AMD to be competitive but to squeeze everything out of a company being desperate enough to give equity to customers.
And AMD will feel this. Nvidia will remain dominant because of ecosystem and supply. AMD can't easily replace Nvidia in supply chain and Nvidia is already strongly entrenched in many AI compute operations. And on the other side Hyperscalers are focused on their own chips (even OpenAI LOL) so they will tell AMD "Give us equity or no deal". This deal might be really the worst AMD deal yet because AMD is telling the world "here, you can get free AI compute from us financed by our equity". And while it might push AMD share price the very share price will drop 80-90% like any other one in case of an AI bubble pop.
You have put your finger on the AI bubble's biggest problem right now. Companies are making promises that they are currently completely incapable of fulfilling, in the hopes that someday they can, and the stock market are valuating these promises as done deals.
Predicting the end of bubbles is well known to be a fool's errand, but if this AI bubble is still going in a year I can only imagine how casually these companies will have to be throwing around multi-trillion dollar promises to each other to keep the stocks pumped up.
> Companies are making promises that they are currently completely incapable of fulfilling, in the hopes that someday they can, and the stock market are valuating these promises as done deals.
That reminds me a lot of Enron. As long as the stock keeps going up everything is fine but when it does t everything comes crashing down.
How's Microsoft's Direct3D moat working out for them now? It's turned out to have been much less of a moat than it once was. Triple-A titles that are developed for Windows using Direct3D 12 are getting support on Linux through Proton within days of release, or even at launch sometimes.
There is not. AMD didn't invest in tooling and interconnect technology the way Nvidia has, probably because of antitrust fears (or maybe mismanagement). But in terms of core GPU technology and fab, AMD is close to being a peer.
I've been saying this for several years now and it seems that someone finally listened :)
Try to use AMD GPU's for AI and you'll understand. Unless you have lots of your own engineers to throw at making their stuff work, it's easier for most companies just to keep throwing money nVidia's way.
I understand that it's that way today. But I am talking about "potential". If OpenAI and AMD engineers get their heads together and make some new software etc, couldn't AMD in theory become as valuable as Nvidia or at least half as valuable?
It seems like to take a 350M market cap company to 2B+ or a 6x+ increase in stock price would be worth doing for a few hundred million dollar investment in software and such?
By the time that could feasibly come to fruition, I suspect the AI bubble will have long since popped. Despite making decent GPUs for graphics, AMD can't seem to get its act together on the GPU compute front.
> I also don't get why there commiting so much to the future, are they sure of the quality of the products and their demand that much?
It's one big game of musical chairs, and everyone can hear the phonograph slowing down.
OpenAI is making these desperation plays because they've ran out of hype. GPT-5 "bombed", the wider public doesn't believe AI is going to keep getting exponentially better anymore. They're out of options to generate new hype beyond spewing ever larger numbers into the news cycle.
AMD is making this desperation play because soon, once the AI bubble pops, there'll be a flood of cheap unused GPUs & GPU compute. Nobody's going to be buying their new cards when you can get Nvidia's prior gen for pennies on the dollar.
I find it funny how people say GPT-5 "bombed". I noticed a significant improvement in maths and coding with GPT-5. To quantify were I've found the models useful:
- GPT 3.5: Good for finding reference terms. I could not trust anything it said, but it could help me find some general terms in fields I was unfamiliar with.
- GPT 4: Good for cached, obscure knowledge. I generally could trust the stuff it said to be true, but none of its logic or conclusions.
- GPT 4.5: Good for reference proofs/code. I cannot trust its proofs or code, but I can get a decent outline for writing my own.
- GPT 5: Good for directed thinking. I cannot trust it to come up with the best solution on its own, but if I tell it what I'm working on, it's pretty decent at using all the tricks in its repertoire (across many fields) to get me a correct solution. I can trust its proofs or code to be about as correct as my own. My main issues are I cannot trust it to point out confusion or ask me, "is this actually the problem we should be solving here?" My guess is this is mostly a byproduct of shallow human feedback, rather than an actual issue with intelligence (as it will often ask me at the end of spending a bunch of computation if I want to try something mildly different).
For me, GPT 5 is way more useful than the previous models, because I don't have a lot of paper-pushing problems I'm trying to solve. My guess is the wider public may disagree because it's hard to tell the difference between something better at the task than you, and something much better.
I used scare quotes for a reason. It didn't "bomb" in the sense of failing [insert metric], it bombed in the sense that OpenAI needed it to generate exponentially more hype and it just didn't. (And on a lesser level, GPT-5 was supposed to cut OpenAI's costs but has failed to do so)
> I can trust its proofs or code to be about as correct as my own.
I have little to say about this, as I find such claims to be broadly irreplicable. GPT-5 scores better on the metrics, but still has the same "classes" of faults.
Gemini 2.5 was the first breakthrough model, people didn't know how to use it but it's incredibly powerful. GPT5 is the second true breakthrough model, it's ability to deal with math/logic/etc complexity and its depth of knowledge in engineering/science is amazing. Every time I talk to someone who stans Claude and is down on GPT5 I know they're building derivative CRUD apps with simple business logic in Python/Typescript.
On the flip side of it (and where most institutional investors are mentally) is that if OpenAI is to ever achieve AGI, it must invest nearly a trillion dollars towards that effort. We all know LLMs have their limitations, but next phase of AI growth is going to come from OpenAI, Anthropic, Google, maybe even Microsoft, and not some stealth startup. E.g., Only Big Tech can get us to AGI due to sheer massive amounts of investments, not a traditional silicon valley garage startup looking for their Series A. So institutional investors have no choice but to continue to throw money into Big Tech hoping for the Big Payoff, rather than investing in VC funds like 10 years ago.
AMD did this deal because it's literally offering financing to them. OpenAI doesn't have access to capital markets like AMD does. So it's selling off shares of its own stock to finance the purchase of billions of dollars worth of GPUs. And the trick appears to be working since the stock is up 30% today, meaning it has paid for itself and then some.
That “only big tech can solve AGI” bit doesn’t make sense to me - the scale argument was made back when people thought just more scale and more training was gonna keep yielding results.
Now it seems clear that what’s missing is another architectural leap like transformers, likely many different ones. That could come from almost anywhere? Or what makes this something where big tech is the only potential source of innovation?
Yup. LLMs can get arbitrarily good at anything with RL, but RL produces spiky capabilities, and getting LLMs arbitrarily good at things they're not designed for (like reasoning, which is absolutely stupid to do in natural language) is very expensive due to the domain mismatch (as we're seeing in realtime).
Neurosymbolic architectures are the future, but I think LLMs have a place as orchestrators and translators from natural language -> symbolic representation. I'm working on an article that lays out a pretty strong case for a lot of this based on ~30 studies, hopefully I can tighten it up and publish soon.
The barrier of entry is too high for traditional SV startups or a group of folks with a good research idea like transformers. You now need hundreds of billions if not trillions to get access to compute. OpenAI themselves have cornered 40% of global output of DRAM modules. This isn't like 2012, where you could walk into your local BestBuy, get a laptop, open an AWS account, and start a SaaS over the weekend. Even the AI researchers themselves are commanding 7- and 8-figure salaries that rival NFL players.
At best, they can sell their IP to BigTech, who will then commercialize it.
Are you saying you disagree that a new architectural leap is needed and just more compute for training is enough? Or are you saying a new architectural leap is needed and that or those new architectures will only be possible to train with insane amounts of compute?
If the latter I dont understand how you could know that about an innovation that’s not yet been made
I’m saying it is highly likely that the next leap in AI technology will require massive amounts of compute. On the order of tens of billions per year. I’m also saying that there are a small number of companies that would have access to that level of compute (or financial capital).
In other words, it’s is MORE likely that an OpenAI/Google/Microsoft/Grok/Anthropic gets us closer to AGI than a startup we haven’t heard of yet. Simply because BigTech has cornered the market and has a de facto monopoly on compute itself. Even if you had raised $10 billion in VC funding, you literally can not buy GPUs because there is not enough manufacturing capacity in the world to fill your order. Thus, investors know this and capital is flowing to BigTech, rather than VC funds. Which creates the cycle of BigTech getting bigger, and squeezing out VC money for startups.
If it comes from anywhere else but it needs a lot of capital to execute, big tech will just acquire them right? They'll have all the data centers and compute contracts locked up I guess.
no amount of investment is going to make AGI just appear. It's looking more and more like current architectures are a dead end and then it's back to the AI drawing board just like the past 30 years.
The difference this time is that it's global coordinated collusion, and it's not just the superwealthy, it's states that are willing to go all in on this. If you thought the banks were too big to fail, the result here is going to be a nationalization of AI resources and doubling down.
Maybe if instructed, but how would it know it needs to use python in this case vs just answer? Perhaps you'd instruct it to always attempt using code to reduce errors.
But the idea of trivial problems like this potentially causing issues for LLMs might mean other aspects of intelligence could also be a struggle for LLMs (which could impact it's coding ability as well).
This is very interesting.
1. Authors mention the attention mechanism being perhaps unable to attend to the location of gaps since the gaps aren't tokens. But I would've expected a good LLM transformer to be at least a bit close to the gap location. I don't understand why mathematically the architecture is less suitable for that, it could attend to a region that may contain gaps. I wonder if fine-tuning on a task like this could help?
2. Shorter inputs with less omissions were harder to solve. That is not completely surprising, as a human doing this task, if 1 word was missing it would be harder to notice. And similarly 1 line would be harder than 10 lines. But still interesting for an LLM to have this problem.
3. Reasoning models do better, as they can write out the documents and potentially solve this easily. It still very surprising that this doesn't lead to 100% accuracy. This should be a trivial task. Like the paper says, a trivial program can be written to solve this. Perhaps ChatGPT (or similar agent) could read this paper while training, and know to write and run python when solving an issue like this.
The most interesting thing though, is what other aspects of intelligence we may not have identified explicitly, and whether LLMs and current AI is very bad at them. This paper suggests that there likely are many of those, and it seems in general a pretty fun time for people working building benchmarks.
Also just more accessibility in terms of being able to pay for it. As a viewer in Canada, there was no service whatsoever that was showing the Real Madrid vs Barcelona Spanish cup final a few months ago (Copa del Rey). I had to signup on a Spanish state tv website and use a VPN to access it for free.
I would have gladly paid, but there was no opportunity.
Content fragmentation and some sports rights not being bought and resold by anyone is also a big problem.
"Content fragmentation and some sports rights not being bought and resold by anyone is also a big problem."
I dont watch soccer. But if you want to watch soccer with a decent exposure (national league, Champions league etc.) in some EU countries you pay easily 100 Euros. And for Europe, this is a lot!
The Emerson quartet version blew me away when I first heard it. I recommend it over the piano versions because I think you hear the full extent of the voices better (coming from someone that loves and play the piano).
reply