Learning stuff that doesn’t help in work(calculus is not helpful for 99% of software engineering) is really hard if you don’t base it in reality I find. Maybe it’s just me but I would never read a text book for fun so suggesting learning by reading a text book seems crazy. Calculus can be fun and interesting but the teacher has to actively try to make it so. The learning will take longer but you’re more likely to see it through and I think it’s more likely to stick long term too.

I learned way more reading crafting interpreters than I did in my compiler class for example.

That’s interesting because I can’t imagine learning a subject without a textbook. I have a hard time believing another medium would have the depth and density to get all the points across. Although it does depend on the subject matter and one’s learning goals.

But I also do read textbooks for fun… Now that I have a few decades of experience in a lot of these subjects I get way more out of the books. And I can start to understand more of the meta information. Like, of all the things the author could’ve used as an example, why did they pick that. Also, it’s hugely interesting for me to look at the homework problems and theorize why this particular problem was picked. Especially fun for electrical engineering books. But ya, I’m weird like that.

No one is saying it’s impossible to build fully remote teams from scratch, it’s just very hard and requires strong leadership. Most companies have crappy management so they can’t pull it off.

I don’t even think it’s about leadership. Isn’t it weird for somehow to care about people who they only know through the phone or zoom calls? Some people won’t find that weird but I think a significant number of people would, or just can’t do that at all.

My teams velocity is up around 50% because of ai coding assistants.

I think you are getting caught in marketing semantics. The scale is all you need movement was mostly a way to funnel money to LLMs. Did Ilya ever actually believe transformers with no other improvements would lead to AGI, or did he believe that it would lead to a much more useful AI and wanted to raise money for that but found it hard without claiming it would lead to AGI? At the end of the day it is probably a good thing that so much money went into scaling recently, because it did work, as long as your measure of success is more nuanced than “did it lead to AGI”. And even then it may lead to AGI as the amount of money spent on ai research is much higher now and that new money may be what is actually needed.

My issue isn't that they are wrong, my issue is with rewarding those who are wrong. But your argument is it's fine to reward those who lie? I'm not sure how this isn't worse.

I'm sure that you're right that many people used it as a vehicle rather than being just true believers (I know some people that do), but there were also a lot of true believers.

The movement also stopped a lot of research. It has also resulted in a lot of money being dumped into companies betting on it being true. If we are in fact in a bubble (and it looks this way) then all that damage is on the hands of the SIAYN crowd.

Not being a true believer makes it better, it makes it worse. A lie is far worse than being wrong. Being wrong isn't a big issue, especially in the world of research. But lying is a major issue. It ruins it for everyone. We don't have to do this cycle of boom and bust to get things done. That's literally destructive

The issue as I see it is that google invented transformers way before they were released publicly. Clearly there were not enough resources being spent on them which is why the scale movement came about. Would google still be hoarding transformer based LLMs today without Ilya’s hype? Seems like a real possibility to me.

I'm not sure how you get there. The SIAYN movement happened after the AIAYN paper. The latter influenced the name of the former.

So what do you mean by secret? It was published in a paper. That's public

No one was releasing a product based on the paper though. Ilya had to go and raise a bunch of money for that to happen. Maybe I’m just more cynical and accepting of lying as the way things are done compared to you.

Well the question here if LLMs are the cotton gin or if they’re the combine/tractor thing that killed all the farming jobs.

I think it was the GPS, automation (robotics), bioengineered crops, and conglomerates. My point is, I'm pretty sure it's a lot of factors. Even in the cotton gin case. It's probably naïve to give so much credit to one thing

When I was in high school none of my teachers actually read any of the homework we turned in. They all skimmed it, maybe read the opening and closing paragraph if it was an essay. So I guess the question is if having an ai grade it is better than having a teacher look at it for 15 seconds, because that’s the real alternative.

The problem with this strategy is that homework is mostly a tool for learning, not checking progress. Most teachers use homework as a way to get an extra hour or two of learning in each week for the students. If we remove it there will be less learning time available. So you’re gonna have to expand the school day or school year which means more teachers which is expensive.

If homework is not used to check progress, then cheating on it is not a problem.

Don't grade homework. Only grade work done in school. Students who cheat on their homework are just deterring themselves and won't do as good at the in-class, graded exams.

Well it comes down to what the point of school is. If the goal is just to figure out how competent a student is, sure, but if the goal is to educate students then reducing the amount of time they spend on school work will lead to worse outcomes.

Too expensive. Baumols cost disease has made teachers expensive in the US. There aren’t enough to do verbal exams

Every standardized test I have ever taken requires critical thinking, and none of them have been about regurgitating facts.