Reading this was a good reminder not to be intimidated by assumptions about complexity. (Without giving it much thought) I would have assumed that it would be hard to replace malloc for such fundamental applications as ls, but it's surprisingly simple.
There's usually an easy-ish way to override malloc/calloc/realloc/free on Unix, as it's very useful to do when debugging issues or just to collect allocation metrics.
In ELF objects (i.e. on Linux) this is usually done with the "Weak" symbol binding. This is an optional flag for symbols in ELF format that let you override a symbol by providing a competing non-weak symbol, which the linker will prefer when there is a conflict. https://en.wikipedia.org/wiki/Weak_symbol
You can see the list of Weak symbols by looking for a 'W' in the output of `nm` on linux hosts.
This applies to a lot of things unfortunately. There is a cult of just being afraid and scaring other people.
"You can't do it, just use a library.". "Just use this library, everyone uses it.". "Even google uses this library, do you think you are better." etc.
To add another example to this, you will read that memcpy is super mega optimized on libc and you shouldn't do it yourself etc. etc. etc.
But if you just check clickhouse [1] as an example. They implemented it, it is pretty basic and they say it works well in the comments of the code.
Also you can check musl libc code etc. and it is fairly simple.
People still would argue that you used some intrinsic so it isn't portable or you just benchmarked on one case so it won't work well overall.
Well you CAN benchmark as wide as you want inside your project and have a different memcpy code per project. This kind of thing isn't as bad as people make it out to be in my opinion.
Ofc memcpy is just an example here and it applies similarly to memory allocation, io etc.
As a negative note, imo this is one of the major reasons why most software is super crappy now. Everything uses some library -> those libraries change all the time -> more breaking -> more maintenance. Similar chain happens in terms of performance because the person that wrote that library probably doesn't even know how I am using the library.
This is also why people have endless arguments about what library/tool to use while they can be learning more and more things every day.
The killer feature of DTs is how fast they can be. I worked very hard on a project to try and replace DT based classifiers with small NNs in a low latency application. NNs could achieve non-trivial gains in classification accuracy but remained two orders of magnitude higher latency at inference time.
Also, decision trees (but not their boosted or bagged variants) are easy (well, easy-ish) to port manually to an edge device that needs to run inference. Small vanilla NNs are as well, but many other popular "classical" ML algorithms are not.
> It's always funny how many people think that the only font of altruism is taking care of children who have your DNA, like that's some kind of selfless act
This is a strawman position in my opinion. I don't think there's that many people who think they're carrying out some selfless act by having children. It's simply biologically true that the children you'll probably have the easiest time raising are your own and, assuming we want to continue as a species, we do need people to have children. It's fine to have them, fine to not, neither side has some moral high ground.
Personally I want to have my cake and eat it here. Tech has amazing potential to make the world a better place to live in and genuinely bring people together. The crowning achievement of AI so far to me is not Claude Code, it’s AlphaFold. I find the documentary DM released about developing it inspiring both as a technology story but also a team achieving things together that make the world better. I want to see more of that and hope I can steer my career in that direction.
Yeah, hopefully an outgrowth of this will be new amazing applications like that, that we never could've dreamed of before. I imagine "distributed services" will be "solved" by EOY, and the days of glorified CRUD app coders making 200K straight out of college are over for good.
But I think there will be new opportunities for people who are willing and able to learn. Entirely new fields will pop up and somebody will have to work on them. Most likely, the CS grads who are out of a job, or just frustrated and want to do something else.
So I don't think the opportunity to do innovative things and make a difference in the world is gone. But the opportunity to do so by typing code into a text editor may have breathed its last.
> author (pilot?) hasn't generally thought too much about the problem space
I’ve stopped saying that “AI is just a tool” to justify/defend its use precisely because of this loss of thought you highlight. I now believe the appropriate analogy is “AI is delegation”.
So talking to the vibe coder that’s used AI is like talking to a high level manager rather than the engineer for human written code
AIUI this might not be as bigger problem as you’d imagine. Aerodynamics dominate most of the resistance at medium to highway speeds. This was discussed in the recent Rivian R1T cannonball run conversion videos where they doubled the battery capacity (and therefore drastically increased the weight of the vehicle) without substantially affecting efficiency.
Yes, weight alone is not that important for fuel economy at highway speeds. Aerodynamic efficiency of a vehicle is determined by the cross section area + drag coefficient. A 90s Honda Civic has a worse drag coefficient but a vastly smaller cross section than the Rivian, so it needs to move less air out of the way to drive forward. Smaller EVs can go farther with smaller batteries, but the American consumer has decided for one reason or another that cars must be tall, and therefore less aerodynamically efficient.
I currently have a landscape designer planning our yard landscaping. When I see the impressive renderings they produced I immediately thought that it’s some idealized version of how it will look on a sunny day after 10 years of bedding in. I asked them to also produce renders of how it’ll look on a gloomy winter day 6 months after planting everything. Seems they don’t have the tools to produce those images really though
I get stuck on asking “why am I solving this problem” too much. I am surrounded by technical problems that it would give a dopamine hit to solve and I’d feel the pleasure of helping my fellow man, but 99% of them feel like they shouldn’t even exist and solving them doesn’t really lead to any meaningful progress beyond providing me job security and money. (How) do people deal with this?
Deciding which problems should be solved, identifying where there is business value in solving them, is pretty much the definition of business leadership.
I think the only real answer is moving into management, where you can more effectively argue against spending effort on things that aren't worthwhile.
Well that’s not what I wanted to hear! I think you’re right though, you get to choose your challenge: do you want your problem to be possibly working on things that don’t really matter or be responsible (and empowered) to figure out what really matters.
Yeah it sucks but unfortunately this is the conclusion I came to after pondering on this for my own career. I think you phrased it as well as I've ever seen it put though.
And of course there is always the other options. For myself I didn't relish either choice and now I teach programming and CS. But I'm old and this feels like a good way to end my working years.
Thanks for sharing this detail. I've been interested in taking some form of flight training for a long time and finally have the financial means to do it, but I haven't decided whether to go with glider or powered flight. Your comments makes me realize that the time commitment might be larger than I can manage at this stage (two kids 1 and 5) and so may need to wait until a little later in life.
I want so badly to try and talk you into flying gliders, because it's amazing: way, way more fun (IMO) than the overwhelming majority of the power flying you can do as a civilian. And there are people who make it work! But ... probably this is wise, and better to make thoughtful decision.
It seems like this gets easier as the kids get older; I've seen parents of teenagers make it work. For some folks it's a family affair---several kids & one or both parents learning to fly. These families have been uniformly super fun to have at the field! For others I think it's a matter of the kids being old enough to have some independence, + I'm sure a very supportive partner.
So don't give up on gliders forever! I'm expecting times in my life when I can't get out to the field enough to stay proficient & safe, and I'll have to quit flying for a little---but a dearly hope not forever. It's just that, likewise, now's maybe not the right time for you.
reply