An unfortunate title, completely devoid of any hint about the Very Interesting content in this post:
This is an insider story chronicling the evolution of Twitter 1.0's approach to build infrastructure and developer efficiency strategies at scale on teams of hundreds to thousands of SWEs.
-- long hauler here - I used to be the smartest person people knew - everyone would always say "wow.. you're so smart" - my memory was extremely good - I could remember most conversations I'd had - how I'd learned the things I knew came with ease - I'd be the one people would always pull into conversations to help problem solve - now - I don't know how to describe it - I know the stuff is still in there - but it's like the map to the knowledge has faded - now I spend a lot of time remembering how to remember - it's been particularly difficult as - for better or worse - I really defined myself as the smart person in the room - I really wish I could get back to how I used to be - I've tried everything I can think of but so far nothing has really helped - to be honest when I first realized what was going on I became suicidal - now I've learned to accept it - however I still hide myself away from everyone --
It's worth noting that a lot of the early database designs, including this 2018 video pre-date some dramatic improvements to dynamodb usability.
I think the biggest ones were:
- an increase in the number of GSIs you can create (Dec 2018) [1]
- making on-demand possible [2]
- an increase in the default limit for number of tables you can create (Mar 2022) [3]
I don't think these new features necessarily make the single-table, overloaded GSI strategy that's discussed in the video obsolete, but they enable applications which are growing to adopt an incremental GSI approach and use multiple tables as their data access patterns mature.
Some other posters have recommended Alex DeBrie's dynamodb book and I also think that's an excellent resource, but I'd caution people who are getting into dynamodb not to be scared by the claims that dynamodb is inflexible to data access changes, since AWS has been adding a lot of functionality to support multi-table, unknown access patterns, emerging secondary indexes, etc.
I have quarter of century experience working on Linux including running it on all my laptops and desktops for 23 years.
Over the years keeping it working started becoming what seems like a full time job and I jumped the ship for M1 MacBook Pro. It still has a lot of problems but at least I have more time to do my actual work.
I understand I might just becoming less tolerant of wasting time on things that should just work (my actual work being more and more distant from tinkering with OS internals).
But, if I with all my experience am spending time keeping it alive, what has to do a person that has no idea about systemd or pulse audio or dealing with failed upgrades or anything else? Essentially -- you are using it until first problem happens and then you can either reinstall it from scratch or switch to Windows or Mac.
I still keep a Linux laptop dedicated for critical security tasks (I don't trust Apple this much) and a number of Linux VMs (I am running small datacenter at home).
Every time these stories pop up, I'm amazed at how much infrastructure has built up around crypto. Entire websites with reporters that do nothing but cover crypto stories. Layers upon layers of derivatives built on top of coins I've never heard of, which themselves are built upon structures on top of ETH or BTC. Hedge funds. Analysts. Exchanges. Fintech. So much money swirling around a Greater Fool scam.
The fact that it can do more is in no way a negative. Can even make a limited JSON parser with reduced capabilities. And with JSON can do more definitions like header names vs column names vs just arrays of arrays.
What I found interesting is that the period for reappearance of “failed ideas” is roughly the length of a career. Coincidence or due to any real effect I don’t know. I wrote a little about it here:
“Don’t be discouraged by the failure of technology or approaches of the past. If the problem is still important after all this time then it’s a problem worth solving. Don’t avoid unsuccessful solutions. Avoid insignificant problems. Success this time around could be unlocked by advances in any number of unrelated disciplines.”
My guess is that this advice of "raise less money" is a result of hanging around too many successful founders.
That is, if you talk to successful founders, they will generally wish they raised less money (due to dilution).
And, if you talk to failed founders, they will generally wish they raised more money (to increase likelihood of true PMF).
Also, I think that fear is a useful mental state when there is real and imminent danger. In startup land, you are right to be fearful, because you are much more likely to die than stay alive.
If you don't have the luxury of unlimited shots at success (due to a lackluster safety net or other life goals), it makes sense to maximize the likelihood of success in your current venture. There is a lot of "startup cost" to working on a new idea, and in a lot of fields, you will eventually succeed if you just stay alive/don't die.
* Gut feel. This doesn't say much, but sometimes that is a good heuristic. It isn't one thing, but you'll just get a feeling.
* Custemers are not signing up and not paying. The most obvious one. But this can be hard to find. But say if you are used to add features or fix bugs customers find, and then all of the sudden nothing. It could mean product is in good shape, or nobody is using it.
* Senior people leaving. Especially developers whose judgement you trust.
* Drastic changes to the product. "Wait, why are we pivoting?" Corrolary: quite often the pivot fails. It is always in the news if it succeeds. but that is only because it is exceptional.
* No bonuses, and salaries have not been going up. If you somehow find out that nobody's salary went up and nobody got a bonus (this is hard often, companies don't want you to discuss those things).
* Maybe a sharp increase in team-building activities. Taking everyone for lazer tag is cheaper than increasing salaries. But it presents this image of "everything is fine". I have seen that -- a few senior people left. All of the sudden a sudden surge of minigolf, bbq, ski trips and other to mask away the issues. But then again, this can be a sign that the team is doing so well and are being rewarded. I guess this is more of a gut feel and depends on the large context (so it is a secondary sign).
* Owners start avoiding meetings and questions. Because they know they might have to lie. It is up to you, but you can try to ask directly. Then you sort of force their hand. That could backfire. You are now "not a team player" and not a "culture fit" so beware.
This is an insider story chronicling the evolution of Twitter 1.0's approach to build infrastructure and developer efficiency strategies at scale on teams of hundreds to thousands of SWEs.