Not a single mention of planning poker and story points?
They're not perfect (nothing is), but they're actually pretty good. Every task has to be completable within a sprint. If it's not, you break it down until you have a part that you expect is. Everyone has to unanimously agree on how many points a particular story (task) is worth. The process of coming to unanimous agreement is the difficult part, and where the real value lies. Someone says "3 points", and someone points out they haven't thought about how it will require X, Y, and Z. Someone else says "40 points" and they're asked to explain and it turns out they misunderstood the feature entirely. After somewhere from 2 to 20 minutes, everyone has tried to think about all the gotchas and all the ways it might be done more easily, and you come up with an estimate. History tells you how many points you usually deliver per sprint, and after a few months the team usually gets pretty accurate to within +/- 10% or so, since underestimation on one story gets balanced by overestimation on another.
It's not magic. It prevents you from estimating things longer than a sprint, because it assumes that's impossible. But it does ensure that you're constantly delivering value at a steady pace, and that you revisit the cost/benefit tradeoff of each new piece of work at every sprint, so you're not blindsided by everything being 10x or 20x slower than expected after 3 or 6 months.
One of the other markers of "true MVC" I look for is that you ought to have pervasive mixing and matching of the pieces. It is common for models to see some reuse in multiple "views" and "controllers", but if all or almost all of your controllers match up to precisely one view, then you're burning design budget on a distinction that is buying you nothing. If you've got strictly one-to-one-to-one matches for each model, view, and controller, then you're just setting your design budget on fire.
Another major aspect of the original "true" MVC is multiple simultaneous views on to the same model, e.g., a CAD program with two completely live views on the same object, instantly reflecting changes made in one view in the other. In this case MVC is less "good idea" than "table stakes".
I agree that MVC has fuzzed out into a useless term, but if the original is to be recovered and turned into something useful, the key is not to focus on the solution so much as the problem. Are you writing something like a CAD program with rich simultaneous editing? Then you probably have MVC whether you like it or realize it or not. The farther you get from that, the less you probably have it... and most supposed uses of it I see are pretty far from that.
Demonstration of capability will get you hired, capability comes only through practice.
I built a hobby system for anonymously monitoring BitTorrent by scraping the DHT, in doing this, I learned how to build a little cluster, how to handle 30,000 writes a second (which I used Cassandra for - this was new to me at the time) then build simple analytics on it to measure demand for different media.
Then my interview was just talking about this system, how the data flowed, where it can be improved, how is redundancy handled, the system consisted of about 10 different microservices so I pulled the code up for each one and I showed them.
Interested in astronomy? Build a system to track every star/comet. Interested in weather? Do SOTA predictions, interested in geography? Process the open source global gravity maps, interested in trading? Build a data aggregator for a niche.
It doesn’t really matter that whatever you build “is the best in the world or not” - the fact that you build something, practiced scaling it with whatever limited resources you have, were disciplined to take it to completion, and didn’t get stuck down some rabbit hole endlessly re-architecting stuff that doesn’t matter, this is what they’re looking for - good judgement, discipline, experience.
Also attitude is important, like really, really important - some cynical ranter is not going to get hired over the “that’s cool I can do that!” person, even if the cynical ranter has greater engineering skills, genuine enthusiasm and genuine curiosity is infectious.
> maintenance activities after the taxpayer places the computer software into service
This is the part that I think makes this whole jig of treating software development like a purely capitalizable expense so nuts.
I previously worked at a public company that wanted software developers to treat as much work as possible as CapEx - it makes you look more profitable than you actually are, which is bad for taxes but good for your stock price. Developers hated it. The problem with it is that with modern web based software, CI/CD, A/B testing, etc. that the line between "new software" (i.e. CapEx) and "maint" (i.e. OpEx) is so blurred as to be pointless. E.g. many times I'd be fixing a bug (technically OpEx) but that would often lead to some new features ideas, or ways to structure the software to make it easier to add new features (technically CapEx). Software is fundamentally different from capital expenditures in other areas, and assuming a 5 year straightline depreciation schedule for software is laughably absurd.
What other sort of capital expenditure has you do releases every day, or requires 24/7 monitoring? I would argue that the business of software has changed so drastically over the past 20 or so years that it makes much more sense just to categorize it as OpEx by default (for both tax and GAAP purposes), and only have it be capitalized as CapEx for very small and specific reasons.
Another interesting direction you can take reservoir sampling is instead of drawing a random number for each item (to see whether it replaces an existing item and which one), you generate a number from a geometric distribution telling you how many items you can safely skip before the next replacement.
That's especially interesting, if you can skip many items cheaply. Eg because you can fast forward on your tape drive (but you don't know up front how long your tape is), or because you send almost your whole system to sleep during skips.
For n items to sample from, this system does about O(k * log (n/k)) samples and skips.
Conceptually, I prefer the version of reservoir sampling that has you generate a fixed random 'priority' for each card as it arrives, and then you keep the top k items by priority around. That brings me to another related interesting algorithmic problem: selecting the top k items out of a stream of elements of unknown length in O(n) time and O(k) space. Naive approaches to reaching O(k) space will give you O(n log k) time, eg if you keep a min heap around.
What you can do instead is keep an unordered buffer of capacity up to 2k. As each item arrives, you add it to the buffer. When your buffer is full, you prune it to the top k element in O(k) with eg randomised quickselect or via median-of-medians. You do that O(2k) work every k elements for n elements total, given you the required O(n) = O(n * 2*k / k) runtime.
Over all of history, there is no accounting for what "the mainstream" decides to believe and do. Many people (wrongly) think that "Darwinian processes" optimize, but any biologist will point out that they only "tend to fit to the environment". So if your environment is weak or uninteresting ...
This also obtains for "thinking" and it took a long time for humans to even imagine thinking processes that could be stronger than cultural ones.
We've only had them for a few hundred years (with a few interesting blips in the past), and they are most definitely not "mainstream".
Good ideas usually take a while to have and to develop -- so the when the mainstream has a big enough disaster to make it think about change rather than more epicycles, it will still not allocate enough time for a really good change.
At Parc, the inventions that made it out pretty unscathed were the ones for which there was really no alternative and/or no one was already doing: Ethernet, GUI, parts of the Internet, Laser Printer, etc.
The programming ideas on the other hand were -- I'll claim -- quite a bit better, but (a) most people thought they already knew how to program and (b) Intel, Motorola thought they already knew how to design CPUs, and were not interested in making the 16 bit microcoded processors that would allow the much higher level languages at Parc to run well in the 80s.
Pay a lot of attention to realizing you (and all of us) are in "boxes". This means what we think of as "reality" are just our beliefs" and that "the present" is just a particular construction. The future need have no necessary connection to this present once you realize it is just one of many possible presents it could have been. This will allow you to make much more use of the past (you can now look at things that didn't lead to this present, but which now can be valuable).
Relational databases are not the preferred storage mechanism at Amazon. If a team wants to use an OLTP relational database it’s possible that it will be a decision they will need to defend at any kind of design review.
Of course there are relational databases running OLTP workloads, but it’s far away from the norm. There was a program a while ago to shift many RDBMS systems onto something else.
> Trying to boot the full service on a single machine required every single developer in the company installing ~50ish microservices on their machine, for things to work correctly. Became totally intractable.
This is certainly one of the critical mistakes you did.
No developer needs to launch half of the company's services to work on a local deployment. That's crazy, and awfully short-sighted.
The only services a developer ever needs to launch locally are the ones that are being changed. Anything else they can consume straight out of a non-prod development environment. That's what non-prod environments are for. You launch your local service locally, you consume whatever you need to consume straight from a cloud environment, you test the contract with a local test set, and you deploy the service. That's it.
> I guess one can grumble about bad architecture all day but this had to be solved.
Yes, it needs to be solved. You need to launch your service locally while consuming dependencies deployed to any cloud environment. That's not a company problem. That's a problem plaguing that particular service, and one which is trivial to solve.
> Both FAANG companies I’ve worked at had remote dev environments that were built in house.
All FANG companies I personally know had indeed remote dev environments. They also had their own custom tool sets to deploy services locally, either in isolation or consuming dependencies deployed to the cloud.
This is not a FANG cargo cult problem. This is a problem you created for yourself out of short-sightedness and for thinking you're too smart for your own good. Newbies know very well they need to launch one service instance alone because that's what they are changing. Veterans know that too well. Why on earth would anyone believe it's reasonable to launch 50 services to do anything at all? Just launch the one service you're working on. That's it. If you believe something prevents you from doing that, that's the problem you need to fix. Simple. Crazy.
On the "worse is better" divide I've always considered you as someone standing near the "better" (MIT) approach, but with an understanding of the pragmatics inherent in the "worse is better" (New Jersey) approach too.
What is your actual position on the "worse is better" dichotomy?
Do you believe it is real, and if so, can there be a third alternative that combines elements from both sides?
And if not, are we always doomed (due to market forces, programming as "popular culture" etc) to have sub-par tools from what can be theoretically achieved?
* Best introduction: Financial Intelligence, Revised Edition: A Manager's Guide to Knowing What the Numbers Really Mean (Berman, Karen, Knight, Joe, Case, John, imusti)
* Then read this, all of it, except perhaps most of the exhibits: Most recent Costco annual report (10-k).
* Berkshire Hathaway annual letters, Transcripts from Berkshire annual meetings
* Great follow-up: Financial Shenanigans, Fourth Edition: How to Detect Accounting Gimmicks and Fraud in Financial Reports (Schilit, Howard, Perler, Jeremy, Engelhart, Yoni, McGraw-Hill Education)
* Absolute classic (but needs to be updated): The Intelligent Investor: The Definitive Book on Value Investing. (Graham)
* On investing in general: Margin of Safety (Seth Klarman)
* Great intro to valuation, with the caveat that modern finance and concepts like WACC may not necessarily be useful to your investing strategy: Valuation: Measuring and Managing the Value of Companies (Tim Koller, Marc Goedhart, David Wessels)
* Another classic (but needs to be updated): Security Analysis (Graham and Dodd)
I've read those in more-or-less that order, interspersed with plenty of other useful books that you'll discover depending how deep you go. The most important thing is reading 10-ks.
My biggest problem with 3rd party feature flag setups is that I have high expectations for them and it is technically difficult to meet all of them:
- local/static access: it should not have to call out to a 3rd party server to get basic runtime config
- unused-flag detection: flags should have three reported states: never used, recently used, not recently used. These will be different from the user-controlled states of active, inactive, etc.
- sticky a/b testing: should follow the logged in user until the flag is removed
- integration with logger: I should be able to use it with my logger out of the box to report only relevant feature flags. Alternatively can provide a packed value of all relevant flags, would probably have to do flag state versioning.
- integration with linter: should warn me if flag has not recently been used or I used a flag in the code that is not in our database (alternatively, will upsert the flag automatically if it doesn’t exist)
- hashed flag names on frontend build: prevent the leakage of information, not a perfect solution, but I would want to avoid writing “top-secret-feature” where we can.
I fully acknowledge that a lot of solutions come close, but I haven’t looked at the current state of things in the last few years so it may have improved.
I would focus on theory first, then tools. I recommend the following, in order:
1) Data modeling: Fully read Kimball's book, The Data Warehouse Toolkit, 3rd edition. It's outdated and boring but it's an excellent foundation, so don't skip it.
2) Data modeling: After #1 above, spend at least 30 minutes learning about Inmon's approach, the ER Model, 3NF, etc. You can find some decent YouTube videos. You don't need to learn this deeply, but you should understand how it's different than Kimball's approach.
3) Data warehousing & data lakes: Read the academic paper titled "Lakehouse: A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics." For an academic paper, it's surprisingly approachable.
4) If you don't know this already, do 15 minutes of googling to understand ETL vs ELT and the pros and cons of each approach.
5) Putting it all together: Once you've done the things above, then read Kleppman's book, Designing Data-Intensive Applications.
6) Focus on the future: Spend a little time learning about streaming architectures, including batch/microbatch/streaming and pros and cons for each. You may also want to learn about some of the popular tools, but you can easily google those.
One related aspect that the article doesn't go into too much is that there is a tension in the size of the unit of code review: there are reasons for reviewing big chunks at once, but also reasons for reviewing individual changes that are as small as possible. I've gone into more detail on this in the past.[0]
Stacked diffs make that possible because you can review either individual commits or an entire stack at once.
The irony is that this is largely the way that Linux kernel development works -- and the Linux kernel was the first user of Git! Most projects who later adopted Git seem to have never learned this lesson that was there from the beginning, and have since been reinventing wheels.
I am in a math-learning Discord server led by a PhD guy, where everyone works through math books recommended by him, solves the exercises and posts the solutions to get checked. Of course, you can always ask questions.
All of the math is proof-based, so we start with books that teach just that: proof writing, basic logic and set theory. Then you can branch out and learn what you like. Each person goes at their own pacing.
One thing is that you will have to put a lot of effort into learning yourself; there is no silver bullet, regardless of whether you know proofs or not.
If you want to join, you can PM u/CheapViolin on Reddit.
Having TL'ed both Frontend and Backend at Google, I don't think it's front first or back first. It's "contract first", and here is why:
- Assuming PRD and UX are ready. The Frontend team take a look and come up with the Frontend data model they want. The focus is on making sure it's easiest for them to maintain and develop the client side.
- The Backend team take a look and come up with the Storage data model that is flexible enough for short term future. The Backend should anticipate the growth of the product and try not to be put in a situation where they have to redesign the database/infrastructure.
- Then both teams meet to negotiate the "Contract", which is the API layer. In some case, the API is more similar to the Frontend data model. In some case, the API is more similar to the Storage data model. In other case, there is a translation layer between Storage <-> API <-> Frontend (not as desirable, but not the end of the world). There are standard on how API should be designed (https://aip.dev/).
- When the contract is established, both teams can run more independently to each other, release at different times, and only needs to sync when there is a need for the contract to change. The API is designed to be easily extended, but not changed or deleted. The Backend team doesn't care if the Frontend data model exists. The Frontend team doesn't care what kind of Storage the Backend uses. It's an abstraction.
The complexity of the project can't be entirely eliminated. A good TL makes explainable trade offs on which parts of the system bear the complexity. It's their job to think not only about the architecture, but also how the development can be sustained in the future. This is especially true when the product is developed by a medium-large team. The focus is no longer on having the leanest/optimal code base. The focus is on making sure many parts of the system can move more independently, in parallel, and that it's scale-able to the growing business needs. It's a classic case of Amdahl's Law applicable to real life.
Of course, this is not always applicable to small projects that is never intended to have more than a couple engineers. Even then, when I work on solo full stack projects, I still unconsciously do this contract first approach.
I'd add two things, from a decade of experience at Google:
Review code in this order: protocol buffers, unit tests, headers, implementation. It's common for a new employee to be an expert on C++ or Java or whatever languages but it's very uncommon to meet anyone who knows how to define a decent protocol message. The tests should give the reviewer a very clear idea of what's happening in the code (and this should be congruent with the description of the change), if not send back to author at this point. The headers should contain clear interfaces, types, and comments and should not contain anything that's not part of the API (when this is technically possible). Finally look in the CC file; at this point the reviewer should see things they were already expecting and no funny business.
Any claims about speed, efficiency, or performance whether in comments or during the review must have accompanying microbenchmarks or they should be deleted. Wrong ideas about software performance abound, and even correct beliefs become incorrect with time, in which case the microbenchmarks are critical to evaluating the continued value of the code.
When sending a changelist for review, always clear all automated warnings or errors before wasting the reviewers' time. Nobody wants to see code that doesn't build, breaks a bunch of tests, doesn't lint, etc.
They're not perfect (nothing is), but they're actually pretty good. Every task has to be completable within a sprint. If it's not, you break it down until you have a part that you expect is. Everyone has to unanimously agree on how many points a particular story (task) is worth. The process of coming to unanimous agreement is the difficult part, and where the real value lies. Someone says "3 points", and someone points out they haven't thought about how it will require X, Y, and Z. Someone else says "40 points" and they're asked to explain and it turns out they misunderstood the feature entirely. After somewhere from 2 to 20 minutes, everyone has tried to think about all the gotchas and all the ways it might be done more easily, and you come up with an estimate. History tells you how many points you usually deliver per sprint, and after a few months the team usually gets pretty accurate to within +/- 10% or so, since underestimation on one story gets balanced by overestimation on another.
It's not magic. It prevents you from estimating things longer than a sprint, because it assumes that's impossible. But it does ensure that you're constantly delivering value at a steady pace, and that you revisit the cost/benefit tradeoff of each new piece of work at every sprint, so you're not blindsided by everything being 10x or 20x slower than expected after 3 or 6 months.