Does anyone have any good resources for learning how to better architect systems? Other than, say, The Architecture of Open Source Applications: http://aosabook.org/en/index.html
Reading source code of prominent projects is a good way to get better, but it's hard to understand the architecture if you aren't aware of the design constraints/requirements that led it to be the way. Surely there must be a way to fast track this to some extent. Under the deliberate practice model by Anders Ericsson's (i.e. the 10,000 hour rule), simply reading and understanding code is suboptimal because of the lack of immediate feedback on your understanding. I think it'd be interesting to see tutorials/guides that showed how a system design has changed throughout its history as well as the reasons for those changes.
I beleive that there is one answer to this question which does 80% of the job, it is easy to express and yet in the same time impossible to grasp for so many programmers/managers which leads to so many awful codebase.
As the post says very well: start by thinking about what you would do if you had infinite resources. That's the ideal case. Find a real bottleneck (money, cpu, memory, ?) and rethink your architecture accordingly, find the next real bottleneck and so on.
This is gold. I remember a quote from a prominent game programmer, I think it's John Carmack but I'm not sure, that when thinking of an approach to a new game engine he would start on the basis of an imaginary computer with instant CPU calculation and infinite memory. I can see the parallels here.
In my experience, there is no better way than to build it. You learn quickly from your failures and become aware of subtle issues that no text-book or article will ever touch since they are so domain specific.
The best way to learn by doing, IMO, is by joining an organization that works on a large and complex code base. It won't be as exciting as writing new code, but adding to an existing complex system will force you to take into consideration things that you otherwise would not.
parent has some great points. You only will really learn by doing.
A couple of things you can do is start reading up on well designed systems and the gotcha stories. Personally, I track highscalability.com since I spend a lot of time working in distributed systems. They tend to have interesting insights and will link to tech talks and blogs that contain a wealth of information. Also go read up on some of the papers explaining the designs behind systems you use (e.g. Amazon dynamo) and why they made the choices they did. It'll give you some insight into how people measure tradeoffs and figuring out what's important.
Take a task of some inherent complexity but that can still be coded in 10-20 pages of code or less, create one solution, and then try out various different versions of it, or even complete rewrites, changing coding style, decomposition into modules, performance tradeoffs taken, data structures chosen and so forth, and then compare those with respect to length, code clarity, create benchmarks, etc.
Example projects: ray tracer, hash table (way more choices and trade-offs to be made than you would expect), toy grep, toy pseudo-pascal or lisp compiler, basic web crawler, ... It's hard, but you will learn a ton. Even putting substantial effort into improving a one page program, incubating and trying out ideas over a few days, can be beneficial if there is some "meat" in that program to chew on.
In general I believe software architecture is much more low level and less philosophical than a most aspiring architects believe. It's really about knowing the trade-offs of a decision before making it, and this rests on knowing a lot about data structures, algorithms, existing components (like databases, operating systems) etc. Those little programs in non-trivial problem areas often already show up a lot of those things.
It applies to all creative work, but I've had good experience with "forced alternatives." That is to say,
Forcing myself or my team to come up with three options for architecture even if the first one seems to work. The process often leads to great instead of good outcomes because in scenarios 2 and 3 you're free to moon-shoot as you have the safety net of a workable option #1.
One strategy I've found works for me is to keep a document explaining why you designed the system the way you did. What were the alternative possibilities? Why didn't you pursue them instead? Do so without referencing projects or implementation details, and derive your reasoning from first principles.
I revisit the document as I'm working on a project, so when I go to code a new feature, I make sure I can justify the need to create it with respect to the larger design of the system it lives in.
This reminds me of a tip Guy Steele gave in his talk on the Language Design panel of the Dynamic Languages Wizards Series in 2001 [1]. He said: It's important to establish design principles early on, that is in effect to do meta-design, because as you evolve the design, or as the language grows later, you want to remain true to an initial vision. This has to do with keeping the description small, and if you can resort to a meta-principle to help you resolve a particular design decision, it keeps the language more consistent. (17:09-17:29) [2]
This is a good book, though a few years old now:
Beautiful Architecture: Leading Thinkers Reveal the Hidden Beauty in Software Design
(http://shop.oreilly.com/product/9780596517984.do)
Disclaimer: I'm the lead developer for one of the featured projects (though wasn't at the time).
It would be faster to be mentored by (/work with) someone who knows how to architect systems. The better your mentor, the better you will be. If one thinks he is already good enough at architecting systems, obviously he will not improve very quickly. There is always someone better.
Reading source code of prominent projects is a good way to get better, but it's hard to understand the architecture if you aren't aware of the design constraints/requirements that led it to be the way. Surely there must be a way to fast track this to some extent. Under the deliberate practice model by Anders Ericsson's (i.e. the 10,000 hour rule), simply reading and understanding code is suboptimal because of the lack of immediate feedback on your understanding. I think it'd be interesting to see tutorials/guides that showed how a system design has changed throughout its history as well as the reasons for those changes.