One of the fun things about linear algebra is that you’re working with lots of simple equations all at once. It’s like moving from single variable calculus to multi-variable calculus. You realize that the additional complexity was there all the time, and you were just studying a special, restricted case before.
Yes, and it's super useful. The first times you find yourself working with several equations and you 'just' put that into a matrix form and solve that system is an amazing feeling of 'truth' having been hiding in plain sight.
Absolutely. It’s easy to forget how useful it is to be able to solve a system of equations like that. It lets you find optimal settings for complex systems with multiple constraints. Fitting a machine learning model is just one application for this... there’s a whole academic discipline called Operations Research that focuses on using optimization to find solutions to practical problems.
Just another example of the huge power in creating accurate digital models of the world.
We touched a bit of that. As an aside, when I was a student, we also had an applications module called "Numerical Analysis". We went through a bunch of algorithms (Cholesky, Seidel, Newton-Raphson, Gauss, etc.) and ran them by hand. We basically were "human computers". The exams were also solving problems using these algorithms, and given the iterative nature of many of them, if you make a tiny mistake it'll ripple and ruin everything for you.
As an aside, my background is in optimal control/control theory and instrumentation, and operational calculus, state space representations, and matrices were a huge part of the "Jiu-Jitsu" we did. Tinkering with RST control, robust systems, finding optimal systems and sequences (Hamilton-Jacobi-Bellman-Pontryagin) with matrices flying around on our exam papers solving these systems. It was a nice abstraction.
Very, very, powerful tools that command immense respect for Laplace, Lagrange, and people like them who invented things to solve problems we're facing now.
