There seems to be something similar in other branches of mathematics too (and lots of other fields). A clear example I recall was studying Fourier transforms in math and I couldn't make any sense of it. To me it was just some by-rote algebra with integrals of exponentials of complex numbers.
But then I happened to do some audio signal analysis, and when I saw a magnitude spectrum of a waveform, it was instantly obvious what's going on (and understanding the phase part after this was no problem). Such practical examples seem to be almost banned from university math, I'm guessing to make sure everything is very abstract and rigorous (e.g. you have to work with finite length and discretized signals where the maths don't strictly apply). And after getting this intuition the formal math started to make sense too.
But when one begins to teach, it becomes quite easy to see why things are like this. All of these things are so obvious to the teacher that it's hard to understand how one thinks before these are obvious and the standard notation/vocabulary is typically a good way to work with these, but only after understanding the stuff.
What I try to do is to probe out something that the student already knows, which may be from a totally different field, and find a simple example in the new topic so I can say that "this is exactly the same thing, but with this different notation/abstraction". This very often causes the things to "click" for them.
This is of course very hard to do with a textbook or a mass lecture. And probably the main thing why we need humans to do teaching instead of just giving out material.
It's hard if you don't know any practical examples. Anecdote: I taught a college math course for a semester while I was between jobs. I shared an office with some other teachers including a bright grad student who was TA'ing differential equations.
My degree was in physics, and I had worked in industry. I told him that I wished the math courses had included some engineering applications of differential equations. He looked at me with a straight face and said: "There are no engineering applications of differential equations."
Lol. That's every teacher I had, including in university. It wasn't until I dropped out and years later resumed reading physics and maths for my own interest that I properly understood three point of some of the maths they were teaching.
> Such practical examples seem to be almost banned from university math
For background, I studied theoretical math. I am also amateur electronics engineer.
I agree, theoretical math students and staff are a bit dismissive about everything else. At least where I studied. Where I studied, we would regularly joke about other students claiming that math is hard saying, "what they are learning isn't even math, it is just a bunch of formulae and rote learning". And this included physics or applied math students. Don't even get started about CS students...
It is unfortunate, because math concepts were frequently invented to solve a real physical problem. Or they were invented and we found them extremely useful for a physical problem. Historically people would not put much distinction and would usually do both physics and math and there would be a lot of cross-pollination.
Just read a bit about how Einstein came up general relativity -- it is fascinating story about how Einstein wasn't good at math but needed a problem solved so he reached out to some friends and those friends basically gave him some private lessons until Einstein clicked and figured it out.
For me, I pretty much understood Fourier analysis before I started doing electronics. But the usefulness and practicality of it only clicked after I moved to higher frequency problems and started using frequency domain when working with my circuits. You don't get much significance of it just studying theoretical math.
> I agree, theoretical math students and staff are a bit dismissive about everything else.
I think this does go somewhat both ways. E.g. in the current discussion there are quite a few people complaining about "useless maths". I think a big problem is that different areas of studies don't appreciate that they are, well, different, and it's not trivial to transfer ideas across fields.
> Just read a bit about how Einstein came up general relativity
Also an interesting story about special relativity: Minkowski (Einstein's math professor) proposed very early on to Einstein that special relativity should be formulated geometrically (with what's now called Minkowski spaces), but Einstein deemed it too mathy and unintuitive, only to accept later that it was the right way to go. Without the geometric formulation it's very possible Einstein wouldn't have developed general relativity later on.
> But the usefulness and practicality of it only clicked after I moved to higher frequency problems and started using frequency domain when working with my circuits.
One argument for teaching FT more abstractly is that the "useful parts" are quite different in different fields. E.g. in many applications of sound and acoustics (and probably in EE) the main interest is in the spectrum. But in e.g. statistics the main thing is the convolution theorem and the spectrum has practically no use.
I'd say that the Fourier Transformation doesn't belong in the calculus program at all.
It should be taught in linear algebra, where it does not only make sense, but is a non-trivial example of application that the textbooks have so little of.
Why should it be in linear algebra? You mean as in doing linear algebra with functions in infinite-dimensional spaces? What I recall mostly (not) learning from linear algebra were various classifications of matrices and their properties (which made very little sense to me at the time too).
I'm not exactly sure where FT should belong in the math syllabus. It's heavily related to trigonometry of course but it is an integral transform, so needs a bit of calculus. Although discrete versions are probably easier to grasp with just multiplications and sums, and there is quite rarely much actual integrating as in find-the-closed-form-antiderivative going on.
Maybe trying to have too "unified" syllabus for engineers/scientists is not the best way in practice. Instead there could be more discipline-specific teaching (something TFA hints to too). With ML (and 3D graphics earlier) something like this seems to be happening to linear algebra. The "mathy" linear algebra (that I studied at least) is mostly about properties and decompositions of (complex) matrices, but in ML/3D engineers very rarely use such things (but do need stuff outside "traditional" linear algebra like tensor products, projective coordinates and rotation groups).
In fact, there is a lack of linear algebra in infinite-dimensional spaces on the general curriculum, and it is an important subject for physics and a few engineering areas besides the relevance for mathematicians. It wasn't clear to me how the dependency between calculus and algebra was supposed to be, but your comment makes it crystal clear.
The discrete transformation is quite fitting for finite-dimensional algebra, and honestly I can't understand how anybody every thought teaching the continuous transformation first and the discrete one never was a good idea. The only explanation is that since everything is thrown on the calculus package without any consideration, there's only time for one, so people kept the most general one.
Infinite-dimensional spaces do indeed come up in e.g. Gaussian processes and the kernel trick. I have to admit I've never really understood them the rigorous form. For ML for example it would be likely more useful than e.g. most matrix decompositions.
I'd guess math uses continuous forms because it's where the mathematical tools are and many things tend to get simpler in mathematical sense when you let something go infinitesimal or infinite. This could maybe be different if digital computers would have been invented before calculus.
I've learned to appreciate that mathematicians think of maths quite differently to engineers or scientists. To them the interest is in the "mathematical objects" and their (provable) properties, not the applications or relationships to "the real world" (and this is probably a good thing in itself) and for engineers/scientists it's the opposite. Maybe something like how linguists vs novelists approach language.
It also comes up a lot in the foundations of RL - the basis of how it is justified (or in some cases proven) to work is contraction mappings and functional operators.
Is there a good book for learning ML Linear Algebra? I've taken linear algebra math courses in uni and most of it went in and out one ear.
I'm looking for a book that helps me understand ML algebra, so that when I read research papers I'm not just lost and nodding my head aimlessly. Such a book may not exist, maybe it's a group of books, but if ANYONE would have pointers in this direction, I would be in your debt
At the end of the day, analytical Fourier analysis doesn't get nearly as much practical use as numerical approaches, and you don't really need THAT much background to make sense of DFTs.
I know that, for myself, revisiting Fourier Analysis after going through DFTs in my numerical analysis classes made a lot more sense, and I kinda wish I had started with that angle in the first place.
Analytical fourier analysis is needed for many other fields of math though. For example in statistics it's very central for dealing with density functions. I'd guess in many fields of physics it's also very crucial.
But for a lot of fields you just need the DFT and the calculus stuff can be mostly a distraction. How I finally figured out FT is something like the infinitesimal limit of DFT.
If there's one place for Fourier analysis to be is on linear algebra.
The calculus course is way too bloated for historical reasons that haven't mattered for more than a century. Pushing everything into that context only serves to make the contents hard to understand and seemingly useless.
But then I happened to do some audio signal analysis, and when I saw a magnitude spectrum of a waveform, it was instantly obvious what's going on (and understanding the phase part after this was no problem). Such practical examples seem to be almost banned from university math, I'm guessing to make sure everything is very abstract and rigorous (e.g. you have to work with finite length and discretized signals where the maths don't strictly apply). And after getting this intuition the formal math started to make sense too.
But when one begins to teach, it becomes quite easy to see why things are like this. All of these things are so obvious to the teacher that it's hard to understand how one thinks before these are obvious and the standard notation/vocabulary is typically a good way to work with these, but only after understanding the stuff.
What I try to do is to probe out something that the student already knows, which may be from a totally different field, and find a simple example in the new topic so I can say that "this is exactly the same thing, but with this different notation/abstraction". This very often causes the things to "click" for them.
This is of course very hard to do with a textbook or a mass lecture. And probably the main thing why we need humans to do teaching instead of just giving out material.