How to Think About Analysis

Category: Mathematics
Author: Lara Alcock
This Month Reddit 3


by adventuringraw   2019-11-17

haha, well... I've been self studying for maybe 18 months now. Two years next November I guess. I've had to fill in my own prereqs, and it's always the first thing I look at. I bought a book on matrix calculus for econometrics and statistical applications like... over a year ago. I couldn't even get through the first ten pages, the 'linear algebra review' got into things like block matrix algebra and some gnarly identities I still haven't seen yet. That book's probably going to have to wait until I'm all through Terence Tao's Analysis, and maybe up through a few more linear algebra books, haha. I felt like I 'knew' linear algebra after my undergrad, but I mostly worked in R2 and R3. Even after finishing Axler's 'linear algebra done right' though, looks like I've still got a ways to go. There's an intermediate linear algebra book by a guy named Roman I'm eyeing, maybe that'll help close the gap. Ah well, someday. Either way, I try not to buy books wildly out of my range anymore, haha.

So yeah, Prereqs.

Wasserman's might be a bit much. Even harder than the calc will be the rigorous proofs. Have you worked with formal proofs before? A good book to ease into things would be how to think about analysis. I highly recommend it. It will arm you for what it means to tackle higher level math, what high level math even 'is' (it's WAY more about proofs than solving equations) and as a bonus, it'll walk you through the 'true' foundations for calculus. Most college classes just teach you like... here's you how take a derivative, here's a ton of integral tricks. The book above will ground what an integral and derivative even is, it's cool stuff if you're like me, and like to understand the tools you're using. The book will probably only take 10~15 hours given your level, so it's a pretty cheap investment compared to the hundreds of hours Wasserman's would take if you were to finish the whole thing.

Alright. Next thought, calc as it's usually taught in college is a massive collection of tricks for solving various kinds of integrals. In stats and probability, you only need a few tricks for the most part. especially integrals involving an e^-x^2 term, haha. So you don't exactly need all of calc 2 to get around... the problem is it'll be really hard to fill in the gaps when you find them. I was able to bootstrap my way into things I'd forgotten/never learned, but a lot of stats teaching stuff assumes your calc isn't the best, so you can often find detailed proofs where you get confused. I picked up a fair bit that way (l'hospital's rule for example, it was used in a derivation of the gamma distribution I saw).

Incidentally, I remember things I learn by using anki. It's not for everyone, but maybe check it out and try it for a bit. Don't go overboard, and know there's a learning curve around how to make good cards, and there's some work setting it up where you can type in math (LaTex support) and the app for iphone costs $25 (free for android though) but it's helped me to cover as much ground as I've covered. Since I started this stuff again, I've been through a few thousand pages of textbooks (four of them math), and I've got pretty good recall through all of it.

I don't know. Honestly, the real trick of stats is learning to think intuitively about it... I started at Kahn's academy, but feel like I didn't get much insight there. It felt like it was introducing a grab bag of techniques for specific problems, instead of a framework for thinking. Maybe someone else would have a more appropriate suggestion for you... I don't know. I definitely took the hard road, and I'm way better for it, but the textbook might feel more discouraging for you than it did for me. I'm kind of weird... I self taught German and Russian a few years back so I could read through Kafka, Hesse, Dostoevsky and such in the original. I kind of like grinding through impenetrable tomes for some reason, so definitely pick your path based on what will keep you motivated, not just what'll get you there fastest. You can always come back and hit Wasserman's in a year or two after you get more foundation in place. Do a google search for 'wassermans all of statistics pdf'. Many textbooks have pdfs available free online (legally or otherwise) so you can go through the first chapter and see how it feels. If you're intruiged, buy a copy. If it's discouraging, find another road. All roads lead to Rome. The book I used was sub-optimal, but I got here just fine. The real important work is just to find something that's good enough, and throw yourself at it until it's done. Consistent effort even in the not-the-best directions will get you a hell of a long ways. Finishing Kahn's Academy stat class would certainly not be the worst choice.

by adventuringraw   2019-11-17

if math has been a struggle for you, I highly recommend how to think about analysis. It's a book meant for high schoolers heading into college, arming them about what to expect from 'real' math classes. If you want to make a career of this, you have some very serious math ahead of you. The good news though, 'real' math likely looks very different than you think it does... it's often far more about logic than anything else. There's even a proof (the curry howard correspondence) that shows that algorithms and proofs are in some sense the 'same'. That book's a great little quick tour through the 'real' foundations of calculus, as a way of exploring how to learn higher level math. it's a great bit of prep. From there you can tackle linear algebra or whatever else you need to fill in. If you have a math area you want to know, let me know and I can point you toward some good books.

For real though, the life of a coder (and a researcher it seems) is one of constant education. The sooner you learn how to learn on your own, how to manage your time effectively, and how to choose the 'right' rabbit holes while still staying in line with your long term direction... it's a big deal. You won't directly learn that in college, so the sooner you start finding a lifestyle that works for you, the better off you'll be. Figuring out how to work and study and still have a life and stay sane is something I wish I'd gotten better at a whole hell of a lot earlier in my life.

by adventuringraw   2019-11-17

it'll get better, you just have to roll with it. Considering spending some time outside class (or in open office time with the professor) to talk about the guts behind the ideas. If you want some help just muscling through though... check out anki. Read that post and think about trying it. It's the only way I've been able to cover such an absurd amount of ground in the last 18 months... got over 5,000 cards. Takes around 10 minutes a day to keep up with review (long as I don't get behind) and makes it real easy to keep things handy for later.

The reason that's valuable... there are two ways to learn to understand complex systems. The first is from the bottom up... start with the axioms, trace out a DAG of definitions, lemmas, and so on that outlines a map of the space you're working in.

The other though, is top down... learn to use a system until it's in your bones. A lot of puzzle games will be presented more like this. The more complex symmetries and pieces of the puzzle emerge as a flash of insight once you've got hours spent wrestling with a wide range of specific problems. Calc 2 is annoying, I know... but when you hit analysis, this bullshit work you're doing now will ironically be a source of inspiration for understanding more general concepts. I know it sucks, but with anki you can do it with little pain, with extra time outside class you can fit in philosophical understanding if you wish, or you can just coast through rote with an A using the flash cards and come back later to fill in the gaps. If you're in calc 2, you don't have many 'crappy' math classes left, just knuckle down and get through it.

One thing I'd highly recommend though... think about getting into some actual math as a hobby. I know it can be hard to fit in if you have a busy schedule, but it can be really encouraging to have something you're actually excited about, and see where it intersects with the things you have to learn. For a really quick and easy read, I'd suggest you check out how to think about analysis for a taste of what calculus 'actually' is. It rigorously grounds differentiation and integration, explores what it feels like to get into higher math, and introduces a little of what some of you'll see in your first analysis course whenever you get to it. It's pretty elementary, so you could blow through it in a week or two... might give you some encouragement while you're sucking it up through this class.

If you want something more meaty to ground calc with by the way... Spivak's 'Calculus' has been great for the parts I've poked through. I'd like to go through it properly soon. It's basically an intro to real analysis text, going a little slower and more methodically than usual (making sure you 'get' all the ideas) and slowly building up a proper foundation for calculus. If you really want to understand what calculus is all about, those two books are a great place to look if you can free up some time.

by adventuringraw   2019-11-17

what would you like to know?

I jumped back in after a decade a little under two years ago. I had enough calc that I started in with a mathematical statistics text. There was a ton I had to backtrack on (logarithm rules, basic trig stuff, some basic algebra stuff, proof methods) but as I went, it all slowly clicked together, especially since I took notes and scheduled regular review so once I saw something again, I got to keep it.

Do you have any particular thing you're excited to head towards? 'Math' is a giant area. It helps if you have some practical reason, even if it's just an abstract question or a thing you want to understand. That's my two cents at least.

As for where to start... I like books personally. how to think about analysis is a great place to start. You can read through the whole thing in a few weeks, it's not a terrible investment, but it'll ease you into thinking about what math 'is', why you care, and how to pursue it. If you enjoy Alcock's book, a concise introduction to pure mathematics would be a great followup. It'll still be accessible, but a lot more rigorous and in depth than what you'll get in how to think about analysis. It's written for someone with just a high school level background, and builds a bridge up into thinking in terms of proofs, and goes through a number of interesting results.

Beyond that, there's a really cool thing called the infinite napkin project that you might have fun checking out as well. It's written by an Olympiad coach that struggled with talking about his research to high school students. Math is SO hierarchical, it's absolutely insane, so to get into some crazy topics you might be interested in (quantum computing algorithms) you might need a seemingly absurd amount of background knowledge first (linear algebra, complex numbers, hilbert spaces...) so... the infinite napkin project is meant to be a whirlwind tour through 'higher math' for a fairly accomplished high schooler. It's absolutely not meant to get you functional anywhere (his section on group theory is about 50 pages long. I'm currently working through a text on the topic that's 500 pages) but it DOES give you a good flavor for different topics, and his resources list is excellent, I've been really happy with the ones I picked up that he said he enjoyed. You could go through a chunk of the napkin, see what you're excited by, and then pick a resource yourself to really dig in. I've self studied my way through a number of math texts in the last two years. It can get a little lonely if you don't have any friends that share your hobby (so make some!) but it beats doing sudoku and crossword puzzles, haha. And if you do it for long enough, this weird little hobby can add some serious money to your paycheck if you're already an engineer.

by adventuringraw   2019-11-17

You should read this. I found it here a month ago on this subreddit, and it really stuck with me. I love those stories that help round out abstract concepts I've been thinking of.

More generally though... simple algebra used to be for the greatest thinkers alive. 'Ars Magna'. "The Great Art" written by Geromalo Cardano in the 1600s or whatever was the first European mathematical work that advanced beyond what was known by the Greeks... it gave a partial solution to how to find solutions to homogenous cubic polynomials (ax^3 + bx^2 + cx + d = 0)

he solved it with hilarious methods. Galileo used some incredibly painful notation where you're juggling ratios instead of... you know. Doing algebra the way we think of it. Fibonacci tried to encourage a switch to our standard number system, because arithmatic is RADICALLY easier when dealing with a simple base ten system instead of whatever crappy roman numeral type language they were using before. Took them 400 years to adopt our modern number system from the time the 'better' alternative was introduced.

All this is to say... you're absolutely right. The crystal core of the ideas we use can radically change our reach. What was impossible with one way of working becomes elementary when you can look at it right. But you've got a few layers of problems here. First... what's the right way of looking at it? I just read Judea Pearl's "Causality", and it's fascinating seeing a branch of math that's still so young, that there are arguments about what the definitions and axioms should even be. It's still a bubbling cauldron of ideas more so than an established branch. But even once you've gotten the 'right' way of looking at things (often there are many possible ways, you need to pick the right one for the job) now you're left with the arguably harder task of communication. How do you build a bridge to efficiently transmit a new way of thinking? I love 3blue1brown just because his whole shtick is finding new ways to graphically describe concepts that most people only vaguely understanding. The article I linked above (Ars Longa, Vita Brevis: 'long art, short life') breaks down the emergence of an art as being in 3 tiers... the inventors, the teachers, and the teacher teachers. The 'best' teachers I think are what you're asking about partly, but the right 'inventors' (what is the perfect framing that should be taught?) is part of the problem too.

Anyway, a related article you might also enjoy... thought as technology. A cool little exploration by Michael Nielson about the fact that 'how to think about things' is itself a technology, just one that's a pain in the ass to pass on compared to physical goods. He had some cool things to say on the topic you might also enjoy.

Also also... from a math perspective, I highly recommend you check out Alcock's how to think about analysis if you're looking for something fun to read. It's a very, very light introduction to real analysis, looking at the foundations of calculus, limits, series and convergence and so on. If you're interested in the 'heart' of what it means to learn math, I think you'll find that to be a pretty fun, approachable little book. You'll be able to blow through it in a couple weeks, but it'll give you some good framing for continuing the journey, if you're interested in doing so.

by adventuringraw   2019-07-21

I've found the theory to be really helpful, but that's a long ass road. Maybe on the side (while honing your practical skills) considering going through Wasserman's 'all of statistics' followed by Bishop's 'pattern recognition and machine learning'. Those two books alone could easily take 18~24 months of relatively consistent study (if you're not used to self-studying through serious mathematical texts) but it's worth the journey, and after your first few math books, I've found it's MUCH easier to go on from there. That kind of struggle will help you see the road towards studying your way into understanding white papers too, and build the confidence it'll take to even think to try such a crazy thing. I don't think you need that level of depth to be a practical engineer, but at the same time... I can't imagine doing this stuff without having the clear foundational insight into what the hell exactly I'm trying to do. If you're patient and motivated enough to add those two books on to your other studies, I think you'll find it'll pay off really well long-term.

Don't skip the exercises though, 'reading' those book is worth far less than struggling through all the problems, that's the real work that'll give you the insight and ability to reason with this stuff. If you're new to self-studying math entirely, I highly recommend you start first with 'how to think about analysis'. It's a quick read and will do a lot to arm you for the journey. Also, if you're weak with calc/linear algebra, you'll need to start there first.

by adventuringraw   2019-07-21

math is a funny thing... our culture gets so hung up on 'good at' and 'bad at', but the more I get into neurobiology and ML, the more amazing our general learning abilities seems to be. My partner and I are radically different, she's better at chess than I am in spite of having a poor ability with 'traditional' chess thinking, she relies almost entirely on pattern recognition, so she has to stand over the board looking down so her brain can feed up ideas from the books she's read (since chess layouts are always shown in those books from the top down).

All this is to say... there's a goddamn giant mountain in front of you, and it's easy to think that you're 'bad' at it because of where you're starting, or even because of base talents and interests that might not seem to line up with math at first glance. Just wanted to start out by saying that's horse shit. You're also 'bad' at judo and chinese (presumably), but given a few years of regular practice, you could get those reasonably under your belt as well. Math is a way of thinking and looking at problems, and it's incredibly helpful. It's kind of mind blowing the doors it can open... information theory, statistics, linear algebra, calculus, game theory, graph theory, group theory, representation theory, category theory... every branch opens up mind blowing new insights, tools, and models for looking at new problems. Don't look at it like this 'thing' you have to learn though. You can't learn all of math. You can just slowly learn new tools, get better at understanding what it even 'means' to learn one of those new fields, and how to organize your study to make real progress as you're slowly getting deeper.

So... my recommendation for where to start? Start with the meta learning. What is math? How can you learn it? How should you study? The best glimpse into those questions I've found is how to think about analysis. It takes a complete beginner's perspective (explaining how to read the standard math notation even... the summation symbol, epsilon, etc) slowly builds up an introduction to the guts of what calculus is, basically. You can read it in a week or two, so it's not a huge time investment, and it'll do a lot I think to arm you for the road ahead.

I'm personally a fan of bottom up learning as much as possible, but that's just because I hat trying to play with half a deck. There's plenty of people though that just treat pieces they're working with like 'black boxes'. You can use a decision tree without any fucking clue about information theory, or even what the decision surface actually looks like for the resulting tree. Finding good visualizations when wrapping your head around that stuff can be really helpful... so if you're struggling with one resource, don't be afraid to look for another. Sometimes a git article with some good graphs can make all the difference.

I don't know what road is best for you, but the only barrier in front of you is your patience, and your willingness to spend time every single week, and turn this into a practice instead of just a hobby. I started a year and a half ago after ten years in an unrelated industry, and while I still have a long way to go, I've also covered a ton of ground too. I'd never even had stats before at all, even in high school... now I'm comfortably following some pretty gnarly multivariate derivations in Bishop's pattern recognition and machine learning. You just keep putting one foot in front of the other, pay attention to your goal, follow your curiosity, and before you know it... people start looking at you funny, because you know things most people don't know, and you can build things most people don't even understand. I can't imagine a more exciting thing to be learning, especially at this time in history. If you have the patience and interest for it, whichever road you take I think you'll find it well worth your time.

My own personal suggestion by the way... take a little time for fundamentals on the regular (starting with linear algebra, a proper textbook with a lot of exercises if possible) and practical (actually implementing stuff, doing Kaggle competitions, whatever). Eventually in the distant future, you'll meet in the middle, and find you have the insight to start pursuing your own questions... possibly even questions no one has ever solved before, and you'll have an enormous amount of practical good to bring to whatever field you've been working in, if you choose to continue there. Good luck!

by adventuringraw   2019-07-21

I 'didn't apply' myself for the first most of my life too, haha. I've slowly gotten better at being disciplined and putting in the work though... I'm currently maybe 15 months or something into a heavy deep dive back in though. I had a little bit better of a foundation than you from my computer science degree, but there was still a huge amount of new stuff... all of statistics, for example, haha. Didn't even have the basics from high school.

Here's the thing though... you're not only learning the math itself, you're also learning how to learn. One part of that is ritual and schedule. You know how some people are yo-yo dieters? Or they work out like crazy in January and then drop it until next January? One quote I like... most people overestimate what they can do in one year, and underestimate what they can do in ten. It's fine to have short term goals, and some short term goals can be blitzed with enough grinding, but my personal recommendation is to learn a lifestyle that you can maintain long term. If you do decide to do 6+ hours a day for some crazy reason, know that this is unsustainable, unless you find you really, really enjoy what you're doing. I'd personally suggest you set a goal of an hour (or whatever is sustainable for you... half an hour even) of math, and focus on coding for the rest of your time.

For math... sounds like one big piece (as other people have mentioned) is getting comfortable with understanding what 'higher math' even is. Arithmetic is not what it's all about. I can't do those math problems comfortably either. It's much more about the kind of logical problem solving you'll need while coding too actually. I highly, highly recommend you get a copy of how to think about analysis. It'll give you a little rough intro into calculus (from an intuitive perspective even, the best road in) but the book spends the bulk of it's time talking about study habits and goals for high schoolers coming into an undergrad math degree. What does it even mean to study math, really? How can you make sure you learn it in a useful way? How can you learn to read proofs? Why are proofs important even? It's a really great little book, and you could blow through it in a week, easy. A day or two even if you sink your full '6 hours' in, but like I said... find a road to live, not to crash through.

by dblurk2   2019-07-21

I am sure this is the book you're referring to

by adventuringraw   2019-07-21

to add another recommendation... check out how to think about analysis. Most freshman calc stuff (as others have said) is a collection of hand waving justifications and tricks for solving complicated integrals/derivatives. There IS a real foundation for calc though. 3blue1brown has a video series (that someone else mentioned) exploring some of the justifications from a more graphical perspective... the book I gave is a great resource for tackling that too, I highly recommend it. It's a quick read, you could blow through it in a couple days, but the way of thinking it explores might just give you your first taste of math as something 'real' instead of some bullshit to memorize.

by adventuringraw   2019-07-21

right on man, good luck with the journey! And for what it's worth... math is probably best approached as an ongoing side hobby. A little every week. It's a long ass journey, and you can't really pound it down, it takes a lot of time to sink in. If you're excited to get into math again, why not pick up a book? If it's been a while and you're looking for a good place to start, I recommend how to think about analysis for a light primer. You could blow through it pretty quick, and it'll get you in the right mindset to tackle something more challenging... of which there's plenty. Calc (Strang's) linear algebra (Strang or Boyd) and statistics (wasserman's all of statistics) are all good choices after, or 'a walkthrough combinatorics' if you'd rather ease back in with more of a problem solving/puzzle bent instead of a more guided tour through hard concepts. Either way, I'd encourage you to just... you know. Pick something and get started if it's something you're interested in. If all that is stuff you already know though, Bishop's pattern recognition and machine learning is a good next choice.