HN Academy

I'm e-learning Linear Algebra right now to have a good math foundation for Machine Learning.

I was a History and Sociology major in college - so I didn't take any math.

If you are like me, and working off an initial base of high school math, I would recommend the following (all free):

Linear Algebra Foundations to Frontiers (UT Austin) Course: https://www.edx.org/course/linear-algebra-foundations-to-fro... Comments: This was a great starting place for me. Good interactive HW exercises, very clear instruction and time-efficient.

Linear Algebra (MIT OpenCourseware) Course: https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra... Comments: This course is apparently the holy grail course for Intro Linear Algebra. One of my colleagues, who did an MS in EE at MIT, said Gilbert Strang was the best teacher he had. I started off with this but had to rewind to the UT class because I didn't have some of the fundamentals (e.g. how to calc a dot product). I'm personally 15% through this, but enjoying it.

Linear Algebra Review PDF (Stanford CS229) Link: http://cs229.stanford.edu/section/cs229-linalg.pdf Comments: This is the set of Linear Algebra review materials they go over at the beginning of Stanford's machine learning class (CS229). This is my workback to know I'm tracking to the right set of knowledge, and thus far, the courses have done a great job of doing so.

⬐ earthicus

> This course [Strang] is apparently the holy grail course for Intro Linear Algebra.

I haven't watched his lectures, but I TA'd a linear algebra course that used his text book, and strongly disliked his presentation. I've heard that's a fairly common reaction actually - it's one of those love it or hate it books. I'm bringing it up because if you (or someone else reading this) turn out to be in the group that doesn't love it, you should not give up on loving linear algebra! You are definitely still allowed to have a different 'holy grail course'!

⬐ the_clarence

Gilbert Strang is probably the best teacher on videos, up there with Dan Boneh.

⬐ selimthegrim

Where’s the love for Lax?

⬐ earthicus

Page after page of mathematical insights and delights! I've never had the opportunity to work through it systematically, but have frequently read excerpts and have never been let down. I would expect nothing less from a figure so great as Lax!

It's worth pointing out in the context of this discussion that the book is, by the author's own design, not an introduction to linear algebra. It is a second course that Lax used to teach his advanced undergraduates and beginning graduate students at the Courant Institute. For example, OP with a high school math background will surely be very puzzled by page two, when a linear space is defined as a field 'acting on' a group. Which is, i think, the 'right' way of thinking about the algebraic structure, in the sense that it greatly simplifies all the intricate moving parts of linear algebra. Anyhow, I second your recommendation!

⬐ muhneesh

Awesome input! Learning isn't linear (tee-hee...)

⬐ ivan_ah

That's a pretty good list, here are some things I'd add.

Amazing js visualizations/manipulatives for many LA concepts: http://immersivemath.com/ila/index.html

LA Concept map: https://minireference.com/static/tutorials/conceptmap.pdf#pa... (so you'll know what there is to learn)

Condensed 4-page tutorial: https://minireference.com/static/tutorials/linear_algebra_in... (in case you're short on time)

And here is an excerpt from my book: https://minireference.com/static/excerpts/noBSguide2LA_previ... (won't post a link to it here, but check on amazon if interested)

⬐ muhneesh

Wow - nice stuff. I'm a fan of the reference map.

⬐ espeed

Good recommendations. In addition to the UT, MIT and Stanford courses you recommend above, for developing your visual intuition, 3Blue1Brown's Essence of Linear Algebra video series is second to none. [0]

Another good one is MathTheBeautiful [1] by MIT alum Pavel Grinfeld [2]. He approaches Linear Algebra from a geometric perspective as well, but with more emphasis on the mechanics of solving equations. He has a ton of videos organized into several courses, ranging from in-depth Intro to Linear Algebra courses to more advanced courses on PDEs and Tensor Calculus.

Esp note his video on Legendre polynomials [3] and Why {1,x,x²} Is a Terrible Basis: https://www.youtube.com/watch?v=pYoGYQOXqTk&index=14&list=PL....

Gilbert Strang was Greenfield's PhD advisor: https://dspace.mit.edu/handle/1721.1/29345. Pavel has a clear and precise teaching style like Strang, and he makes reference to Prof's Strang and his MIT course from time to time.

NB: Prof Strang has a new book Linear Algebra and Learning from Data that just went to press and will be available in print by mid Jan 2019. A few chapters are available online now, and the video lectures from the new MIT course should on YouTube in a few weeks. [4]

[0] Essence of Linear Video Series (3Blue1Brown): https://www.youtube.com/channel/UCYO_jab_esuFRV4b17AJtAw/pla...

[1] MathTheBeautiful https://www.youtube.com/watch?v=pYoGYQOXqTk&index=14&list=PL

[2] https://en.wikipedia.org/wiki/Pavel_Grinfeld

[3] https://en.wikipedia.org/wiki/Legendre_polynomials

[4] MIT Linear Algebra and Learning from Data (2018) http://math.mit.edu/~gs/learningfromdata/

⬐ peteretep

A bit weird to add a negative review, but here goes:

https://www.coursera.org/learn/linear-algebra-machine-learni...

Is _not_ a good introduction. The instructors are all over the damn place, and you will spend much of your time finding better explanations from other sources. Wish I hadn't started with this. On the plus side, you will get a certificate at the end.

⬐ pumanoir

Indeed weird, because the course you mentioned is actually excellent. However, it was designed for people who had (somehow) already seen the subjects in an abstract and unapplied setting (such as a math class at uni). They refresh or refocus the subjects with a geometric intuition and with some concrete applications in mind; which I found quite useful and beautiful. This class is more like a more developed version of 3b1b videos on LA.

⬐ elbear

It's funny, because as I was reading your comment, I was thinking of 3b1b. He's doing great work by visualizing abstract concepts, but I think what he does mainly helps people who have already gone through the material. If it's your first time encountering the topic, you'll likely feel lost or not see the point.

What 3b1b does still brings a lot of value, so I don't want to take away anything from his work.

⬐ lalaithion

3blue1brown has a great youtube series on both Calculus and Linear Algebra that provide excellent intuitive backing for the concepts in both areas.

⬐ kharak

Out of curiosity, do you feel you can compete with people who have advanced degrees in more quantitative sciences?

Although I'm in the process of plugging several holes in my own math education, I don't believe I'd be able to get any interesting, ML related jobs. I also don't see myself able to perform well in comparison, given that I lack the mathematical intuition one builds after several years of (almost) daily practice.

(I hope I don't sound discouraging, relearning math has been quit fun so far and made me able to understand more of everything).

⬐ muhneesh

The direct answer is - not at the research level but yes, in terms of application as the field matures and abstracts over time.

There are also adjacent jobs to ML engineer (product management and so on...).

⬐ dsiegel2275

I agree, Strang's Linear Algebra course is excellent. I worked through the entire course in March/April this year.

I just completed my final exam at CMU in their graduate intro to ML class (10-601). Having gone through the LA course was essential to my success. But equally important (if not more) to ML is a solid foundation in probability.

⬐ hnuser355

If you care about anything that runs on a computer, linear algebra is one of the best maths.

⬐ roymurdock

Would be interested to hear why you’re studying machine learning. Do you see important problems you think it can solve, are you looking to make more $$ as an ML data scientist, or just generally interested in stats/data?

⬐ muhneesh

Part of a broader effort - I committed to learning to code about 3 years ago. At that point in time, I didn't really know why I was doing it... really out of curiosity. I kept going because it was addicting, and really an antithesis to my day job at the time (investment banking) - which I felt was corporate / bureaucratic and unintellectual.

That said, eventually, I want to start a startup. I'm building out small side projects now. I'm generally comfy with web + mobile dev, and I wanted to upskill in a "newer" technology that was more "mathy".

⬐ roymurdock

Thanks for sharing, good luck on your journey

⬐ all2

I bet you could flow-chart your entire firm into a process that's mostly automated. Something like wealthfront, maybe?

⬐ thanatropism

What's the difference between learning and e-learning? Is the latter faster?

⬐ CBLT

Orders of magnitude cheaper.

⬐ rasmus1610

I just love the MIT linear algebra course with Gilbert Strang. Awesome teacher

⬐ jackallis

did you find LAFF too mathy? i got turned off by the mathyness of it and i quit in 2 weeks. does it get any better? All the math notations and lines got so dry that i vapourised trying to understand.

⬐ muhneesh

I didn't think about it during the time. It's a fair comment, and probably true.

What it did really well (for me) was integrate HW with each lecture video, and start at a really basic foundation. It took me from 0 -> something.

⬐ flor1s

Don't forget to review calculus as well. Khan Academy is a good start for learning about single variable calculus (http://www.khanacademy.org), but their content on multivariable calculus is a bit lacking (neural networks / deep learning use the concept of the derivatives and the gradient a lot). A good supplement for multivariable calculus would be Terence Parr and Jeremy Howard's article on "All the matrix calculus you need for deep learning": https://explained.ai/matrix-calculus/index.html

⬐ muhneesh

Thanks - I am doing that as well! I've been using MIT OpenCourseware for single variable calculus (and will do the same for multivariable). I fenced the parent post to Linear Algebra to not go too far away from the OP.

I will certainly check out the Terrence Parr / Jeremy Howard site, and am super familiar with Khan Academy.

⬐ lkrych

I'm going to plug Calculus: Single Variable from the University of Pennsylvania on Coursera (https://www.coursera.org/learn/single-variable-calculus).

This was the best Calculus course I've taken online.

⬐ thrwmlaccnt

When you say you’re going through the MIT OCW calculus courses are you watching the videos or also doing practice problems?

What other calculus resources are you using?

⬐ muhneesh

Watching videos, reading the (indicated portions of) the text, doing practice problems, eventually exams - relying on the resources provided in the OCW site.

To be transparent - I just started the calculus class. I finished the UT Austin Linear Algebra class two weeks ago, and am 7 lectures + readings + 2 problem sets in on the MIT Linear Algebra class and 3 lectures in on the Calculus class.

⬐ Connarhea

I'm coming to the end of my first year (6 year part time) Comp Sci course and have seen that we have options for AI and Machine Learning modules in future years. Where should I go to find something like a list of what I should be brushing up on, or learning completely from scratch, in order to not fall flat on my face during those type of modules.

I understand there are very set starting points in math subjects because concepts build on one another but I don't know what I should be starting with and where to go afterwards.

⬐ anonymous_i

I came across-Calculus Made Easy by Silvanus Thompson,on someones twitter feed. Published in 1910 and far less scary and far more interesting to read than a lot of math text books.

https://www.gutenberg.org/files/33283/33283-pdf.pdf

⬐ ajot

There is also a web goodlooking version, discussed previously in HN: https://news.ycombinator.com/item?id=18250034

⬐ claudiawerner

Similarly, Stroud's "Engineering Mathematics" takes you right from addition all the way up to Fourier transforms... a fantatsic book.

⬐ FranzFerdiNaN

This should be your first calculus book if you are learning from scratch. Much better than those 1000+ page behemoths colleges use.

⬐ Cheyana

“Considering how many fools can calculate, it is surprising that it should be thought either a difficult or a tedious task for any other fool to learn how to master the same tricks.”

I really wish technical books were still written like this. Though if Thompson posted this on HN as a comment he probably would have been downvoted.