Published on

Free ML Resources to Become a FAANG ML Engineer

Authors

Here is a list of the free resources I used during my two year journey transitioning from a Java backend software engineer to a machine learning engineer who works on model research and development and deployment into various products. I have also included recent resources I have used to keep myself updated in the field.

Fundamentals

  • Linear Algebra – I watched 3Blue1Brown's Essence of Linear Algebra series on YouTube as a refresher on linear algebra. At that point, it had been six years since I had taken the course, so I pretty much had forgotten most of the concepts. I binged all these videos on a one hour train ride visiting my parents, and they laid the foundation for me to tackle linear algebra concepts in ML. Linear algebra is used virtually everwhere in ML (see my Linear Algebra 101 for AI/ML series to get an idea of linear algebra's application to ML).
  • Multivariable Calculus – Khan Academy's Multivariable Calculus lessons were a great refresher of what I had learned in college. Looking back, I just needed to have reviewed Unit 1 – intro and Unit 2 – derivatives. This knowledge is mainly needed to understand how training a neural network with backpropagation works.
  • Calculus for ML – Speaking of calculus and backpropagation, you can also watch this amazing animated video that explains both concepts with stunning visualizations.
  • Information Theory – I recommend this easy-to-understand book on information theory called Information Theory: A Tutorial Introduction. Information theory pops up in ML feature selection, model evaluation, and regularization, among many other areas.
  • Statistics and Probability – I never found a great condensed resource for stats and probability, but in general, I recommend watching the StatQuest YouTube channel. The creator cracks cheesy jokes, but if that fits your humor, all the better.

Machine Learning

  • Stanford Intro to Machine Learning by Andrew Ng – No pain no gain here. Eventually in your ML journey you'll need to be willing to do the hard work of integrated your refreshed math knowledge with machine learning fundamentals. Stanford's CS229, the intro to machine learning course, published their lectures on YouTube for free. I watched lectures 1, 2, 3, 4, 8, 9, 11, 12, and 13, and I skipped the rest since I was eager to move onto deep learning. The course also offers a free set of course notes, which are very well written. I printed out the course notes chapter by chapter, marking them with my notes. I remember feeling frustrated that I was encountering so many new math concepts, but I simply took the time to google each math concept. Nowadays, I look back at the notes with fondness because the concepts in the notes are so simple and intuitive to me now, making me proud that I had climbed that hill.
  • Caltech Machine Learning – When I got frustrated with CS229, I'd sometimes watch Caltech's machine learning lectures on YouTube. These were less mathematical and more intuition based, so ultimately I knew I would have to return to Stanford's math heavy lectures, but Caltech's lectures provided me with fundamental ML concepts that I needed to know.

Deep Learning

  • Andrej Karpathy's Zero to Hero Series – Andrej Karpathy, an AI researcher who graduated with a Stanford PhD and led Tesla AI for several years, released an amazing series of hands on lectures on YouTube. I highly highly recommend watching the engaging lectures and following along in a Jupyter or Colab notebook. If there's only one education resource you can pick, this is the one.
  • Neural networks – Stanford's CS231n course notes and lecture videos were my gateway drug, so to speak, into the world of deep learning. In particular, the free coding assignments, all of which are available as Google Colab notebooks, which you can run on Google's servers for free, were fun and educational. I highly recommend this course.

Transformers and LLMs

  • Transformers – After the release of ChatGPT, I knew I had to understand the Transformer neural network architecture better. I watched this lecture from the University of Waterloo, which gave me lots of insights into how Transformers worked. Then I solidified that knowledge by watching yet another lecture, this time from the University of Michigan. Each lecture is about an hour long, so I simply used times when I was waiting in transit/commute time to soak in the knowledge. I rewatched several sections over and over again until I understood the concepts. These two lecturers explain the attention mechanism and transformers very well. I have also heard good things about Jay Alammar's The Illustrated Transformer guide, though I never used it because the lectures had taught me the fundamentals.
  • ChatGPT Explainer – Wolfram's YouTube explainer video on ChatGPT is one that I share with friends and have gotten good feedback that it's a useful video to watch.
  • Interactive LLM Visualization – This LLM visualization that you can play with in your browser is hands down the best interactive experience with an LLM.
  • Financial Times' Transformer Explainer – The Financial Times released a lovely interactive article that explains the transformer very well.
  • Residual Learning – This is an important concept that pops up in neural network architecture design, and who better to explain it than the researcher who introduced it, Kaiming He. This is his 2023 Future Science Prize Laureates Lecture on residual learning.
Subscribe to access free Google Colab notebook on building ML image search step by step. Unsubscribe any time.

Efficient ML and GPUs

  • How are Microchips Made? – This YouTube video by Branch Education is one of the best free educational videos on the internet, regardless of subject, but also, it's the best video on understanding microchips. This knowledge is important in the industry because a senior FAANG engineer is expected to know and predict industry trends to stay ahead of the curve.
  • CUDA – My FAANG coworkers who have risen to L8 and L9 (Principal and Distinguished Engineer) levels acquired their CUDA knowledge from this series of lectures.
  • TinyML and Efficient Deep Learning Computing – MIT published their 2023 lectures on efficient ML techniques online. After watching them, I couldn't believe that they had released such valuable resources for free because these were state of the art techniques being used at FAANG and responsible for valuable technology that could be the basis of Silicon Valley startups.

Alas, the resources below are not free, but they're just as good as the ones listed above, and I can't find a free replacement for them. If you do decide to purchase them, please use my links below, which helps to support this website at no additional cost to you. Thanks!

  • Designing Machine Learning Systems, https://amzn.to/40bIgrj – When people think of machine learning practitioners, they often think of scientists and engineers training sophisticated neural networks on large amounts of neatly formatted data. However, the reality is machine learning engineers and data scientists largely deal with engineering problems and system design on a day to day basis. After all, where does that training data come from? How do you ensure it has little to no issues with it? Where do you store that data? How do you fetch that data efficiently? How do you deal with shifts in the distribution of the data? This best selling book by Stanford graduate Chip Huyen is the best book I know that imparts a lot of this practical knowledge to readers who want to be familiar with what machine learning looks like in industry. I refer to it from time to time even as a machine learning practitioner at FAANG and have found it very helpful many times.

  • Machine Learning System Design Interview, https://amzn.to/3BS8rJx – When you're ready to interview for a machine learning job at FAANG, I highly recommend purchasing this book. It contains 10 real world machine learning system design interviews. If you don't encounter these exact ones during your interview, you'll very likely encounter a variant because these are straight from multiple FAANG internal interview question banks.

  • Chip War, https://amzn.to/3YBsKDX – Chip War is a bestselling book published in 2022 about microchip technology. I recommend reading all of it, but crucially, its beginning chapters on the invention of the microchip actually explain CPUs very well. And this gives you a good idea of how those concepts extend to GPUs. I was surprised that the author, a trained historian, was able to write about the technical topic of microchips so well.

I'll keep this list of resources updated as I find better ones.