How to keep up with AI/ML research
(Without burning out)
AI/ML research is hard to keep up with. I have found three main reasons for this:
Many engineers are not used to reading math-heavy research papers
The language of research papers is dense and requires strong foundations to understand
There is just too much research being published every day, and it’s tough to read it all
Let’s address each of these issues one by one to make AI/ML research easy to follow.
Math-heavy papers & what to do about them?
Check out a paragraph from the research paper titled ‘Improving Language Understanding by Generative Pre-Training’, which introduces the GPT architecture.
The paragraph describes the pretraining objective that is maximized during LLM training.
It looks intimidating, but it actually isn't. It is written this way because math is a great way to condense ideas and concepts and avoid verbosity.
To make it easy to understand, simply pass it to an LLM with a simple prompt as follows.
And this is what you get as a response.
You could also prompt an LLM further to explain a concept in more detail or, with step-by-step examples, if you’re still struggling to understand something.
All thanks to LLMs, this is one of the best (and judgment-free) ways to understand math-heavy research papers.
“Should I read a math book before reading a research paper?”
There are two ways to approach the mathematics in AI/ML research:
Bottom-up: Read all the foundational math first (Linear algebra, calculus, control theory, mathematical optimization, and so on), and then start reading AI/ML research.
Top-down: Read an AI/ML research paper first and then learn the relevant math.
I’ve followed both approaches in the past and found that the ‘Bottom-up’ one simply took too much time and effort, making it impractical. (I was stuck trying to learn all the different ways to integrate equations and multiply matrices, which later proved to be of not much use.)
I have long since dropped this approach and now read research papers first, then go ‘Top-down’ to read only the math essential to understanding the paper. This is what I find most efficient and recommend to you, unless you’re absolutely new to math (which is usually not the case for engineers).
Dense language & what to do about it?
Research papers are deliberately written concisely, avoiding verbosity. Otherwise, a single one would have pages enough to fill an entire textbook.
Here’s an excerpt from ‘Attention is all you need’, the famous research paper that introduces the Transformer architecture.
This short text introduces you to roughly a dozen concepts, including:
Recurrent neural networks
LSTM
GRU
Sequence modeling/ Language modeling
Machine translation
Encoder-Decoder architectures
Sequential computation precluding parallelization
Memory constraints limiting batching across examples
Factorization tricks
Conditional computation, and so on.
To a person just getting introduced to AI/ML research, this is a nightmare.
Here are three ways to read and understand such dense language:
Scan and read in passes
Give yourself time and read the research paper in multiple passes rather than in one go. First, scan the research paper to get its basic layout and reassure yourself that the paper does not go on and on forever. (Sounds silly, but it works.) Then, patiently read each section one at a time.
If you’re completely new to AI/ML research, there’s a high chance that none of it will make any sense in the first go. This is completely normal.
Let an LLM expand the dense sections
Pass each dense paragraph one at a time to an LLM and ask it to “Explain this paragraph line by line in simple words.” Take your time to understand the concepts, and keep returning to the earlier sections until everything fits together in a coherent sequence.
Don’t let the jargon overwhelm you
Look up unfamiliar terms with a simple web search and follow references.
If you follow these steps patiently, I promise everything will eventually start making sense.
Too long, too many, no time
Now, if you’re completely comfortable with reading AI/ML research, there’s a chance that you might feel overwhelmed by the number of research papers being published. Each paper is 15-20+ pages long, and hundreds to thousands land on arXiv every day.
Finding the relevant ones, let alone reading everything published each day, seems impossible.
I solve this by simply curating my social feeds (I mostly use LinkedIn and Substack) towards AI/ML research, releases, and developments.
Here are some accounts that you could follow on LinkedIn that talk about AI and ML:
Here are some of my recommendations from Substack in the AI/ ML engineering, trends, and macroeconomics space:
I also have an AI agent that brings me some interesting research papers every week, and you could build one for yourself following this tutorial.
Alongside this passive approach of simply letting my feed decide what I am updated on, I look out for recently published research papers on:
I write about what I found important each week in a newsletter edition titled “This Week in AI Research” and send it out every week. This could be a source you can use to stay up to date on AI/ML research.
How to quickly filter through AI/ML research?
Now that you have a curated feed and other trusted sources of AI/ML research, there’s still the problem of going through them all.
I use a simple system to find important research papers that you can steal.
Quickly read through the abstract, check the figures, and read the conclusion.
If a paper seems promising or directly relevant to your work or curiousity, then go deep into the methodology and evaluation sections. This is where the most important details of a paper lie: whether baselines are fair, whether ablations are present, and whether benchmarks were cherry-picked.
How to actually learn from a research paper?
Simply reading a research paper end-to-end is less useful than applying what it teaches. Some of the ways to make the learnings stick are:
Implement them in code: Manually code up the core idea from a research paper. Use an LLM to review your code, but don’t rely on vibe-coding. I frequently use this approach to learn better. (Example)
Use them at work or in your personal projects: Always ask how you can apply your learnings to the product or pipeline you’re working on. This will give you compounding returns in your career.
Start writing: Writing personal notes or an explainer/tutorial for others is another great way to learn better. Writing forces you to really get good at something before you can simplify it and explain it to others.
“Which papers to read if I am starting out?”
Pick 2-3 areas that deeply interest you or are relevant to your current work or career ambitions. Prioritize the early work and foundational papers in these areas and read them first.
A great way to build a strong foundation is to keep following and reading the research papers in the references. This will help you master a field’s foundational concepts faster than you might expect.
For example, before reading about Heavily Compressed Attention (HCA), I would follow references and read about:
How to build a research reading habit?
Curiosity and necessity are two great forces that might motivate you to read AI/ML research. For me, curiosity has been a big one.
Curate your feed and connect with others who love reading and keeping up with AI/ML research.
If you’re a beginner, set aside 1 to 2 30- to 60-minute sessions dedicated to reading AI/ML research every week. Put them in your calendar as protected time and do not skip them.
Reading for just an hour a week beats a heroic weekend binge every few months, and this is how you build a habit and win with slow compounding without burning out.
No one keeps up with all AI/ML research
My final advice to you before we conclude is to drop the unrealistic goal of keeping up with everything happening in AI/ML. No one can do this, including full-time researchers. Let go of this goal, be easy on yourself, and take your time.
👋🏻 Feel free to leave a comment or DM me if you have any questions, and share this with others if you found it valuable!
Join the paid tier today to get access to all posts in this newsletter, including:
and so many more!