My day-to-day varies a lot depending on what stage of the project we are in versus what the immediate deliverables are.
At OpenAI and Meta, you have these milestones — say, a big training or reinforcement-learning run — in 10 months. It gets intense when we’re approaching the deadline.
Whatever work I identify is always based on the current iteration of the model. If I say the model isn’t good at X and my solution helps fix X, it is based on that version of the model. If I miss the deadline, I don’t know whether the next version will have the same issues or not.
If we are further away from that deadline, then we’re mostly working on evaluations and trying to find failure cases and issues with the existing model.
The work is super dynamic. Sometimes you think something is super easy and you’ll get it done in a day. Other times, it’s the opposite — because there are so many unknowns, it might take a week.
Working at frontier labs feels very different from Big Tech
What we’re limited by in these foundational labs is compute. It’s not like Big Tech or other places where you can keep hiring a bunch of people and give them small pieces of a task to do.
Everyone needs compute to actually do something, and as soon as you have a lot of people, the compute gets divided, so no one will be able to do anything.
You also want high-bandwidth communication between stakeholders — you don’t want 10 different layers of communication. The speed of iteration is much faster. These core groups tend to be much smaller and tighter.
The idea of a “team” is also very fluid. Each person has their own projects, but they collaborate with others to work on joint projects. At Meta and OpenAI, there are a lot of senior people and not a lot of junior people, so everyone has a decent scope of projects.
Sometimes I collaborate more with people outside my immediate team than within it. Your scope isn’t restricted to four or five people. Your scope is the problem you’re trying to solve.
Communication and going deep with coding are key
Communication is the most important aspect in these labs. Because a lot of things aren’t documented, you need to be able to articulate what you’re doing, why you’re doing it, what the next steps are, convey your results, and get feedback on your work.
Becoming comfortable going through the code and identifying the specifics is one of the most important skills I’ve seen. The speed at which the code evolves is much faster than the documentation. If you’re stuck on something, read the code and try to understand it yourself.