When you talk to Jakob Uszkoreit you have to bring your “A“ game. It's not because he's evasive. It's because the stuff he's working on is cutting edge enough that you need every neuron firing to keep up with what he's talking about.
Uszkoreit is one of the fathers of modern AI. In 2017 he was one of eight authors of Attention is All You Need, known to many as the Transformer paper. Its publication, while he and his coauthors were top software engineers at Google, is one of the seminal moments of the AI revolution. It’s arguably akin to the creation of the domain name system of the internet in the 1980s or Tim Berners Lee’s invention of the World Wide Web itself in the early 1990s.
Now together with Rhiju Das, a star biochemistry professor at Stanford and one of the earliest scientists to appreciate the power of RNA (Ribonucleic Acid), he’s mining the intersection of AI and biology with a three-year old company named Inceptive.
Das did postdoctoral research in computational protein folding at the University of Washington with David Baker. Das's and Uszkoreit's work helped Baker along with Demis Hassabis and John Jumper at Google’s DeepMind division win the Nobel Prize in chemistry last year.
Their ambitions are massive even by Silicon Valley standards. Google wanted to organize the world’s information and make it searchable. Yawn. Inceptive’s goal is to reorganize biology to revolutionize the way we make drugs and vaccines. It’s teaching its AI system cutting edge biology so it can spit out redesigned RNA molecules as easily as ChatGPT answers questions or writes software code.
Scientists’ growing ability to manipulate RNA has become a critical part of the drug and vaccine development process. It’s partly why scientists were able to develop Covid vaccines so quickly. CRISPR is perhaps the best known example of RNA's powers.
Imagine a smart cancer vaccine - one that can provide immunity to all different forms of the disease, without side effects. Uszkoreit and Das think they can get us there, or certainly close. Along the way they think they can help drug companies and biotech firms make existing drugs better, with fewer side effects, and cut years and hundreds of millions of dollars off the cost of developing new ones.
Grandiose? Crazy? Maybe. But because of the founders’ pedigrees and initial successes, they’re nonetheless attracting some big time attention. A slew of big drug companies and biotech firms are already paying them millions of dollars to partner on early stage research and development.
Meanwhile top investors - Andreessen Horowitz, Obvious Ventures, and Section 32 along with Nvidia - have been impressed enough with Inceptive’s early results that they’ve invested an unheard of $120 million in two early rounds.
Uszkoreit told me they’ve just recently used AI to create an RNA molecule that performs as expected in mice. "It's amazing in a sense. But it's not enough. Right? We have to do this not just ... in mice, we have to do this for proteins that are therapeutically relevant. We have to show that this works for a broad range of proteins," he said.
Inceptive is a truly post-pandemic operation. It’s based both in Berlin, where Uszkoreit lives, and Silicon Valley, where Das lives. It also has operations in Zurich, as well as employees in London, Vancouver and in cities on the east coast of the US. Uszkoreit said there are now 50 people scattered over these locations.
Their hill is particularly steep for the moment because most of the data Inceptive needs to train its models on doesn’t yet exist. So that’s what you see when you visit Inceptive’s Palo Alto labs: Teams of biologists and chemists performing RNA tests and experiments, and then with the help of top data scientists and deep learning geeks, feeding that data into Inceptive’s AI systems. Traditional AI systems by comparison have an almost limitless amount of data to train on.
The reason there isn’t enough RNA training data is because scientists once thought RNA, a cousin of DNA, wasn't good for much. The field is only about two decades old. “For proteins we had 200,000 experimentally obtained conformations (samples of the many different 3D structures/folds that these molecules can assume), as they call them. For RNA. We didn't even have 2,000,” Uszkoreit said.
Think of it as creating biological software. The upfront development costs may be huge. But once developed, assuming it works, you’ve created a license to print money.
"For all the praise that's heaped upon him (Uszkoreit), I still think he's underestimated - as crazy as that might sound, given that he is more or less the guy who came up with the idea that created Transformers," says Vijay Pande, who created and runs Andreessen Horowitz's $3 billion health and biotech fund.
Three years ago when Pande bumped into Uszkoreit and Das at a conference and learned they were teaming up, he agreed to help fund their $20 million seed round almost on the spot. "Das was originally a grad student at Stanford. And I was on his thesis committee. And I collaborated with him when he was a grad student. He's a superstar in applying computational methods to RNA,” Pande said.
It all still might not work. This is truly cutting edge science. And big pharma, with its bottomless bank accounts, is also working on this problem. Ultimately, it’s Inceptive’s approach that will give it the biggest edge over competitors, Uszkoreit says. For starters, he’s building a company with top talent from two vastly different professional cultures and forcing them to work together - biologists and chemists working side by side with AI geeks. Silicon Valley is much more rooted in the “Move fast and break things” culture that Facebook popularized. Biologists and chemists are most certainly not.
This different way of looking at problems and their solutions is top of mind every day at Inceptive. “When you saw their labs (in Palo Alto) did you notice that they have these two parallel rows? A wet lab row and a computational lab row? Pande asked me. “Well, in the middle they have something they refer to as the beach - between the dry and wet - where the two groups interact. It’s one of the exciting things about them - they force these two groups to interact.”
“That is really hard to do,” Uszkoreit says. “And it's really hard because it doesn't happen with just people from one discipline. Right. You can't just have a bunch of bio people in a room figure out the right asset because then they may generate data that is too clean.” He said neural networks typically learn better when they are fed noisier data and must develop methods to parse it. “So what you now need is you need an organization that excels in both of these fields, and you need these people to work together very well, even if they're nine time zones apart.”
You can count the number of interviews Uszkoreit’s done about Inceptive on one hand. And Inceptive’s website reinforces that low, inscrutable profile. It’s a single page that tells you how to apply for a job there. “We are creating tools to develop increasingly powerful biological software for the design of novel, broadly accessible medicines and biotechnologies previously out of reach,” it says. It almost seems intentionally worded to say “Only applicants who actually understand our website need apply.”
Below is an edited version of our conversation the Wednesday before Thanksgiving.
Fred Vogelstein: What was the original plan for Inceptive when you started?
Jakob Uszkoreit: So let's start with the long term and work backwards to the short term. A compelling way to think about a foundation model for life is to build a model for generating and for designing RNA (Ribonucleic Acid)
It is totally conceivable that RNA was the first molecule family of life. And the reason why that's conceivable is because you can do everything in life with RNA. That is not true for DNA. Nor is it true for proteins.
RNA can actually be a little biological machine (that makes proteins) and at the same time it can encode the blueprint of a protein for messaging that information (within and among cells). It just spans that entire gamut.
So it's definitely quite special. Most gene editors (like CRISPR) that are in trials today are RNA based, right? And those things can enable truly personalized precision medicine, such as personalized cancer vaccines, right? And so that is why in the long term, having a model to design RNAs is something super interesting and attractive.
There's already tens of companies building protein design models. It's a pretty crowded space. But not only are there fewer companies doing this for RNA, it also seems to me to be the more elegant way of deploying some of this technology in the longer term.
FV: What’s useful about this work in the short term? Can it help pharma companies make drugs better, faster, and cheaper?
JU: The technology may allow them to make drugs faster. It may allow them to make drugs more cheaply because they can spend more time designing compounds that they know work rather than screening dozens of compounds via trial and error. Ultimately, it will allow us to make drugs that nobody could have made before.
Imagine a medicine where you take a teeny tiny amount of something that you could easily inhale once - say because you have some genetic predisposition to something chronic happening over time. And what this thing does is it just hangs out in your system until a certain condition is met.
At that stage it then starts initial defenses. And maybe also reports in some routine tests that you undergo, just like liquid biopsies are becoming kind of more realistic for detecting certain forms of cancer.
FV: In other words, you're not just talking about helping drug companies do stuff better, faster, cheaper, but also creating new medicines?
JU: Exactly that. It’s not an overnight thing. That development work will take us many, many years. But the interesting thing about this direction is that these incremental steps stack pretty well. And that then forms a reasonably direct course towards this completely sci fi vision of medicines.
FV: So things like vaccines that cure all cancer or antibiotics that do not have resistance?
JU: Yes. Antibiotic resistance is a very particular problem of highly resistant microbes. But you could very well imagine other ways of combating bacteria that don't have as much of that risk. So there's a very interesting whole kind of other avenue around phage therapies where effectively you infect the bacteria with viruses or bacteria, which supercharges them. That then maybe even obviates the need for traditional antibiotics at all. This is a bold statement, but it is not inconceivable.
FV: How did you stumble on this idea in the first place? You’ve played a critical part in developing the foundation for artificial intelligence. But you’re not a biologist.
JU: Three things happened in about three months. The first thing was that my first child was born, my daughter, during the pandemic.
And then a little over a month later the CASP 14 (Critical Assessment of Structure Prediction) results were released. I realized the stuff that I'd been working on for the last five or six years actually was ready for prime time in molecular and structural biology.
The third thing that happened several weeks later. The first phase one and two trials of the MRNA Covid vaccines came out. Both of them had efficacies in excess of 90%.
Finally a very wise person (who I won’t name) asked me how I’d feel if, 10 years from now, my daughter learned how I stayed at Google despite understanding where this technology was going and despite having the clout to go after that opportunity. The MRNA Covid vaccines probably saved in excess of 20 million lives. So at that stage it became a moral obligation. If I don't do this, I risk somebody telling her that I could have done this, but did not.
A few months later (while I’m at a conference in Palo Alto with Rhiju Das) Vijay Pande, who started and runs the A16Z Bio and Health fund, walks past. And he says “If the two of you do a company, I'm going to write you a check, any check, actually.” And so we were like, “OK, we no longer have an excuse.”
FV: How far along are you in proving you can actually do what you’re working on?
JU: We've only taken baby steps. The good thing is these baby steps are already useful. This is super hot off the presses. But we've for the first time now proven this in animals, that we can take a molecule entirely out of a deep learning system, and we can put it into a piece of mouse or a part of a mouse and it behaves much more favorably than something created with state of the art algorithms employed by our competitors.
So now we have a large neural network and you can tell it, “Hey, design an MRNA that expresses a lot of protein for the following protein.” And it spits out something.
So that's amazing. Now we have to do this not just for a protein that allows us to measure this well in mice, we have to do this for proteins that are therapeutically relevant (in humans). And we have to show that this works for a broad range of proteins.
FV: Who are your competitors? What’s your competitive advantage?
JU: There have been somewhat similar results from academia and industry. The big difference in our case is that we've done this in a very data driven way. We've done this in a way that is very end-to-end, machine learned.
Big pharma tends to do this in a way that is much more screening heavy. So they just try a whole bunch of things, and they then choose the best ones. But that gets exponentially harder the more variables you have. Most of the other startups that I know of rely heavily on publicly available data.
We use that publicly available data too. But we've gotten pretty good at these validation assays (tests/experiments), evaluation assays for comparing models, and increasingly data generation assays, and the scale is increasing. (The results get fed into our AI system)
That is really hard to do. And it's really hard because it doesn't happen with just people from one discipline. Right. You can't just have a bunch of bio people in a room figure out the right asset because then they may generate data that is too clean. And they're not cutting the right corners to generate data that is noisier. (Noisier data is better at training AI systems.) So what you now need is you need an organization that excels in both of these fields, and you need these people to work together very well, even if they're nine time zones apart.
FV: How do you manage such a far flung team on two continents?
JU: It definitely adds to the challenge. There are some aspects that are just very predictable, such as the amount of overlap time that you have with people is just very limited. But then there are other things that are more subtle and don't become clear to everybody right away, even though they're suffering from them.
For example, our US staff tend to wake up to decisions that have already been made because I - working in Berlin, which is 9 hours ahead - have already had an entire workday to do stuff. So they (the staff in the US) are constantly waking up to a wall of messages and wondering whether they should have had the chance to be heard. We try to correct for that, but sometimes people forget.
For the Europeans, on the other hand, there is also the challenge to not just stay at the office working late in the night to have more time in common with work going on in California. That’s super challenging because you end up sacrificing your social life. And when you do that for a while, you wake up one day and you feel insane regret.
And then, (unless you correct for it) if you have people hailing from different academic backgrounds who don't naturally speak the same language culturally, you (can create a workplace) that promotes camp forming. I was incredibly paranoid about this for the first one and a half years of the company.
FV: Why don’t you merge your operations then? Is it because it’s harder to hire top AI people in Silicon Valley than in Europe?
JU: Yes, very much. (It’s harder). For me, at this stage of my career, the startup route makes the most sense. But if you take the average case, then it's impossible to beat Big Tech in Silicon Valley at the moment.
It’s not so much the compensation gap as the liquidity gap. Google is now starting to let people vest immediately. Sure, their equity packages are much less attractive in terms of growth potential. But they're very sizable. There are people coming out of university with easily seven digit compensation packages that are immediately liquid.
And that's not something a startup can compete with. You can give employees 10x the equity. But the liquidity of that is zero. Meanwhile the cost of living in the Bay Area, especially for people who are planning to have a family or have a family is just so astronomically high. So you hear things like, “Well, if I were to move to Europe, I could take your offer, but if I stay here, I just cannot make do with anything that has only six digits.”
FV: I appreciate the time. Thank you for talking with me.
