AI8/10
Sergey Brin Says Even Google Doesn't Fully Understand Gemini's Capabilities
▶
In an unscripted Q&A, Google co-founder Sergey Brin describes Gemini's convergence across scientific domains, unexpected skill transfer between tasks, and admits uncertainty about how best to prompt the models.
Original post · 5 min read
Sergey Brin rarely speaks publicly. He sat down for an unscripted Q&A on Frontier AI.
He admits even the people building these models do not fully understand what they have created:
1. All the specialized AI models are converging into one. Google used to need separate models for different scientific problems. Now the main Gemini models are becoming state-of-the-art for math and other scientific questions at the same time. Brin says he would not have predicted this convergence at the outset, and watching it happen has been incredible.
2. Training an AI on one skill mysteriously improves unrelated skills. This is the concept of transfer. Train a model on coding, and its math reasoning gets better, and vice versa. Teaching it to process images can improve its ability to think through geometric word problems. The capabilities bleed into each other in ways nobody fully engineered.
3. Even Sergey Brin does not know how to prompt these models. He says he is genuinely confused about what level to prompt at. Do you tell it to debug a specific chunk of code, or ask it to write a better neural net training algorithm, or just say, " What should I do today. He admits that even at Google, they do not know exactly where the edges of Gemini's capabilities are.
4. One of the biggest leaps in AI came from the dumbest sounding trick. Chain-of-thought prompting is just telling the model to think step by step before giving your problem. Brin says it seemed like the dumbest thing ever, and there was no obvious reason it should work. But it did, and it spurred a significant increase in AI capability. Some of the most straightforward requests turn out to unlock the most.
5. Brin would not modify his own biology for today's AI. Asked how humans can keep up with the accelerating bandwidth of models, he acknowledged neural links and direct brain connections are being pursued. But he said he would personally wait for the technology to mature a lot before doing anything to change his biology. Today's models do not justify it.
6. Super intelligence does not mean solving the impossible. An audience member argued that true super intelligence would mean solving NP complete problems like the travelling salesman. Brin pushed back. Most computer scientists believe P is not equal to NP, which means no algorithm can reliably solve those problems optimally, and it does not matter how smart the AI is. Impossible stays impossible. Super intelligence just means being smarter than humans.
7. Computers mastering a skill has never stopped humans from pursuing it. Deep Blue beat Kasparov at chess in the 1990s, and people kept playing chess. After AlphaGo, the human game of Go advanced dramatically, and the players who lost to it became vastly better. Brin's point: AI does not retire human ambition in an area; it often pushes the state of the art and pulls people up with it.
8. Brin thinks something close to transformers could get us to AGI. Asked directly if transformers are sufficient, he said his guess is yes, largely because they have proven weirdly flexible, working for image and video far beyond their original text purpose. But he was careful to note they have quietly changed a lot along the way and are not the same architecture as the original transformer paper.
9. AGI means two different things, and one requires understanding the physical world. Brin personally thinks of AGI as AI that can improve itself. But he concedes others define it as AI that can do anything a person can, and he thinks they are probably more correct. To do everything a person can, the AI must understand and interact with the physical world, which is why world models, and robotics, become essential.
10. Inside Google, they now use the AI to build the AI. Brin says the team has shifted a lot of energy toward having the AI do things like monitor training runs and generate its own training data. You start to use the tool to build the tool. That is most of what he spends his time on now, what he calls the self-improvement game.
11. Brin is unusually candid about where Google trails its competitors. He admits Google was a little late to focus deeply on coding. He says Gemini 3.0 and 3.1 were on top across the board six months ago, but other labs have since made strides, particularly in coding. He gives a competitor's model the edge now on deep coding and overnight tasks, while pitching Gemini's flash model as far faster for rapid interactive iteration. hindsight, he says, is that they should have focused on code earlier.
12. He sees his own role as a rabble-rouser, not a manager. Brin is honest that delivering Gemini is Corey and Demis's responsibility, not his. he describes his job as poking and prodding the team, asking, are you really doing that, reminding them of priorities they might be missing and ideas they are not paying enough attention to. He admits this is sometimes a little disruptive.
13. Confidence comes from ignoring the monthly temperature. Brin says if he judged Google's position every month by which competitor just shipped a model, he would lose his confidence very quickly. Instead, he watches the longer arc. Things shift around constantly; one lab leads on one thing, another pulls ahead somewhere else, and he feels good about where Gemini actually is despite the day-to-day noise.
He admits even the people building these models do not fully understand what they have created:
1. All the specialized AI models are converging into one. Google used to need separate models for different scientific problems. Now the main Gemini models are becoming state-of-the-art for math and other scientific questions at the same time. Brin says he would not have predicted this convergence at the outset, and watching it happen has been incredible.
2. Training an AI on one skill mysteriously improves unrelated skills. This is the concept of transfer. Train a model on coding, and its math reasoning gets better, and vice versa. Teaching it to process images can improve its ability to think through geometric word problems. The capabilities bleed into each other in ways nobody fully engineered.
3. Even Sergey Brin does not know how to prompt these models. He says he is genuinely confused about what level to prompt at. Do you tell it to debug a specific chunk of code, or ask it to write a better neural net training algorithm, or just say, " What should I do today. He admits that even at Google, they do not know exactly where the edges of Gemini's capabilities are.
4. One of the biggest leaps in AI came from the dumbest sounding trick. Chain-of-thought prompting is just telling the model to think step by step before giving your problem. Brin says it seemed like the dumbest thing ever, and there was no obvious reason it should work. But it did, and it spurred a significant increase in AI capability. Some of the most straightforward requests turn out to unlock the most.
5. Brin would not modify his own biology for today's AI. Asked how humans can keep up with the accelerating bandwidth of models, he acknowledged neural links and direct brain connections are being pursued. But he said he would personally wait for the technology to mature a lot before doing anything to change his biology. Today's models do not justify it.
6. Super intelligence does not mean solving the impossible. An audience member argued that true super intelligence would mean solving NP complete problems like the travelling salesman. Brin pushed back. Most computer scientists believe P is not equal to NP, which means no algorithm can reliably solve those problems optimally, and it does not matter how smart the AI is. Impossible stays impossible. Super intelligence just means being smarter than humans.
7. Computers mastering a skill has never stopped humans from pursuing it. Deep Blue beat Kasparov at chess in the 1990s, and people kept playing chess. After AlphaGo, the human game of Go advanced dramatically, and the players who lost to it became vastly better. Brin's point: AI does not retire human ambition in an area; it often pushes the state of the art and pulls people up with it.
8. Brin thinks something close to transformers could get us to AGI. Asked directly if transformers are sufficient, he said his guess is yes, largely because they have proven weirdly flexible, working for image and video far beyond their original text purpose. But he was careful to note they have quietly changed a lot along the way and are not the same architecture as the original transformer paper.
9. AGI means two different things, and one requires understanding the physical world. Brin personally thinks of AGI as AI that can improve itself. But he concedes others define it as AI that can do anything a person can, and he thinks they are probably more correct. To do everything a person can, the AI must understand and interact with the physical world, which is why world models, and robotics, become essential.
10. Inside Google, they now use the AI to build the AI. Brin says the team has shifted a lot of energy toward having the AI do things like monitor training runs and generate its own training data. You start to use the tool to build the tool. That is most of what he spends his time on now, what he calls the self-improvement game.
11. Brin is unusually candid about where Google trails its competitors. He admits Google was a little late to focus deeply on coding. He says Gemini 3.0 and 3.1 were on top across the board six months ago, but other labs have since made strides, particularly in coding. He gives a competitor's model the edge now on deep coding and overnight tasks, while pitching Gemini's flash model as far faster for rapid interactive iteration. hindsight, he says, is that they should have focused on code earlier.
12. He sees his own role as a rabble-rouser, not a manager. Brin is honest that delivering Gemini is Corey and Demis's responsibility, not his. he describes his job as poking and prodding the team, asking, are you really doing that, reminding them of priorities they might be missing and ideas they are not paying enough attention to. He admits this is sometimes a little disruptive.
13. Confidence comes from ignoring the monthly temperature. Brin says if he judged Google's position every month by which competitor just shipped a model, he would lose his confidence very quickly. Instead, he watches the longer arc. Things shift around constantly; one lab leads on one thing, another pulls ahead somewhere else, and he feels good about where Gemini actually is despite the day-to-day noise.
♥ 1.4K · ⟲ 254 · 👁 275.2KView on X ↗
