Before Illia Polosukhin left Google in 2017, he had a brainstorming lunch and then returned to his desk to build what may have been the very first transformer, the neural network architecture that makes generative artificial intelligence possible.
Now, Polosukhin is considered one of the founding fathers of modern AI.
Polosukhin co-wrote the now famous 2017 paper, “Attention Is All You Need” along with seven Google colleagues, who have collectively become known as the “Transformer 8.” Seven of them appeared on stage together for the first time at Nvidia‘s annual developer conference in March, where CEO Jensen Huang said, “Everything that we’re enjoying today can be traced back to that moment.”
Seven of the “Transformer 8” joined Nvidia CEO Jensen Huang at GTC, Nvidia’s annual developer conference in San Jose on March 20, 2024. From left to right: Lukasz Kaiser, Noam Shazeer, Aidan Gomez, Jensen Huang, Llion Jones, Jakob Uszkoreit, Ashish Vaswani and Illia Polosukhin.
Nvidia
Polosukhin said Google started utilizing transformers in 2018 in Google Translate, which made for a “massive improvement.” But a broadly popular use of the technology didn’t come until OpenAI launched ChatGPT in November 2022.
“OpenAI had very little to lose by opening this up,” Polosukhin told CNBC. “If, for example, any other company, especially public company, opened it up and the first question you ask there, it was like an inappropriate answer, that would be in the news.”
By the time the formative paper was published at the end of 2017, Polosukhin had exited Google to start his own AI company, Near, with fellow software engineer Alexander Skidanov. All eight of the authors have now left Google, although Polosukhin was the first to depart.
“Google research is an amazing environment,” Polosukhin said. “It’s great for learning and kind of this research. But if you want to move really fast and, importantly, put something in front of a user then Google is a big company with a lot of processes and, very rightfully so, security protocols, etc., that are required.”
Ultimately, he said, “for Google it doesn’t make sense to launch something that’s not a $1 billion idea.”
While at Google, Polosukhin was a proponent of open source.
“At the time, opening it up and making it available to everyone to build on top of it was the right decision,” he said.
With Near, Polosukhin is focused on what he calls user-owned AI, “that optimizes for the privacy and sovereignty of users.”
Watch the video to hear the full conversation between CNBC’s Katie tarasov and and Illia Polosukhin.