What motivates the commercial AI companies to share their research results and know-how?
Why did Google published the Transformer architecture instead of keeping it to themselves?
I understand that people may want to do good things for humanity, facilitate progress, etc. But if an action goes against commercial interest, how can the company management take it and not get objections from shareholders?
Or there is a commercial logic that motivates sharing of information and intellectual property? What logic is that?
My understanding is that frontier researchers will work for companies that will let them publish papers and discuss them with their peers.
When you're an engineer at the tier of these AI researchers, winning an extra 100k/year on top of you current 500k (numbers out of my ass) is not worth it vs getting name recognition. Being known as one of the authors that made the transformer for example will enable you work with other bright minded individuals and create even better things.
So essentially these commercial companies have "we'll let you publish papers when you work for us" as a perk.
> When you're an engineer at the tier of these AI researchers, winning an extra 100k/year on top of you current 500k (numbers out of my ass) is not worth it vs getting name recognition. Being known as one of the authors that made the transformer for example will enable you work with other bright minded individuals and create even better things.
Also, instead of an extra 100k a year, you get to raise a billion dollars in VC funds for your next company
& then sell it back to Google, in this case https://www.axios.com/2024/08/05/google-characterai-venture-...
There are a few commercially valid strategies.
1. Goodwill and mindshare. If you're known as "the best" or "the most innovative", then you'll attract customers.
2. Talent acquisition. Smart people like working with smart people.
3. Becoming the standard. If your technology becomes widely adopted, and you've been using it the longest, then you're suddenly be the best placed in your industry to make use of the technology while everyone retools.
4. Deception. Sometimes you publish work that's "old" internally but is still state of the art. This provides your competition with a false sense of where your research actually is.
5. Freeride on others' work. Maybe experimenting with extending an idea is too expensive/risky to fund internally? Perhaps a wave of startups will try. Acquire one of them that actually makes it work.
6. Undercut the market leader. If your industry has a clear market leader, the others can use open source to cooperate to erode that leadership position.
> Or there is a commercial logic that motivates sharing of information and intellectual property? What logic is that?
There absolutely is a sound commercial justification to share research: long-term growth through advancement of the field. (Deep learning would never have made the progress it has without open research!)
If this seems quaint, it’s because we’re too accustomed to short-term, transactional, Wall Street thinking.
Plus, it's probably going to leak anyway. If it's really just an idea, and you need to hire humans to work on it who will move in and out or your org.
Might as well get some dubious medium term gain rather than spend a bunch of money on security for nothing.
It's not so much that it seems quaint, it's that we are accustomed to short-term, transactional, Wall Street thinking from companies like Google.
For very good reason, because that's exactly how they behave in all other areas. The question remains, why do they appear altruistic when it comes to sharing papers?
I find it hard to believe that it's actual altruism. It's far more likely that it's transactional behavior that just appears altruistic from the outside.
> it's that we are accustomed to short-term, transactional, Wall Street thinking from companies like Google.
Out of all of the companies in the world, I wouldn't put Google near the bottom of the list in terms of stuff they've discovered and released to the world.
It's such a rapidly developing field with much of the progress happening in small labs on the open source models. Eventually, the field will coverage and stabilize. For now, the bet is too be open and supportive, to be close to the progress and be in best position when the dust settles.
It isn’t altruism! It’s good business: it pursues economic gain through mutual benefit.
People may be altruistic, but in a company setting they may have no possibility for altruism. CEO decisions influence property of others (the shareholders) so he can not freely pursue altruistic goals.
I heard that Dodge v. Ford Motor Co. was an important precedent in the US. https://en.m.wikipedia.org/wiki/Dodge_v._Ford_Motor_Co.
These days the courts give wide latitude to companies to offer virtually any plausible reason why superficially altruistic acts are in fact good long term for shareholder value. Anyone wanting to do what Ford did just needs to keep their mouth shut about the real reasons.
Interesting. Do you know any particular court cases?
My wikipedia link above in turn links to https://en.m.wikipedia.org/wiki/Shareholder_primacy, which says in the last paragraph: "The doctrine waned in later years."
This probably confirms what you say, but I'd be interested to learn about specific cases.
Everyone benefits from the gains, everyone gets more customers and more investment.
The ACL, NeurIPS, ICLR and the rest of AI professional organizations are why this happens. Forced open sourcing of everything. No pay to access. It’s the ideal open academic environment for rapid innovation. We must jealously defend our current system, as it will soon come under attack by those who get angry about democratization of the means of computation.
Also, lots of copyright abolitionists in AI. Many people who work in the space delight in the idea of making information, especially their own, free.
The ghost of Aaron Swartz runs through every researcher in this space.
Indeed, is there a chance Google did not evaluate properly what the transformer will eventually be used for/become. It was created for translation as an improvement on seq2seq, right? Which was for translation, not for thinking, and to a certain extent... still is about translation, and are not other emergent capabilities actually a side-effect, only observed later when parameter size grew?
I would guess it comes down to that the best researchers in the world want their work out in the open
Ilya doesn't. He's a strong proponent of closed source and censorship.
He is just one person. He happens to be the most famous scientist working on this field at the moment it became a gold rush, but it's work built on the shoulders of those who came before, whose discoveries are just as important.
Let's hope Totally Safe Intelligence doesn't end up having the same relationship with their name as Totally Open AI.
The more people copy your outdated thing, the better for you, because they always gonna lag behind you.
This may be related to Google's business model. Google's main businesses - search engine and advertising - both rely on an open web ecosystem. Therefore, Google has long maintained a friendly attitude toward open source and the open web, such as with Chromium, Noto fonts, Go, Flutter, and others. By providing infrastructure tools that benefit the open web, Google extends the reach of its searchable content and advertising. When the entire Web ecosystem benefits, Google ultimately benefits as well. This model also aligns with the philosophy of the open source community, where everyone is a beneficiary and naturally becomes a contributor.
All of the major labs have one thing in common: they have nearly unlimited data and money, but what they don’t have unlimited is talent and ideas. It’s just a way of progressing without having to „hire every idea“.
If you don't allow them to publish research work, your greatest talents will leave.
I used to work in such a restrictive environment. Nobody worth their salt stayed long.
It's worth noting that, while a noteworthy paper, nobody really expected the Transformer at the time to be the breakthrough it eventually became.
Transformer is just an example. We observe a constant stream of information shared by companies, even now, when "AI" is booming.
and back then in 2017, AI hasn't really been productized yet, people behind Transformer were researchers, publishing their results were the norm.
Well Deepseek's survival also depends on the giant amount of hype they can generate, and they won't get more investor money just by having done a one-hit wonder. Becoming deeply integrated in the AI ecosystem with various tools and innovative discoveries will most like be more beneficial than protecting the secrets of their first success.
Deepseek doesn't need hype to survive. They are bankrolled by their now billionaire founder.
> Why did Google published the Transformer architecture instead of keeping it to themselves?
Because they make their money from advertisements. Not their AI models. Same for Meta.
Compare that to e.g. OpenAI who's trying to make money from their AI models, and are thus underbid by Google and Meta.
Just as the company's name DEEPSEEK, it's commercial company and invest their based on AI, but the company's founder has more targets which are more common for human. Money is number for them, they want to do more, especially for DEEPSEEK.
If google never published it (and we pretend like it would not have leaked) then we would never have the LLM:s we have today (including Googles). Everyone would loose.
Deepseek is not a commercial AI company. They are the hobby of a hedge fund, something they do on the side for fun and glory.