This vaguely-nationalist world view around tech that’s emerging in Europe is dangerous, man.
On the brain drain problem in particular, one way to ensure talent sticks around is to create a good environment for people to do their best work. In much of Europe, getting bureaucracy out of the way and encouraging real investment would go a long way. People leave because they can make more money and they want to be surrounded by the best people. People would trade some of that off to stick around their home countries, however if you go to California and talk to folks from e.g. NL or DE working on this stuff, they have a lot to say about innovation and working culture back home.
Assuming us Europeans finally get our act together, I think it is better for our long-term future (and the ethical problems) if we manage to get a baseline of training input and data ourselves, from scratch, with everything being ethically sourced.
Oh and, while we're at it, the EU has 24 official languages plus a host of minority languages. Most LLMs focus on the English, German, French and Chinese languages, but everything else is... left behind at best. An European model with actual funding and proper data sources might be able to significantly reduce that.
[1] https://www.taiwannews.com.tw/news/6245677
[2] https://www.theguardian.com/technology/2024/apr/16/techscape...
Yeah China and US models have baises but so will any model. The biases do not get in the way of the product though. You don't open those models just to ask for what happened in Taianaman square or if Taiwan is a state. You dont ask ChatGPT to generate CASM. But they are very good at the tasks you actually expect from a LLM. If you fail at that, nobody will use your model no matter how "ethically sourced" a colonizer-based entity like Europe made it.
I'm pretty sure no county taking a stab at making their own model for sovereignty purposes will let "proper licensing" stand in their way.
that is not true, so please read before make an opinion. The French Mistral project shipped seven+ years ago with 140 languages for example.. language translation was the first LLM task from 2015
Current frontier models (closed and open) are already really good at small languages too. I use them in Finnish sometimes, and the language is immaculate. They underestand even somewhat obscure dialects. Multilinguality seems to be a mostly solved problem.
How do people not know about it and keep making stuff from scratch?
1. Free of controversy like unlicensed training materials
2. Free of exploitative rlfh loops by people in low-wages countries
3. The leasons learned (and published) from going through the entire training process on "European" hardware: "AI factories" (the term for Slurm HPC/HTC systems with lots of heavy GPU nodes, heavily subsidized by our government [0])
1 and 2 are strong counter-LLM arguments at the moment, and hold back some groups of potential users. Another is energy/water use, so going for maximum green energy would be a nice boon as well. 3 is something I consider to be highly useful for our European identity and "way of the ninja" (for you Naruto fans out there).
but in general, yes, as someone who is vehemently anti-ai GPT-NL has piqued my interest specifically because of the ethical protections / measures they're talking about. question is whether they stick to it.
You take current version and build on top of it. You have the weights.
You might not get some n+1 version at some point but the n version you will have will be still most likely much better than whatever you come up with burning good will money of people believing in „sovereignty”.
You are not getting ahead in this game by being „true to your local values” capital expenditure is insane in this game.
Its purpose is not to become some kind of OpenAI or global foundation offering services/tools on that scale.
There is a lot of critisism on this project, not invalid, but mostly based in lack of understanding of what the goals are of the organization as well as the people building the thing.
The people building it, are well aware of how it will be less capable than other LLMs on a general reasoning aspect, not only due to having actually purchased _all_ licensed data that has been used as inputs. Not being a multi billion dollar corporation, this means having very little data and should be an obvious signal to observers that it has not the goal to outdo other models.
In my opinion (personal) its a project that has a learning and demonstration value that is not 'look how well our model performs against others', but still offers value.
(Of course states can ignore copyright in a legally polite manner, such as asserting that training on all published material in the National Library is fair game)
Unlike the US, Europe has no California-level VCs. I don't expect hundreds of billions of Euros to be poured into long-shot projects.
Unlike China, Europe has neither cohesive public investment at the global level nor the drive to grow. Long-term investments have a lot of words, a lot of regulations, a lot of proxy goals, but there is neither a lot of money nor urgency. It was captured by this post: https://x.com/piotrsankowski/status/2065795919623438546
So yeah, both in economy and warfare, Europe dooms itself to be in the hands of the US, China, or a mix of both.
Some would consider that a good thing. There is a lot to be said for VC in recent years not being beneficial for the economy, certainly on an individual level, other than "number go up".
At the same time, it made in many cases EU dependent on the US. A lot of governments are basically dependent on MS Office or Google Cloud.
With AI, it is even more strategic.
What a wild statement, VC's are behind most of the growth in the US economy, and they directly drive up wages in tech. I'd be fascinated to hear a valid complaint of VC's that isn't just money envy
What's ironic and sad at the same time is that pre-2022 Russia's Yandex(domestic Russian variant of Google) was lightyears ahead of what EU, a significantly richer and more capable block, had. IIRC, their reverse image search was so good, they had to nerf it because people were using it to find the identity of people from photos.
Same for Israel, their tech sector is probably greater than the EU one combined
Absolutely shameful how the EU kept managing to snatch defeat from the jaws of victory over and over.
Arguably, staying out of the AI "race" is a good thing
My ex-neighbor (when I was a teenager, living in Belgium) and very good friend really wanted to make it big. He became a chip engineer, moved to California, raised money for a first startup (it tanked) then raised money for a second startup. He made the world a better place (he created some very specific micro-inverters for solar panels) and made a $$$ exit.
The EU saw exactly zero of the wealth he created and he's never ever coming back to what he considers a failure of a continent.
That's the problem: many of the great minds with the mindset required to do great things already left the EU.
> So yeah, both in economy and warfare, Europe dooms itself to be in the hands of the US, China, or a mix of both.
And in energy (economy is energy and energy is economy, and China really understood that) the EU doomed itself to be in the hands of Russia.
We are a failure of sinking continent.
The US is a great place to live if you have talent, want to work, and want to reap the rewards.
In former times the energy monopoly was called "The Power Company"; we intend to give this name an entirely new meaning."
– CEO Nwabudike Morgan, "The Centauri Monopoly"
I don't know if it is the right strategy but there's certainly a legitimate strategy in there.
You also can't just spin up a research team out of nowhere.
Regulations are not even throughout each of the 27 member states. Each country is relatively small in the world stage.
Until EU progresses towards federalization, discussing this is a moot point.
The problem is that there are a lot, at least 30, of these small projects scattered around, funded for a few years as some ad-hoc temporary coalition of universities and businesses. Those simply cannot compete with businesses spending tens of billions on developing these. Especially when you have to bring a spoon to a gunfight restricting to "clean" data.
Multilinguality is essentially a solved problem, and restricting too much on one language with more limited resources is gonna make the model worse in that language too.
I'd rather have smaller european labs try to give it a go at distributed training. If multiple countries got together and said, "look, we tried training a distributed model that speaks in all of our local languages and that is comparable to 1-year-old Chinese open-source models", that, at least, I would find interesting.
Maybe I'm more attuned to this type of thing having grown up as a national of a smaller state living in the shadow of a bigger state but you constantly see actors from the bigger state belittling and condescending anything contributed socially or economically from the smaller state.
And I see this sort of dynamic here in this forum where Americans very frequently talk condescendingly like this about Europe generally and European tech especially (they did it to China too but China smartly ignored this self-interested nonsense and carried on anyway which is what Europeans should do).
It really grates on me and presumably many others. But it serves an agenda too of a lot of the founders and financiers that hang out here that have big fat customers in Europe they'd like to keep sweet and competitors they'd like to keep down.
Sure… they can, except at the end of the day it’s a bit late, regulatory burden will make it comparatively useless, and because of that nobody will ever use it. It will be spending a bunch of taxpayer dollars for press releases.
The running joke is that when these “sovereign” EU models launch, they’re going to refuse to answer anything that might involve personal information such as Elon Musk’s birthday.
I challenge the assumption you can do meaningful work in this field without blatant disregard for intellectual property.
The idea that it’s all down to training size is clearly incorrect, as every expert human learned their craft without nearly the sum total information of the internet. Clearly there are architectural wins to be found.
Besides that, why would everyone just be fine with Opus level AI at best, as that’s all the US is willing to export, and I doubt China will share beyond that.
Sovereign AI is more important than ever after Friday.
I agree there is likely some hubris in this sort of announcement, but investing in European expertise and industrial base in this area is important.
Countries should want control over _where_ the compute is happening rather than _what code_ is running.
What's wrong with a country hosting a Kimi, Qwen or GPT-Oss on their hardware for their government work purpose?
They are not neutral technology, they are a direct representation of the training set that has been chosen and how they are aligned.
In many ways, they are ideology made code.
If we leave building them to the US and China, only their way of seeing things will be digitized.
I don't like the idea of that.
Furthermore, the expertise in designing and training these models is valuable as well. The existing models are good as a starting point in terms of learning from previous mistakes, but we should not just let a handful of American and Chinese people keep the knowledge and expertise.
One problem with this particular project, though, is that copyright has been enforced for Dutch LLM training before, and the AI industry cannot exist without massive scale piracy, the likes of which has never been seen before. A lot of Dutch training material exists in pirated books that AI companies in countries that do not care about copyright have access to, but are exempted from the training set here. The impact of enforcing copyright on an AI model will be quite interesting to see.
1. Huge tax incentives, let the companies get grossly wealthy while paying minimal taxes. Minimum 10 years with clauses protecting "retribution" taxes there after.
2. Tax incentives for the founders/shareholders, just like above.
3. Drop worker protections to a minimum, make it easy to fire people. You only want serious/dedicated employees anyway.
Within 2-3 years there will be at least a trillion dollars looking to get in.
Don't worry though if reading that made you mad. Its absolutely not going to happen. I can think of few things more antithetical to the European ethos than smart skilled people working 80-100hrs weeks with almost no vacation to gas their founders net worth by tens, hundreds, of billions.
1. Pay the workers in company scrip and relocate the workers to a company town. That way, all workers are fully dedicated to the company.
2. Start importing slaves from Africa again. It worked to build up massive wealth. Should do the trick for AI as well.
3. Abolish the 8 hour work day. No comment needed.
With these 3 simple tricks, you too can get 6-7 bazillion euro AI mammoths.
When you reward the most selfish, corrupt, and antisocial behaviours with wealth and power, you're guaranteed to create a selfish, corrupt, and antisocial society. IMHO it's indicative of what I have dubbed Americas "mental illness epidemic"; specifically cluster B personality disorders [0] which are characterised by socially-destructive and self-destructive behaviours.
If that's the world you want for you and your loved ones, congratulations. You've earned it!
[0] https://wikipedia.org/wiki/Classification_of_personality_dis...
If the models are good though, they will have their sovereign AI and should be happy to pay for it instead of American or Chinese models. You may call it greed, but to me it just sounds fair.
[1]: https://mises.org/mises-wire/3-times-many-europeans-move-us-...
No thanks.
Why do you feel grinding insane hours would be beneficial to AI progress?
The government would be far better off figuring out how to take commodity models and applying them to government functions where they can, with deterministic scaffolding and guardrails, to make government more efficient, optionally using RL on traces from their use to improve their performance.
Imagine taking models and fine-tuning them / doing RL rollouts to help automate permit application approvals, as applied specifically to Dutch permit processes. That would be a real help to Dutch businesses!
That type of applied AI is more interesting and effective now than just trying to make another foundational model that isn't going to work well or do anything of economic value.
Because then the USA can't just turn it off.
https://www.siliconcontinent.com/p/nineteen-thoughts-on-ai-a...
I love it! So this is our answer to America and China denying foreigners access to their frontier models.. a massive 13,5M€ founding to develop souvereign european ai, trained exclusively on legally obtained documents and highest moral standards as defined in EU AI Act.
[0]: https://www.quotenet.nl/zakelijk/a71588202/techondernemers-m...
> GPT-NL was never built to compete with Claude or ChatGPT. It was trained exclusively on licensed data, and is intended more for governments and companies where privacy and compliance matter more than raw performance.”
That's it? That it didn't aim to compete with SOTA models? Maybe this is something you have to start with something, then ramp up, rather do what only a select few labs been able to do, start with really big models. Especially if you're resource constrained, which since this is a government project, I really hope for the sake of the tax payers it was.
And they're going to train an LLM with all kinds of extra difficulties compared to OpenAI for just 13.5M?
The very first Llama was 16M for one training.
All these tiny niche models are perhaps fun as an academic exercise or great for the researchers resume but I highly doubt that they'll add any value or will be used for anything serious.
Even if this becomes a somewhat decent model with a fantastic understanding of "gezellig", "kring verjaardag" or "pannenkoeken", how many people will interact with it before the limits of it will drive them back to a frontier model?
Even if the purpose of this is government & other regulated industries, do we really want our government to use a poor model? Either do it right or don't do it at all.
I guess we’re going for GPT2 level capability?
> This public investment underlines the importance of an independent, trustworthy and future‑proof Dutch language model.
It does, but not in the way you think it does.
They're training a model, not funding a startup. €13.5 million is plenty to pre- and post-train a decent model.
I was actually surprised by how little it was.
#define(HARMFUL)
[edit] Downvoters please tell me what the problem is with specifying this?
Also, when training models, you create talent that then could go to other countries (brain drain). Restricting that brain drain without imposing authoritarian restrictions on the movements of people seems hard, so it seems hard to keep talent as a competitive advantage. If instead the competitive advantage is datacenters with chips, power capacity building, fast path to building datacenters, I think they are easier to retain while preserving the rights of everyone involved.
An ecosystem is the tribal knowledge, revolving door of talent, known processes etc.
If the end goal is to make a half assed Dutch speaking model, I think it won’t cut it. I don’t see anyone using it over Gemma 4b that runs on my laptop.
An ecosystem is more durable and has desirable second order effects.
This looks like a good step in that direction.
Why don't they work together on it? Companies like Airbus have already been able to do that with aircraft.
This is not even funny. If you want a competitive AI industry, you need to invest much more heavily in infrastructure first, building models second.
