Federated AI Still Has a Central Coordinator Problem

Federated Learning Isn’t Decentralized AI (Yet)

Here's the problem federated learning is trying to solve: Most AI training starts with pooling data. That fails wherever the data is sensitive or legally stuck. In 2023, Samsung reportedly restricted employee use of ChatGPT after engineers pasted internal source code into it. In late 2024, Italy’s data protection authority fined OpenAI €15M over how personal data was handled in training. Hospitals hold exactly the data that could improve diagnostic models, but they can’t simply merge patient records.

Federated learning is the standard answer: instead of moving data to the model, move the model to the data. Each site trains locally and sends only model updates to a coordinator, which aggregates them. It’s not theoretical. Google has used it for on-device keyboard prediction in Gboard, and a multi-hospital study (EXAM) trained a COVID-19 oxygen-needs model across roughly 20 institutions without sharing patient records.

Where it leaks

“The data never leaves” is not the same as “nothing leaks.” Model updates are computed from private data, and researchers have shown that training examples can sometimes be reconstructed from shared gradients (see Deep Leakage from Gradients, Zhu et al., 2019). Membership inference attacks can also reveal whether a specific record was in the training set.

Real protection needs extra layers:

• Secure aggregation, so the coordinator sees only the sum of updates, not any single site’s contribution.
• Differential privacy, which adds calibrated noise to updates and gives a measurable privacy guarantee at a cost in accuracy.

Both help. Neither is free. Any claim of “private AI” should come with the accuracy cost attached.

It doesn’t decentralize power

This is the part the marketing skips. In most federated systems, one party runs the coordinator, owns the global model, and decides who participates. The data is distributed, but control isn’t. That’s centralized AI with extra steps.

If the goal is the open, user-owned AI that this contest is about, federation alone doesn’t deliver it. You also need verifiable records of what happened, independent compute, and governance people can contest.

A stack that could actually work

None of this is hypothetical. Each layer has open-source tooling today:

The gap nobody talks about: causal structure

Almost all federated learning is about prediction: classify this image, predict the next word. But decisions in medicine and policy depend on causal questions: does this treatment help, or do sicker patients just receive it more often?

Learning causal structure across silos is harder. Sites may measure different variables, confounders differ from site to site, and local graphs may disagree. Federated causal discovery work exists, but it’s far less mature than federated prediction. A practical starting point is to learn a causal graph locally at each site (for example with pgmpy), merge the graphs in an aggregation step, and test whether the merged structure stays consistent across sites.

What this means for you

• If you build: tools like Flower make a federated prototype an afternoon’s work. The hard part is knowing what to measure: the accuracy you lose, the privacy you actually gain, and who runs the coordinator.
• If you’re a student or researcher: the open problems are at the edges, such as heterogeneous data across sites and causal discovery without pooling records. A small, well-measured experiment there can be a real contribution.
• If you just use AI products: treat “private AI” as a testable claim. Ask what’s protected, from whom, and at what cost.

If we get this right

A hospital in Delhi, one in Nairobi and one in São Paulo improve a shared model without any patient record leaving the building. The coordinator is a protocol anyone can audit. Every training round leaves a verifiable record of which sources, configuration and model version produced the result. Contributors are rewarded for useful updates instead of having their data harvested for free.

The honest version of this future isn’t “AI owned by everyone.” It’s AI where you can check who trained what, on whose data, under which rules, and where leaving doesn’t mean losing your own contribution. Getting there means solving the dull problems first: measuring the privacy-accuracy tradeoff, handling messy data across sites, and making governance something people can contest.