The Economics of AI Infrastructure: Compute Costs, Cloud Lock-In, and Competition

We are all watching the AI industry race from one frontier to another. Every few weeks, there's a new model creating hype, a new benchmark that becomes the industry standard, a new agent that promises to do the work of an entire team, or a new approach that makes yesterday's breakthrough look ancient.

I enjoy following these developments as much as anyone here, but there’s a question that is often ignored by us:

What happens when everyone can build with AI, but the infrastructure required to run it remains expensive, scarce, and controlled by a relatively small number of players?

That last part interests me most. We spend a great deal of time asking which model will win. I want to understand what happens when thousands of companies build useful products with AI, then discover that running those products at scale is an expensive business in its own right.

Let us have a look at it through a practical scenario.

A founder has built an AI assistant for accountants. The prototype works fine, and a few early access customers love it, which makes the founder envision a business that could actually run. Everything goes well with the first few customers. Then the usage grows. Customers anticipate quick responses; the service has to stay available, and with that the monthly compute bill begins to look rather different from the one generated by a handful of testers.

A successful start was just the beginning; the founder now has a new set of problems at hand. He needs to rent more computing capacity to keep the business running, hoping that suitable hardware is available at a workable price. To buy new equipment, he needs more capital and the expertise that are necessary to operate it. Luckily, if he finds a new provider with less cost, the migration could consume engineering time the company needs elsewhere.

All this decision-making, along with the founder juggling between hiring more staff, finding new customers, and convincing the current ones to pay more for the products, is a hefty task. And it makes founders wonder if having an excellent business model is enough.

I think the AI conversation needs to get more interesting. Building a capable product is becoming possible for more people. Delivering it reliably, at a competitive price, introduces another set of constraints. To understand those constraints, I started following the money and the hardware.

The first surprise is how many things have to go right before an advanced AI chip reaches a data centre.

Many of us know processor efficiency as something that just needs to perform calculations quickly, but calculations are only part of the job. The machine running on these processors must move enormous amounts of data between its processor and memory. Usually, High-bandwidth memory, shortened to HBM, helps feed data to the processor at the speeds workloads demand. Advanced packaging brings multiple components together so they can operate as one system.

I know these terms are hard for most of the people to understand. Here, let me break them down this way:

Think of a busy restaurant. Hiring a brilliant chef does little for the service if ingredients keep arriving slowly or the kitchen cannot get the equipment it needs. In an AI accelerator, the processor is only one part of the kitchen.

The supply chain behind these components is highly specialized. A shortage in one stage can hold up the finished product, even when another stage has room to expand.

Research from Epoch AI illustrates the scale of this problem. In its March 2026 analysis, the organization estimated that four leading AI chip designers collectively consumed around 90% of global HBM supply and advanced CoWoS packaging capacity in 2025. Its estimate for their share of advanced logic-die production was about 12%.

The contrast in the figures is revealing. Your company can have all the money ready, but if the manufacturing process isn’t up to speed, your orders will remain on a waiting list. Expanding chip production takes investment, equipment, skilled workers, and time. While a software team can deploy an update in days, Semiconductor capacity follows a different clock.

That manufacturing clock becomes more relevant when the entire industry is trying to expand at once. And it goes in favour of those with ample investments in their hands. Making companies with the strongest financing and the earliest access to suppliers have more options and leaving smaller buyers have to work around the capacity that remains available to them.

Even with this race, getting hold of a chip is only the beginning. The way companies buy access to computing power can shape their competitive position long after the initial purchase.

FTC’s staff report from January 2025 raised concerns about potential effects on competition, including increased switching costs and unequal access to important inputs. It identified risks for regulators to examine rather than declaring every partnership anticompetitive.

The commercial logic behind these arrangements is easy to understand. An AI developer who needs computing capacity can get it supplied by the cloud provider, perhaps alongside financing or other support. It looks like a win-win for both parties. The developer gets resources to build its business, while the provider secures revenue and strengthens a commercial relationship.

The complication appears when that relationship becomes demanding to leave.

Think of our accounting-assistant startup that has built its service around a particular cloud environment. Its deployment tools, monitoring systems, security controls, and model-serving software have all been configured to work together. But they found a competitor who is offering a lower price, tempting the founder to consider moving. But even though the new provider may have the right hardware, the transition still requires testing, engineering work, and careful handling of live customer service.

The migration bill and operational risks associated with the transition force the founder to stay with the current provider.

Cloud customers have faced versions of this problem for years. The UK's Competition and Markets Authority has highlighted that AI workloads add their own technical complications, especially when performance depends on particular accelerators and software configurations.

This is the distinction that keeps haunting me: access to compute and freedom to choose compute are two different things.

Yes, I tend to be more optimistic, but this is reality. A provider that offers a startup the capacity it needs today with some commitments. But these technical and contractual commitments narrow the startup’s choices tomorrow. The initial arrangement may be entirely reasonable for both parties. The bigger concern is when many businesses make similar commitments and a small group of suppliers becomes difficult to avoid. What happens then?

The same pressure appears on the other side of the market. Companies that own or rent large fleets of accelerators must turn that expensive equipment into useful work customers will pay for. A warehouse full of GPUs sounds impressive. The economics depend on what those GPUs actually deliver.

I found this particularly interesting while reading a 2025 research from the IEEE CLOUD conference. Researchers affiliated with IBM Research and other institutions investigated GPU bottlenecks in large-batch language-model inference. Their findings showed that memory bandwidth can limit performance even when much of a GPU's computational capacity remains underused. They also explored ways to improve memory allocation and serve additional workloads on the same hardware.

Let’s look at the practical implications of these findings through an example. A provider serving thousands of requests might combine them into batches to improve throughput. Processing more requests together can make better use of the hardware. But waiting for a larger batch increases delay, and customers expect quick responses. The operator has to juggle between efficient hardware use and the service its customers expect.

On top of that, model architecture and workload scheduling add further issues. Two AI services running on similar hardware can produce very different economics. One may use the available resources efficiently; another might spend more time moving data or waiting for work to arrive.

This is why instead of judging the strength of an AI infrastructure business by its installed GPU count alone, the best way is to analyze how effectively it converts expensive equipment into reliable computing services at a price customers will pay.

By having better software and scheduling, those economics can improve. It gives skilled operators an advantage without requiring them to buy another fleet of machines.

But efficiency isn’t the only factor. The money used to build the fleet in the first place is the actual issue.

You all know building an AI infrastructure requires large, long-lived investments. Data centres need buildings, electrical connections, cooling systems, networking, and specialized hardware. Many operators use financing structures that spread these costs across investors, lenders, landlords, and customers.

In his September 2026 working paper, Columbia Business School economist Stijn Van Nieuwerburgh examined how the AI infrastructure boom is being financed. He describes a shift toward arrangements that include leases, joint ventures, project debt, private credit, and other financing structures.

These can help fund expansion while spreading financial exposure across multiple parties. But at the same time, they also introduce risks around customer concentration, construction delays, technological obsolescence, and the future value of specialized assets.

Consider an operator that borrows money to build a data centre and signs a long-term agreement with a major customer. The contract gives the operator a revenue stream that helps justify the investment. The lender has a clearer basis for assessing repayment, and the customer secures access to capacity.

Everyone has a reason to like the arrangement. The risks still differ.

The customer may have committed to more capacity than it eventually needs. The operator must keep paying its obligations while finding enough demand to use its equipment. The lender depends on the borrower meeting its commitments and the underlying assets retaining value.

I find this part of the AI boom especially worth watching. Even with a booming industry, individual investors can still lose money. Those outcomes are perfectly compatible.

There is, however, a strong counterargument to the idea that infrastructure will remain a durable barrier. But even with discussions like:

1. AI models are becoming more efficient.
2. Quantization and distillation are helping to reduce the resources needed to run workloads.
3. Open-weight models give developers more choice, while better inference software can squeeze more useful work out of existing hardware.
4. New suppliers can enter the market, and additional manufacturing capacity can ease bottlenecks.
5. Large providers can also achieve genuine efficiencies through scale.

Today's shortages and prices tell us little on their own about the market several years from now.

I would be making a mistake if I treated every current constraint as permanent.

The more interesting question is how quickly those improvements spread, who can take advantage of them, and whether they give smaller businesses enough room to compete.

Imagine our founder returns six months later. The product has improved with inference costs fallen, and the team has found a more efficient way to serve customers. Those changes could transform the business. Yet the founder still needs dependable capacity, commercially sensible terms, and enough financial room to survive a period of slower growth.

That is the part of the AI story I want to follow. Yes, we all know that Intelligence is becoming easier to access, and there are real opportunities here. The ability to turn that intelligence into a sustainable business still depends on

• the infrastructure underneath it,
• the economics of running it,
• and the freedom to choose how it is delivered.

The next AI winners may well build better models. Others may win by serving those models more efficiently or finding customers the larger players have overlooked. Some will build the infrastructure that everyone else depends on.

As I watch the next wave of releases arrive, I keep returning to one question:

When more companies can build with AI, who will have the economic freedom to keep building?

I would love to have people from industry share their insights on this matter and share their perspective; we can publish our conversation here on Hackernoon. Don’t hesitate to reach out if you have insights to share.