Why Your LLM Keeps Matching the Wrong Products (And Why a Better Prompt Won't Fix It)

Imagine you have two product catalogs. One contains 10,000 items, the other 50,000. Your task is to find matching products, even though suppliers use different names, abbreviations, and formats.

Sounds like a perfect job for an LLM, right?

Modern language models understand context, recognize synonyms, and handle unstructured text pretty well. Just give the model two lists, ask it to find matching items, and you're done.

Except it doesn't quite work that way.

An LLM can understand both product names correctly and still return the wrong match.

The reason is simple: similar product names, identical products, and commercially acceptable substitutes are three different things.

While working on automated data matching problems, we've encountered several challenges that can't be solved simply by switching to a more powerful model or writing a better prompt.

Let's look at what goes wrong and how to build a more reliable matching process.

1. Similar Names Don't Mean Matching Products

Let's start with a simple example.

Catalog A contains:

Apple Juice, 1L, No Added Sugar

Catalog B contains:

Apple Juice Drink, 1L, Sweetened

The lexical similarity is high. Most words overlap, the volume is the same, and both products are apple-based drinks.

They're semantically close, too.

But they're not identical products. One is juice with no added sugar, while the other is a sweetened juice drink.

** **

If we're looking for an exact product match, this pair should be rejected.

Now consider the opposite situation:

Extra Virgin Olive Oil, 750 ml

and

EVOO, 0.75L

The names look different, but EVOO is a common abbreviation for Extra Virgin Olive Oil, and the volumes are equivalent once normalized.

If the other relevant attributes match, this pair could be a valid match.

** **

These are simplified examples, but they illustrate a fundamental problem.

An algorithm can be very good at measuring textual similarity without understanding whether two products are actually equivalent.

That requires looking at product attributes and the requirements of the matching task.

And sometimes the business doesn't need an identical product at all. It needs an acceptable substitute.

In that case, two products may have different brands or some different attributes and still be considered a valid match.

So we need to separate two questions:

How similar are these product names?

And:

Should these products be considered a match?

They're not the same question.

2. Sometimes the Answer Isn't in the Data

Consider a product listed as:

Premium Coffee, 250g

We know it's coffee, and we know the package weight. But that's hardly enough to match it confidently with a specific product in another catalog.

We don't know the manufacturer, roast level, coffee variety, or other potentially important characteristics.

A human working with the catalog might open the supplier's product page, look up the missing information, and then make a decision.

An LLM that only receives the original product name doesn't have that information.

It might guess the missing attributes based on its general knowledge. But a guess is not the same as verified product data.

** **

This matters when matching results are used for purchasing, price comparisons, or updating ERP records.

A wrong match can lead to a wrong order, an invalid price comparison, or incorrect data in a business system.

That's why it's worth checking whether the available information is sufficient before trying to make a matching decision.

If an important attribute is missing, the system may need to retrieve additional information, consult an external source, or send the case for human review.

Not every AI error can be fixed with better AI. Sometimes the system simply needs more data.

3. Why One Giant Prompt Is a Bad Idea

The most obvious way to automate product matching is to give an LLM two lists and ask it to return matching pairs.

This can work for small datasets, especially when product names are clear and contain enough information.

But things get complicated as catalogs grow.

** **

First, the model has to do several jobs at once: find potential candidates, interpret product attributes, and decide which products match.

Second, processing costs increase. Sending large catalogs into the model's context for every request is expensive and often impractical.

Third, debugging becomes much harder.

Suppose the model returns the wrong product. What exactly went wrong?

Did it misunderstand an abbreviation? Miss an important attribute? Fail to consider the correct candidate? Or apply the wrong matching rule?

When everything happens inside one prompt, answering those questions isn't easy.

That's why we find it more practical to split large-scale catalog matching into several specialized steps.

Conventional algorithms can handle text cleaning and normalization. Lexical and semantic search can identify potential matches. An LLM can interpret ambiguous names and extract product attributes.

Final matching decisions can then be based on structured features, business rules, and a separate ranking algorithm.

There's no reason to use an LLM for every operation when a simpler method can do the job reliably and at a lower cost.

4. A More Practical Way to Match Product Catalogs

Let's look at a general approach that can be used when designing a product matching system.

** **

Start by Finding Candidates

Instead of comparing every product against every other product using an LLM, we can first narrow down the search space.

A common approach is to combine lexical and semantic similarity.

Lexical search works well for similar words, product codes, and naming conventions.

Semantic search helps find items with related meanings, even when their wording differs.

But there's an important limitation: if the correct product doesn't make it into the candidate list, the next stages won't be able to select it.

That's why candidate retrieval quality should be evaluated separately from final ranking quality.

Some systems need a short list of the most likely matches. Others need to find as many acceptable alternatives as possible.

These are different requirements, and they call for different retrieval strategies.

Then Compare Product Attributes

Once we have a set of candidates, we can extract and normalize the attributes needed for matching.

For food products, these might include product type, ingredients, weight, and packaging.

For electronic components, the important attributes could be manufacturer, model, technical specifications, and compatibility.

** **

LLMs are particularly useful here because they can interpret unstructured names that are difficult to process with regular expressions or dictionaries alone.

But extracting the attributes is only half the job.

We still need to compare them correctly.

For example:

• "750 ml" and "0.75 L" should be treated as equivalent volumes.
• "Sugar-free" and "sweetened" may indicate an important difference.
• Different names for the same country shouldn't automatically count as a conflict.

And not all attributes matter equally.

In one product category, a different manufacturer may be unacceptable. In another, products from different brands may be perfectly valid substitutes.

That's why simply checking whether every attribute matches is often not enough.

Separate Ranking From Language Understanding

After extracting and comparing attributes, we can build a feature set for each product pair.

These features might include lexical similarity, semantic similarity, manufacturer agreement, matching attributes, and other signals.

A separate classifier or rule-based system can then evaluate how well the pair meets the matching requirements.

This way, the LLM doesn't have to handle every part of the decision-making process.

It also becomes possible to evaluate retrieval, attribute extraction, and ranking independently.

And that's especially useful when something goes wrong.

Instead of trying to fix the entire system by rewriting one prompt, we can investigate the specific component responsible for the error.

5. Stop Retraining the Model for Every New Rule

Even with a well-designed matching process, there's another challenge: product catalogs never stop changing.

Suppliers introduce new abbreviations. New brands appear. Product attributes and matching requirements evolve.

One way to adapt is to collect new examples and periodically fine-tune the model.

That makes sense when we actually need to change the model's learned behavior.

But if the change is simply a new synonym or business rule, fine-tuning may be unnecessary.

Suppose a company decides that products from different manufacturers can be considered equivalent in a particular category, as long as certain attributes match.

It's easier to store that rule in configuration than to teach an LLM to reproduce it through another training cycle.

But what about unusual matching cases that can't be described by one simple rule?

This is where retrieval-augmented generation, or RAG, can help.

** **

When a specialist validates a difficult product pair, that decision can be stored in a database of verified examples.

The next time a similar case appears, the system can retrieve the relevant historical example and provide it to the LLM as additional context.

This gives us two ways to manage changing knowledge:

• Explicit business rules and reference data, enforced programmatically.
• Validated historical examples, retrieved when relevant.

It doesn't mean we'll never need fine-tuning again. But new data doesn't always have to mean changing model weights.

Sometimes updating a dictionary, a configuration rule, or a set of validated examples is enough.

Of course, RAG needs its own quality controls. An outdated or irrelevant example can lead to the wrong decision, too.

6. Before You Change the Model, Check These Five Things

Let's say your matching system keeps making mistakes with a certain group of products.

The first instinct might be to improve the prompt or switch to a more powerful LLM.

Before doing that, it's worth checking a few things.

** **

1. Was the correct product in the candidate list?

If the correct item never appeared in the search results, the final model had no chance of selecting it.

The problem may be in lexical or semantic retrieval, not ranking.

2. Was there enough information in the input data?

If the original product name doesn't include important attributes, the model may not have enough evidence to make a confident decision.

Data enrichment or human review may help more than another prompt.

3. Were the attributes extracted and normalized correctly?

Even if the LLM identified the attributes correctly, the next component might compare their values incorrectly.

Equivalent units or alternative names may be treated as conflicts.

4. Were the right business rules applied?

The same attribute can have different importance across product categories.

If the system applies a universal rule where a category-specific rule is needed, the problem may be in the matching logic rather than the LLM.

5. Is it actually a model capability problem?

If retrieval, input data, attributes, and rules all look correct, then it's worth examining the model itself.

Maybe it struggles to interpret certain terminology or follow the required instructions.

At that point, changing the prompt, providing additional context, trying another model, or fine-tuning may be reasonable options.

These checks won't solve every matching problem, but they help identify where the error starts before spending time and money changing the entire system.

7. Won't This Make the System Too Expensive?

Splitting matching into several stages makes the system easier to control, but it also adds complexity.

And cloud LLM calls come with recurring costs.

One option is to extract detailed attributes for every product in the entire catalog before matching begins.

This could help find acceptable substitutes with very different names.

But processing hundreds of thousands of products this way can get expensive.

An alternative is to use relatively cheap candidate retrieval first and apply the LLM only to the selected products.

That reduces costs but introduces the risk of missing valid matches at the first stage.

So there's a trade-off between recall, processing speed, and cost.

For some businesses, missing a valid match can be a costly mistake. For others, sending a small number of difficult cases for human review may be perfectly acceptable.

The same reasoning applies when choosing between cloud and local models.

Local inference may be cheaper at high and stable processing volumes, but it requires infrastructure, maintenance, and, when fine-tuning is involved, ongoing work with training data.

Cloud models reduce the need for some of that infrastructure but introduce usage-based costs.

That's why the comparison shouldn't be limited to the price of an API call.

What matters is the total cost of producing results at the required quality level.

Conclusion: The Model Isn't Always the Problem

When an LLM matches two products incorrectly, the natural reaction is to improve the prompt, choose a more powerful model, or prepare a new dataset for fine-tuning.

Sometimes that helps.

But before changing the model, it's worth understanding why the mistake happened.

Maybe the correct candidate never appeared in the search results. Maybe an important attribute was missing from the input data. Or maybe two equivalent values were incorrectly treated as conflicting.

Sometimes the system simply doesn't know the business rule that determines whether two products should be considered a match.

These problems require different solutions.

Reliable product catalog matching depends on more than the language model's capabilities. It also depends on data quality, candidate retrieval, structured attributes, and matching rules.

Instead of trying to teach one model everything, it's often more practical to build a system where each component does its own job and changing knowledge can be updated without rebuilding the entire architecture.