Prevent Retrieval Dilution in Alfresco RAG with Metadata Filters

A RAG application over a small set of documents often works surprisingly well. This can change once the same application is connected to an actual Alfresco repository.

We observed this while building an example with the Sample: Web Site Design Project site. This site is pre-installed in new Alfresco deployments, which makes it a convenient source for a reproducible example.

We loaded its content into a vector store and asked:

What is the state of localisation of our new website design?

A normal similarity search returned meeting notes from the project. These notes contain the decision to include localisation in phase 1, so the RAG application could answer the question.

def retrieve(state: State) -> dict[str, list[Document]]: documents = vector_store.similarity_search( state["question"], k=10, ) return {"context": documents} 

This works pretty well so far.

However, the sample site contains only a handful of documents. Most of them have little content beyond "lorem ipsum". There is not really a haystack where we need to find our needle in.

When more content makes retrieval worse

For the next step, we added an Engineering site with a document about the design of a product web interface. This document discusses localisation extensively, but intentionally does not describe the state of localisation of the corporate website project.

The document conversion and chunking pipeline split this one file into 368 LangChain documents. We added them to the same vector store and asked the same question again.

This time, all ten results came from the Engineering document:

Title: product-webui-design.txt state of localisation of the new website design ... ---------------------------------------------------------------------- Title: product-webui-design.txt localisation of the new website design ... ---------------------------------------------------------------------- Title: product-webui-design.txt localisation of the new website design ... 

The meeting notes containing the answer were no longer part of the retrieved
context. Consequently, the application answered:

I do not know. 

The Engineering document was specifically created to make this behaviour easy to reproduce. In a real Alfresco repository, the same problem appears less artificially. Numerous teams store meeting notes in the same system. There may be thousands of contracts, agreements, policies, product specifications, and project documents containing similar terminology.

In this example, we call the effect retrieval dilution: text chunks which are similar to the question, but irrelevant to its actual scope, occupy the limited number of retrieval results. The relevant chunks are displaced before the LLM gets a chance to see them.

Increasing k moves the boundary, but does not solve the underlying problem. It also sends more text to the model, increases cost, and may add even more noise to the context.

Alfresco already knows the document context

The vector embedding represents the text of a chunk. It does not know that one document belongs to the Engineering site and another one to the Web Site Design Project.

Alfresco does know this.

Documents loaded through OpenDMA carry the repository metadata alongside their text. The LangChain integration preserves this metadata on every resulting Document, including chunks produced by Unstructured or Docling.

for document in documents[:2]: print("Title:", document.metadata.get("opendma:Title")) print("Site:", document.metadata.get("alfresco:Site")) print("Path:", document.metadata.get("alfresco:Path")) 

For a document from the sample site, this produces information like:

Title: Meeting Notes 2011-02-10.doc Site: swsdp Path: /Company Home/Sites/swsdp/documentLibrary/Meeting Notes 

For the additional document, it produces:

Title: product-webui-design.txt Site: engineering Path: /Company Home/Sites/engineering/documentLibrary 

The site is only one part of the available information. OpenDMA can also expose the Alfresco content type, aspects and property values, together with common document properties and information derived from the repository.

Most Alfresco customers use sites to organise at least part of their content. Restricting a search to a site is therefore useful beyond this sample. More importantly, it demonstrates a general pattern: information already maintained by the ECM system can guide retrieval.

Adding metadata to the prompt after retrieval would be too late. The wrong chunks have already occupied the top ten results. The metadata needs to influence which chunks are retrieved.

A three-step metadata-aware RAG

We extend the normal two-step RAG with one query analysis step:

question -> analyse query -> retrieve with filter -> generate answer 

The analysis separates two different parts of the information need:

• a query for semantic similarity search;
• metadata which restricts the search scope.

For our example, the result should look like this:

{ "query": "state of localisation of new website design", "site": "swsdp", } 

The application first reads the available sites, including their titles and descriptions, from Alfresco through OpenDMA. The list might contain:

swsdp Sample: Web Site Design Project engineering Engineering - All product engineering related documents 

We then use structured LLM output for the query analysis. The schema restricts site to one of the site IDs returned by Alfresco or no filter instead of allowing the model to invent an arbitrary value.

class AnalyzedQuery(TypedDict): query: str site: str | None def analyze_query(state: ExtendedState) -> dict[str, AnalyzedQuery]: structured_llm = llm.with_structured_output(search_schema) analyzed = structured_llm.invoke(state["question"]) return {"query": analyzed} 

The query analysis is not an agent. It runs once and produces a fixed structured result. There is no tool loop and no sequence of attempts to resolve missing information.

The retrieval step uses both fields:

def retrieve(state: ExtendedState) -> dict[str, list[Document]]: analyzed = state["query"] documents = vector_store.similarity_search( analyzed["query"], k=10, filter=lambda document: ( ( analyzed["site"] is None or document.metadata.get("alfresco:Site") == analyzed["site"] ) and document.metadata.get("opendma:CheckedOut") is False ), ) return {"context": documents} 

The optimized text is used for semantic similarity. The site ID is used as a metadata filter. The exact filter syntax depends on the vector database, but the separation is independent of LangChain.

LlamaIndex supports the same approach with MetadataFilters and a custom
retriever. The framework changes, but the three steps do not.

Same vector store, different result

The vector store still contains all chunks from the Engineering site. We do not remove them and we do not build a separate vector store.

When the query analysis selects swsdp, similarity search is restricted to chunks from the Sample: Web Site Design Project site. The retrieved context once again contains the relevant meeting notes:

Title: Meeting Notes 2011-02-10.doc Site: swsdp Key Decisions ... Decided to include localization in phase 1 ... 

The application can answer the question again:

The state of localisation of the new website design is that it has been decided to include localisation in phase 1 of the project. 

The metadata filter does not make the embedding model better. It gives the similarity search a more appropriate set of candidates.

Filtering by business metadata

Sites are a convenient standard example, but many Alfresco deployments have custom content models. A document type or aspect may carry a customer number, employee ID, project number, claim number, or another business identifier.

The same three-step RAG can use these properties when the value is present in the question.

For example:

What termination clauses apply to customer 4711?

The query analysis can produce:

{ "query": "termination clauses", "filters": { "customer:Number": "4711", }, } 

Or for an HR use case:

Which training certificates expire for employee E-1027?

{ "query": "expiring training certificates", "filters": { "employee:Id": "E-1027", }, } 

These examples do not require the LLM to discover what customer or employee the user means. It only extracts a value which is already present in the question and assigns it to a known metadata field.

If the user asks about “Acme”, but Alfresco stores only the internal customer number 4711, another step is needed to resolve the name to the number. That can involve a repository search, a CRM lookup, or a tool-calling agent. It is a useful agentic RAG pattern, but it is not part of this three-step flow. The LLM must not invent the missing identifier.

Other properties can be used in the same way when they are part of the question and the Alfresco content model:

• a project or case number;
• a document type or aspect;
• a lifecycle value such as approved;
• an effective date, legal entity, region, or language.

The useful filters depend on the actual data model. A customer number is valuable only when it is populated consistently. A lifecycle filter is useful only when the repository reliably distinguishes drafts from approved documents.

Query analysis should also be allowed to return no filter. Applying a guessed filter can remove the document containing the answer and reduce recall instead of improving it. Questions spanning several sites or several customers require a broader filter or a different retrieval workflow.

Metadata filtering is also not an access-control mechanism. It reduces the candidate set for retrieval, but it does not replace the permissions enforced by Alfresco. A production indexing and retrieval design still needs to make sure that users cannot retrieve content they are not allowed to access.

When a RAG application stops finding information after its corpus grows, inspect the retrieval result before changing the prompt or the model. Look for metadata which separates the relevant documents from chunks competing for the same top-k positions. Preserve this metadata during ingestion and apply it as part of retrieval.

Runnable implementations for all three frameworks are available in the
sample repository.

The complete step-by-step versions of the example are available in the
LangChain metadata-aware retrieval tutorial and the equivalent tutorials for LlamaIndex and Haystack.