Playwright WebMCP: Calls Pass 50 of 50, First Read Misses Late Tools
Playwright 1.65.1 can list and call the tools a page registers through WebMCP, and the calls were reliable whenever the tools existed at the moment of the call: the same task passed in 50 of 50 runs, in a median of 9.64 ms. The failures were around the calls. page.webmcp.tools() did not wait for tools the page registered late, nothing checked the input against the schema, and a tool that threw reached the test as "Tool execution failed" with no reason.
The page, the tests and every raw result are in the PoC repo.
The setup
WebMCP lets a page register tools that an agent can call instead of clicking through the UI. Playwright exposes them as page.webmcp. I wanted to know if a test can use them reliably, what breaks, and how that compares with driving the same page through selectors.
- Playwright 1.65.1 and Chromium 159.0.8154.6 (the Playwright build, headless), launched with
--enable-features=WebMCP. Node 24.14.1 on macOS arm64. - A toy shop page with a normal UI (search, add to cart, total, checkout) and seven WebMCP tools backed by the same cart logic:
search_products,add_to_cart,get_cart,checkout, plus three awkward ones for the failure cases (echo_qty,boom,hang). - No model is involved. Every figure is a scripted run, 50 runs per row, each in a fresh browser context. The median is the 26th of the 50 sorted values.
The page registers each tool with a name, an input schema and an execute function. This is add_to_cart. The text helper wraps a value in a content array.
const mc = document.modelContext || navigator.modelContext; mc.registerTool({ name: 'add_to_cart', description: 'Add a quantity of a product to the cart.', inputSchema: { type: 'object', properties: { sku: { type: 'string' }, qty: { type: 'integer', minimum: 1 } }, required: ['sku', 'qty'], }, execute: async ({ sku, qty }) => { if (!Number.isInteger(qty) || qty < 1) return { isError: true, content: [{ type: 'text', text: 'qty must be a positive integer' }] }; try { return text(addToCart(sku, qty)); } catch (e) { return { isError: true, content: [{ type: 'text', text: e.message }] }; } }, }); The test side is plain Playwright. This is a condensed version of the task script. It adds the mugs, and the full script adds the lamp in the same way.
const tools = await page.webmcp.tools(); const found = JSON.parse( (await page.webmcp.callTool('search_products', { query: 'enamel' })).content[0].text ); await page.webmcp.callTool('add_to_cart', { sku: found[0].sku, qty: 2 }); const cart = JSON.parse( (await page.webmcp.callTool('get_cart', {})).content[0].text ); Same task, tools against test ids
The task: add 2 mugs and 1 lamp, expect a total of 54. The DOM script uses getByTestId, written once against UI variant 1. Variant 2 renames every test id and label.
| UI | Method | Passed | Steps | Median task time | p95 |
|---|---|---|---|---|---|
| v1 | WebMCP | 50/50 | 6 | 9.64 ms | 10.63 ms |
| v1 | DOM test ids | 50/50 | 8 | 134.77 ms | 150.8 ms |
| v2, renamed | WebMCP | 50/50 | 6 | 9.12 ms | 9.93 ms |
| v2, renamed | DOM test ids | 0/50 | n/a | 1001.05 ms | 1002.51 ms |
Read this table carefully, because it flatters WebMCP.
- The v2 failure is by construction. I renamed the ids the script depends on, so every run stopped at the 1000 ms action timeout with
locator.fill: Timeout 1000ms exceeded.It shows that test ids are fragile. It does not show that DOM driving is worse in general: role and text locators would survive some renames. - Steps are the calls the script made. The DOM figure is low because the locators were known in advance. An agent without them would also need snapshots to find them.
- I did not measure where the DOM time goes. The likely cause is that a DOM action does more than a tool call: Playwright waits until the element is ready and then sends real input. With a model in the loop, the model's time per step is added to both methods. I did not measure that.
The part that does carry over: the tools are not tied to the markup. Variant 2 changed only the UI, so the same WebMCP script passed on both variants. That is the case for testing the tools directly.
Late registration: tools() does not wait
The page registered the tools 0, 50, 200 and 1000 ms after its script ran. Straight after goto I read page.webmcp.tools() once, then read it again every 10 ms until the tools were there. The time in the last column runs from the first read.
| Registration delay | Found on immediate read | Found when polling | Median time until found |
|---|---|---|---|
| 0 ms | 50/50 | 50/50 | 3.15 ms |
| 50 ms | 0/50 | 50/50 | 59.12 ms |
| 200 ms | 0/50 | 50/50 | 209.21 ms |
| 1000 ms | 0/50 | 50/50 | 1008.5 ms |
With any delay, the immediate read returned an empty list and no error, in all 150 runs. A test that reads the tool list straight after navigation sees no tools, and nothing says they are on the way. Poll until the tools you need are there:
await expect .poll(async () => (await page.webmcp.tools()).length, { timeout: 5000 }) .toBe(7); What callTool does not do for you
Each benchmark case below ran 50 times, with the same outcome every time.
- No schema check.
echo_qtydeclaresqtyas a required integer with a minimum of 1 and no extra properties. ThroughcallTool, a string"3", the value 0, a missingqtyand an extra property all reached the page'sexecutefunction unchanged and returned normally. In this Playwright and Chromium build, nothing between the test and the page checked the input, so the page has to. - Page errors are results.
add_to_cartchecks its own input. For an unknown SKU it returnedisError: truewith the text "Unknown sku NOPE", andcallToolhanded that back as a normal result. It did not throw. A test has to checkresult.isError. - Thrown errors lose the reason. A tool that throws
new Error('boom')reached the test asOperationError: Tool execution failed.The word "boom" is not in it. - Unknown tool throws, and the message lists the available tools. A browser without the flag throws "WebMCP is not enabled". A page that registers nothing returns an empty list.
- A tool that never resolves. With
{ timeout: 500 },callToolthrewTimeout 500ms exceeded.
Without a timeout, the result depends on where the code runs. I ran each of these checks once, outside the 50-run benchmark. In a plain library script, the call threw Timeout 30000ms exceeded. Under the Playwright test runner, with the test timeout raised to 50 seconds, the call was still pending after 42 seconds. With the default settings, the call failed after 30 seconds with Test timeout of 30000ms exceeded. That is the test timeout, not a timeout of the call.
Pass a timeout, and check the result:
const result = await page.webmcp.callTool( 'add_to_cart', { sku: 'NOPE', qty: 1 }, { timeout: 500 } ); expect(result.isError).toBe(true); What didn't work
There are two naming traps, and neither gives an error.
- In this Chromium build the page-side object is
document.modelContext.navigator.modelContextis undefined. A page that only looks atnavigator.modelContextregisters nothing, and nothing says why. - The page-side hint for a consequential tool is
consequentialHint, which Playwright reports asannotations.consequential. A tool that I registered withconsequentialcame back fromtools()with no annotations.
The limits
- No model is involved. This says nothing about whether a model picks WebMCP tools better than DOM clicks, and I did not measure tokens.
- I called the tools through Playwright only. I did not test what a browser's own agent does with the schema before it calls a tool.
- One toy page on localhost, one machine, one browser build, headless. The timings are from a warm local run on macOS. Read them as orders of magnitude.
- Not tested: Playwright MCP's own WebMCP support, Firefox, other Chromium builds, headed mode.
What I'd do with it
- Test your page's tools with
page.webmcp.callTool. Wait for registration before you read the tool list. - Pass a timeout on every
callTool, and checkisErroron every result. - Validate input inside
execute. Do not rely on the declared schema. - When a tool can fail, return
isErrorwith a message. Do not throw: the message of a thrown error did not reach the test.
Repo: qa-gary-parker/poc-playwright-webmcp. The MCP vs CLI experiment measures the token cost of driving a page through Playwright MCP and the CLI. More are in the playbook.
Originally published at https://garyparker.dev/blog/playwright-webmcp