How to Give Your AI Coding Agent Eyes and Hands in Your Chrome Browser

Most AI coding assistants (Claude Code, Cursor, Antigravity, Codex) are trapped inside terminal sandboxes. Whenever your agent needs to check Google Search Console metrics, review a pull request on GitHub, or verify Cloudflare DNS settings, it hits login barriers and 2FA prompts.
Headless browsers (Puppeteer, Playwright) start with blank profiles — no cookies, no extensions, no developer logins. Moreover, synthetic DOM clicks (element.click()) produce event.isTrusted = false, which modern single-page applications actively ignore or block.
To solve this, we built Agent Chrome Bridge — an open-source bridge connecting your AI agent directly to your active desktop Chrome session via Chrome DevTools Protocol (CDP).
Architecture: Lightweight & Local-First
The system consists of three lightweight parts:
- Chrome Extension (Manifest V3): Attaches via
chrome.debuggerto dispatch engine-level CDP input events withisTrusted: true. - Local Loopback Daemon (Python): Listens exclusively on
127.0.0.1:8766(loopback only) and enforces WebSocket origin checks to prevent unauthorized access from external origins. - CLI Tool (
br.py): A clean CLI that agents invoke in one-line shell turns.
Visual Element Indexing: Why Numeric Tags Beat CSS Selectors
AI agents struggle with complex, obfuscated CSS selectors like button.flex.items-center.bg-blue-600.hover\:bg-blue-700.
Instead, Agent Chrome Bridge implements deterministic numeric tagging:
- The agent runs
python br.py look. - The bridge scans the visible viewport and labels interactive elements with clean numeric markers:
[1],[2],[3]. - The agent receives a clean inventory:
[14] button "Create repository" [16] input:text "Repository name" [18] button "Submit" - The agent executes
python br.py click 14orpython br.py type 16 "my-new-project". - If the DOM shifts, the subsequent
lookrecalculates indices automatically.
Real-World Example: Inspecting Google Search Console
Here is what an autonomous inspection turn looks like:
# 1. Open Google Search Console in a background tab python br.py open https://search.google.com/search-console # 2. Inspect visible elements python br.py look # 3. Click on the URL inspection search box python br.py click 7 # 4. Type the URL and submit via trusted hardware enter python br.py type 7 "https://example.com/blog/my-post/" --enter # 5. Capture an annotated screenshot to verify results python br.py shot The entire flow takes seconds and reuses your existing Google login session without prompting for 2FA or exposing session tokens to cloud proxies.
Security Boundaries: Strict Rules for AI Agents
Granting an AI agent browser capabilities requires disciplined guardrails:
- Zero Credentials in LLM Prompts: The agent must never be asked to enter master passwords, API keys, or credit card details. The human logs in; the agent acts within the authenticated tab.
- Human-in-the-Loop for Payments & CAPTCHAs: The bridge does not solve CAPTCHAs or complete payment forms. When encountering a gate, it pauses and notifies the user.
- Local Isolation: The daemon runs on loopback
127.0.0.1only, and the extension validates message origin.
Getting Started
The project is completely free and open source:
- GitHub Repository: https://github.com/Rasr20/agent-chrome-bridge
- Official Release v2.2.0: Download ZIP archive
- Documentation & Web Landing: Digital Vault PRO — Agent Chrome Bridge
Try it with your AI coding agent and let us know what you think in the comments!