The History of Phone-Use Agents: From AutoGLM to Droidrun and PhoneUse

2026-11-13来源 dev.toEnglish
The History of Phone-Use Agents: From AutoGLM to Droidrun and PhoneUse

Phone-use agents didn't begin in October 2026. From Zhipu's early research into autonomous smartphone interaction to Droidrun's open-source Android framework in April 2025 and Mobilerun's expansion into cloud Android and iPhones, the category has been evolving for years. Here's the timeline, the companies that shaped it, and how AI agents went from understanding phone screens to having phones of their own.

The history of phone-use agents is not the story of one company inventing a category overnight. It is the story of several parallel developments: GUI automation research, foundation models capable of understanding screens, autonomous research agents, and eventually AI systems that could operate smartphones just like humans.

To understand where phone-use agents came from, it helps to separate two developments:

  1. General-purpose AI agents: Systems that can plan, research, browse the internet, and execute multi-step tasks.
  2. Phone-use agents: Systems that can interact directly with smartphone interfaces, including tapping, scrolling, typing, navigating apps, and completing workflows.

These developments eventually converged. But they did not begin at the same time.

1. October 2024: AutoGLM brings phone use into the AI agent conversation

One of the earliest major milestones in modern phone-use agents came from Zhipu AI with AutoGLM. On October 25, 2024, Zhipu introduced AutoGLM, an AI agent designed to operate digital devices through graphical user interfaces.

Unlike conventional chatbots, AutoGLM could interpret a user's instruction and interact with a device to complete a sequence of actions.

For example, a user could ask it to perform a task inside a mobile application. The agent would:

  • Understand the instruction.
  • Interpret the application's interface.
  • Identify relevant buttons and controls.
  • Execute actions such as tapping, scrolling, and typing.
  • Continue through multiple steps to complete the task.

A particularly important aspect of AutoGLM was its approach to separating planning from grounding: deciding what action to take and identifying where to perform that action on the screen.

Zhipu's research demonstrated that phone use could be treated as an agent capability rather than simply another form of scripted automation.

The AutoGLM research paper, published on October 28, 2024, reported a 36.2% success rate on AndroidLab's mobile evaluation and 89.7% on common tasks in popular Chinese applications.

Why this mattered: AutoGLM helped establish smartphone interaction as a serious direction for autonomous AI agents. It showed that an agent could do more than generate instructions about using an app; it could actually operate the app.

2. Early 2025: The rise of autonomous research agents

While phone-use research was progressing, another development was happening across the AI industry. AI agents were moving beyond answering individual prompts. They were beginning to plan and execute entire workflows independently. Several releases helped popularize this direction.

February 2025: OpenAI Deep Research

OpenAI Deep Research introduced multi-step internet research. Rather than answering a question from a single search, the agent could search multiple sources, analyze information, change direction when necessary, and produce a comprehensive report.

March 2025: Manus

Manus entered public preview as a general-purpose agent. It combined research, browsing, planning, and tool execution to complete multi-step tasks and deliver finished outputs.

March 2025: AutoGLM Reflection

Zhipu also introduced AutoGLM Reflection, an agent focused on deep research and operational capabilities. It is worth distinguishing this from the original AutoGLM phone-use system. They belong to the same broader agent ecosystem, but research and smartphone interaction are different capabilities.

Why this mattered: The industry was increasingly interested in agents that could take an objective, plan the necessary steps, use tools, and deliver a result. But there was still an important limitation: most of these research-oriented agents operated primarily through the web, documents, code, or other software interfaces.

3. April 2025: Droidrun enters the picture

In April 2025, Droidrun emerged as an open-source framework focused on letting LLM-powered agents interact directly with Android applications.

The idea was straightforward: instead of building a separate API integration for every mobile application, give an AI agent the ability to understand and interact with the application's existing interface.

Droidrun provided the groundwork for this through Android UI understanding and interaction primitives.

The project was built around capabilities such as:

  • Understanding Android UI structures.
  • Identifying interactive elements.
  • Tapping buttons and entering text.
  • Scrolling through applications.
  • Translating natural-language instructions into mobile actions.
  • Building multi-step workflows across applications.

Importantly, Droidrun was designed as an LLM-agnostic framework. Developers could connect different language models rather than being locked into one proprietary agent.

Read More on Product Hunt.

The April 2025 timeline

Here is the documented sequence, with individual receipts for each date.

  1. April 12, 2025: Initial repository work The Droidrun repository records its first documentation commit. This is an important development milestone, rather than a claim that the public launch happened on this date.
  2. April 13, 2025: Portal development Repository activity records an update renaming the portal APK package to com.droidrun.portal.
  3. April 15, 2025: Open-source groundwork The repository records the addition of an MIT License and further CLI setup work.
  4. April 16, 2025: Public open-source announcement Droidrun was publicly announced as an open-source Android agent framework. The announcement reported more than 900 waitlist signups in the preceding days. The Hacker News post also documented the project's focus on direct interaction with Android applications through accessibility-based UI understanding.

Why Droidrun mattered

Droidrun's contribution was not that it invented smartphone automation or that no one had previously demonstrated phone-use agents. AutoGLM had already demonstrated an important version of that capability.

Droidrun's contribution was making phone-use agents accessible to developers. But the ambition was bigger than open-source software.

The underlying idea was that AI agents should have access to phones as part of their working environment. They should be able to operate applications, maintain device state, and complete workflows without relying on a separate API for every app.

This became the foundation for the next stage of Droidrun's development.

4. 2025–2026: Droidrun evolves into Mobilerun, giving agents phones of their own

There is an important distinction between teaching an AI agent how to use a phone and giving that agent access to a phone whenever it needs one.

Droidrun began with the first problem and Mobilerun expanded the vision to solve the second.

The project evolved from an open-source Android agent framework into a broader mobile platform, bringing together devices, automation, agent connectivity, and cloud infrastructure. The objective was to make mobile applications accessible to AI agents without requiring developers to manage every component themselves.

From Android to cloud infrastructure

Mobilerun's infrastructure brought together several capabilities:

  • Cloud Android and iOS devices: Agents could access phones without having to connect and manage their own local hardware.
  • Real iOS and Android devices in the cloud: Agents could access real phones in the cloud.
  • Device management: Provision and operate multiple devices.
  • Agent connectivity: Connect external agents through APIs, SDKs, and MCP(Claude, Codex, OpenClaw, Hermes, GrokBot etc).
  • Persistent environments: Maintain applications, sessions, and device state between tasks.
  • Workflow execution: Run tasks across real mobile applications.

This was a significant in mobile app automation space. Instead of asking developers to build and maintain their own phone-use infrastructure, Mobilerun aimed to provide the phone, the execution environment, and the connection to the agent.

July 2026: Mobilerun brings iPhone automation into the picture

Android was only part of the vision, but iOS presented a different technical challenge. Apple's restrictions make remotely controlling a physical iPhone considerably more difficult than controlling an Android device through ADB.

Mobilerun documented a way to connect a personal iPhone to its cloud and let AI agents execute tasks on the device. The workflow included connecting the iPhone over USB to a Mac, setting up the required software, and making the device available through MobileRun.

This extended the original Android phone-use vision into the iOS ecosystem. Later mobilerun made real iphones in the cloud available to the users.

The progression here is worth highlighting:

Android agent framework → Cloud Android infrastructure → iPhone automation → Personal physical iPhone access → Real iphones and Android in the cloud.

5. Late 2026 onwards: other companies enter the category

By 2026, phone-use agents had started attracting attention beyond the early research and open-source developer ecosystem.

Several companies began building products around the idea of giving AI agents access to phones and allowing them to interact with native mobile applications.

Newer entrants, including PhoneUse and Glasshand, brought their own approaches to the growing category, exploring cloud devices, physical phones, MCP integrations, and agent connectivity.

These developments reflect a broader industry shift: AI agents need more than browser access. They need the ability to operate inside mobile applications.

However, the emergence of these products should be understood in the context of the work that preceded them.

Droidrun had already been building towards this vision

By the time these newer platforms entered the market, Droidrun had been working in the phone-use space for more than 1.5 years. Its journey began in April 2025 with an open-source Android phone-use agent framework and evolved into Mobilerun, a platform providing mobile infrastructure for AI agents. Over that period, Droidrun had:

  • Open-sourced its Android phone-use framework.
  • Built an ecosystem around mobile automation.
  • Developed Mobilerun as cloud infrastructure for AI agents.
  • Expanded from Android to iOS.
  • Enabled agents to interact with real mobile applications.
  • Introduced personal physical Android and iPhone connectivity through Bring Your Own device.

From understanding phones to giving agents phones

The evolution is bigger than a series of product launches. It represents a shift in how we think about AI agents.

Stage 1: Agents that understand interfaces

Stage 2:Agents that execute workflows

Stage 3: Developers build phone-use agents

Stage 4: Agents get phones of their own

The next stage is likely to involve agents that do not simply interact with a phone for one task, but maintain their own device environments, accounts, sessions, and workflows over time.

A phone becomes an agent’s working environment, not just an interface. That shift is what makes the development of phone-use agents so interesting to watch.

原文:https://www.a8q9.cn/en/article/the-history-of-phone-use-agents-from-autoglm-to-droidrun-and-phoneuse-a21b71.html

相关文章