Three of the first 18 GitHub repos I tested were malware

In September I was building a small tool called Legwork. The idea: when your AI assistant needs a tool it doesn't have, such as something to pull tables out of a PDF, it finds an open-source project on GitHub and installs it for you.

Before writing much code I wanted to know whether that could work at all, so I pulled eight real repos from GitHub search and tried them by hand. One of them looked like the easiest of the lot.

"No dependencies, standard library only"

It was a small data-extraction script. Its README said it had no dependencies and showed one command to try: list what it can do, with a --list flag. Nothing about it looked unusual. I ran exactly that command on my Mac.

It crashed with this:

File "extract.py", line 21, in <module> File "<string>", line 102, in run_sync File "<string>", line 91, in bootstrap RuntimeError: win32 only 

File "<string>" means the error came from code that was built and executed at runtime, not from any file in the repo. When I opened the source, the import names were hidden behind XOR, and most of the file was a large encrypted blob. On import, before it even looked at my --list flag, the script decrypted that blob and executed it.

The only reason nothing happened is that the payload checked for Windows first, and I was on a Mac. I didn't investigate what it does on Windows.

So the easiest-looking repo of my first eight, picked from ordinary search results, with a README that gave no hint of trouble, ran hidden code on my machine. A "harmless" read-only flag made no difference, because the payload fired the moment the file was imported.

What I built because of it

Two things went into Legwork before anything else.

A scan before install. It reads every Python and JavaScript file in a repo, without running any of it, and looks for the ways payloads hide: byte arrays decoded with XOR, imports whose names are computed at runtime, eval of decoded strings, code pushed off-screen behind whitespace. A hit stops everything before a single package is installed.

A sandbox for everything else. Every tool Legwork installs runs in the operating system's sandbox (sandbox-exec on macOS, bubblewrap on Linux). In your home folder it can read only the folder you chose. It gets none of your environment variables or keys, and no network.

Then I tried the top ten

Four days later I ran the real test. I took the ten most-starred AI repos created on GitHub in the previous week, exactly as search ranked them, skipping none.

The two blocked repos had different owners and different names: one called itself a text humanizer, the other a short-video generator. Each was three days old and had about 700 stars. Each contained the same 28,649-byte file, identical byte for byte, under a different file name. It was the same template as the script that had run on my Mac four days before, with the hidden names re-randomized.

The scan refused both in about a second. Nothing from either repo was installed, imported or run.

I can't tell you how two three-day-old repos got 700 stars each. I can tell you that sorting by stars put them in the top ten.

All three repos have since disappeared from GitHub.

What this does and doesn't prove

The scan is a tripwire, not a malware scanner. It catches the hiding techniques I've seen in real payloads, and someone determined could get past it. It also reads only the repo's own source, not what an installer downloads later. That's why the sandbox is the real protection and the scan is the early warning.

It also has to stay quiet on honest code. I ran it over the 150 most-starred Python, JavaScript and TypeScript repos on GitHub. It flagged one: lodash, for a vendored debugging script from 2009 that really does call eval(unescape(...)). A later run over 22 more repos produced two false alarms, on a large bundled Vue file and a chiptune synthesizer, and I tightened the rules that caused them.

What I'd take from this even if you never use Legwork

• Stars are not a trust signal. Especially on a repo that's a few days old.
• "Just try --help" is not safe. Code can run at import, before any flag is read.
• AI agents make this worse. An assistant that can run shell commands will happily clone and run a trending repo because its README says it does what you asked for. Whatever it runs has your SSH keys, your files and your environment variables.
• If an agent installs code for you, put a sandbox between that code and your home folder. Any sandbox.

Legwork is open source (MIT) and runs on macOS and Linux: https://github.com/kumarganduri/legwork

The full write-ups, including what failed along the way: the first finding and the top-ten trial. The repo also keeps a defanged copy of the first payload as a test fixture, with every encoded byte replaced by random data.