We scanned 900 MCP configs on GitHub. 75% had security problems.

I expected to find maybe a dozen hardcoded API keys and a handful of overly permissive configurations scattered across the results. The usual negligence you stumble on when you go digging through public repositories looking for things people probably shouldn't have committed.

What I didn't expect was that three out of four configuration files would fail basic security checks, and that the single most popular "MCP package" in the entire dataset wouldn't actually be a package at all.

This is the full account of how I got to those numbers, what the raw data revealed along the way, and where I think the whole MCP configuration ecosystem is quietly heading.

AI creates faster than it can be verified. MCP servers multiply this problem: every tool your agent calls is a new unverified input. The runtime layer, the proxy between your agent and the API, is where verification actually happens, because it's the only place that sees everything.

Why I started poking around in the first place

I've been building AI agent security tooling for the past few months, mostly focused on runtime enforcement, which is basically making sure autonomous agents don't do things they shouldn't be doing when they're making calls to LLM APIs behind your back.

MCP kept surfacing in that work. For anyone who hasn't encountered it yet: MCP is the protocol that Claude Desktop, Cursor, and a growing number of similar tools rely on to connect AI agents to external servers. You define which servers the agent talks to in a JSON config, and then it just... has those capabilities. Reading files, querying databases, calling APIs, running shell commands, whatever those servers decide to expose.

That configuration file is basically the permission boundary for everything the agent can do. Get it wrong and every misconfiguration flows directly into the agent's behavior, which gets uncomfortable when you consider that agents process untrusted input from users, tool outputs, and scraped web content.

I kept running across theoretical discussions of MCP vulnerabilities. Prompt injection through tool results, malicious MCP servers, data exfiltration via crafted tool calls. Plenty of hypothetical attack scenarios had been written up, but I couldn't find anyone who had actually gone and looked at what real developers are configuring in practice. Are most setups reasonably locked down? Is version pinning widespread? Do people genuinely put their API keys directly into these JSON files?

I figured the fastest way to answer those questions was to just go look.

How the scanner was built (and what broke along the way)

The core approach was deliberately unsophisticated. GitHub has a search API for code that I used to look for specific filenames and content patterns across public repos. The scanner grabs claude_desktop_config.json, .cursor/mcp.json, and anything with mcpServers in it, pulls down the raw file, tries to make sense of the JSON, and if it parses okay, runs its 52 checks against it. The checks cover whether there are leaked credentials, what kind of permissions each server gets, whether anyone bothers pinning package versions, and a few other things I kept adding as patterns emerged during early test runs.

The implementation hit several annoying problems that are probably worth mentioning because they ended up shaping the final dataset. GitHub's Code Search caps results at roughly 1000 per query pattern, which I partially worked around by splitting queries using date ranges and file size qualifiers, though there's still an inherent ceiling on coverage. More frustratingly, some file paths on GitHub contain spaces, parentheses, and unicode characters (one particularly memorable path included Portuguese text about "creating your second brain with AI" and I still don't know if it was an actual project or a joke), and the scanner kept crashing on URL encoding issues. It took three separate rounds of fixes before the thing could crawl reliably without dying on edge cases.

After approximately 40 minutes of crawling, waiting on rate limits, and retrying failed downloads, the scanner had collected 900 configuration files from 839 unique repositories. Every repository identifier was SHA256 hashed before being stored. No owner names, no repository URLs, and no actual credential values exist anywhere in the dataset.

The initial results were surprisingly bad

75% of the collected configuration files contained at least one security finding.

I had gone into this expecting something around 30%, maybe 40% if I was being particularly pessimistic about developer habits. Not three quarters. Technically 74.8% but at that point who's counting. The number seemed high enough that I spent a while double-checking the analysis pipeline to make sure I wasn't overcounting or flagging things incorrectly.

The severity split caught my attention too. 1.6% critical, which means actual credential exposure. Then this massive 76.2% chunk classified as high. 21% medium. The rest low, barely worth mentioning. I couldn't figure out why high severity was so dominant until I drilled into the individual check results.

Turns out one specific check was responsible for almost all of it.

Nobody pins versions (43.6%)

Nearly half of all scanned configuration files reference MCP server packages without specifying which version should be installed. This single check, which I had honestly expected to be a minor contributor, accounts for the vast majority of high-severity findings in the entire dataset.

The pattern I encountered over and over again looks like this:

{
  "mcpServers": {
    "filesystem": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-filesystem", "/home/user"]
    }
  }
}

That -y flag is the problem. It tells npm to just grab whatever happens to be the latest version right now and run it. So if someone pushes a bad update to that package tonight, or if the maintainer account gets compromised, your agent loads the new code next time it starts. Nobody reviews it. There's no changelog diff, no approval process, nothing sitting between the compromised dependency and your running agent.

I manually spot-checked a handful of the unpinned configurations (through the public repositories, not stored data) to see whether the projects had lock files elsewhere that might compensate. Some did. The majority didn't. The pattern was clear: developers treat MCP configuration files as quick setup artifacts, something you write once to get the agent working and then largely forget about, rather than as security-critical infrastructure that deserves the same discipline you'd apply to a Dockerfile or a CI pipeline.

The fix, frustratingly, is trivial:

"args": ["-y", "@modelcontextprotocol/server-filesystem@1.2.3", "/home/user"]

What genuinely bothers me about this finding is that the JavaScript ecosystem already went through precisely this lesson. The left-pad incident in 2016, where a single maintainer unpublishing a tiny package broke half the internet's build pipelines, was supposed to have permanently established the principle that you pin your dependencies and maintain awareness of your supply chain. That was ten years ago. And now we're doing the exact same thing again, except the packages involved don't just pad strings or format dates. They read your filesystem and execute shell commands. The blast radius got massively bigger but somehow the config hygiene stayed in 2015.

The shell access problem is worse than it sounds

Roughly one in eleven configuration files grants the AI agent direct access to command execution, which is concerning enough as a headline number. The details underneath it are, I think, considerably more alarming.

The most frequently appearing entry in the "MCP server package" column across the entire dataset is not, in fact, a recognized package with documented behavior and scope controls. It's run. Just that. A bare shell command wrapper that showed up in 136 separate configurations, making it the single most popular choice by a wide margin.

For perspective: @modelcontextprotocol/server-filesystem, which is an official Anthropic-maintained package with actual path scoping and access controls, showed up in only 51 configurations. So the bare unrestricted shell executor beat the official scoped package by almost 3 to 1. I had to recount that because it seemed wrong.

Many of the server-filesystem entries were pointed at absurdly broad paths. Not /home/user/project/data or some appropriately scoped subdirectory, but just /. Or C:\. Or the user's entire home directory, which on most developer machines contains SSH keys, cloud credentials, browser profiles, basically your whole digital identity sitting there for the agent to browse through.

Put simply: giving shell access to a system that routinely processes arbitrary user prompts, tool call results from external APIs, and content retrieved from the open web is basically the same as giving your web application root access to the server it runs on. The information security community spent roughly two decades figuring out why that's a terrible idea. Same logic applies here, except arguably it's worse, because the input surface of an AI agent is much harder to validate than a traditional HTTP request.

Hardcoded credentials: fewer than expected

Twenty configuration files contained patterns that the scanner identified as hardcoded API credentials sitting directly in environment variables or command-line arguments. OpenAI key formats, GitHub personal access tokens, database URLs with the password right there in the connection string.

All of this in public repositories. Fully searchable.

I had honestly anticipated this number being higher, maybe a lot higher. The whole .env plus .gitignore thing has been repeated in every tutorial and starter template for years now, and based on this data it seems like that particular piece of advice actually landed. Which is nice. But twenty exposed credential sets is still twenty credential sets that someone should rotate, and based on the age and activity patterns of the repos I was looking at, I suspect most of those keys are still live.

What actually kept me scrolling through results: the combinations

The scanner was designed to evaluate individual findings in isolation. Surprisingly, the thing that proved most concerning during the analysis wasn't any single category of vulnerability. It was how frequently multiple issues appeared stacked together in the same configuration file.

I kept encountering setups structured roughly like this (reconstructed from patterns, not copied from any specific repo):

{
  "mcpServers": {
    "shell": {
      "command": "run",
      "args": []
    },
    "files": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-filesystem", "/"]
    },
    "github": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-github"],
      "env": {
        "GITHUB_TOKEN": "ghp_xxxxxxxxxxxxxxxxxxxx"
      }
    }
  }
}

Nothing pinned. Shell wide open via run. Filesystem pointed at root. GitHub token just sitting right there in the JSON. One file, all of it committed together, probably in about 30 seconds.

Any single one of those would flag as medium or high severity on its own. But when you stack them, the result is something else entirely. One bad npm update and an attacker can read everything on disk, run whatever commands they want, and push code to your GitHub. The agent handles the whole chain by itself with no human anywhere in the process.

This kind of compositional risk is not hypothetical. In February 2026, an autonomous AI agent operating under the GitHub account hackerbot-claw systematically exploited misconfigured CI/CD workflows across seven major open-source repositories, including projects from Microsoft, DataDog, and the Cloud Native Computing Foundation. The agent described itself as powered by claude-opus-4.5. It achieved remote code execution in five of the seven targets. The worst hit was Aqua Security's Trivy, a security scanner with over 32,000 GitHub stars. Using a stolen PAT from a misconfigured pull_request_target workflow, the attacker deleted releases. Then it published a malicious VSCode extension under Trivy's trusted publisher identity. The full incident was documented by StepSecurity, whose analysis confirmed that every attack relied on the same root cause: overly permissive configs that nobody audited.

We wrote a full analysis of the attack chain: orchesis.io/blog/hackerbot-claw

What I changed in my own configuration after reviewing 900 of them

My own MCP setup looks substantially different now than it did before I started this project.

Every package version was pinned explicitly. I realize this means manual updates and occasional breakage when I fall behind, but that friction is precisely the point. I want to actually see what changed in a package before my agent starts running the new code.

Shell access was removed entirely. I had run configured because it was convenient for quick experiments, and after seeing it appear in 136 other configurations with zero restrictions, I realized that "convenient" is a bad justification for granting an autonomous system unrestricted command execution.

All credentials were moved into .env files, and I double-checked (somewhat sheepishly) that .env was actually present in my .gitignore. During the scan I had noticed repositories where the .env file existed in the commit history but wasn't gitignored, which the scanner didn't specifically flag but which came up during manual spot-checking.

Filesystem paths were scoped down to the specific project directory where I actually need the agent to operate. Not my home folder. Not the root partition.

That's it. Four changes.

Limitations (and what this analysis doesn't tell you)

This dataset is 900 configuration files from public GitHub repositories, and that's it. No private repos, no configs that stayed on someone's laptop and never got committed, no enterprise setups managed through internal platforms.

The sample is almost certainly skewed toward individual developers and hobby projects rather than what you'd find in a production enterprise environment.

GitHub search also has a hard ceiling. Roughly 1000 results per query pattern, and even splitting queries by date ranges and file sizes only gets you so far. So this is a sample. A decent one, I think, but not a census.

Some of these configs are surely just examples or documentation rather than anything connected to a real running agent. I didn't try filtering those out, partly because it's genuinely hard to tell (how would you even distinguish reliably?), and partly because I think insecure examples might actually be worse than insecure production configs. An insecure example gets cargo-culted into dozens of new projects by people who assume the official-looking snippet must be fine.

The checks themselves are heuristic. They catch the obvious stuff like unpinned deps and leaked keys, but they're not going to find a sophisticated backdoor in a server's source code or a subtle permission escalation. The scanner flags "this dependency isn't pinned" but has no idea whether the currently installed version has a known CVE.

Where I think this is heading

75% of public configs failing basic checks isn't really an individual negligence problem at this point. When three quarters of your users get it wrong, the defaults are wrong. If the quick path through the documentation gives you an insecure config and the secure version requires you to know about version pinning and go look up the latest tag number, the insecure version is going to win basically every time. That's a design issue, not an education issue.

I think the MCP ecosystem has maybe a year before one of two things happens. Either the tooling catches up, meaning built-in config validation, automatic version locking, some kind of permission audit integrated into Claude Desktop and Cursor. Or there's a big enough supply chain incident involving MCP servers that the conversation gets forced from the outside. The EU AI Act enforcement begins in August 2026, which is five months away, and audit trails for AI agent behavior are about to become a legal requirement for a lot of companies. Looking at what I found in these 900 configs, my money is on the incident coming first. I hope I'm wrong about that.

The scanning tool, all 52 checks, and the full analysis pipeline are open source. Run it yourself at orchesis.io/scan. Everything runs in your browser, no data sent anywhere. All repository data has been anonymized.

Open source · MIT License

Try the MCP Scanner

Scan your MCP configuration in seconds. Runs entirely in your browser.