AI's the end of the Shell as we know it and I feel fine … but insecure

Model Context Protocol has many fine uses, but then it hinted at becoming a Von Neumann machine

Column Generative AI is changing so rapidly, it’s hard to stay up to date, so I generally avoid the newest shiny thing until it becomes unavoidable.

About six weeks back MCP – the Model Context Protocol – achieved that status, so I dived in.

MCP looks and acts as though it has been designed without any thought to security or bad actors

It's simple, clear, and very easy to get started with MCP. Within an hour, I'd coded my first pure Python MCP "server" - a simple tool, integrated into Anthropic's Claude Desktop, that fetches info about the current cost of electricity at my home.

I've grown a bit bored with language models. They talk a lot, but can't do much about it. Once MCP enters the chat to give them a few tools, their conversation becomes interesting again.

But converse about what? If you can attach any tool to a language model, what tool first?

In retrospect, the answer seems obvious: a Bash shell. My MCP "server" came to all of 50 lines of code (posted here). I fired it up in Claude Desktop to find that it's quite a bit of fun. Ask it to describe all the files in a directory – it corrected a spelling mistake I made in the path! Instruct it to use homebrew to install the ffmpeg media-handling libraries – please sir may I have another! Then, for the pièce de resistance – transcode an old AVI file into a modern H.265 MP4, using ffmpeg – a tool with such a dizzying array of options you do actually need an AI to get across them. It did that, too – although it thought it had failed, because my very simple tool lacks any ability to stream back to the language model any intermediate states, and that transcode took 20 minutes.

This looks and feels like the future of the shell. That said, in my 33rd year of Bash, let's acknowledge how profoundly dangerous such a tool could be at scale. I review and accept every Bash command line generated by Claude before passing it along to the tool for execution, because language models hallucinate – and I'm not running the tool in a container. It's helpful but won't replace my interactions with the shell. Instead, it'll paper over the gaps in my knowledge.

Moreover, MCP looks and acts as though it has been designed without any thought to security or bad actors. Anyone who runs a random MCP server without first inspecting the code, then fires it up in a container to inspect its inputs and outputs, gets what they deserve – a rootkit.

A few days ago, a friend passed me a preprint paper from Fudan University that shocked me out of my complacency. "Frontier AI systems have surpassed the self-replicating red line" it screams, name-checking John von Neumann's famous thought experiment. It goes on to describe the process by which its authors embedded two open source language models – Meta's Llama 3.1 and Alibaba's Qwen 2.5 – inside of an agent framework that turns both into extremely resourceful self-replicators.

Given little more than shell access, these models worked out how to scan the hardware they ran on to find their weights and inferencing software, duplicating both, then firing up a second instance of themselves. Voila! You've got a replicator. Along the way, the models either used tools available to them via Bash (ls, curl, etc), or – when they could not locate a tool they needed, coded up a functional equivalent in Python.

Here we need to note that the wonderful Tobias Mann has written many words for The Register about how to run these models on bog-standard hardware. This is not a theoretical capability of the latest-and-greatest models from OpenAI or Anthropic: It's a capability that can be deployed today on just about any gaming PC.

This has the vibe of an AI-driven update to the Morris Worm – a moment back in the very early days of the internet when folks suddenly realized that having everything on the internet wide open to everyone wouldn't be a survivable strategy. It looks as though we're confronting another such moment. "The future doesn't repeat," Mark Twain noted. "But it does rhyme." ®