Cybersecurity Landscape for New AI Protocols

Cybersecurity Landscape for New AI Protocols.

Dulan Dissanayake

Lead Software Architect

Security measures should become a growing concern as AI emerges with new technologies and protocols. As we all know, there are 2 new AI protocols introduced by Google and Anthropic:

Agent2Agent (A2A) – By Google https://developers.googleblog.com/en/a2a-a-new-era-of-agent-interoperability/
Model Context Protocol (MCP) – By Anthropic https://www.anthropic.com/news/model-context-protocol

Organisations are eager to use these new technologies and protocols, which allow them to do greater implementations. But this comes with a price, where these new technologies and protocols can have security loopholes or unidentified vulnerabilities.

Some research studies showed that as of February 2023, OpenAI GPT-3 uncovered 213 security vulnerabilities in the code repository, and only 4 were false positives.

Today, we will look into understanding Model Context Protocol (MCP) and potential Security risks that can be exploited by threat actors. First of all, let’s dive in and understand what MCP is and its architecture.

What is MCP and its Architecture?

MCP is a Client-Server protocol, and it standardises how applications provide context (data and tools) to LLMs for the operations. Think of MCP like a USB-C port for AI applications. Just as USB-C provides a standardised way to connect your devices to various peripherals and accessories, MCP provides a standardised way to connect AI models to different data sources and tools.

Architecture:

MCP follows a Client-Server architecture host application can connect to multiple servers.

Credit: Image from https://modelcontextprotocol.io/introduction#general-architecture

MCP Hosts – Applications (Claude Desktop, IDE’s etc) or AI tools that require data access. This is the coordinator between the Client-Server.
MCP Clients – Protocol implementation that establishes and maintains the connection between a MCP server. This will be initiated by the MCP Host.
MCP Servers – A Lightweight program that implements the MCP protocol and exposes the data and tools to the MCP clients. This will expose a list of capabilities the Server has via the MCP protocol.
Local Data Sources – Local files, databases, API’s that MCP Servers can access.
Remote Services – External API’s available over the internet for MCP Servers.

We will now look into the MCP ecosystem and try to identify potential security risks and mitigations to avoid these.

Threats/Risks:

Prompt Injection Attack:

This is a new attack vector against Large Language Models (LLMs) where attackers inject malicious inputs into prompts and manipulate GenAI systems. In the MCP ecosystem, Indirect Prompt Injection will be a main area that can be exploited. Imagine an attacker who manages to take control over an external resource and crafts a malicious message containing hidden instructions/commands within this resource (eg, website, API server, etc). When the LLM starts interacting with these resources via an MCP server, it can pass those malicious instructions/commands inside the LLM for execution. Similar to Remote Code Execution.

Session Hijacking:

MCP Server and Client will maintain a stateful, persistent connection to each other by creating a long-lived session. If the session management is not implemented securely, the attacker will be able to hijack the session and potentially allow them to impersonate a real user. This will eventually open the doors for data exfiltration and ransomware attacks.

Tool Poisoning Attack:

This is a specialised form of Indirect Prompt Injection, where LLMs will see the MCP tools’ complete description, including the hidden instructions. These hidden instructions can be in the form of a comment, and users may not notice them inside the tool’s definition. When LLMs execute these tools, the hidden instructions will also be executed. These instructions can be used to read sensitive data from the host and send it to a malicious server or to interact with another MCP server to exfiltrate data from those resources.

Data Exfiltration:

By exploiting potential security vulnerabilities like Prompt Injection, Session Hijacking, and Tool Poisoning, an attacker can easily gain access to users’ data.

These are a few potential security threats/risks if the MCP ecosystem is not implemented with proper security measures and controls. Below are some more potential threats/risks broken down by the MCP components.