Verify

Verify the claims

Every number this site publishes about ue-mcp can be checked on your own machine. The steps below give the exact commands. The surface measurement and the task benchmark apply the same rules to any MCP server, so you can run them against ue-mcp and the servers you are comparing it with.

1. Measure the advertised surface

ue-mcp surface starts any MCP server, sends initialize and tools/list, and prints the tool count, the action count and the size of the input schemas. It needs no Unreal install.

  1. 01Measure ue-mcp as a client gets it by default: npx ue-mcp surface -- npx -y ue-mcp
  2. 02Measure ue-mcp with every action listed inline: npx ue-mcp surface --env UE_MCP_CONTEXT_STRATEGY=full -- npx -y ue-mcp
  3. 03Measure another server with the command that starts it: npx ue-mcp surface -- <command> [arguments], or npx ue-mcp surface --url <url> for a streamable HTTP server. Add --json for machine-readable output.

surface ships in the first ue-mcp release after 1.4.16-beta.1. Until that release is on npm, clone the repository, run npm ci, and use npx tsx src/index.ts surface in place of npx ue-mcp surface.

Actions are counted the same way for every server: a tool whose input schema has a top-level action, operation or command parameter with enumerated values counts one action per value, and any other tool counts as one. Tokens are estimated at four characters per token. A server that hides its actions behind a search tool, or takes the action name as free text, counts one per such tool. ue-mcp's default is one such gateway tool.

ue-mcp 1.4.16-beta.1, no editor runningToolsActionsInput schemas
Default (micro)15491 bytes, ~123 tokens
full323,727109,907 bytes, ~27,477 tokens

The full row includes the flow tool and its three actions, which the category counts quoted elsewhere on this site leave out. It was also measured on a newer build than the release those counts come from, so the two can differ by a few actions.

Measured on 2026-11-21. The surface docs give the same measurement for three other public servers, with their versions and commands.

2. Check the comparison grades

  1. 01Open /compare.md.
  2. 02Under Sources, find the public documentation or repository each column was graded from. A cell with a mark beside it has its own note below its table.
  3. 03Pick a grade and open the source. yes needs a dedicated, typed tool for the task; reaching it only through a generic Python tool is partial.
  4. 04If a grade is wrong, open an issue.

3. Reproduce the test results

From a fresh clone, with Node.js 20 or later and no Unreal install:

git clone https://github.com/db-lyon/ue-mcp.git
cd ue-mcp
npm ci
npm test

npm test runs the unit suite, the multi-editor suite and the integration suite, which spawns the real CLI, server and daemon in sandboxes with local stand-ins for the editor and every network service.

  • tests/coverage/behavioural-coverage.json maps every bridge action to the live and C++ automation tests that run it and check its effect. Print its totals with node -e "console.log(require('./tests/coverage/behavioural-coverage.json').summary)".
  • npx vitest run tests/unit/behavioural-coverage.test.ts fails when an action has neither a test nor a recorded exemption.
  • CI runs the type check, the unit and integration suites and the release gates on every pull request before it merges: CI runs.
  • With Unreal Engine installed: node scripts/deploy.mjs, npm run build, then npm run test:live, which launches its own offscreen editors on copies of the test project tests/ue_mcp and runs every live test against them.

4. Run the task benchmark

The repository's benchmark folder holds 11 editor tasks: Blueprint graphs and components, a material graph and instance, level actors, Sequencer, landscape, PCG, rollback of a failed edit, deleting a referenced asset, and Play In Editor. Each task has an exact prompt, and each check reads the result through Unreal's own Python API and prints PASS or FAIL. The checks never call an MCP server.

  1. 01Create a project from the Blank template in Unreal Engine 5.7 or later, with no starter content. Enable the Python Editor Script Plugin, and turn on Project Settings > Plugins > Python > Enable Remote Execution.
  2. 02Connect your agent to the MCP server you are testing, with the project open in the editor.
  3. 03Give the agent each prompt from benchmark/tasks/, in order, in one editor session. Leave the Play In Editor session from the last task running.
  4. 04Score it: python benchmark/run_checks.py --engine "<engine folder>". With more than one editor open, add --project <name>.
  5. 05Read the agent's transcript as well. The checks see only the final editor state, not whether the server did the work or the agent found another way.

ue-mcp passed 11 of 11 checks on 2026-11-21, on Unreal Engine 5.9 with tests/ue_mcp. That run called ue-mcp's actions directly from a script rather than through an agent, so it shows the actions produce the required state, not how well an agent chooses them. The benchmark docs list the actions used for each task and the two that needed a second attempt.

What these checks do not show

  • Surface counts are what a server advertises, not whether its actions work.
  • The ue-mcp benchmark result was produced by ue-mcp's maintainers. Run the benchmark yourself, with your own agent, against each server you are considering.
  • The checks have been run on Unreal Engine 5.9 only.
  • Eleven tasks sample each area of the editor; they do not cover every action in it.