WP Copilot
Search… /
Wrap-up

Course Recap: A Testing Checklist Before Shipping an Ability Change

⏱ 11 min

Seven lessons, one recurring idea: an ability is plain PHP until the moment it calls an LLM, and that one moment needs a different kind of test than everything around it. This lesson doesn’t introduce anything new, it’s the checklist you actually run through before merging a change to an ability, in the order that catches the most expensive mistakes first.

What this lesson pulls together
The four-layer mental model from Lesson 1
Registration, permission, deterministic execution, and AI output, each needing a different kind of test.
The dependency-injection pattern from Lesson 4
The one architectural decision that makes an AI-calling ability testable at all.
The distinction between blocking tests and informational evals
What gates a merge versus what you read before shipping a prompt change.
Where this course sits relative to Course 19 and Course 8
Preventive testing versus reactive debugging, and where this course goes deeper.
Prerequisites

All seven previous lessons in this course. This one is a reference you’ll come back to, not new material to work through once.

Step 1: The checklist itself

Run through these, in order, before merging any ability change
1
Registration and schema
wp_has_ability() confirms it exists, the schema shape matches what you intended, and meta.mcp.public is exactly what you meant it to be, not accidentally flipped. Lesson 3.
2
permission_callback across real roles
Tested for the role that should pass, the role that shouldn't, and logged-out. Never tested only as an admin. Lesson 3.
3
execute_callback with no AI dependency
Exact assertions on input and output shape, data providers for multiple cases rather than copy-pasted test methods. Lesson 3.
4
If it calls an LLM: is the AI call injectable
A one-method interface between execute_callback and wp_ai_client_prompt(), wired through a filter, with a fake substituted in tests. If this isn't in place yet, stop and do Lesson 4 before writing more tests around this ability. Lesson 4.
5
Error propagation from the AI layer
A failing fake confirms execute_callback surfaces a WP_Error correctly, with no real provider outage needed to prove it. Lesson 4.
6
Evals for the actual generated output
Structured output scored against field-level criteria, run separately from the PHPUnit suite, with a pass-rate threshold rather than a 100% expectation. Lesson 5.
7
Custom handlers contract-tested
Your own McpErrorHandlerInterface or McpObservabilityHandlerInterface implementation tested against the interface directly, plus MCP Inspector for what a real client actually sees. Lesson 6.
8
CI wired and merge-blocking
The deterministic suite runs in GitHub Actions against a real MySQL service container, and a required status check actually blocks the merge, not just reports it. Lesson 7.

Step 2: The one decision that determines how much of this checklist applies

Not every item here applies to every ability. The fork in the road is Step 1’s fourth row: does execute_callback call wp_ai_client_prompt() or anything like it. If it doesn’t, an ability like the ticket-listing example from Lesson 3, you’re done after the first three items, ordinary PHPUnit, no different from testing any other plugin function. If it does, the last four items all apply, and skipping any one of them leaves a real gap: no injection point means no way to test the ability’s own logic without a live API call, no evals means nobody notices the model’s output quality degrading over weeks, no contract test means a custom handler you built could be silently unwired and nothing would tell you.

The most common shortcut, and why it costs more later

The shortcut worth naming explicitly: shipping an AI-calling ability with only a manual, one-time “I tried it and it looked right” check, no injected dependency, no automated test at all. It works, right up until a prompt tweak, a provider switch, or a schema change breaks something nobody re-checked by hand. Every pattern in this course exists specifically to make that manual check unnecessary, replaced by something that runs on every change instead of once.

Step 3: Where this course sits in the wider track

This course is the preventive counterpart to Debug & Troubleshoot the MCP Adapter. That course’s whole premise is reactive: something already broke, reproduce it, check the log, isolate registration, permission, or execution. Everything built across these eight lessons exists to reduce how often you ever need that workflow at all, catching the same three failure categories, registration, permission, execution, before a change reaches production rather than after an agent hits it there. If you haven’t taken that course yet, it’s worth doing regardless, since the debugging workflow it builds is still the right tool for the failures no test suite catches, an environment-specific issue, a client-side quirk, something genuinely new.

This course also goes considerably further than the single CI/CD lesson inside Advanced WordPress MCP Architecture & Enterprise AI, which introduces wp scaffold plugin-tests and a basic GitHub Actions workflow in one lesson as part of a much broader capstone course. Everything from Lesson 2’s Composer-based test library onward, the AI-client injection pattern, evals, contract testing against the adapter’s own real fixtures, is what that single lesson didn’t have room for.

Step 4: What to build next

If you’re starting a new plugin with abilities from scratch, Build a WordPress MCP Server From Scratch is the natural place to register your first real ability, this course’s checklist is exactly what to apply the first time you change it. If you already have abilities in production with no tests at all, start with Lesson 2’s PHPUnit setup and Lesson 3’s non-AI tests first, that’s the highest ratio of protection to effort, before moving on to Lesson 4’s injection pattern for anything that calls an LLM.

Recap

The checklist in Step 1 is the entire course compressed into eight lines: registration and schema, permission across real roles, deterministic execution, an injectable AI dependency, error propagation from that dependency, evals for the actual output, contract-tested custom handlers, and a CI pipeline that actually blocks a bad merge rather than just reporting one. Not every ability needs every line, the fork is whether execute_callback calls an LLM at all, but every ability that does needs all eight, and skipping any one of them is exactly the gap a future regression will find first.

Resources & further reading

← Wiring It Into CI/CD Finish ✓