Why Testing AI Abilities Is Different From Testing Ordinary PHP

Here’s a claim worth being precise about before this course goes any further: an ability is not hard to test. wp_register_ability() registers a namespaced name, a schema, a permission_callback, and an execute_callback, all plain PHP, all testable with ordinary PHPUnit exactly the way you’d test any other WordPress hook-driven code. Nothing about the Abilities API itself resists testing. What resists testing is a specific, common design decision: an execute_callback that reaches out to an LLM through the PHP AI Client SDK before returning. That one line changes what “correct” even means for the rest of the function, and this lesson draws the line between the part that stays ordinary and the part that doesn’t.

Step 1: Most of an ability is already ordinary PHP

Take an ability with no AI dependency at all, one that lists open support tickets by querying a custom post type. Its permission_callback checks a capability. Its execute_callback runs a WP_Query and shapes the result. Both are deterministic: same input, same WordPress state, same output, every single time. A PHPUnit test against either one is exactly as easy as testing any other plugin function that reads or writes WordPress data, assertTrue(), assertSame(), assertCount(), done. The next lesson sets up the test suite, and the lesson after that writes exactly these tests. None of that work is complicated by anything AI-related, because there’s nothing AI-related in it yet.

Step 2: Naming the actual problem precisely

The complication starts the moment an execute_callback calls wp_ai_client_prompt()->generate_text() before returning, the pattern the PHP AI Client SDK course builds as a summarize-post ability. That single call introduces two separate problems, and it matters that they’re separate, because they need two different fixes:

Both problems live inside the same function call, but they don’t share a fix. Fixing non-determinism means changing what you assert on and how you score it, not changing whether the call happens. Fixing the external dependency means changing whether the real call happens in a test at all, replacing it with something local and fast. Lessons 4 and 5 build one fix each.

Step 3: Why “just mock the AI call” isn’t sitting there waiting for you

If you’ve used WP_Mock or Brain\Monkey to stub a WordPress core function in a unit test, the instinct here is reasonable: stub wp_ai_client_prompt() the same way. It doesn’t work as cleanly as it sounds, for a reason worth being explicit about. The PHP AI Client SDK has no built-in test double for this call, no shipped fake provider, no documented “return this canned response instead of calling out” mode. generate_text() is the end of a fluent builder chain (with_text(), using_system_instruction(), using_temperature(), and so on), and stubbing a fluent chain with a general-purpose function-mocking library means stubbing every method in the chain and keeping all of them in sync with however your production code actually calls it, which is exactly the kind of brittle setup that breaks the moment someone adds one more builder call.

Step 4: The real fix is a design pattern, not a mocking library

Since the SDK gives you nothing to mock, the fix has to happen one layer up, in how you write the execute_callback in the first place. Wrap the part that calls wp_ai_client_prompt() behind a small interface of your own, one method, a plain string or WP_Error in, a plain string or WP_Error out. Your production code implements that interface by calling the real SDK. Your tests substitute a different implementation that returns a canned value instantly, no network call, no API key, no non-determinism, because you’re testing your own contract, not the SDK’s fluent chain. This is ordinary dependency injection, the same technique any well-tested PHP codebase uses for a database connection or an HTTP client, applied to the one place in an ability where it’s actually needed. Lesson 4 builds it in full.

Step 5: The four layers, and how each one gets tested

That last row is the one piece of an AI-calling ability that genuinely can’t be tested the way the other three are, and trying to force it into an exact assertion is the single most common mistake teams make here: either skipping testing the AI output entirely, or writing a brittle assertStringContainsString() test against one particular phrase that happens to break the next time the model or the prompt changes slightly. Lesson 5 gives that layer its own approach.

Test it: sort your own abilities into these four layers

Before writing a single test, list every ability your plugin registers and mark which of the four rows in Step 5’s table actually applies to each one. Most abilities will only have the first three rows, no AI call at all. A permission_callback that checks current_user_can() and an execute_callback that runs get_posts() never touch row four. Only abilities matching the PHP AI Client SDK course’s summarize-post pattern need it. Knowing which of your abilities need which kind of test, before you write any of them, is the entire point of this lesson.

Recap

Abilities are plain PHP, and testing registration, permission_callback, and an execute_callback with no AI dependency is ordinary PHPUnit work, nothing about the Abilities API changes that. The real complication is narrower than it sounds: an execute_callback that calls wp_ai_client_prompt() introduces non-determinism and an external dependency, two separate problems needing two separate fixes. There is no built-in SDK mock for the AI call, so the fix is architectural, wrap the AI-calling logic behind your own small interface so it’s injectable in tests, and grade the actual generated output with evals rather than exact assertions. The next two lessons build the ordinary PHPUnit half, Lessons 4 and 5 build the two AI-specific fixes.

Resources & further reading

• MCP Adapter repository, testing guide, GitHub
• PHP AI Client SDK repository, GitHub
• Abilities API reference, developer.wordpress.org
• WP_Mock repository, GitHub
• Brain\Monkey repository, GitHub