Features
Find the part of CertiQA that matches the job in front of you.
CertiQA covers a lot of ground, and a long feature list does not help you decide whether it fits. This page is organised by the job you are trying to do. Each item says what it does in a line and links to the page that explains it properly.
In the product
API tests that chain requests, run against a public API.
A booking lifecycle against restful-booker (a public practice API): create a booking, read it back, update it, delete it, and confirm it is gone. Each request passes values to the next, and the whole suite runs in one report.
Test an AI agent
Your agent calls tools and answers customers, and a single unit test cannot tell you how it behaves when a tool fails or a user pushes back.
- Tool-call mockingIntercept your agent's REST and Model Context Protocol tool calls, and return the responses you define so runs repeat. Chosen tools can pass through to a real system.
- LLM judgeScore groundedness, hallucination and policy safety against rubrics you write in plain English.
- Red-teaming20+ built-in scenarios across seven OWASP LLM Top 10 categories, plus attacks generated from your agent's own tools. Team Pro Plus and Enterprise Ultra.
- Multi-agent checksSend OpenTelemetry GenAI spans from LangGraph, CrewAI or AutoGen, assert who may delegate to whom, and cap hops, tool calls and run time.
- Playground to testChat with your agent, inspect its tool calls and reasoning, then promote a good session into a repeatable test.
- Accessibility-related checksRun an agent with keyboard-only or text-and-role browsing and compare it with a normal run, or add scripted conversation checks to the tests you already run. Experimental, and not a certification.
- Worked examplesComposite example scenarios with no customer names or metrics, to see what a suite looks like.
Test a web UI
Selectors drift, builds go red for reasons that are not bugs, and the first version of a test takes longer than the feature.
- CertiHealEach step keeps ranked locator candidates and repairs drift in a fixed order. Every repair is listed in the run report, and a missing element still fails.
- Record in Chrome or EdgeWalk through a flow in your own signed-in browser and get an editable case.
- Import and exportImport Playwright, Cypress, Selenium and Postman files. Export a case to Playwright TypeScript.
- Chromium, Firefox and WebKitChoose the browser when you run a suite, and the report keeps each run separate.
- Data-driven casesUpload an Excel or CSV file and run a case against its rows.
- Accessibility checksAutomated axe-core scans and a keyboard audit that help test and report toward WCAG, ADA and Section 508. They report findings and do not certify compliance.
Reuse and review tests
The same steps are copied into every case, tests arrive faster than anyone reads them, and nobody can say what is covered.
- CertiBlocksBuild login or setup once. Edit it and every case, suite and agent test that embeds it picks up the change on its next run.
- One case, several suitesA case in Smoke and Nightly is one definition, not two copies.
- Variables and Vault secretsSecrets stay in an encrypted Vault, can be scoped to an environment, and are not written into exports.
- AI assistants over MCPCursor, Claude Code and Claude Desktop draft cases into CertiQA, and reviewers read them as plain steps.
- Requirement coverageLink requirements to cases and see which requirements have none.
- Run comparisonPick two runs and see regressions, improvements and persistent failures with the error and screenshot for each.
- Change ReviewRequire approval of changes to cases and blocks. Team Pro and above.
Show a release is ready
A manager or reviewer asks for proof, and the answer is scattered across pipelines, dashboards and chat.
- Release ReportSign off a run to generate a dated export with a SHA-256 hash, or export a date range through the API. It is evidence, not certification.
- CI triggers and reportsStart runs from your pipeline, receive HMAC-signed webhooks, and download JUnit XML or CTRF JSON.
- Jira and SlackFile defects with step logs attached, and send failure digests to a channel.
- Flake handlingFlake Shield and auto-quarantine keep known flaky cases from blocking a build. Team Pro and above.
- Your own keys and networkBring your own model keys on Team Pro Plus. Customer-hosted runners are part of Enterprise Ultra.
FAQ
Questions about what CertiQA does
What can CertiQA test?
AI agents, web UIs and APIs, in the same suites. For agents it mocks tool calls, injects failures, scores answers with an LLM judge and runs OWASP LLM Top 10 attacks. For web UIs it runs Playwright-based tests and repairs locators that have drifted. A single case can mix browser steps, API calls and agent steps.
AI agent testingWhere should I start?
Start with the job that hurts most. If you ship an AI agent, build one case with mocked tools and an LLM judge check. If you own a flaky browser suite, import it or record one flow and read the run report. In both cases the 14-day trial includes Team Pro Plus features.
Can CertiQA generate tests from a requirements document?
Your connected AI assistant (such as Cursor or Claude Code) can read requirements or PRDs in your repo and draft cases over MCP, using the validate_case tool to check steps before saving. Nothing is added to your suite until a person reviews the draft. You can also add requirements to a project and see which ones have no linked case.
Coverage and review for automation leadsCan I get my tests out of CertiQA?
Web UI test cases export to Playwright TypeScript, one case at a time, and secrets referenced in steps become process.env lookups in the exported file. The export is plain Playwright and does not include self-healing.
Web UI automationHow does CertiQA fit into CI and issue tracking?
Trigger runs from your pipeline, receive HMAC-signed webhook callbacks, and download results as JUnit XML or CTRF JSON. Failures can be filed in Jira with step logs attached, and digests sent to Slack.
Can I run tests inside my own network?
Customer-hosted runners are part of Enterprise Ultra. They connect outbound over HTTPS so staging endpoints, test data and model keys can stay behind your firewall. The security page describes what leaves your network.
Security overviewWhich models can CertiQA use for judging and authoring?
You can connect your own keys for Anthropic, OpenAI, Gemini, Grok and DeepSeek and route each workload to a model. Multi-provider bring-your-own-key is part of Team Pro Plus and Enterprise Ultra, and the 14-day trial includes Team Pro Plus features.
Compare plansWhat is a Release Report, and is it a certification?
It is a dated export of what CertiQA measured for a project, for a signed-off run or a date range, stamped with a SHA-256 hash so later edits are detectable. It is evidence for a human reviewer, not a certification.
For engineering managersPick one job and try it on your own work.
Build one agent case with mocked tools, or import one flaky browser suite, and see how it behaves. Not sure which? Read the brief for managers or compare plans.