📝contract-first-api-testing-openapi.md

Contract-First API Testing with OpenAPI

The Failure Pattern

Most API test suites start the same way.

Someone writes a few happy-path integration tests. Then a few negative tests. Then a few auth tests. A new endpoint gets added, so somebody copies an old spec, changes a URL, and adjusts a couple assertions.

Six months later you have:

duplicate request logic
inconsistent payload generation
endpoints with zero negative coverage
test files that do not match the current API shape
a contract in one place and actual coverage in another

That is the point where the suite starts feeling expensive instead of useful.

The OpenAPI Contract Should Do More Than Generate Docs

Most teams use OpenAPI for one of two things:

interactive docs
client SDK generation

Both are useful. Neither is enough.

The contract should also drive the test strategy.

If the contract is the source of truth for the API surface, it should inform:

which endpoints exist
which parameters are required
which request bodies are valid
which status codes are expected
which auth rules apply

At that point, you stop inventing test coverage from memory.

What Contract-First Actually Means

It does not mean "generate every test automatically and hope for the best."

It means the contract becomes the truth that your test system reads from.

In practice, I want four layers.

1. Surface Discovery

Read the contract and build a normalized list of operations.

type OperationContract = {
  operationId: string;
  method: 'GET' | 'POST' | 'PUT' | 'PATCH' | 'DELETE';
  path: string;
  tags: string[];
  requiresAuth: boolean;
  requestSchema?: unknown;
  responseSchemas: Record<string, unknown>;
};

That lets the test system reason about the API systematically.

2. Scenario Matrix Generation

The contract tells you what is possible. The test matrix turns that into behavior coverage.

For a typical create endpoint, I usually want rows for:

valid payload
missing required field
invalid type
boundary length/value cases
unauthorized request
forbidden request when roles matter

3. Validation

The test should validate the response against the contract, not just check a couple fields.

4. Domain-Specific Assertions

The contract does not know business rules. Your tests still need those.

Examples:

record was actually created
audit log was emitted
side effects happened correctly
duplicate operations are rejected in the right way

That is the balance.

Use the contract to define the shape. Use domain assertions to prove the behavior.

A Practical Flow

Here is the structure I prefer.

Step 1: Parse the OpenAPI spec

import fs from 'fs';
import yaml from 'yaml';
 
const raw = fs.readFileSync('./openapi.yaml', 'utf8');
const spec = yaml.parse(raw);
 
const operations = Object.entries(spec.paths).flatMap(([path, methods]) => {
  return Object.entries(methods as Record<string, any>).map(([method, operation]) => ({
    operationId: operation.operationId,
    method: method.toUpperCase(),
    path,
    tags: operation.tags ?? [],
    requestSchema: operation.requestBody?.content?.['application/json']?.schema,
    responseSchemas: Object.fromEntries(
      Object.entries(operation.responses ?? {}).map(([status, response]: [string, any]) => [
        status,
        response.content?.['application/json']?.schema,
      ])
    ),
  }));
});

Now you have a machine-readable operation list instead of tribal knowledge.

Step 2: Generate scenario rows

type ScenarioRow = {
  name: string;
  payload?: unknown;
  expectedStatus: number;
  auth: 'none' | 'user' | 'admin';
};
 
const createUserScenarios: ScenarioRow[] = [
  {
    name: 'creates a user with a valid payload',
    payload: { email: '[email protected]', password: 'Test1234!' },
    expectedStatus: 201,
    auth: 'admin',
  },
  {
    name: 'rejects missing email',
    payload: { password: 'Test1234!' },
    expectedStatus: 400,
    auth: 'admin',
  },
  {
    name: 'rejects unauthenticated create',
    payload: { email: '[email protected]', password: 'Test1234!' },
    expectedStatus: 401,
    auth: 'none',
  },
];

This is where the suite becomes readable. Not because the spec got bigger, but because the behavior is now data.

Step 3: Validate responses against the contract

import Ajv from 'ajv';
 
const ajv = new Ajv({ allErrors: true, strict: false });
 
function validateResponse(schema: object, body: unknown) {
  const validate = ajv.compile(schema);
  const valid = validate(body);
 
  if (!valid) {
    throw new Error(`Contract validation failed: ${ajv.errorsText(validate.errors)}`);
  }
}

This catches a whole category of drift that assertion-by-assertion tests miss.

What the Contract Can and Cannot Tell You

This distinction matters.

The contract can tell you:

shape
required fields
enum values
parameter formats
documented status codes

The contract cannot tell you:

whether duplicate requests are handled safely
whether pagination is stable under changing data
whether side effects are correct
whether authorization rules are actually enforced properly
whether the response is semantically right for the business rule

So the contract is the coverage scaffold, not the whole test strategy.

The Biggest Benefits I Have Seen

1. Coverage Stops Being Accidental

When the operation list comes from the contract, you can answer simple but important questions:

which endpoints have tests
which status codes are covered
which operations still have no negative scenarios

That makes coverage governable.

2. New Endpoints Get a Default Testing Path

A new operation enters the spec. The test system sees it. It is now visible work, not a forgotten corner.

3. Drift Gets Caught Earlier

If the implementation no longer matches the contract, the suite tells you. If the contract changed and the tests did not, the suite tells you that too.

4. The Specs Get Smaller

This is one of my favorite side effects.

A contract-first suite usually means:

less copy-pasted request code
fewer giant test files
more shared validation
clearer scenario tables

The tests become easier to reason about because they stop hiding the pattern.

A Structure That Scales Well

I like organizing contract-driven API tests like this:

tests/
  contracts/
    openapi.yaml
  domains/
    users/
      UserScenarios.ts
      UserValidations.ts
      UserApi.spec.ts
    orders/
      OrderScenarios.ts
      OrderValidations.ts
      OrderApi.spec.ts

The names matter less than the separation:

contract parsing
scenario rows
reusable validations
thin execution specs

That keeps the suite honest as the API grows.

Where Teams Usually Overdo It

There are two common overcorrections.

1. Trying to Fully Generate All Tests

Pure generation sounds elegant. In practice, it usually produces shallow tests with weak intent.

The contract should generate the scaffold. Humans should still design meaningful behavior cases.

2. Trusting the Contract Too Much

If the OpenAPI spec is stale, a contract-first suite can give you false confidence.

That is why I want the contract in the normal engineering workflow:

updated in the same PR as the endpoint
reviewed like code
treated as a release artifact, not a side file

My Rule of Thumb

If an API exists in production, I want three things tied back to the contract:

at least one happy path
at least one auth or permission path when relevant
at least one invalid-input path for the main request shape

That baseline alone catches a surprising amount of real drift.

The Main Takeaway

Hand-written API tests drift for the same reason hand-written docs drift: they rely on memory and discipline without enough structure.

OpenAPI gives you that structure.

Not by replacing thoughtful testing, but by making it systematic.

That is what contract-first means to me:

the contract defines the surface
the scenario matrix defines the behavior coverage
validation proves shape correctness
domain assertions prove real behavior

Once you set the suite up that way, API coverage stops feeling like endless manual upkeep and starts feeling like an actual system.