# Testing and verification

URL: https://ship-and-scale.vercel.app/docs/operations/testing

Know which checks are local, which touch hosted services and what each passing result proves.

## Start from a clean release copy [#start-from-a-clean-release-copy]

Use the Node version required by `package.json` and install with `npm ci`. That command uses the lockfile and generates the MDX collection. A clean copy should not depend on another developer's `.next`, `.source`, `node_modules` or private environment. Configure your own authorized hosted target only for tests that need it.

This guide describes the **template's** commands. The standalone seller-manual project has its own tests and no provider runtime. Neither a docs build nor a mocked API response proves that a buyer can create a real project, receive an email or complete a payment.

## Command and effect table [#command-and-effect-table]

| Command                                     | Service effects / cleanup                                                                     | What a pass establishes                                                   |
| ------------------------------------------- | --------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------- |
| `npm run typecheck`                         | No provider operation; local TypeScript output/cache                                          | Types and imports agree with this source/configuration                    |
| `npm run lint`                              | No provider operation                                                                         | Selected code rules pass                                                  |
| `npm run build`                             | Produces `.next`; may execute build-time reads in your app                                    | Production compilation and generated routes; not payment/inbox acceptance |
| `npm test`                                  | Local setup/provider fixtures; no real provisioning                                           | Orchestration, failure, retry and isolated service logic                  |
| `npm run test:ui -- tests/e2e/docs.spec.ts` | Uses an already-running browser/server; reads public routes                                   | Actual documentation interactions in Chrome                               |
| `npm run kickstart:check`                   | Read-only provider probes                                                                     | Required observable account/project access; future writes remain unproven |
| `npm run kickstart:deploy -- --check`       | Read-only source/provider probes                                                              | Deployment prerequisites visible before writes                            |
| `npm run test:hosted`                       | Writes SQL fixtures inside rolled-back transactions on explicitly selected hosted test target | Database policies, grants and lifecycle invariants covered by fixtures    |
| Opted-in `*.live.spec.ts` browser tests     | May create/delete accounts, send messages, upload files or call paid AI                       | The specific exercised integration paths; cleanup depends on the script   |

Read a test before enabling it. “Test” in a filename does not mean read-only. Rollback covers the SQL transaction, not automatically an external email, file-storage operation or paid provider request. Use dedicated targets and fixture identities, never shared production customer records.

## Static checks and local fixtures [#static-checks-and-local-fixtures]

```sh
npm run typecheck
npm run lint
npm test
npm run build
```

`npm test` runs `tests/setup/*.test.mjs` through Node's test runner with `--test-isolation=none`. Its fixture services cover denied account checks, uncertain responses, idempotent retries, source boundaries and provider orchestration. A useful failure test asserts that no write happens after a required preflight failure, not just that an error string appears.

When you change permissions or billing synchronization, test the adversarial boundary: another workspace, a demoted member, a stale session, a repeated event or two concurrent operations. Avoid a test that merely repeats the implementation's happy-path string.

## Browser and Chrome setup [#browser-and-chrome-setup]

The shipped `playwright.config.ts` uses installed Chrome, one worker and `http://localhost:3000` by default. It does not start the app automatically. Start `npm run dev` in another terminal and wait for the ready message. To use another port, set the active local `APP_URL` to that same origin and pass the port through the design watcher, for example `npm run dev -- --port 3108`. `APP_URL_DEV` is only a saved reference; do not change the production origin for routine local testing.

Point browser checks at the same origin. In PowerShell, set `$env:PLAYWRIGHT_BASE_URL = 'http://localhost:3108'`; in POSIX shells, prefix the test command with `PLAYWRIGHT_BASE_URL=http://localhost:3108`. Run a focused suite with `npm run test:ui -- tests/e2e/docs.spec.ts`. The origin guard checks scheme, host and port. Routine local development does not need `npm run kickstart -- --restore-local`; that command changes the shared hosted Auth/email/MCP origin.

```sh
npm run test:ui -- tests/e2e/docs.spec.ts
npm run test:ui -- tests/e2e/seo.spec.ts
```

For PowerShell, set `$env:PLAYWRIGHT_BASE_URL = 'http://localhost:3000'` before the command if needed; POSIX uses `export PLAYWRIGHT_BASE_URL='http://localhost:3000'`. Chrome must already be available. A missing browser executable is an environment failure, not a passing UI check.

Check desktop/mobile, both themes, keyboard focus, long names, loading/empty/error states and browser console errors. A screenshot proves the captured view rendered, not that every control works. Keep traces, videos and screenshots containing personal data private; review before sharing.

## Hosted opt-ins and transaction fixtures [#hosted-opt-ins-and-transaction-fixtures]

For `npm run test:hosted`, the existing script requires a valid configured installation and `HOSTED_TEST_PROJECT_REF` equal to `SUPABASE_PROJECT_REF`. It uses the Management API and verifies the migration ledger before fixture work. This opt-in confirms the intended target; it does not make a production target safe to test.

The SQL suites cover ordinary signup, tenant isolation, account/settings, workspace lifecycle/ownership, notifications, feedback, blog publication and installed integration/billing boundaries. Fixtures insert synthetic records, switch database roles/claims to exercise access, make assertions, and roll back. No local Supabase or Docker is involved.

For [the workspace-notes recipe](/docs/build/first-feature), copy its fixture and target-guarded runner, apply the new migration on the test target, then run `node scripts/test-notes.mjs`. Its denial tests should cover owner A reading/creating only in A, owner B isolation, member create denial and unverified-user denial. Also test immutable ownership/timestamps and replay identity. The paid-create variant needs its own subscription/privileged-path tests after closing direct authenticated inserts.

## Real provider acceptance [#real-provider-acceptance]

Hosted browser suites are skipped unless explicitly opted in. Current switches include `RUN_HOSTED_AUTH`, `RUN_HOSTED_SETTINGS`, `RUN_HOSTED_WORKSPACES`, `RUN_HOSTED_ONBOARDING`, `RUN_HOSTED_NOTIFICATIONS`, `RUN_HOSTED_FEEDBACK`, `RUN_HOSTED_INTEGRATIONS` and `RUN_HOSTED_AI`. Read the chosen suite's setup, recipient restrictions, external effects and cleanup before setting its switch. Do not set every switch just to make a summary look complete.

Some acceptance still requires a person and real accounts: verification-on signup, passwordless/recovery/invitation inboxes, Google consent, Stripe purchase/payment events, deployed MCP client consent/revocation, AI failure/refund and clean module removal. Use test/sandbox modes where supported and explicitly approve real external effects. Record provider, mode, source identifier and outcome without customer payloads or keys.

## Test a new tenant boundary [#test-a-new-tenant-boundary]

For notes, establish two workspaces and at least an owner and ordinary member. Create a note as owner A. List as owner B and assert no A note. Attempt a forged B `org_id` as A and expect denial. Repeat after removing A's membership and with an unverified identity. Check that a direct REST/Data API call cannot bypass a hidden UI control.

For the API/MCP extension, old credentials must receive no new notes capability. Grant only one credential, read the expected workspace, remove the grant and verify the next request fails. For paid creation, an unpaid direct database insert must fail even when a caller bypasses the action. A user-scoped list can remain available according to the documented product policy.

## CI and recording results [#ci-and-recording-results]

This release does not provide a general ready-to-run CI pipeline. You can design one around the local checks, but provider fixtures need isolated secrets/targets and browser setup. Do not copy an untested workflow and describe it as included automation. If adding `.github/workflows`, account for the GitHub token's Workflows permission during initial source upload.

Keep a concise verification record: command, source version, environment/target category, pass/fail/skipped count, artifacts and remaining gap. Preserve failed results and skipped reasons. A new source change affecting a boundary invalidates the relevant prior result; unrelated successful checks do not need endless repetition. Continue with the [production launch checklist](/docs/operations/launch).