Plan: Production Smoke Tests with Seed Data

Overview

A Playwright smoke test suite that runs against the live site (https://ai-mee.uk) after deploys. Each run seeds its own ephemeral test user and full data set (client, campaign, posts) via the Supabase service-role key, runs 9 critical page checks, then deletes everything — leaving zero test artifacts in production.

Reuses the tracked-record cleanup pattern from tests/bot-e2e/helpers/supabase.ts.

Data Lifecycle

Setup:    seedAuthUser → seedCustomer → seedCampaign → 3x seedPost → write state JSON → login via real form
Tests:    9 page checks using seeded IDs from state JSON
Teardown: cleanupTestData() → deletes platform_posts → posts → campaign → customer → auth user

Auth is handled via the actual login form (email + password). Playwright's storageState setup-project pattern saves the session once and all tests reuse it.

Steps

Phase 1 — Extend Existing Seed Helpers

File: tests/bot-e2e/helpers/supabase.ts

1. Add seedCampaign(customerId, name, description) — inserts into customer_campaign, tracks the ID for cleanup, exports the function.
2. Add optional campaign_id param to seedPost() so posts can be linked to a campaign.

Phase 2 — Playwright Smoke Config

Create: front-end/playwright.smoke.config.ts

• testDir: './tests/smoke'
• baseURL from SMOKE_BASE_URL env var (default https://ai-mee.uk)
• No webServer block — tests run against the live site
• globalTeardown: './tests/smoke/global-teardown.ts'
• Two Playwright projects:
  • setup — runs auth.setup.ts, no storageState dependency
  • smoke — runs smoke.spec.ts, depends on setup, uses storageState: './tests/smoke/.auth/user.json'
• Chromium only, expect.timeout: 20_000, retries: 2 on CI, screenshot: 'on'

Modify: front-end/package.json

"test:smoke": "playwright test --config playwright.smoke.config.ts"

Phase 3 — Seed Module + Auth Setup

Create: front-end/tests/smoke/seed.ts

Self-contained seed/cleanup module (cannot cross-import from tests/bot-e2e/ due to separate workspace packages — the bot-e2e helpers serve as the reference implementation).

Functions:

• initSeedClient() — creates Supabase admin client using service-role key
• seedAuthUser(email, password) — creates auth user via admin REST API, emails confirmed automatically
• seedCustomer(name, userId) — inserts into customer_customer, resolves user UUID
• seedCampaign(customerId, name) — inserts into customer_campaign
• seedPost(customerId, options) — inserts into customer_posts + customer_platform_post
• cleanupTestData() — deletes all tracked records in reverse FK order, then clears the auth user via admin API
• writeState(state) / readState() — persists seeded IDs to tests/smoke/.smoke-state.json for sharing between setup and spec

Cleanup deletion order (respects FK constraints):

1. customer_platform_post
2. customer_posts
3. customer_campaign
4. customer_customer
5. Auth user (via DELETE /auth/v1/admin/users/:id)

Create: front-end/tests/smoke/auth.setup.ts (Playwright setup project)

1. Fail fast with a clear error if SUPABASE_URL, SUPABASE_SERVICE_ROLE_KEY, or SMOKE_TEST_PASSWORD are missing
2. Generate a unique email: smoke-{Date.now()}@test.invalid
3. Seed data:
   • seedAuthUser(email, password) → userId
   • seedCustomer('Smoke Test Co', userId) → customerId
   • seedCampaign(customerId, 'Smoke Campaign') → campaignId
   • seedPost(customerId, { status: 'pending_review', campaignId }) → postId
   • seedPost(customerId, { status: 'approved' })
   • seedPost(customerId, { status: 'sent' })
4. writeState({ customerId, campaignId, postId })
5. Navigate to /auth, click "Use password instead" (.forgot-link-btn), fill email + password, submit
6. Wait for redirect to /app (confirms login succeeded)
7. page.context().storageState({ path: 'tests/smoke/.auth/user.json' })

Create: front-end/tests/smoke/global-teardown.ts

import { cleanupTestData } from './seed'
import { unlink } from 'node:fs/promises'

export default async function globalTeardown() {
  try {
    await cleanupTestData()
    await unlink('tests/smoke/.smoke-state.json').catch(() => {})
    await unlink('tests/smoke/.auth/user.json').catch(() => {})
  } catch (err) {
    console.error('[smoke teardown] cleanup error (non-fatal):', err)
  }
}

Phase 4 — Smoke Test Spec

Create: front-end/tests/smoke/smoke.spec.ts

Uses test.describe.serial() — tests run in order and share the saved auth state.

Reads .smoke-state.json in test.beforeAll() to get seeded customerId, campaignId, postId.

Every test also asserts: no Vue error overlay, no 404 catch-all page ("Page not found" text absent).

Phase 5 — GitHub Actions Workflow

Create: .github/workflows/smoke.yml

Triggers:

• deployment_status — fires automatically when Cloudflare Pages GitHub app reports a successful deploy
• workflow_dispatch — manual trigger with an optional base_url input for staging
• schedule: '0 */6 * * *' — fallback cron every 6 hours (catches manual wrangler deploys)

Condition: skip if deployment_status event and .state != 'success'.

Job steps (ubuntu-latest):

1. Checkout
2. pnpm 10.10.0 + Node 20 (cache on front-end/pnpm-lock.yaml)
3. cd front-end && pnpm install --frozen-lockfile
4. pnpm exec playwright install chromium --with-deps
5. Health check: curl --retry 10 --retry-delay 5 -sf $SMOKE_BASE_URL
6. pnpm test:smoke
7. Upload playwright-report/ artifact on failure (7-day retention)

Required GitHub secrets:

Phase 6 — Gitignore

Modify: front-end/.gitignore

tests/smoke/.auth/
tests/smoke/.smoke-state.json

Files Summary

Verification Steps

1. Local dry-run: Set env vars and run pnpm test:smoke against production. All 9 tests pass. Query customer_customer with service-role key afterwards — zero smoke records should remain.
2. Cleanup resilience: Kill the test mid-run (Ctrl+C), re-run — unique timestamp email avoids collision; globalTeardown handles cleanup.
3. Manual CI trigger: Fire workflow_dispatch from GitHub Actions UI — full pipeline runs end-to-end.
4. Intentional failure: Break one assertion → verify workflow fails, report artifact uploads, cleanup still runs.

Decisions

Ephemeral user per run, not a shared test account.
No persistent test user to manage. Timestamp in email (smoke-{Date.now()}@test.invalid) prevents collisions between concurrent runs. Full isolation between runs.

Self-contained seed module in front-end/tests/smoke/seed.ts.
Cannot cross-import from tests/bot-e2e/helpers/supabase.ts due to separate workspace packages. The bot-e2e helper is the reference implementation; the smoke module mirrors the same tracked-records + reverse-FK-deletion pattern.

globalTeardown for cleanup, not afterAll.
Playwright's globalTeardown runs even if tests crash or are cancelled. The cleanupTestData() function deletes records in reverse FK insertion order: customer_platform_post → customer_posts → customer_campaign → customer_customer → auth user.

Real login via the actual form.
Tests the auth flow itself (not just page rendering). Playwright's storageState setup-project pattern is the official recommended approach. .forgot-link-btn toggles email+password mode on /auth.

Service-role key as a CI secret.
Required to bypass RLS for seeding and to delete the auth user at teardown. Same pattern used by the bot-e2e test suite. The key is never logged.

Further Considerations

Cloudflare Pages trigger:
If CF Pages isn't connected via the GitHub app (manual wrangler deploy), deployment_status events won't fire. The 6-hour cron catches drift. Alternatively, append the following to your local deploy script:

gh workflow run smoke.yml --ref master

Orphaned data safety net (future enhancement):
If teardown fails (network error), orphaned smoke data accumulates in production. A future sweeper could delete customer_customer rows where:

email LIKE 'smoke-%@test.invalid' AND created_at < now() - interval '1 hour'

Not needed for v1 — unique timestamps mean orphans don't interfere with future runs.