QA Suite Structure - Scola LMS¶

Version: 1.1
Date: January 26, 2026
Status: SSOT - Canonical Structure

⚡ Quick Reference: quick-reference.md (1-page cheat sheet for daily work)

Purpose¶

Define the canonical test suite organization, naming conventions, tagging strategy, and execution policies for Scola LMS E2E tests.

Directory Structure¶

tests/e2e/
├── flow/            # ⚡ P0: Critical end-to-end flows (2-5 min) - NEW!
├── smoke/           # P0: Fast, stable, critical path (< 2 min total)
├── critical/        # P0-P1: Business-critical workflows (< 10 min total)
├── rbac/            # P0: Security & access control (< 5 min total)
├── regression/      # P2: Feature regression (< 30 min total)
├── performance/     # P2: Performance benchmarks (< 15 min total)
├── fixtures/        # Shared fixtures (auth, data, helpers)
└── support/         # Shared utilities (selectors, API clients)

0. Flow Tests (`tests/e2e/flow/`) ⚡ NEW in Phase 2.1¶

Purpose¶

Critical end-to-end user journeys that validate complete workflows across multiple roles. These are the HIGHEST priority tests - if they fail, deployment is blocked.

Characteristics¶

Duration: 2-5 minutes total
Scope: Complete user journeys spanning multiple roles
Retries: 0 (must be 100% stable - failures indicate real issues)
Parallelization: Yes (tests use API-based auth, fully isolated)
Priority: P0 (blocks deployment)

Naming Convention¶

<module>_<role1>_<role2>_<role3>_flow.spec.ts

Examples:
- lms_student_teacher_student_flow.spec.ts
- lms_teacher_publish_idempotent_ui.spec.ts
- enrollment_admin_student_teacher_flow.spec.ts

Content Guidelines¶

Each flow test should: - Cover complete user journey (e.g., student submits → teacher grades → student sees grade) - Use API-based authentication (session storageState, NOT UI login) - Be resilient to existing data (check API state, not UI badges) - Include version tracking for stateful operations - Validate idempotency where applicable

Example Flow:

test.describe('LMS P0 Flow: Student → Teacher → Student @p0', () => {
  test('student submits assignment → teacher grades → student sees published grade', async ({ studentPage, teacherPage }) => {
    // Student: Submit assignment
    await studentPage.goto('/assignment/1')
    await studentPage.getByTestId('submit-button').click()
    const version = await getLatestSubmissionVersion(studentPage, 1)
    expect(version).toBeGreaterThan(0)

    // Teacher: Grade submission
    await teacherPage.goto('/faculty/lms/gradebook/1')
    await teacherPage.getByTestId('grade-input').fill('85')
    await teacherPage.getByTestId('save-grade').click()

    // Teacher: Publish grades
    await teacherPage.getByTestId('publish-grades').click()
    await expect(teacherPage.getByText('Successfully published')).toBeVisible()

    // Student: Verify published grade visible
    await studentPage.goto('/grades')
    await expect(studentPage.getByText('85')).toBeVisible()
  })
})

Tags¶

test.describe('LMS P0 Flow Tests @p0 @flow @lms', () => {
  // tests here
})

Fast Path Execution¶

# Run only P0 flow tests (2-5 minutes)
npx playwright test tests/e2e/flow --grep "@p0"

# Expected output: ✅ 11 passed

Known Issues & Solutions¶

See testing-guidelines.md section "Known Gotchas & Battle-Tested Solutions" for: - Assignment type field naming hell - Submission version not incrementing - Publish grade average always zero - Idempotency key format mismatch - Submission modal not visible - File upload rejecting .txt fixtures - Session auth vs UI login flakiness

1. Smoke Tests (`tests/e2e/smoke/`)¶

Purpose¶

Fast sanity checks to verify core system is operational. Run on every commit.

Characteristics¶

Duration: < 2 minutes total
Scope: Minimal happy path per module
Retries: 0 (must be 100% stable)
Parallelization: Yes (tests are isolated)

Naming Convention¶

<module>_smoke.spec.ts

Examples:
- lms_smoke.spec.ts
- auth_smoke.spec.ts
- navigation_smoke.spec.ts

Content Guidelines¶

Each smoke test should: - Login → Navigate → Verify 1 critical element visible - NO data creation (use seeded baseline) - NO complex interactions (single page validation)

Example Flow:

test('LMS smoke: teacher sees course list', async ({ page }) => {
  await page.goto('/faculty/lms')
  await expect(page.getByTestId('lms-course-card')).toBeVisible()
})

Tags¶

test.describe('LMS Smoke Tests @smoke @lms', () => {
  // tests here
})

2. Critical Tests (`tests/e2e/critical/`)¶

Purpose¶

Business-critical workflows that MUST work for system to be functional. Run before PR merge.

Characteristics¶

Duration: < 10 minutes total
Scope: End-to-end user journeys
Retries: 1 (allow flaky network recovery)
Parallelization: Yes (per test file)

Naming Convention¶

<feature>_<workflow>_flow.spec.ts

Examples:
- grade_publish_flow.spec.ts
- assignment_submission_flow.spec.ts
- student_enrollment_flow.spec.ts

Content Guidelines¶

Each critical test should: - Cover complete user journey (multi-step) - Verify data persistence across pages - Include assertions on notifications/side effects

Example Flow:

test('End-to-end: teacher grades and publishes', async ({ page, request }) => {
  // Step 1: Teacher creates assignment
  // Step 2: Student submits work
  // Step 3: Teacher grades submission
  // Step 4: Teacher publishes grade
  // Step 5: Student sees grade in gradebook
  // Step 6: Student receives notification
})

Tags¶

test.describe('Grade Publish Critical Tests @critical @lms @grade', () => {
  // tests here
})

3. RBAC Tests (`tests/e2e/rbac/`)¶

Purpose¶

Security gates to verify access control and data isolation. Zero-tolerance failures.

Characteristics¶

Duration: < 5 minutes total
Scope: Negative tests (unauthorized access)
Retries: 0 (security issues are blockers)
Parallelization: Yes (independent checks)

Naming Convention¶

rbac_<module>_<scenario>.spec.ts

Examples:
- rbac_no_leak.spec.ts
- rbac_student_isolation.spec.ts
- rbac_faculty_boundaries.spec.ts

Content Guidelines¶

Each RBAC test should: - Use multiple user contexts (storageState) - Verify DENY scenarios (403, redirect, hidden UI) - Test both UI and API layers

Example Flow:

test('Student cannot access gradebook route', async ({ page }) => {
  // Login as student
  await page.goto('/faculty/gradebook')
  // Expect: redirect to /student/home OR 403 page
  await expect(page).toHaveURL('/student/home')
})

test('Student cannot call grade publish API', async ({ request }) => {
  const response = await request.post('/api/grade/publish', { data: {...} })
  expect(response.status()).toBe(403)
})

Tags¶

test.describe('RBAC No Leak Tests @rbac @security @p0', () => {
  // tests here
})

4. Regression Tests (`tests/e2e/regression/`)¶

Purpose¶

Feature stability to catch regressions in existing functionality. Run nightly.

Characteristics¶

Duration: < 30 minutes total
Scope: Edge cases, UI details, integrations
Retries: 2 (allow intermittent failures)
Parallelization: Yes (by feature area)

Naming Convention¶

<module>_<feature>_regression.spec.ts

Examples:
- lms_ui_regression.spec.ts
- lms_notifications_regression.spec.ts
- lms_calendar_regression.spec.ts

Content Guidelines¶

Each regression test should: - Test non-critical but important features - Cover UI details (badges, tooltips, formatting) - Verify integrations (notifications, calendar sync)

Example Flow:

test('Assignment shows active submission badge', async ({ page }) => {
  await page.goto('/student/lms/course/1')
  const badge = page.getByTestId('active-submission-badge')
  await expect(badge).toBeVisible()
  await expect(badge).toHaveText('Active')
})

Tags¶

test.describe('LMS UI Regression @regression @lms @ui', () => {
  // tests here
})

5. Performance Tests (`tests/e2e/performance/`)¶

Purpose¶

Performance benchmarks to catch degradation. Run nightly or pre-release.

Characteristics¶

Duration: < 15 minutes total
Scope: Load times, API response times, caching
Retries: 3 (network variance acceptable)
Parallelization: No (avoid resource contention)

Naming Convention¶

<module>_performance.spec.ts

Examples:
- lms_performance.spec.ts
- gradebook_performance.spec.ts
- upload_performance.spec.ts

Content Guidelines¶

Each performance test should: - Measure specific metrics (ms, MB, queries) - Use consistent test data (seeded baseline) - Assert against defined SLAs

Example Flow:

test('Gradebook loads in < 500ms (10 students)', async ({ page }) => {
  const startTime = Date.now()
  await page.goto('/faculty/gradebook/1')
  await page.waitForLoadState('networkidle')
  const loadTime = Date.now() - startTime
  expect(loadTime).toBeLessThan(500)
})

Tags¶

test.describe('LMS Performance Tests @performance @lms @nightly', () => {
  // tests here
})

Shared Resources¶

Fixtures (`tests/e2e/fixtures/`)¶

Auth Fixtures¶

auth.fixture.ts
- teacherAuth()  → storageState for teacher1
- studentAuth()  → storageState for student1
- parentAuth()   → storageState for parent1

Data Fixtures¶

data.fixture.ts
- getCourse()    → seeded course data
- getAssignment() → seeded assignment
- getSubmission() → seeded submission

Support Utilities (`tests/e2e/support/`)¶

Selectors¶

selectors.ts
- LMS_COURSE_CARD = '[data-testid="lms-course-card"]'
- GRADEBOOK_CELL = '[data-testid="gradebook-cell"]'

API Helpers¶

api.ts
- publishGrade(request, payload) → Promise<Response>
- createSubmission(request, file) → Promise<Response>

Tagging Strategy¶

Primary Tags¶

Tag	Purpose	Run Frequency
`@smoke`	Fast sanity	Every commit
`@critical`	Business gates	Pre-PR merge
`@rbac`	Security gates	Pre-PR merge
`@regression`	Feature stability	Nightly
`@performance`	Perf benchmarks	Nightly

Secondary Tags¶

Tag	Purpose
`@lms`	LMS module tests
`@grade`	Grade-related tests
`@submission`	Submission-related
`@security`	Security-critical
`@p0`	Priority 0 (blocker)
`@p1`	Priority 1 (critical)
`@p2`	Priority 2 (important)
`@nightly`	Run in nightly suite

Usage Example¶

test.describe('Grade Publish Flow @critical @lms @grade @p0', () => {
  test('Teacher publishes grade successfully @smoke', async ({ page }) => {
    // ...
  })

  test('Idempotency prevents duplicate grades @security', async ({ request }) => {
    // ...
  })
})

Retry Policy¶

Configuration (playwright.config.ts)¶

export default defineConfig({
  projects: [
    {
      name: 'smoke',
      testMatch: '**/smoke/**/*.spec.ts',
      retries: 0, // Smoke must be 100% stable
    },
    {
      name: 'critical',
      testMatch: '**/critical/**/*.spec.ts',
      retries: 1, // Allow 1 retry for network flakes
    },
    {
      name: 'rbac',
      testMatch: '**/rbac/**/*.spec.ts',
      retries: 0, // Security gates are strict
    },
    {
      name: 'regression',
      testMatch: '**/regression/**/*.spec.ts',
      retries: 2, // Allow 2 retries for UI flakes
    },
    {
      name: 'performance',
      testMatch: '**/performance/**/*.spec.ts',
      retries: 3, // Network variance acceptable
    },
  ],
})

Execution Examples¶

Run Smoke Tests Only¶

npx playwright test tests/e2e/smoke --project=chromium

Run Critical + RBAC (Pre-PR)¶

npx playwright test tests/e2e/critical tests/e2e/rbac --project=chromium

Run by Tag¶

npx playwright test --grep "@smoke"
npx playwright test --grep "@p0"
npx playwright test --grep "@lms.*@grade"

Run Nightly Suite¶

npx playwright test tests/e2e/regression tests/e2e/performance --project=chromium

Run Full Suite¶

make qa-all
# OR
npx playwright test

Naming Conventions Summary¶

Type	Pattern	Example
Smoke	`<module>_smoke.spec.ts`	`lms_smoke.spec.ts`
Critical	`<feature>_<workflow>_flow.spec.ts`	`grade_publish_flow.spec.ts`
RBAC	`rbac_<module>_<scenario>.spec.ts`	`rbac_no_leak.spec.ts`
Regression	`<module>_<feature>_regression.spec.ts`	`lms_ui_regression.spec.ts`
Performance	`<module>_performance.spec.ts`	`lms_performance.spec.ts`
Fixture	`<purpose>.fixture.ts`	`auth.fixture.ts`
Support	`<purpose>.ts`	`selectors.ts`, `api.ts`

CI Integration¶

GitHub Actions Example¶

name: E2E Tests

on: [pull_request]

jobs:
  smoke:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - run: make qa-reset
      - run: make qa-seed
      - run: npx playwright test tests/e2e/smoke --project=chromium

  critical:
    runs-on: ubuntu-latest
    needs: smoke
    steps:
      - uses: actions/checkout@v3
      - run: make qa-reset
      - run: make qa-seed
      - run: npx playwright test tests/e2e/critical tests/e2e/rbac --project=chromium

Document Owner: QA Architect
Review Cycle: Monthly
Last Updated: January 25, 2026

QA Suite Structure - Scola LMS¶

Purpose¶

Directory Structure¶

0. Flow Tests (tests/e2e/flow/) ⚡ NEW in Phase 2.1¶

Purpose¶

Characteristics¶

Naming Convention¶

Content Guidelines¶

Tags¶

Fast Path Execution¶

Known Issues & Solutions¶

1. Smoke Tests (tests/e2e/smoke/)¶

Purpose¶

Characteristics¶

Naming Convention¶

Content Guidelines¶

Tags¶

2. Critical Tests (tests/e2e/critical/)¶

Purpose¶

Characteristics¶

Naming Convention¶

Content Guidelines¶

Tags¶

3. RBAC Tests (tests/e2e/rbac/)¶

Purpose¶

Characteristics¶

Naming Convention¶

Content Guidelines¶

Tags¶

4. Regression Tests (tests/e2e/regression/)¶

Purpose¶

Characteristics¶

Naming Convention¶

Content Guidelines¶

Tags¶

5. Performance Tests (tests/e2e/performance/)¶

Purpose¶

Characteristics¶

Naming Convention¶

Content Guidelines¶

Tags¶

Shared Resources¶

Fixtures (tests/e2e/fixtures/)¶

Auth Fixtures¶

Data Fixtures¶

Support Utilities (tests/e2e/support/)¶

Selectors¶

API Helpers¶

Tagging Strategy¶

Primary Tags¶

Secondary Tags¶

Usage Example¶

Retry Policy¶

Configuration (playwright.config.ts)¶

Execution Examples¶

Run Smoke Tests Only¶

Run Critical + RBAC (Pre-PR)¶

Run by Tag¶

Run Nightly Suite¶

Run Full Suite¶

Naming Conventions Summary¶

CI Integration¶

GitHub Actions Example¶

0. Flow Tests (`tests/e2e/flow/`) ⚡ NEW in Phase 2.1¶

1. Smoke Tests (`tests/e2e/smoke/`)¶

2. Critical Tests (`tests/e2e/critical/`)¶

3. RBAC Tests (`tests/e2e/rbac/`)¶

4. Regression Tests (`tests/e2e/regression/`)¶

5. Performance Tests (`tests/e2e/performance/`)¶

Fixtures (`tests/e2e/fixtures/`)¶

Support Utilities (`tests/e2e/support/`)¶