QA Quality Gates - Scola LMS

Version: 1.0
Date: January 25, 2026
Status: SSOT - Enforceable Gates

Purpose

Define the quality gates that code MUST pass at different stages of the development lifecycle. Failures at any gate are blockers.

---

Gate Overview

Gate	Trigger	Duration	Failure Action
Local Dev	Before git push	< 5 min	Fix immediately
PR Gate	On Pull Request	< 15 min	PR blocked
Nightly	Scheduled 02:00 UTC	< 60 min	Slack alert + Jira ticket

---

1. Local Dev Gate (Pre-Push)

Purpose

Fast feedback loop for developers before pushing code. Prevents broken code from reaching CI.

Required Checks

1. ✅ Smoke Tests - Core functionality works
2. ✅ Linting - Code style compliance
3. ✅ Type Checking - No TypeScript errors

Commands

Option A: Full Local Gate (Recommended)

cd /home/scola/odoo/scola-fe-v2
make qa-all

This runs: 1. make qa-reset - Restore scola_test from baseline 2. make qa-seed - Seed LMS test data 3. make qa-run - Run Playwright smoke + critical tests 4. npm run lint - ESLint 5. npm run type-check - Validasi seluruh file TypeScript aktif pada konfigurasi Playwright dan suite E2E 6. npm run test:contract:portal - Kontrak JSON-RPC student/parent portal

Option B: Smoke Only (Fast)

cd /home/scola/odoo/scola-fe-v2
npx playwright test tests/e2e/smoke --project=chromium

Option C: Quick Lint + Smoke

npm run lint && npx playwright test tests/e2e/smoke --project=chromium

Option D: Starter FE Gate yang realistis

npm run lint
npm run type-check
npm run test:contract:portal
E2E_ODOO_URL=https://be-dev.gcgscola.id npm run test:e2e:smoke:starter

Pass Criteria

• Smoke tests: 100% pass (0 retries allowed)
• Lint: 0 errors, 0 warnings
• Type check: 0 errors

Expected Duration

• make qa-all: about 5 minutes
• Smoke only: about 2 minutes
• Lint + Smoke: about 3 minutes

---

2. PR Gate (CI Pipeline)

Purpose

Automated verification on every Pull Request to ensure changes do not break critical functionality.

Required Checks

1. Lint + Type Check - kode FE valid dan type-safe
2. Contract Tests - kontrak API/portal tetap stabil
3. Build - produksi dapat dibundel
4. E2E Starter Smoke - manual server gate jika perubahan menyentuh auth, routing, atau flow utama

Current Workflow Policy

Workflow aktif .github/workflows/deploy-unified.yml tidak menjalankan Playwright pada push/PR/deploy default. Job E2E hanya berjalan saat workflow_dispatch dipicu dengan input run_e2e_smoke=true; default CI cukup memberi guidance manual gate di step summary.

GitHub Actions Workflow

Catatan: contoh pr-gate.yml di bawah adalah referensi untuk dedicated QA workflow, bukan jalur default deploy-unified.yml.

# .github/workflows/pr-gate.yml
name: PR Quality Gate

on:
  pull_request:
    branches: [main, develop]

jobs:
  lint-and-build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-node@v3
        with:
          node-version: '20'
      - run: npm ci
      - run: npm run lint
      - run: npm run type-check
      - run: npm run test:contract:portal
      - run: npm run build

  smoke-tests:
    runs-on: ubuntu-latest
    needs: lint-and-build
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-node@v3
      - run: npm ci
      - run: npx playwright install --with-deps chromium
      - run: make qa-reset
      - run: make qa-seed
      - run: npx playwright test tests/e2e/smoke --project=chromium --reporter=html
      - uses: actions/upload-artifact@v3
        if: failure()
        with:
          name: smoke-test-results
          path: playwright-report/

  critical-tests:
    runs-on: ubuntu-latest
    needs: smoke-tests
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-node@v3
      - run: npm ci
      - run: npx playwright install --with-deps chromium
      - run: make qa-reset
      - run: make qa-seed
      - run: npx playwright test tests/e2e/critical --project=chromium --reporter=html
      - uses: actions/upload-artifact@v3
        if: failure()
        with:
          name: critical-test-results
          path: playwright-report/

  rbac-tests:
    runs-on: ubuntu-latest
    needs: smoke-tests
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-node@v3
      - run: npm ci
      - run: npx playwright install --with-deps chromium
      - run: make qa-reset
      - run: make qa-seed
      - run: npx playwright test tests/e2e/rbac --project=chromium --reporter=html
      - uses: actions/upload-artifact@v3
        if: failure()
        with:
          name: rbac-test-results
          path: playwright-report/

Commands (Local Simulation)

Run Full PR Gate Locally

cd /home/scola/odoo/scola-fe-v2

# Step 1: Lint + Build
npm run lint
npm run type-check
npm run test:contract:portal
npm run build

# Step 2: Reset + Seed
make qa-reset
make qa-seed

# Step 3: Run Smoke
npx playwright test tests/e2e/smoke --project=chromium

# Step 4: Run Critical
npx playwright test tests/e2e/critical --project=chromium

# Step 5: Run RBAC
npx playwright test tests/e2e/rbac --project=chromium

Run PR Gate with Tag Strategy

# Smoke + Critical + RBAC in one command
npx playwright test --grep "@smoke|@critical|@rbac" --project=chromium

Run P0 Tests Only

npx playwright test --grep "@p0" --project=chromium

Pass Criteria

• Smoke: 100% pass (0 retries)
• Critical: 100% pass (1 retry allowed)
• RBAC: 100% pass (0 retries)
• Lint: 0 errors
• Build: Success

Expected Duration

• Total: ~15 minutes
• Parallel execution (CI): ~10 minutes

Failure Action

• PR status: ❌ Blocked
• Developer action: Fix and push again
• Merge: Not allowed until green

---

3. Nightly Gate (Full Regression)

Purpose

Comprehensive validation to catch regressions, performance issues, and edge cases. Runs every night.

Required Checks

1. ✅ All Smoke + Critical + RBAC - Core functionality
2. ✅ Regression Tests - Feature stability
3. ✅ Performance Tests - SLA compliance
4. ✅ Integration Tests - Cross-module scenarios

Schedule

Cron: 0 2 * * *  # 02:00 UTC daily

GitHub Actions Workflow

# .github/workflows/nightly.yml
name: Nightly Regression

on:
  schedule:
    - cron: '0 2 * * *'
  workflow_dispatch: # Allow manual trigger

jobs:
  full-regression:
    runs-on: ubuntu-latest
    timeout-minutes: 90
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-node@v3
      - run: npm ci
      - run: npx playwright install --with-deps
      - run: make qa-reset
      - run: make qa-seed

      # Run ALL test suites
      - run: npx playwright test tests/e2e/smoke --project=chromium
      - run: npx playwright test tests/e2e/critical --project=chromium
      - run: npx playwright test tests/e2e/rbac --project=chromium
      - run: npx playwright test tests/e2e/regression --project=chromium
      - run: npx playwright test tests/e2e/performance --project=chromium

      # Upload results
      - uses: actions/upload-artifact@v3
        if: always()
        with:
          name: nightly-test-results
          path: playwright-report/

      # Notify on failure
      - uses: slackapi/slack-github-action@v1
        if: failure()
        with:
          payload: |
            {
              "text": "🚨 Nightly regression FAILED",
              "attachments": [{
                "color": "danger",
                "text": "Check artifacts: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
              }]
            }
        env:
          SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK }}

Commands (Local Simulation)

Run Full Nightly Suite

cd /home/scola/odoo/scola-fe-v2
make qa-reset
make qa-seed

# Run all test categories
npx playwright test tests/e2e/smoke --project=chromium
npx playwright test tests/e2e/critical --project=chromium
npx playwright test tests/e2e/rbac --project=chromium
npx playwright test tests/e2e/regression --project=chromium
npx playwright test tests/e2e/performance --project=chromium

Run with Nightly Tag

npx playwright test --grep "@nightly" --project=chromium

Run Full Suite (All Tests)

npx playwright test --project=chromium

Pass Criteria

• Smoke + Critical + RBAC: 100% pass
• Regression: ≥ 95% pass (known flakes acceptable)
• Performance: ≥ 90% pass (SLA variance acceptable)

Expected Duration

• Total: ~60 minutes
• Parallel workers: 4 (configurable)

Failure Action

1. Slack notification to #qa-alerts channel
2. Jira ticket auto-created with failure details
3. Team review during daily standup
4. Priority: P1 if >5% failure rate

---

Tag Strategy for Gates

Tags Mapping to Gates

Tag	Local Dev	PR Gate	Nightly
@smoke	✅ Required	✅ Required	✅ Included
@critical	⚠️ Optional	✅ Required	✅ Included
@rbac	⚠️ Optional	✅ Required	✅ Included
@p0	✅ Required	✅ Required	✅ Included
@p1	⚠️ Optional	✅ Required	✅ Included
@p2	❌ Skip	❌ Skip	✅ Included
@regression	❌ Skip	❌ Skip	✅ Included
@performance	❌ Skip	❌ Skip	✅ Included
@nightly	❌ Skip	❌ Skip	✅ Included

Tag-Based Execution

Local Dev (Fast)

npx playwright test --grep "@smoke" --project=chromium

PR Gate (Medium)

npx playwright test --grep "@smoke|@critical|@rbac|@p0|@p1" --project=chromium

Nightly (Comprehensive)

npx playwright test --grep "@nightly|@regression|@performance" --project=chromium
# OR
npx playwright test --project=chromium  # Run ALL

---

Makefile Shortcuts

Existing Commands (from Makefile)

qa-backup:
    @scripts/qa/db/backup.sh

qa-reset:
    @scripts/qa/db/restore.sh

qa-seed:
    @E2E_DB_NAME=scola_test \
     E2E_ODOO_CONF=/home/scola/odoo/config/scola-dev.conf \
     /home/scola/odoo/odoo-bin shell -c /home/scola/odoo/config/scola-dev.conf -d scola_test \
     < scripts/qa/seed/seed_lms.py

qa-run:
    @npx playwright test tests/e2e/smoke tests/e2e/critical --project=chromium

qa-all: qa-reset qa-seed qa-run

Recommended Additions

# Add to existing Makefile

qa-smoke:
    @npx playwright test tests/e2e/smoke --project=chromium

qa-critical:
    @npx playwright test tests/e2e/critical --project=chromium

qa-rbac:
    @npx playwright test tests/e2e/rbac --project=chromium

qa-regression:
    @npx playwright test tests/e2e/regression --project=chromium

qa-performance:
    @npx playwright test tests/e2e/performance --project=chromium

qa-pr-gate: qa-reset qa-seed
    @npx playwright test tests/e2e/smoke tests/e2e/critical tests/e2e/rbac --project=chromium

qa-nightly: qa-reset qa-seed
    @npx playwright test --project=chromium

qa-p0:
    @npx playwright test --grep "@p0" --project=chromium

qa-report:
    @npx playwright show-report

Usage

make qa-smoke       # Run smoke tests only
make qa-critical    # Run critical tests only
make qa-rbac        # Run RBAC tests only
make qa-pr-gate     # Simulate PR gate locally
make qa-nightly     # Simulate nightly suite
make qa-p0          # Run all P0 tests
make qa-report      # Open HTML report

---

Troubleshooting Gate Failures

Smoke Test Fails

Likely Cause: Environment not seeded correctly

Solution:

make qa-reset
make qa-seed
make qa-smoke

RBAC Test Fails

Likely Cause: Security regression or incorrect role setup

Action: IMMEDIATE FIX - Security issues are P0 blockers

Debug:

npx playwright test tests/e2e/rbac --debug

Performance Test Fails

Likely Cause: Network variance, cache miss, or real regression

Action: Review performance metrics, re-run 3 times

Debug:

npx playwright test tests/e2e/performance --repeat-each=3

Flaky Test

Definition: Test passes on retry but not first run

Action: 1. Identify flaky test from report 2. Add .skip() temporarily if blocking PR 3. Create Jira ticket with flaky-test label 4. Fix race condition (add proper waits)

Example Fix:

// ❌ Flaky (implicit wait)
await page.click('button')

// ✅ Stable (explicit wait)
await page.waitForSelector('button', { state: 'visible' })
await page.click('button')

---

Gate Exemptions (Emergency Only)

When to Request Exemption

• Production blocker fix needed urgently
• Test infrastructure is down (not code issue)
• Known flake already has Jira ticket

Exemption Process

1. Create Jira ticket explaining reason
2. Get approval from QA Lead + Engineering Manager
3. Add [GATE-EXEMPT: JIRA-123] to PR title
4. Merge with manual override
5. MUST fix within 24 hours

---

Metrics & Monitoring

Key Metrics

• Gate Pass Rate: Target ≥ 95%
• Average Duration: Smoke <2min, Critical <10min, Nightly <60min
• Flaky Test Rate: Target <5%
• RBAC Failure Rate: Target 0% (zero tolerance)

Dashboards

• CI Dashboard: GitHub Actions workflow history
• Test Report: make qa-report (Playwright HTML)
• Slack Alerts: #qa-alerts channel (failures only)

---

Document Owner: QA Architect
Review Cycle: Bi-weekly
Last Updated: January 25, 2026