QA Test Contract Matrix - Scola LMS¶

Version: 1.0
Date: January 25, 2026
Status: SSOT - Single Source of Truth

Purpose¶

This matrix establishes the enforceable contract between LMS specifications and test coverage. Every requirement MUST have corresponding test cases with clear evidence artifacts.

Matrix Legend¶

Column	Description
Spec Ref	Reference to specification document and section
Requirement	Concrete requirement from spec
Risk	Business/Security risk (High/Med/Low)
Test Level	Unit / Integration / E2E
Test Type	Smoke / Critical / RBAC / Regression / Performance
Playwright Spec	Target test file
Data Precondition	Seed data requirements
Pass Criteria	Assertions that MUST pass
Evidence	Report/Trace/Screenshot artifact
Gate	P0 (Blocker) / P1 (Critical) / P2 (Important)

1. Course SSOT & Visibility¶

Spec Ref	Requirement	Risk	Test Level	Test Type	Playwright Spec	Data Precondition	Pass Criteria	Evidence	Gate
lms-rbac.md §3.1	Student sees only enrolled courses	High	E2E	RBAC	`rbac/rbac_no_leak.spec.ts`	2 students, 2 courses (student1 in course1 only)	`student1` sees course1, NOT course2	Trace + Screenshot	P0
lms-rbac.md §3.1	Faculty sees only assigned courses	High	E2E	RBAC	`rbac/rbac_no_leak.spec.ts`	2 teachers, 2 courses (teacher1 in course1 only)	`teacher1` sees course1, NOT course2	Trace + Screenshot	P0
lms-rbac.md §3.1	Non-member cannot view course detail	High	E2E	RBAC	`rbac/rbac_no_leak.spec.ts`	Course with students, attempt access from non-member	403/404 or redirect to home	Trace	P0
lms-performance.md §2	Course list loads <200ms (cached)	Med	E2E	Performance	`performance/lms_performance.spec.ts`	10 courses seeded	`await page.waitForLoadState('networkidle')` < 200ms	Performance report	P2
lms-audit-and-active-submission.md §1	Course item shows active submission badge	Low	E2E	Regression	`regression/lms_ui_regression.spec.ts`	Course with 1 assignment + 1 active submission	Badge visible with text "Active"	Screenshot	P2

2. Assignment + Submission Versioning¶

Spec Ref	Requirement	Risk	Test Level	Test Type	Playwright Spec	Data Precondition	Pass Criteria	Evidence	Gate
lms-assignment-submission-flow.md §2.1	Student can submit assignment v1	High	E2E	Critical	`critical/grade_publish_flow.spec.ts`	Assignment published, student enrolled	Submission created, file uploaded	Trace + Screenshot	P0
lms-assignment-submission-flow.md §2.2	Student can resubmit (v2 supersedes v1)	High	E2E	Critical	`critical/grade_publish_flow.spec.ts`	Existing v1 submission	v2 created, v1 marked superseded	Trace	P0
lms-assignment-submission-flow.md §2.3	Only latest submission is gradeable	High	E2E	Critical	`critical/grade_publish_flow.spec.ts`	v1 + v2 exist	Teacher grades v2, v1 ignored	Trace	P0
lms-rbac.md §3.2	Student cannot view other student submissions	High	E2E	RBAC	`rbac/rbac_no_leak.spec.ts`	2 students, 2 submissions	student1 API call for student2 submission = 0 records	API Response	P0
lms-audit-and-active-submission.md §2	Submission shows version history	Med	E2E	Regression	`regression/lms_ui_regression.spec.ts`	v1 + v2 + v3	History shows 3 versions with timestamps	Screenshot	P2

3. Grade Publish Idempotency + Audit¶

Spec Ref	Requirement	Risk	Test Level	Test Type	Playwright Spec	Data Precondition	Pass Criteria	Evidence	Gate
lms-grade-publish.md §4.1	First publish creates grade + event	High	Integration	Critical	`critical/grade_publish_idempotency.spec.ts`	Submission graded (80), no grade exists	`POST /publish` → 200, grade=80, event created	API Response	P0
lms-grade-publish.md §4.2	Same key + same payload = idempotent (200)	High	Integration	Critical	`critical/grade_publish_idempotency.spec.ts`	Event with key='A' exists (score=80)	`POST /publish` (key='A', score=80) → 200, no new event	API Response	P0
lms-grade-publish.md §4.3	Same key + diff payload = conflict (409)	High	Integration	Critical	`critical/grade_publish_idempotency.spec.ts`	Event with key='A' exists (score=80)	`POST /publish` (key='A', score=90) → 409	API Response	P0
lms-grade-publish.md §4.4	Update without reason = rejected (409)	High	Integration	Critical	`critical/grade_publish_idempotency.spec.ts`	Grade=80 exists	`POST /publish` (key='B', score=90, no reason) → 409	API Response	P0
lms-grade-publish.md §4.4	Update with reason = success + audit	High	Integration	Critical	`critical/grade_publish_idempotency.spec.ts`	Grade=80 exists	`POST /publish` (key='B', score=90, reason="Typo") → 200, event with reason	API Response + DB Query	P0
lms-audit-and-active-submission.md §3	Grade event audit trail queryable	Med	E2E	Regression	`regression/lms_grade_audit.spec.ts`	3 grade events (publish, update, update)	Query returns 3 events, ordered by timestamp	API Response	P1

4. Gradebook Update¶

Spec Ref	Requirement	Risk	Test Level	Test Type	Playwright Spec	Data Precondition	Pass Criteria	Evidence	Gate
lms-gradebook.md §2	Teacher sees gradebook for assigned course	High	E2E	Smoke	`smoke/lms_smoke.spec.ts`	Teacher with 1 course, 5 students	Gradebook loads, shows 5 rows	Screenshot	P0
lms-gradebook.md §3	Published grade appears in gradebook	High	E2E	Critical	`critical/grade_publish_flow.spec.ts`	Grade published (score=85)	Gradebook cell shows 85	Screenshot	P0
lms-gradebook.md §4	Gradebook updates after republish	High	E2E	Critical	`critical/grade_publish_flow.spec.ts`	Grade=80, then updated to 90	Gradebook cell shows 90 (not 80)	Screenshot	P0
lms-performance.md §3	Gradebook query cached (backend)	Med	Integration	Performance	`performance/lms_performance.spec.ts`	100 students, 10 assignments	Second query <50ms (cache hit)	API timing	P2

5. Notifications¶

Spec Ref	Requirement	Risk	Test Level	Test Type	Playwright Spec	Data Precondition	Pass Criteria	Evidence	Gate
lms-notifications-calendar.md §2.1	Student receives notification on grade publish	Med	E2E	Critical	`critical/grade_publish_flow.spec.ts`	Grade published	Notification exists in student inbox	Screenshot	P1
lms-notifications-calendar.md §2.2	Notification contains grade value	Low	E2E	Regression	`regression/lms_notifications.spec.ts`	Grade=85 published	Notification text includes "85"	API/UI	P2
lms-notifications-calendar.md §3	Assignment due date in calendar	Med	E2E	Regression	`regression/lms_calendar.spec.ts`	Assignment with due_date=2026-02-01	Calendar shows event on Feb 1	Screenshot	P2

6. RBAC Isolation (Security Gates - Zero Tolerance)¶

Spec Ref	Requirement	Risk	Test Level	Test Type	Playwright Spec	Data Precondition	Pass Criteria	Evidence	Gate
lms-rbac.md §3.2	Student cannot create course content	High	E2E	RBAC	`rbac/rbac_no_leak.spec.ts`	Student login	`POST /api/lms/item` → 403 or Access Error	API Response	P0
lms-rbac.md §3.2	Student cannot access gradebook route	High	E2E	RBAC	`rbac/rbac_no_leak.spec.ts`	Student login	Navigate to `/faculty/gradebook` → redirect or 403	URL check	P0
lms-rbac.md §3.3	Faculty cannot publish grades for other courses	High	E2E	RBAC	`rbac/rbac_no_leak.spec.ts`	Teacher1 (course1), attempt publish for course2	`POST /publish` → 403 or 0 records	API Response	P0
lms-rbac.md §3.4	Parent is read-only (cannot publish)	High	E2E	RBAC	`rbac/rbac_no_leak.spec.ts`	Parent login	Publish button hidden OR `POST /publish` → 403	UI + API	P0

7. Performance Smoke (Non-Blocker)¶

Spec Ref	Requirement	Risk	Test Level	Test Type	Playwright Spec	Data Precondition	Pass Criteria	Evidence	Gate
lms-performance.md §1	Course list initial load <2s	Low	E2E	Performance	`performance/lms_performance.spec.ts`	50 courses	`page.goto()` → DOMContentLoaded <2s	Performance timing	P2
lms-performance.md §4	Gradebook fetch <500ms (10 students)	Low	Integration	Performance	`performance/lms_performance.spec.ts`	10 students, 5 assignments	API `/gradebook` response time <500ms	API timing	P2
lms-performance.md §5	Submission upload <5s (1MB file)	Low	E2E	Performance	`performance/lms_upload.spec.ts`	1MB test file	Upload completes <5s	Network timing	P2

Coverage Summary¶

Gate	Total Requirements	Spec Files	Status
P0	19	3 (smoke, critical, rbac)	🟡 Skeleton Ready
P1	2	1 (critical)	🟡 Skeleton Ready
P2	10	3 (regression, performance)	🔴 Not Started
Total	31	7	Phase 1: P0 Focus

Next Steps¶

Phase 1 (Current Sprint)¶

✅ Smoke: lms_smoke.spec.ts
✅ Critical: grade_publish_flow.spec.ts, grade_publish_idempotency.spec.ts
✅ RBAC: rbac_no_leak.spec.ts

Phase 2 (Next Sprint)¶

Regression: Notification, Calendar, UI components
Performance: Gradebook caching, Upload timing

Phase 3 (Future)¶

Regression: Full version history, audit trail queries
Integration: Batch grade import, export

Evidence Artifact Standards¶

Evidence Type	Storage	Retention
Playwright Trace	`playwright-report/`	7 days (overwrite)
Screenshots	`playwright-report/`	7 days
Videos	`playwright-report/`	Fail-only, 7 days
API Responses	Test output logs	30 days
Performance Metrics	CI artifact JSON	90 days

SPMB Test Schedule & Results API Contract¶

`POST /api/SPMB/my/test-schedules`¶

Request: { admission_id: number }

Response shape (per item in data[]):

Field	Type	Required	Notes
`slot_id`	number	✅
`schedule_id`	number	✅
`test_type.id`	number	✅
`test_type.name`	string	✅
`test_type.code`	string	✅
`test_type.duration_minutes`	number	✅
`test_type.max_score`	number	✅
`test_date`	string\|null	✅	ISO date
`test_time_start`	string	✅	HH:mm
`test_time_end`	string	✅	HH:mm
`location`	string	✅	May be empty
`state`	enum	✅	draft\|open\|closed\|done\|cancel
`attendance_status`	enum	✅	pending\|present\|absent\|late
`attendance_time`	string\|null	✅	ISO datetime
`score`	number	✅
`notes`	string	✅
`cbt_exam_id`	number\|null	✅
`cbt_exam_name`	string	✅
`cbt_exam_state`	string\|null	✅
`cbt_start_time`	string\|null	✅	ISO datetime
`cbt_end_time`	string\|null	✅	ISO datetime
`cbt_launch.enabled`	boolean	✅
`cbt_launch.exam_id`	number\|null	✅
`cbt_launch.exam_name`	string	✅
`cbt_launch.return_to`	string	✅
`cbt_launch.reason`	string	✅	Empty when enabled=true

`POST /api/SPMB/my/test-results`¶