# Study Buddy beta test plan

Environment: https://beta.skillpixel.vn. Desktop 1440×900, light mode.

Use only the isolated QA course. Do not change global tenant credit, credentials, limits or existing customer lessons. No SQL. Cases needing a separate tenant stay not run until that fixture exists. Every pass requires dated evidence; merged code and local tests do not prove beta behavior.

## Release gate

All P0 cases must pass, or have an explicit owner and an accepted exception. A passing short-video smoke test is not full release acceptance. Keep billing reconciliation, independent quality grading and lecture-video coverage visible.

## Shot plan

1. Instructor off and empty. 2. Generation pending. 3. Machine transcript ready. 4. Instructor upload ready. 5. Learner panel ready. 6. Streaming answer. 7. Final answer and citation. 8. Player at cited timestamp. 9. Disabled panel with playable video. 10. Language mismatch.

| ID | Priority | Area | Steps | Expected result |
| --- | --- | --- | --- | --- |
| REL-01 | P0 | Release | Confirm merge and active beta deployment | PR 602 is merged; deployed frontend commit contains it. |
| CFG-01 | P0 | Instructor | Open a ready video lesson | Study Buddy card shows off, no transcript, and an upload control. |
| GEN-01 | P0 | Transcription | Turn on a ready video without a transcript | Pending state appears; exactly one job is requested. |
| GEN-02 | P0 | Transcription | Wait for real Gemini transcription | Ready state shows machine origin, language and nonempty segments. |
| GEN-03 | P0 | Transcription | Enable again with a saved transcript | Existing transcript is reused; no new generation is pending. |
| UP-01 | P0 | Transcript upload | Upload a valid SRT | Instructor origin replaces machine origin; timings and cue count match. |
| UP-02 | P1 | Transcript upload | Upload a valid VTT | VTT is parsed; instructor origin and cue count are correct. |
| UP-03 | P0 | Transcript upload | Upload reversed cue timing | 400 with a stable error; previous transcript stays available. |
| UP-04 | P1 | Transcript upload | Upload a file over 2 MB | 413; previous transcript stays available. |
| UP-05 | P1 | Transcript upload | Upload text with an unsupported extension | 400; previous transcript stays available. |
| UP-06 | P1 | Transcript upload | Upload a cue after the video ends | 400; previous transcript stays available. |
| VIS-01 | P0 | Learner | Open the lesson while enabled and ready | Enrolled learner sees attribution, question input and suggestions. |
| VIS-02 | P0 | Learner | Turn off, then reload as learner | Panel is hidden; video still plays; saved transcript remains. |
| VIS-03 | P0 | Learner | Turn on again, then reload | Panel returns without regeneration. |
| ANS-01 | P0 | Answers | Ask for the three review intervals | Answer includes one, three and seven days and a valid source. |
| ANS-02 | P0 | Answers | Ask a follow-up about active recall | Answer uses earlier context and the correct lesson evidence. |
| ANS-03 | P0 | Answers | Ask an unrelated question | Answer refuses or clearly says the lesson does not support it. |
| STR-01 | P0 | Streaming | Observe the real beta reply stream | Ordered version-1 start/status/text/final events arrive; text precedes final. |
| STR-02 | P0 | Streaming | Observe learner answer during delivery | Provisional text is visible before final; final answer enables source controls. |
| CIT-01 | P0 | Citations | Click a timestamp in a validated answer | Bunny player seeks to the cited time; evidence supports the answer. |
| CAN-01 | P1 | Recovery | Leave the lesson during a reply | Request cancels; no stale answer appears in the next lesson. |
| REC-01 | P1 | Recovery | Interrupt the browser network during a reply | Clear retry state; retry uses the original question; no automatic paid resend. |
| SEC-01 | P0 | Access | Read config without authentication | 401 before any sensitive config is returned. |
| SEC-02 | P0 | Access | Read and change config as a student | 403; student cannot manage transcript or toggle. |
| SEC-03 | P0 | Access | Ask as a student without course enrolment | 403 before stream headers and provider work. |
| SEC-04 | P0 | Access | Use the QA resource under another tenant host | Request fails and returns no QA lesson or transcript. |
| SEC-05 | P1 | Access | Request a nonexistent resource | 404 without leaking another resource. |
| VAL-01 | P1 | Validation | Send an empty question | 422 before a provider call. |
| SRC-01 | P1 | Source priority | Try JSON replacement after instructor upload | 409; instructor transcript remains the source. |
| CAP-01 | P1 | Captions | Inspect Bunny captions after saving transcript | Caption track is attached in transcript language and selectable. |
| LANG-01 | P1 | Language | Change course language after an upload | Existing transcript stays unchanged; instructor sees a mismatch warning. |
| LEC-01 | P0 | Lecture video | Create a fresh lecture-video lesson in the QA course | Narration transcript is saved automatically with lecture_script origin. |
| ERR-01 | P1 | Failure states | Use a QA-only failed transcription fixture, then retry | Readable error; Retry succeeds; video remains playable. |
| QUO-01 | P0 | Quotas | Exhaust credit on a separate QA tenant | 429 AI_ALLOWANCE_USED before stream headers; no provider work. |
| KEY-01 | P1 | Provider modes | Use a separate own-key tenant without Gemini | Upload remains available; automatic transcription explains the missing key. |
| RATE-01 | P1 | Quotas | Reach the configured request rate on a separate QA tenant | 429; no second billed request is started. |
| RACE-01 | P1 | Concurrency | Upload while a QA transcription is pending | Instructor upload wins; late machine result cannot overwrite it. |
| PERF-01 | P1 | Performance | Measure first event, first text and final completion | Record actual timings; do not claim the deferred five-second target. |
| BILL-01 | P1 | Accounting | Compare QA operation usage with ledger and provider invoice | Call count, token usage and charged amount reconcile. |
| QUAL-01 | P1 | Quality | Grade Vietnamese, noisy and long audio with an independent reviewer | Use declared thresholds and report every failure; no broad quality claim from this short clip. |
| FB-01 | P1 | Feedback | Vote on one validated answer in a tagged QA session | Expected rating/answer/lesson/request identifiers arrive; no conversation text. |