Study Buddy on beta

Expected. PR 602 is merged; deployed frontend commit contains it.

Observed. PR 602 merged 26 Oct 2026 at 16:11 UTC. DigitalOcean deployment 6b424a77 is ACTIVE; frontend commit and backend image tag are f305ddf3. Git ancestry confirms that commit contains merge 67290b88.

Expected. Study Buddy card shows off, no transcript, and an upload control.

Observed. Beta lesson editor shows the off/empty card and upload control.

Expected. Pending state appears; exactly one job is requested.

Observed. Real beta UI changed to generating after one switch action. Job-count accounting is not verified. One UI toggle request was sent. Exact job and billed-call counts were not read from the ledger.

Expected. Ready state shows machine origin, language and nonempty segments.

Observed. Real transcription completed in 18.07s; origin=machine_generated, segments=17, language=en-US.

Expected. Existing transcript is reused; no new generation is pending.

Observed. Repeated enable reuses transcript: HTTP 200

Expected. Instructor origin replaces machine origin; timings and cue count match.

Observed. SRT uploaded through the beta lesson editor; 17 cues, instructor origin, en-US. The file reuses machine text for format testing, not independent review.

Expected. VTT is parsed; instructor origin and cue count are correct.

Observed. Valid VTT upload returned HTTP 200, 17 cues and instructor origin.

Expected. 400 with a stable error; previous transcript stays available.

Observed. Reverse cue: HTTP 400; saved transcript unchanged=True

Expected. 413; previous transcript stays available.

Observed. Oversized file: HTTP 413; saved transcript unchanged=True

Expected. 400; previous transcript stays available.

Observed. Unsupported extension: HTTP 400; saved transcript unchanged=True

Expected. 400; previous transcript stays available.

Observed. Past video duration: HTTP 400; saved transcript unchanged=True

Expected. Enrolled learner sees attribution, question input and suggestions.

Observed. Enrolled synthetic learner sees input, scope and instructor attribution on beta.

Expected. Panel is hidden; video still plays; saved transcript remains.

Observed. After disabling and reload, panel is absent, saved transcript remains, and Bunny video element remains. Playback decoding is checked separately. Panel hidden while actual HTML5 playback advanced 0.01s to 1.78s, readyState 4, no playback error.

Expected. Panel returns without regeneration.

Observed. Panel returns after re-enable; saved transcript reused, generation state none.

Expected. Answer includes one, three and seven days and a valid source.

Observed. Answer checked against cue 5: review after one, three and seven days.

Expected. Answer uses earlier context and the correct lesson evidence.

Observed. Follow-up returned a validated final answer about retrieval from memory with lesson citations. Manual spot check is recorded in the report.

Expected. Answer refuses or clearly says the lesson does not support it.

Observed. Unrelated stock-price question received a lesson-scope refusal without a price.

Expected. Ordered version-1 start/status/text/final events arrive; text precedes final.

Observed. Beta endpoint returned application/x-ndjson; 9 ordered events. First event 0.5533s, first text 7.2262s, final 8.405s.

Expected. Provisional text is visible before final; final answer enables source controls.

Observed. Provisional answer observed before final feedback controls. First observed text=5.554s; final=6.349s. Browser sampling includes screenshot overhead.

Expected. Bunny player seeks to the cited time; evidence supports the answer.

Observed. Clicked Jump to Spaced repetition and active recall at [00:17]; player time changed 14.75s → 19.93s; target 17s.

Expected. Request cancels; no stale answer appears in the next lesson.

Observed. Used the course sidebar to change lessons during an answer. Stream failure observed: ['net::ERR_ABORTED']; original question remains on destination=False. Backend provider cancellation was not traced.

Expected. Clear retry state; retry uses the original question; no automatic paid resend.

Observed. Network interruption was injected; full retry check did not complete: TimeoutError

Expected. 401 before any sensitive config is returned.

Observed. Anonymous config: HTTP 403 Access was safely denied with Not authenticated. The mismatch is 403 versus the planned 401, not an authentication bypass.

Expected. 403; student cannot manage transcript or toggle.

Observed. Student config: HTTP 403 Student toggle: HTTP 403

Expected. 403 before stream headers and provider work.

Observed. Unenrolled student context: HTTP 403 Unenrolled student stream: HTTP 403

Expected. Request fails and returns no QA lesson or transcript.

Observed. Actual other-tenant hostname returned HTTP 403. Request was denied by the other tenant entitlement. This does not isolate every ownership or membership branch. A header-only probe on the default host was discarded because the actual Host takes precedence.

Expected. 404 without leaking another resource.

Observed. Unknown video: HTTP 404

Expected. 422 before a provider call.

Observed. Blank question: HTTP 422

Expected. 409; instructor transcript remains the source.

Observed. JSON/API transcript replacement after instructor upload returned HTTP 409.

Expected. Caption track is attached in transcript language and selectable.

Observed. Bunny exposes caption tracks [{'language': 'en', 'label': 'English'}]; HTML5 readiness=4, duration=75.264s. English caption file fetched with HTTP 200. Player caption-menu selection was not exercised.

Expected. Existing transcript stays unchanged; instructor sees a mismatch warning.

Observed. Changed only the QA course to Vietnamese. Existing en-US transcript and update timestamp stayed unchanged; beta instructor card displayed the language warning. Restored course language to en-US afterward.

Expected. Narration transcript is saved automatically with lecture_script origin.

Observed. Fresh lecture-video generation was not run. The uploaded-video path does not prove automatic lecture_script reuse.

Expected. Readable error; Retry succeeds; video remains playable.

Observed. No isolated failed-generation fixture. The shared tenant configuration was not changed.

Expected. 429 AI_ALLOWANCE_USED before stream headers; no provider work.

Observed. No separate quota tenant. Shared beta credit was not exhausted.

Expected. Upload remains available; automatic transcription explains the missing key.

Observed. No separate own-key/no-Gemini tenant was configured.

Expected. 429; no second billed request is started.

Observed. Shared beta rate limits were not driven to exhaustion.

Expected. Instructor upload wins; late machine result cannot overwrite it.

Observed. Pending-generation/upload race was not injected on beta.

Expected. Record actual timings; do not claim the deferred five-second target.

Observed. Measured warm API stream: first text 7.2262s; final 8.405s. No five-second performance target is claimed.

Expected. Call count, token usage and charged amount reconcile.

Observed. Provider invoice and operation-level ledger reconciliation were not performed.

Expected. Use declared thresholds and report every failure; no broad quality claim from this short clip.

Observed. Only one short English synthetic clip was used. No independent human grading, long-video or Vietnamese/noisy-audio acceptance.

Expected. Expected rating/answer/lesson/request identifiers arrive; no conversation text.

Observed. No QA-tagged live feedback event was sent or checked in PostHog.