Credential Rot
An AI study app that turns course material into quizzes and job-specific practice tasks, then brings them back on a schedule so the learning sticks.
Certificates don't expire. Your memory does.
The problem
People finish courses and certificates, then lose most of what they learned. Most study tools quiz you on facts. Very few help you practice applying the material to the job you actually do.
What it does
- Upload anything you studied: a PDF, a web page or a YouTube lecture.
- Get questions written from that material: multiple choice, short answer and practice tasks set in your own field and role.
- Get graded with feedback: answers are judged on whether you understood the idea, not on whether you used the right technical terms.
- Review over time: each lecture has its own section, and questions come back after 1, 3, 7 and 14 days.
How it works
The app cleans the uploaded text, finds the main topics, writes questions tied to an exact quote from the source, checks their quality, grades answers and schedules reviews.
Built with Python, FastAPI, SQLite, the Claude API and plain HTML/CSS/JS. I built it with Claude Code as my coding partner.
How I tested it
I rated 45 generated questions by hand for accuracy, usefulness and whether they stuck to the source, and tested the grader with 10 answers I wrote myself. Testing turned up real problems:
- About 79% of the text pulled from my slide PDFs was hidden formula code, not readable content.
- The AI sometimes stated facts that weren't in the course material.
- Wrong answers in multiple-choice questions had giveaway patterns.
- Some practice scenarios revealed the answer.
I fixed these by checking every question against an exact quote from the source, cleaning the extracted text, adding question-quality rules and improving the grader. Then I tested again:
| Measure | Round 1 | Round 2 |
|---|---|---|
| Questions both accurate and useful | 80% | 93% |
| Questions with made-up facts | 2 | 0 |
Key decisions
- Grade understanding, not vocabulary. A correct idea in plain words should pass.
- Check in code, not just in the prompt. When the AI sometimes ignored a rule, I added a code check and a retry rather than a longer instruction.
- Don't force a task on every topic. Short, descriptive topics get questions only, not an artificial job scenario.
- Apply, don't memorize. Questions test using a formula, not reciting it.
- One lecture at a time. Reviews stay grouped by source instead of mixing everything together.
What's next
- Read image-heavy slides with vision.
- Remove the remaining multiple-choice giveaways and scenarios that reveal answers.
- Support more than one user.
Links
- Code: github.com/lanaalref/credential-rot
- Demo video: Loom walkthrough