To test a README's install steps, run them on a machine that has never had your project, exactly as written, the way a new user would, and note every place you get stuck. Sai does that on a fresh Windows cloud computer, typing each command into a visible PowerShell window you can watch and record, and reports what the README leaves out with a suggested fix for each gap.
What a README install test is
A README install test is a walkthrough of a project's getting-started steps on a clean machine, following only the documentation, to find missing prerequisites, unclear steps and surprises before new users do.
It is different from a CI install job. CI proves the commands can succeed when run by a script. A README test checks what a person sees: how long each step takes, what the terminal prints, which prompts appear and what the docs forgot to say.
Why install steps that work for you fail for new users
- Your machine is not clean. It already has the runtimes, build tools, caches and environment variables your project needs, so missing prerequisites never show up.
- Silent steps look broken. A large download or first-time build that prints nothing for minutes looks frozen, and many users cancel it.
- PATH changes need a new terminal. On Windows, a running PowerShell window keeps the environment it started with. After an installer adds to PATH, the new command is not found until a new window is opened, and few READMEs say so.
- Defaults nobody mentioned. Telemetry, auto-updates, config files in the home folder and background services are often on by default and missing from the README.
- Windows is the afterthought. Many READMEs are written on macOS or Linux. Shell syntax, quoting and line continuation differ in PowerShell.
README install checklist for Windows
- Clean start: no earlier version of your project, and only the tools the README asks for.
- Prerequisites: every runtime and version is named, with how to install it on Windows.
- Commands as written: each command works when pasted into PowerShell, not only bash.
- Long steps: anything that takes more than a minute says how long and how big.
- Restarts: any step that needs a new terminal, a sign-out or a reboot says so.
- Prompts and warnings: UAC, SmartScreen, firewall and execution-policy prompts are mentioned.
- Defaults: telemetry and anything written outside the project folder are documented, with how to change them.
- First example: the README's first example gives the output it shows.
Ways to test a README, compared
| Option | Clean machine | Sees what a user sees | Recordable | Effort |
|---|---|---|---|---|
| Your own machine | No | Partly | Yes | Low, but misses most gaps |
| A friend or new contributor | Often | Yes | If they record | Hard to repeat each release |
| A CI job on a Windows runner | Fresh VM, but with developer tools preinstalled | No; a script, no screen | Logs only | Low once set up |
| A VM you reset by hand | Yes, after a revert | Yes, if you do it | Yes | An hour or more each time |
| Sai on a Windows cloud computer | Yes | Yes, in a visible PowerShell window | Watch live and record; screenshots kept | One prompt per release |
Keep the CI install job for every commit. Use Sai for the human walkthrough on release candidates and whenever the README changes.
How to test your README with Sai
- Add Sai to your coding agent with the
@simular-ai/sai-mcpserver, or call the API directly. - Point Sai at the README, or the docs page with the install steps, and the release to install.
- Ask for a visible terminal. Sai types each command into a PowerShell window on screen. Open the
watch_urlto watch live and record your own screen if you want a video. - Tell it to follow only the README. When a step fails, Sai notes what a new user would see first, then tries the obvious fix and says what it was. The full prompt is in the Start with a prompt tab.
- Fix the README and re-run. Each run starts on a fresh computer, so the second run tells you whether the fix is enough.
Run it from CI
Send the walkthrough to POST /v1/agents/message when you tag a release. success_criteria lists what must be true, and recordTrajectory keeps a screenshot of every step:
curl -s https://api.simular.ai/v1/agents/message \
-H "Authorization: Bearer $SAI_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"message": "On a Windows computer without mytool, open a visible PowerShell window and follow only the install steps in {{readme}}, typing each command. Then run the first example. Report every gap in the README.",
"options": {
"inputs": { "readme": "https://github.com/acme/mytool#readme" },
"success_criteria": [
{ "id": "installed", "check": "mytool --version prints a version", "returnEvidence": true },
{ "id": "example", "check": "The first README example gives the output the README shows", "returnEvidence": true }
],
"recordTrajectory": true
},
"wait": false
}'Poll /turns for the result and the report. The quick start covers keys, and computers explains which machine a task runs on.
Limitations
- One configuration. A cloud computer is one Windows setup, not every edition, language and corporate policy your users have.
- Slower than a script. A visible walkthrough takes as long as the install itself; keep it for releases.
- Sai will try obvious fixes. It reports them, but a real new user might give up instead. Treat every reported fix as a README gap.
- Windows only here. For macOS steps, run the walkthrough on your own Mac.
Related use cases: test your Windows installer on a clean computer, test your desktop app from Claude Code and reproduce a bug report step by step.
Last reviewed October 23, 2026.