System Prompt Test Harness

System Prompt Test Harness

A method for changing a system prompt without breaking it — fixed cases, one edit at a time.

google-ai-studio prompt-engineering evaluation api testing

À quoi ça sert En anglais

Changing a system prompt without a fixed set of test cases is guessing, and the regression shows up two edits later when you can no longer tell which change caused it. This is the harness: the same cases every time, one edit per run, results written down.

Le skill

Use this when tuning a system prompt in AI Studio, before shipping it as an API
call.

Set up:
1. Write down 8-12 inputs you actually expect, including the two or three that
   have gone wrong before. These are your cases; they do not change while you
   are editing the prompt.
2. Save the prompt in Drive after every edit. AI Studio has no diff, so an
   unsaved good version is a lost one.

Loop:
1. Change ONE thing in the system instructions.
2. Run all cases at temperature 0. Anything above 0 makes a regression look
   like sampling noise and you will chase it for an hour.
3. Record, per case: pass, fail, or changed-for-the-worse.
4. If two cases got better and one got worse, you did not improve the prompt —
   you moved the failure. Keep going or revert.

Ship:
- Raise temperature LAST, and only if the task benefits from variety.
- Use **Get code** to export, and check that the exported call carries the
  system instructions in the `system_instruction` field rather than as the
  first user turn. They behave differently.

Comment l’installer

Google AI Studio

1. Open [aistudio.google.com](https://aistudio.google.com) → Create Prompt.
2. Paste the text below into System instructions (not the user turn).
3. Set temperature and output length for the task, then Save —
   the prompt is stored in your Drive and can be reopened.
4. Get code exports the same configuration as an API call.

Ça a marché pour vous ?

D’où ça vient

Skills app Original