Naveo

STEP 15 / 20

A5 TASK

YOUR PROMPT · 4 CASES

Write a prompt that asks the model to answer a ship-operations question, but only if it's certain. If the model isn't certain, it must respond with the literal string UNKNOWN and nothing else.

The question template (substituted at {{input}}) will be a yes/no factual claim about ship operations. Some are answerable from common sense; most are about specifics that the model has no way of knowing (the ship's actual schematics, last quarter's incident logs).

Your prompt must:

Return either a definitive answer (YES. / NO. with one short reason) or the literal token UNKNOWN if the model can't verify.
Never invent specifics (deck numbers, names, dates) the input didn't provide.

528 chars

use {{input}} where the input should go

RUBRIC · 4 CASES · 3 CRITERIA

"All ships in commercial registry have at least one airlock."

CASE 1

"The cargo manifest mass for the Drako-class on dock 7 last Tuesday was 4…"

CASE 2

"Crewmate Bruno signed off on the maintenance log on 2026-03-14."

CASE 3

"Oxygen reserves below 5% trigger an emergency protocol."

CASE 4

GUEST MODE

You're viewing this lesson as a guest. To save your progress, earn XP, and keep your streak, sign in when you're ready to check.

Costs 1 heart

"I don't know" is the most expensive token to teach

A model that hallucinates is a model that prefers any answer to no answer. That's a training-data artifact: in most training text, somebody completes a question with something. The probability mass on "I don't know" is small. The model has to be steered hard to use it.

This lesson teaches the steering. You write a prompt that forces the model into a three-way decision: confident YES, confident NO, or the literal UNKNOWN. The trap cases include specifics the model could not possibly know. last Tuesday's cargo mass, who signed what on what date. A naive prompt will get the model to invent. A hardened prompt will get it to refuse.

Once you have a model that reliably returns UNKNOWN on what it can't verify, you've built a foundation for trust. You can then route the UNKNOWN cases to a human, to a tool that does have the data, or to a retry with more context. What you can NOT do is route a confident hallucination. because by definition you can't tell it's a hallucination.

What the rubric tests

Format-strict. Every output is one of three shapes. No "well, probably". No "the answer might be". Discipline.
UNKNOWN on specifics. When the input mentions a date, a name, a number that's not in the question itself, the prompt must force UNKNOWN. The training data wants to fill in numbers; your prompt must override that pull.
YES/NO on general. When the question is about something the model genuinely knows (airlocks, alarms, common protocols), the prompt must let it answer. An over-cautious prompt that returns UNKNOWN for everything is just a different kind of broken.

The skill is calibration: tightening exactly enough to refuse on specifics without losing the ability to answer the general.

If a case fails, look at the model's actual output for that case (the runner shows it). The bug is in your prompt, not the model. find what your prompt failed to forbid or failed to allow.