Naveo

STEP 21 / 22

A5 TASK

YOUR PROMPT · 1 CASES

Capstone of the full onboarding. Atlas assigns you a real system:

The "Watch Officer Assistant". a system that receives messages on the ship's comms channel during a shift and processes them in a full pipeline, keeping the shift log clean and actionable.

Your job: design the complete system architecture in YAML, integrating everything you learned in Track 4:

Router (step 03). decides which sub-flow each message goes to. Routes: incident_report, shift_info, crewmate_lookup, escalate_to_human.
Chain per route (step 02). each route has its sub-chain. The incident_report one classifies severity, extracts fields, files the ticket. The shift_info one queries tools (Track 3). The crewmate_lookup one queries tools. The escalate_to_human one just assembles a human message.
Agent loop (step 04). for incident_report, after the ticket, the system can decide whether to run follow-up actions (notify oncall, page maintenance) using tools. Loop with stop conditions.
RAG (step 05). optional, integrated in shift_info when the question can't be answered with tools alone (e.g. "what's the procedure if coolant drops 30%?". manual lookup).
Eval criteria (step 06, step 07). at the end of the YAML, define what evals the system runs to validate behavior. You don't write the full cases. just what test categories you want and what LLM-judge you use to evaluate end-to-end behavior.

Where the user message goes, use {{input}}. Your output is the full system configuration YAML.

936 chars

use {{input}} where the input should go

RUBRIC · 1 CASES · 9 CRITERIA

"meta-evaluation"

CASE 1

GUEST MODE

You're viewing this lesson as a guest. To save your progress, earn XP, and keep your streak, sign in when you're ready to check.

Costs 1 heart

Your complete system.

Atlas doesn't sign the capstone until your system shows all the track's skills integrated:

Unit	How it shows up in this capstone
A · System vs prompt	You decided this is worth being a system (router + chains), not a monolithic prompt
B · Chain design	Each `flow` has 2-4 focal steps, with state passed explicitly between steps. Think about caching where the same input repeats.
C · Dynamic routing	`router` with 4 routes + safe fallback `escalate_to_human`. Decide: rules-based router or LLM router? Cheap classifier up front?
D · Agents with budgets	`incident_report` can enter a loop. Define a budget (max steps / max tokens) and retry policy per error type. Think about graceful degradation if a sub-step fails.
E · Honest RAG	`shift_info` integrates retrieval for procedural questions
F · Observability + evals	Final section with eval categories + judge. Declare what you trace (spans per step, cost, PII redaction).

The system in one image

code

[comms message]
        ↓
   [ROUTER]  → 4 routes + fallback
   ↙ ↓ ↓ ↘
incident   shift   lookup   escalate
chain     (RAG)   chain    chain
   ↓
[optional agent_loop with stops]
   ↓
[result to the shift log]
        ↓
   [EVALS: control + edge + adversarial, judged by LLM]

What has to be in your YAML

We don't ask you for the full prompts of each step. the judge will evaluate structure and composition, not the exact wording of each prompt. What has to be there:

The router with 4 routes and a safe default.
4 flows defined, each with logical steps.
incident_report includes agent_loop with stops.
shift_info integrates RAG (at least in part).
escalate_to_human is conservative.
evals section declaring what gets measured.

The capstone, not the wording

Atlas signs when he sees the whole system working as a single piece. He doesn't care that your YAML is perfect at every bullet. he cares that the pieces are there, connected, and that the system is designable, evaluable, maintainable.

When it passes

The system records it as Track 4 complete. You move from "student with onboarding" to systems engineer with LLMs. The crew can hand you real problems. not to solve them, but to design the architecture to solve them.

Good luck on the shift.