Subject 04 · Builds on Models
Prompting & Directing Models
How to steer a model you have chosen: prompts, system prompts, structured output, in-context examples, reasoning, and the common failure modes.
21 pages across 6 topics
Prompting basics
How to ask for what you want.
- A good prompt An effective prompt is specific and direct. The model cannot read your mind, so the more precisely you say what you want, the better the result tends to be.
- Context and format Add the context behind your request and show a concrete example of the output you want. Both steer the answer far more reliably than piling on adjectives.
- Iterate and refine Prompting is iterative by nature. Send a first attempt, read the output, then refine the wording and add what was missing. Treat the first try as a draft.
- Common mistakes Most beginner trouble comes from vague requests, missing context, no example format, and telling the model what to avoid instead of what to do. Each has a direct fix.
System prompts
Setting who the model is.
- What a system prompt is A system prompt is the standing instruction set once at the start of a session. It tells the model who to be and what rules to follow, while the actual requests come later as user messages.
- System vs user message A system prompt is set once and shapes the whole session; a user message is one turn carrying a specific request. APIs label them with separate roles so the model treats them differently.
- Why it takes priority Models are trained on an instruction hierarchy: system instructions outrank user messages, which outrank tool and third-party content. This keeps a session's rules in force, though it is a defense, not a guarantee.
Structured output
Getting JSON and schemas back reliably.
- Why structured output Models write prose, but the programs that consume their output need predictable data. Structured output means asking the model for machine-readable formats like JSON so your code can parse it without guessing.
- Schemas and strict mode JSON mode guarantees parseable JSON but not its shape. Passing a JSON Schema with strict mode pins the exact fields, types, and allowed values, so the output matches what your code expects.
- Constrained decoding Strict schema modes work through constrained decoding: at every step the model is only allowed to pick tokens that keep the output valid against the schema, so a parse error becomes impossible.
In-context learning
Teaching with examples in the prompt.
- In-context learning In-context learning is the ability of a large language model to pick up a new task from what is written in the prompt, at the moment it answers, with no retraining and no change to its weights.
- Zero-shot vs few-shot Zero-shot prompting asks the model to do a task from an instruction alone. Few-shot prompting adds a few worked examples first, which usually makes the answer more accurate and consistent.
- How examples steer Examples in a prompt mostly teach the model the format, the set of possible answers, and the kind of inputs to expect. One study found that even wrong labels in the examples barely hurt results, which says the format does much of the work.
Reasoning & chain of thought
Letting a model think before it answers.
- Chain of thought Chain-of-thought prompting tells the model to show its work, writing out a series of intermediate reasoning steps before the final answer. On harder problems this raises accuracy.
- Let's think step by step Yes. Just adding the phrase 'Let's think step by step' prompts a model to reason out loud before answering, with no examples in the prompt. This is zero-shot chain of thought.
- Reasoning models Reasoning models are trained to do the step-by-step thinking on their own, producing an internal chain of thought before the final answer instead of waiting for you to ask.
- When it helps Step-by-step reasoning pays off on multi-step problems such as math, logic, and planning. On simple recall or lookup questions it adds time and cost without making the answer better.
Common failure modes
Hallucination, drift, and how to catch them.
- How prompting fails Prompting usually fails in a few recognizable ways: the model invents facts, drifts off your instructions, gets buried in too much context, or guesses at something you left vague. Each has a known fix.
- Hallucination A model's job is to produce likely text, not verified truth, so it sometimes states false things confidently. The main fixes are letting it admit uncertainty and grounding its answers in supplied sources.
- Over-long context When a prompt gets long, models tend to use information at the start and end better than information stuck in the middle. Keep prompts short and place the important parts at the edges.
- Ambiguity and drift When a request is vague, the model fills the gaps with its own guesses; when instructions are buried, it drifts off them. Be specific, state what you want done, and make the key rules easy to find.