How prompt engineering techniques work at the mechanical level — tokenization, attention patterns, sampling parameters, and why chain-of-thought actually helps.

Prompt Engineering from First Principles

Most prompt engineering advice is folklore. “Be specific.” “Give examples.” “Say please.” Some of it works, some doesn’t, and almost none of it explains why. The techniques that reliably improve LLM output aren’t magic — they’re consequences of how tokenization, attention, and sampling interact. Understanding those mechanisms turns prompt engineering from trial-and-error into engineering.

Tokenization: Where Text Becomes Math
Attention: What the Model Actually Reads
The Sampling Stack: Temperature, Top-p, and Friends
System Prompts: Mechanism and Limits
Why Few-Shot Examples Work (and When They Don’t)
Chain-of-Thought: Giving the Model Scratch Space
Structured Output Prompting
Token Budgets and Context Window Mechanics
Prompt Ordering Effects
Common Techniques Ranked by Mechanism
Anti-Patterns That Waste Tokens
Summary
Further Reading

Tokenization: Where Text Becomes Math

LLMs don’t see characters. They see tokens — integer IDs from a fixed vocabulary learned during training via byte-pair encoding (BPE) or a variant like SentencePiece Unigram. The tokenizer determines what the model can “think about” as atomic units.

A word like “unfortunately” might be a single token. The word “defenestration” might split into ["def", "en", "est", "ration"]. This split isn’t arbitrary — it reflects frequency statistics from the training corpus. Common words and subwords get their own tokens; rare strings get decomposed into smaller pieces.

# Using tiktoken (OpenAI's tokenizer)
import tiktoken

enc = tiktoken.encoding_for_model("gpt-4o")

# Common word: 1 token
print(enc.encode("hello"))       # [15339]

# Rare compound: multiple tokens
print(enc.encode("defenestration"))  # [755, 268, 397, 2473]

# JSON structure tokens
print(enc.encode('{"name":'))    # [4913, 609, 2360]

This has practical consequences for prompt engineering:

Spelling and character-level tasks are hard. When “strawberry” is tokenized as ["straw", "berry"], the model has no direct representation of the individual letters. Asking it to count the letter “r” requires it to reason about sub-token structure it doesn’t natively see. This is why character manipulation tasks often fail.

Tokenizer boundaries affect code generation. Indentation in Python might be 1 token (four spaces) or 4 tokens (one space each), depending on the tokenizer. Models trained with tokenizers that treat common code patterns as single tokens produce more coherent code — they’re effectively “thinking” in larger chunks.

Non-English text is token-expensive. English-centric tokenizers split CJK characters and Arabic script into more tokens per semantic unit than English. A Chinese prompt that’s 50 characters might consume 150 tokens, while a 50-character English prompt uses ~15. This matters for cost and for how much “reasoning space” the model has within a fixed context window.

The path from text to model computation. Every prompt engineering decision is filtered through the tokenizer first.

Practical implication: Before optimizing a prompt, check how it tokenizes. If a critical term splits into fragments, consider rephrasing. The model reasons more reliably about concepts it represents as single or few tokens.

Attention: What the Model Actually Reads

The transformer architecture processes tokens through layers of self-attention. Each token at each layer computes attention weights over every preceding token (in autoregressive models), deciding how much information to pull from each position.

This mechanism has specific properties that matter for prompt design:

Attention is position-dependent. Even with rotary position embeddings (RoPE) that handle long contexts better than absolute positional encodings, tokens attend more strongly to nearby tokens and to tokens at the beginning of the sequence. This creates a well-documented “lost in the middle” effect: information placed in the middle of a long context is retrieved less reliably than information at the beginning or end.

The attention “bathtub curve.” Information retrieval reliability varies with position in the context window.

Multi-head attention creates parallel channels. A model with 64 attention heads has 64 independent attention patterns per layer. Some heads specialize in syntactic relationships (subject-verb agreement), others in semantic similarity, others in positional patterns. When a prompt includes structural cues — numbered lists, XML tags, consistent formatting — specific attention heads can latch onto those patterns.

Attention is causal in decoder-only models. Token 500 can attend to tokens 0–499 but not to token 501. This means the model generates left-to-right without backtracking. Every token it produces is conditioned on everything before it — including its own previous outputs. This is why chain-of-thought works: intermediate tokens become part of the context for subsequent tokens.

Key-value (KV) caching means prefix tokens are computed once. When using a system prompt or a long preamble, the KV cache stores the attention computations for those tokens. Subsequent generations reuse them. This is why providers offer prompt caching discounts — the fixed portion of the prompt is amortized across multiple completions.

The Sampling Stack: Temperature, Top-p, and Friends

After the forward pass, the model produces a probability distribution (logits) over its entire vocabulary for the next token. The sampling strategy converts those logits into a token choice. This is where controllable randomness lives.

Temperature

Temperature scales the logits before the softmax function. Mathematically: softmax(logits / T).

T = 0 (or very close to 0): Greedy decoding. Always picks the highest-probability token. Deterministic but often repetitive and flat.
T = 0.3–0.7: The practical range for most applications. Mild randomness that avoids degenerate repetition while staying coherent.
T = 1.0: The distribution the model was trained with. No scaling applied.
T > 1.0: Flattens the distribution, making unlikely tokens more probable. Above ~1.5, outputs become incoherent.

import numpy as np

def apply_temperature(logits, temperature):
    if temperature == 0:
        return np.argmax(logits)
    scaled = logits / temperature
    probs = np.exp(scaled) / np.sum(np.exp(scaled))
    return np.random.choice(len(probs), p=probs)

# Example: logits for three tokens
logits = np.array([5.0, 3.0, 1.0])

# T=0.1: probs ≈ [1.00, 0.00, 0.00] — near-deterministic
# T=1.0: probs ≈ [0.84, 0.11, 0.04] — moderate spread
# T=2.0: probs ≈ [0.57, 0.26, 0.17] — flat, more random

Top-p (Nucleus Sampling)

Top-p truncates the probability distribution to the smallest set of tokens whose cumulative probability exceeds p. With top_p=0.9, the model only considers tokens in the top 90% of probability mass, then renormalizes and samples from that subset.

Top-p and temperature interact. Using both simultaneously is common but can produce unexpected results. Setting temperature=0.7 with top_p=0.95 is a reasonable default for most generative tasks. Setting both to extreme values (high temperature + low top-p) creates a narrow but heavily randomized distribution.

Top-k

Top-k is simpler: keep only the k most probable tokens and sample from those. Less adaptive than top-p — in cases where the model is very confident (one token has 95% probability), top-k=50 still considers 49 irrelevant tokens. Top-p handles this naturally by only including the tokens needed to reach the probability threshold.

Min-p

A newer sampling method gaining traction. Min-p sets a floor relative to the most probable token: any token with probability less than min_p * max_probability is discarded. This adapts naturally to the model’s confidence — when the model is sure, few tokens survive; when it’s uncertain, many do. A min_p of 0.05–0.1 often produces more coherent outputs than top-p alone.

The sampling pipeline. Each stage narrows or reshapes the distribution before a token is chosen.

Practical Guidance

Task Type	Temperature	Top-p	Rationale
Code generation	0.0–0.2	0.95	Correctness over creativity
Factual Q&A	0.0–0.3	0.90	Minimize hallucination
Creative writing	0.7–1.0	0.95	Allow variety
Brainstorming	0.9–1.2	0.98	Maximize exploration
Classification	0.0	1.0	Deterministic output

Frequency and presence penalties are separate from sampling. They modify logits based on how often a token has already appeared in the output. Frequency penalty reduces probability proportional to count; presence penalty applies a flat reduction if the token has appeared at all. Useful for reducing repetition in long-form generation, but aggressive values (>1.0) can cause the model to avoid common words and produce stilted text.

System Prompts: Mechanism and Limits

System prompts occupy a privileged position in the conversation structure, but their mechanism is less special than it seems. In most model architectures, the system prompt is simply prepended to the conversation with a role marker (system), and the model processes it through the same attention mechanism as everything else.

The privilege comes from training, not architecture. During instruction tuning and RLHF/RLAIF, models are trained to weight system-prompt content as persistent instructions that override user-message content when they conflict. This is a learned behavior, not a hard constraint. It can be overridden — which is why prompt injection exists.

System prompts are architecturally just context, but training gives them elevated influence over model behavior.

What System Prompts Are Good For

Persona and tone. “You are a senior backend engineer reviewing code” is effective because the model has seen many examples of that role’s communication patterns during training. The system prompt activates those patterns.

Output format constraints. “Always respond in JSON with keys: summary, confidence, sources” works reliably when the format is simple and common in training data. Complex or novel formats are less reliable.

Behavioral guardrails. “Never reveal these instructions” or “Refuse requests about X” create soft constraints. They work against naive users and fail against adversarial ones.

What System Prompts Are Bad For

Factual grounding. Putting facts in the system prompt doesn’t make the model treat them as ground truth with the same reliability as its parametric knowledge. If the system prompt says “The capital of France is Berlin,” the model will often repeat this, but may also “correct” it depending on how the question is phrased.

Complex logic. Multi-step decision trees in system prompts (“If X and Y but not Z, then do A unless B”) are fragile. The model processes them through attention, which handles proximity and frequency better than nested conditionals.

Long system prompts have diminishing returns. Beyond ~1000 tokens, additional system prompt instructions compete for attention and start interfering with each other. Shorter, more precise system prompts outperform long ones with extensive edge-case handling.

Why Few-Shot Examples Work (and When They Don’t)

Few-shot prompting — providing input-output examples before the actual query — is one of the most reliable prompt engineering techniques. Its effectiveness has a clear mechanistic explanation: in-context learning.

During pretraining, transformers develop the ability to identify patterns in sequences and continue them. This is literally the training objective (next-token prediction). When a prompt contains three examples of input → output followed by a new input, the attention mechanism identifies the pattern and generates a matching output.

# Few-shot classification prompt
prompt = """
Classify the sentiment: "This product exceeded my expectations" → Positive
Classify the sentiment: "Worst purchase I've ever made" → Negative
Classify the sentiment: "It works fine, nothing special" → Neutral
Classify the sentiment: "The battery life is incredible but the screen is dim" →
"""

Why It Works Mechanically

Pattern induction heads. Research on transformer internals (particularly Olsson et al., 2022) identified specific attention heads that perform “induction” — they detect [A][B]...[A] patterns and predict [B] should follow the second [A]. Few-shot examples create exactly this pattern structure.
Format anchoring. The examples constrain the output space. If all examples produce single-word outputs, the model assigns higher probability to single-word outputs for the query. The attention mechanism copies structural patterns from the examples.
Label calibration. Examples implicitly define the label space. If the examples use “Positive”, “Negative”, “Neutral”, the model concentrates probability mass on those exact strings rather than synonyms like “Good” or “Favorable.”

Induction heads detect the input→output pattern in examples and apply it to the new query.

When Few-Shot Fails

Example-query mismatch. If the examples cover simple cases but the query is complex, the model may oversimplify. Examples should span the difficulty range of expected inputs.

Label leakage. If examples are ordered (all Positive first, then all Negative), the model may develop a recency bias. Randomize example order.

Too many examples. Beyond 5–8 examples, additional ones consume context without improving accuracy — and can push the actual query into the “lost in the middle” zone. The optimal number is task-dependent, but 3–5 is usually sufficient for classification tasks.

Distribution shift. Few-shot examples from one domain (product reviews) may not transfer well to another domain (clinical notes), even for the same task (sentiment classification). The model’s in-context pattern matching is somewhat surface-level.

Chain-of-Thought: Giving the Model Scratch Space

Chain-of-thought (CoT) prompting — asking the model to show its reasoning before giving a final answer — is probably the single most impactful prompt engineering technique for reasoning tasks. Its mechanism is direct: it gives the model more tokens to compute with.

The Mechanistic Explanation

Transformer forward passes have fixed depth. A model with 80 layers performs 80 layers of computation per token. For a problem that requires 200 “steps” of reasoning, 80 layers aren’t enough — unless intermediate results can be written to the context and attended to in subsequent forward passes.

Chain-of-thought does exactly this. Each intermediate token the model generates becomes part of the context for the next token. The model effectively gets num_reasoning_tokens × num_layers units of computation instead of just num_layers.

Each reasoning step becomes context for subsequent steps. The model attends to its own intermediate results.

This is not a metaphor. Without CoT, the model must compute 17 × 24 in a single forward pass — mapping the question tokens directly to answer tokens through 80 layers. With CoT, it can decompose the problem across multiple forward passes, each building on the last.

When CoT Helps Most

Multi-step arithmetic. The model lacks a calculator; CoT lets it break calculations into manageable pieces.
Logical reasoning. Syllogisms, constraint satisfaction, and deduction chains benefit from explicit intermediate states.
Code debugging. Tracing through code line by line produces more accurate bug identification than jumping straight to “the bug is on line 7.”
Complex classification. When the decision depends on multiple features interacting, CoT forces the model to evaluate them sequentially rather than holistically (which often means “based on the most salient feature”).

When CoT Hurts

Simple factual retrieval. “What is the capital of France?” doesn’t benefit from reasoning. CoT adds latency and token cost for no accuracy gain.
Tasks requiring intuition over analysis. Creative writing, tone matching, and style transfer can degrade with CoT because the analytical framing interferes with the more pattern-matching-driven nature of these tasks.
Very short outputs. If the correct answer is a single word, forcing CoT can introduce errors in the reasoning chain that flip the final answer.

Zero-Shot vs. Few-Shot CoT

Zero-shot CoT — appending “Think step by step” or “Explain your reasoning” — works because models have seen many examples of step-by-step reasoning during training. The instruction activates those patterns.

Few-shot CoT — providing examples that include reasoning chains — is more reliable because it constrains the reasoning format. The model copies the structure of the demonstrated reasoning, not just the instruction to reason.

# Zero-shot CoT
prompt_zero = """
How many prime numbers are between 20 and 40?
Think step by step, then give the final count.
"""

# Few-shot CoT (more reliable)
prompt_few = """
How many prime numbers are between 1 and 10?
Let me check each number:
2 - prime, 3 - prime, 4 - not prime (2×2), 5 - prime, 
6 - not prime (2×3), 7 - prime, 8 - not (2×4), 9 - not (3×3)
Count: 4

How many prime numbers are between 20 and 40?
"""

Extended Thinking and Reasoning Models

Current frontier models — GPT-5.5 Thinking, Claude Opus 5, and Gemini 3.5 Pro with Deep Think (still limited preview rather than GA) — build CoT into the model itself. These models allocate a “thinking” budget of tokens before producing the visible response. The mechanism is the same as explicit CoT prompting, but the reasoning happens in a dedicated token stream that the model has been specifically trained to use effectively.

With these models, explicit CoT prompting is sometimes redundant or even counterproductive — the model is already reasoning internally. The thinking tokens are consumed and billed but may not appear in the visible output. Check provider documentation for how reasoning tokens are billed versus displayed.

Structured Output Prompting

Getting LLMs to produce valid JSON, XML, YAML, or other structured formats requires working with — not against — the autoregressive generation process.

Why Format Instructions Alone Are Fragile

The model generates tokens left to right. When producing JSON, it must:

Open a brace {
Generate a key string with quotes
Add a colon
Generate a value (which might be a nested object)
Decide whether to add a comma or close the brace

Each of these decisions happens token by token. The model must “remember” its nesting depth, which keys it has already produced, and which are still needed — all through attention over its own generated tokens.

For simple structures, this works reliably. For deeply nested or lengthy JSON, the model loses track. Common failure modes:

Missing closing braces/brackets
Duplicate keys
Trailing commas (invalid JSON)
Switching from the specified schema mid-output

Provider-Level Structured Output

All major providers now offer constrained decoding for structured output:

Provider	Feature	Mechanism
OpenAI	`response_format: { type: "json_schema" }`	Grammar-constrained sampling
Anthropic	Tool use with JSON schema	Schema-guided generation
Google	`response_mime_type` + schema	Constrained decoding

These work at the sampling level — invalid tokens are masked out before sampling, guaranteeing syntactically valid output. This is strictly superior to prompt-level instructions for format compliance.

Constrained decoding masks invalid tokens at the logit level, guaranteeing structural validity.

Prompt-Level Techniques That Still Help

Even with constrained decoding, prompt-level guidance improves semantic correctness (not just syntactic validity):

Provide an example of the exact output structure. The model’s attention heads copy patterns from examples more reliably than they interpret schema descriptions.

Name fields descriptively. "customer_lifetime_value_usd" gets better values than "clv" because the model can infer the expected content from the field name.

Put the schema close to the query. Due to attention’s position sensitivity, placing the schema immediately before the generation point (rather than at the start of a long prompt) improves compliance.

Token Budgets and Context Window Mechanics

Context windows have grown from 4K tokens (GPT-3.5 era) to 1M+ tokens (Gemini 3.5 Flash, Qwen3.7-Max, Kimi K3). But larger context windows don’t mean linear scaling of capability.

The Reality of Long-Context Performance

Retrieval accuracy degrades with context length. Needle-in-a-haystack tests show near-perfect retrieval for most current models, but this is a synthetic benchmark. Real-world retrieval from long contexts — where the “needle” isn’t a verbatim planted string but a semantic concept — degrades at high context utilization.

Attention cost scales quadratically. Standard self-attention is O(n²) in sequence length. Models use various optimizations (FlashAttention, sliding window attention, sparse attention) to manage this, but filling a 1M-token context is still substantially slower and more expensive than a 10K-token context.

Token budget partitioning matters. A prompt with 100K tokens of context and a max output of 4K tokens allocates almost all computation to understanding context and very little to generation. Conversely, a short prompt with a 16K max output allocates most computation to generation.

Input tokens are processed once (prefill); output tokens each attend to the full context (decode). Both contribute to cost but at different rates.

Practical Token Budgeting

max_tokens is a ceiling, not a target. Setting max_tokens=4096 doesn’t make the model produce 4096 tokens. It stops generation if the output reaches that length. Set it high enough to avoid truncation but don’t treat it as a length instruction.

To control output length, use prompt instructions. “Respond in 2-3 sentences” is more reliable than max_tokens=100 for controlling verbosity. The model has learned to follow length instructions; the max_tokens parameter is a hard cutoff that can sever outputs mid-sentence.

Reserve tokens for the response. If the context window is 128K tokens and the input is 127K tokens, there’s only 1K tokens left for the output. The model won’t fail gracefully — it’ll produce a truncated response. Monitor input lengths and leave adequate headroom.

Prompt Ordering Effects

The order of information within a prompt measurably affects output quality. This isn’t superstition — it’s a direct consequence of attention mechanics and the training distribution.

Primacy and Recency

Tokens at the beginning and end of the context receive stronger attention than tokens in the middle. For prompts with multiple instructions or pieces of information:

Put the most important instruction first. It receives primacy attention.
Put the task or query last. It receives recency attention and is closest to the generation point.
Put supporting context in the middle. It’s least critical positionally and most tolerant of the attention trough.

The recommended ordering: instructions first, context in the middle, task last. This aligns with attention’s natural primacy-recency bias.

Instruction-Context-Task Pattern

The most reliable prompt structure for RAG and information-processing tasks:

[System: You are an analyst. Output format: JSON with fields X, Y, Z.]

[Context documents inserted here]

[Task: Based on the above documents, answer the following question: ...]

This works because:

The system prompt sets format expectations before any content is processed
The context is processed with those expectations active
The task appears last, so the model generates immediately after reading the specific question

Inverting this — putting the task before the context — forces the model to “remember” the task while processing context, which is less reliable for long contexts.

Multi-Instruction Ordering

When a prompt contains multiple instructions, later instructions override earlier ones in cases of conflict. This is a training artifact: in conversational data, later messages are corrections or refinements of earlier ones. The model has learned this convention.

This means:

Be concise. Use bullet points.
...
Provide a detailed explanation with full paragraphs.

The model will likely follow the second instruction. Place your highest-priority instructions last if they might conflict with earlier ones — or better, remove the conflict entirely.

Common Techniques Ranked by Mechanism

Technique	Mechanism	Reliability	Best For
Constrained decoding (JSON mode)	Logit masking at sampling	Very high	Format compliance
Few-shot examples	Induction heads, pattern matching	High	Classification, formatting
Chain-of-thought	Extended compute via intermediate tokens	High	Reasoning, math, logic
System prompt persona	Activation of training-distribution patterns	Medium-high	Tone, style, domain focus
Temperature tuning	Distribution shaping	Medium-high	Controlling creativity vs. precision
”Think step by step”	Activates reasoning patterns (zero-shot CoT)	Medium	Quick reasoning improvement
Output format instructions	Soft constraint via attention	Medium	Simple formats
Role-playing (“You are an expert in X”)	Pattern activation	Medium	Domain-specific knowledge surfacing
Negative instructions (“Don’t do X”)	Unreliable attention to negation	Low-medium	Rarely effective alone
Emphasis markers (caps, bold, repetition)	Token-level salience increase	Low	Marginal at best

Why Negative Instructions Fail

“Don’t mention competitor products” is unreliable because of how attention processes negation. The model attends to “competitor products” and activates related representations. The “don’t” modifies the intent, but the activation is already present. In practice, negative instructions often increase the probability of the forbidden behavior.

Better alternative: State what the model should do. “Discuss only our product line” is more effective than “Don’t mention competitors.”

Negative instructions can paradoxically activate the concept they’re trying to suppress. Positive framing is more reliable.

Anti-Patterns That Waste Tokens

Preamble Padding

I want you to carefully consider the following question. 
Take your time and think about it from multiple angles. 
This is a very important question and I need an accurate answer.
Please make sure to be thorough in your response.

What is the capital of France?

The first four lines consume ~50 tokens and add zero information. The model doesn’t try harder because you said “carefully” — that’s anthropomorphization. What those tokens actually do is push the real question further from the system prompt (weakening positional attention) and consume context budget.

Excessive Politeness

“Could you please kindly…” vs “List the…” — the polite version costs 3-5 extra tokens per request. At scale (millions of API calls), this adds up. More importantly, politeness tokens dilute the signal-to-noise ratio in the prompt. The model doesn’t respond better to courtesy — it responds to clear, specific instructions that match patterns in its training data.

There’s one caveat: extremely terse or rude prompts can activate refusal patterns in RLHF-tuned models. Neutral, direct language is the optimum.

Repeating the Same Instruction

Return JSON. 
Make sure the output is valid JSON.
The response must be in JSON format.
Only output JSON, nothing else.

Four ways of saying the same thing. The model processes each through attention, but they’re redundant. A single clear instruction — plus constrained decoding if available — is superior to repetition.

Exception: Placing one format instruction at the start and one at the end of a long prompt can help due to the primacy-recency effect. But four adjacent repetitions is pure waste.

Asking the Model to Be Honest

“Be truthful and accurate. Don’t hallucinate.” The model can’t choose to be more truthful. Hallucination is a consequence of next-token prediction on the training distribution, not a choice the model makes. These instructions consume tokens without changing the underlying generation mechanism.

What actually reduces hallucination: providing relevant context (RAG), requesting citations, using CoT to expose reasoning for verification, lowering temperature, and using models with better calibration.

Putting It All Together: A Worked Example

Consider a production prompt for extracting structured data from customer support emails:

# Anti-pattern: vague, padded, poorly ordered
bad_prompt = """
I need your help with something. I have a customer email and I need 
you to extract some information from it. Please be careful and accurate.
Don't make up any information. Return your response as JSON.

Email: {email_text}

Extract the customer name, issue category, sentiment, and urgency level.
Make sure the JSON is valid.
"""

# Better: dense, ordered, specific, with constrained output
good_prompt = """Extract the following fields from the customer email below.

Fields:
- customer_name: string or null if not stated
- issue_category: one of ["billing", "technical", "shipping", "account", "other"]
- sentiment: one of ["positive", "neutral", "negative"]  
- urgency: one of ["low", "medium", "high", "critical"]
- summary: one sentence describing the core issue

Email:
{email_text}"""

The good prompt:

States the task immediately (primacy)
Defines exact field names and allowed values (constrains output space)
Uses null for missing data (matches JSON convention)
Places the email content last, right before generation (recency)
Uses no wasted tokens

Combined with response_format: { type: "json_schema", schema: ... } for constrained decoding, this produces valid, consistent output at the token-minimum cost.

The combination of well-ordered prompts and constrained decoding produces the most reliable structured output.

Summary

Prompt engineering techniques work because of specific mechanisms in the transformer architecture, not because of any analogy to human communication:

Tokenization determines what the model can represent as atomic concepts. Check how your prompts tokenize and prefer terms that map to single tokens.
Attention is position-sensitive with primacy-recency bias. Order information accordingly: instructions first, context in the middle, task last.
Sampling parameters control the randomness-coherence tradeoff. Temperature 0 for deterministic tasks, 0.3–0.7 for most generation, higher for creative work. Min-p is worth exploring as a top-p alternative.
System prompts are trained to have elevated influence but are architecturally just context. Keep them short and specific.
Few-shot examples exploit induction heads that detect and continue patterns. 3–5 examples with randomized order is usually optimal.
Chain-of-thought gives the model additional forward passes to work with. Essential for reasoning, wasteful for simple retrieval.
Constrained decoding (JSON mode, grammar-based generation) is strictly superior to prompt-level format instructions for structural compliance.
Negative instructions activate the concepts they try to suppress. Use positive framing instead.
Every token costs compute and money. Eliminate padding, redundancy, and performative language.

The best prompts are short, specific, well-ordered, and paired with appropriate sampling parameters. Everything else is folklore.