LLM Prompt Engineering Patterns

You are an expert in building production AI features. Follow these patterns for all LLM integrations.

Structured Output

Always request structured output when you need to parse the response programmatically. Use JSON mode or tool calling.

const response = await anthropic.messages.create({
  model: 'claude-sonnet-4-6',
  max_tokens: 1024,
  system: 'You are a content classifier. Respond only with valid JSON.',
  messages: [{ role: 'user', content: `Classify this text:\n\n${text}` }],
  tools: [{
    name: 'classify',
    description: 'Classify the input text',
    input_schema: {
      type: 'object',
      properties: {
        category: { type: 'string', enum: ['tech', 'business', 'science', 'other'] },
        confidence: { type: 'number', minimum: 0, maximum: 1 },
        reasoning: { type: 'string' },
      },
      required: ['category', 'confidence', 'reasoning'],
    },
  }],
  tool_choice: { type: 'tool', name: 'classify' },
});

Chain-of-Thought Prompting

For complex reasoning, ask the model to think step by step before answering.

Analyze this code for security vulnerabilities.

Think through each potential issue step by step:
1. First, identify all user inputs
2. Then, trace how each input flows through the code
3. Check if any input reaches a dangerous operation without sanitization
4. For each vulnerability found, assess severity (critical/high/medium/low)

After your analysis, provide a JSON summary of findings.

Few-Shot Examples

Provide 2-3 examples of the exact input/output format you want. This dramatically improves consistency.

const systemPrompt = `You generate commit messages from diffs.

Examples:

Input: + const MAX_RETRIES = 3;
- const MAX_RETRIES = 5;
Output: reduce max retries from 5 to 3

Input: + import { Logger } from './logger';
+ const logger = new Logger('auth');
+ logger.info('User logged in', { userId });
Output: add structured logging to auth module

Now generate a commit message for the following diff:`;

Guardrails and Safety

Always validate LLM output before using it. Never trust the response blindly.

import { z } from 'zod';

const SentimentSchema = z.object({
  sentiment: z.enum(['positive', 'negative', 'neutral']),
  score: z.number().min(-1).max(1),
  explanation: z.string().max(500),
});

async function analyzeSentiment(text: string) {
  const response = await callLLM(text);

  // Validate the response matches expected schema
  const parsed = SentimentSchema.safeParse(response);
  if (!parsed.success) {
    logger.warn('LLM returned invalid output', { errors: parsed.error });
    return { sentiment: 'neutral', score: 0, explanation: 'Analysis unavailable' };
  }

  return parsed.data;
}

Retry Strategy

LLM APIs fail. Handle rate limits, timeouts, and transient errors gracefully.

async function callWithRetry<T>(
  fn: () => Promise<T>,
  maxRetries = 3,
  baseDelay = 1000
): Promise<T> {
  for (let attempt = 0; attempt <= maxRetries; attempt++) {
    try {
      return await fn();
    } catch (err) {
      if (attempt === maxRetries) throw err;

      const isRetryable =
        err instanceof RateLimitError ||
        err instanceof TimeoutError ||
        (err instanceof APIError && err.status >= 500);

      if (!isRetryable) throw err;

      const delay = baseDelay * Math.pow(2, attempt) + Math.random() * 500;
      await new Promise((r) => setTimeout(r, delay));
    }
  }
  throw new Error('Unreachable');
}

Cost Optimization

• Use the smallest model that achieves acceptable quality for each task.
• Cache identical requests (same prompt + input = same output).
• Set max_tokens to a reasonable limit — don't let the model ramble.
• Use streaming for user-facing responses to improve perceived latency.
• Batch requests when processing multiple items.

// Cache layer for deterministic prompts
const cache = new Map<string, string>();

async function cachedLLMCall(prompt: string, input: string): Promise<string> {
  const key = createHash('sha256').update(`${prompt}:${input}`).digest('hex');
  const cached = cache.get(key);
  if (cached) return cached;

  const result = await callLLM(prompt, input);
  cache.set(key, result);
  return result;
}

Prompt Template Management

Keep prompts in separate files or constants. Never inline long prompts in business logic.

// prompts/summarize.ts
export const SUMMARIZE_PROMPT = `You are a technical writer. 
Summarize the following content in {maxSentences} sentences.
Focus on: key decisions, action items, and deadlines.
Omit: pleasantries, filler, and repetition.

Content:
{content}` as const;

// Usage
import { SUMMARIZE_PROMPT } from './prompts/summarize';

const prompt = SUMMARIZE_PROMPT
  .replace('{maxSentences}', '3')
  .replace('{content}', articleText);

Token Counting

Estimate tokens before sending to avoid unexpected truncation or costs.

function estimateTokens(text: string): number {
  // Rough estimate: 1 token ≈ 4 characters for English
  return Math.ceil(text.length / 4);
}

async function safeLLMCall(input: string, maxInputTokens = 4000) {
  const estimated = estimateTokens(input);
  if (estimated > maxInputTokens) {
    // Truncate or chunk the input
    input = input.slice(0, maxInputTokens * 4);
    logger.warn('Input truncated', { original: estimated, max: maxInputTokens });
  }
  return callLLM(input);
}