What You Will Be Able To Do

• Make an API call and read token usage.
• Build a chat loop with history.
• Return validated JSON.
• Estimate cost and cut latency.
• Handle errors and protect keys.

Recap Of Week 9

• An eval set measures real quality.
• Ground answers to reduce hallucination.
• Test bias by group; plan for misuse.
• Minimize data sent to a provider.

What An API Is

• A web service that takes text and returns text.
• Your code sends a request, the provider's model replies.
• You pay per token.
• SDKs wrap the web calls in functions.

Setup Safely

• Python 3.9 or newer, a virtual environment.
• pip install anthropic
• The API key goes in an environment variable.
• Never put a key in code or screenshots.

python3 -m venv .venv
source .venv/bin/activate
pip install anthropic
export ANTHROPIC_API_KEY="paste-your-key"

Your First Call

• Three inputs: model, messages, max tokens.
• The reply text is in the first content block.
• Usage shows tokens in and out.

import os
from anthropic import Anthropic

client = Anthropic()
MODEL = os.environ.get('MODEL', 'claude-sonnet-4-5')

r = client.messages.create(
    model=MODEL, max_tokens=300,
    messages=[{'role': 'user', 'content': 'Explain RAG in two sentences.'}])
print(r.content[0].text)
print(r.usage.input_tokens, r.usage.output_tokens)

System Prompts

• Passed separately from the messages.
• Holds role, rules, tone, and format.
• Reuse your week 6 prompting skills.

r = client.messages.create(
    model=MODEL, max_tokens=300,
    system='You are a patient study helper. Answer in under 80 words.',
    messages=[{'role': 'user', 'content': 'What is an embedding?'}])

The API Has No Memory

• Each call stands alone.
• Your program keeps the message list.
• Send the full list every turn.
• Roles alternate: user, assistant.

A Chat Loop

• Read input, append, call, append, print.
• Type quit to stop.

history = []
while True:
    text = input('you> ')
    if text == 'quit':
        break
    history.append({'role': 'user', 'content': text})
    r = client.messages.create(model=MODEL, max_tokens=500,
        system='You are a study helper.', messages=history)
    reply = r.content[0].text
    history.append({'role': 'assistant', 'content': reply})
    print('bot>', reply)

Streaming

• Text arrives in pieces as it is written.
• Users see words within a moment.
• Total time stays about the same.

with client.messages.stream(model=MODEL, max_tokens=500,
        messages=history) as stream:
    for piece in stream.text_stream:
        print(piece, end='', flush=True)
print()

Structured Output

• Describe keys and types; give one example.
• Say: reply with JSON only.
• Parse with json.loads.
• Validate every field.

SYSTEM = ('Extract tasks. Reply with JSON only: '
  '{"tasks": [{"title": str, "due": str or null}]}')

Validate And Retry

• Models usually follow the format, not always.
• On failure, send the error back and ask for a fix.
• Stop after three tries.

import json

def extract(text, tries=3):
    msgs = [{'role': 'user', 'content': text}]
    for _ in range(tries):
        r = client.messages.create(model=MODEL, max_tokens=500,
            system=SYSTEM, messages=msgs)
        raw = r.content[0].text
        try:
            data = json.loads(raw)
            assert isinstance(data['tasks'], list)
            return data
        except Exception as e:
            msgs += [{'role': 'assistant', 'content': raw},
                     {'role': 'user', 'content': f'Invalid: {e}. Fix it.'}]
    raise ValueError('no valid JSON')

Counting Cost

• Price per million input tokens and output tokens.
• Cost = in x in_price + out x out_price, over 1,000,000.
• Chats, RAG, and agents all grow the input.
• Log usage from every response.

IN_PRICE, OUT_PRICE = 3.0, 15.0   # example: check the price page
total = 0
# after each call:
total += (r.usage.input_tokens * IN_PRICE
          + r.usage.output_tokens * OUT_PRICE) / 1_000_000
print(f'${total:.4f}')

Reducing Cost And Latency

• Smallest model that passes your eval.
• Trim history, retrieve fewer chunks.
• Cap output tokens; ask for concise answers.
• Prompt caching and batching where offered.

Errors And Retries

• Expect timeouts, rate limits, server errors.
• Retry with backoff: 1s, 2s, 4s.
• Show a clear message when retries fail.
• Never log keys or personal data.

import time
for wait in (1, 2, 4):
    try:
        r = client.messages.create(...)
        break
    except Exception as e:
        print('retrying:', e)
        time.sleep(wait)

Recap

• An API call needs model, messages, and max tokens.
• Chat apps keep and resend history.
• Validate JSON and retry on failure.
• Track tokens to know your cost.
• Stream, trim, and retry for a better feel.

Next Week: AI And Society

• Work, education, law, copyright.
• The EU AI Act and responsible use.

Questions And Exercise

• Build the study assistant with streaming.
• Add /extract with validation.
• Log cost and rerun your week 9 eval.