AI-Native Updated 2026-09 12 min read View as Markdown

Type-safe LLM applications in TypeScript

A model returns a string. Your application needs a value. Everything interesting about building LLM apps in TypeScript happens at that boundary — and it is the one place `as` will hurt you most.

TypeScript has a specific advantage in this domain and a specific trap, and they are the same fact: the model's output arrives as an unvalidated string, and the type system cannot see that.

const result = JSON.parse(await llm.complete(prompt)) as Classification;

That line compiles. It asserts a shape that nothing checked, on data produced by a non-deterministic process, from a provider that may have changed its behaviour since you wrote it. It is the single most common defect in TypeScript LLM code, and it is invisible in review because it looks like every other line.

The whole discipline is: one schema, at the boundary, generating both the runtime check and the static type.

Schema first, type second#

import { z } from "zod";

export const Category = z.enum(["billing", "support", "refunds", "other"]);

export const Classification = z.object({
  category: Category,
  confidence: z.number().min(0).max(1),
  reasoning: z.string().max(500),
});

export type Classification = z.infer<typeof Classification>;
export type Category = z.infer<typeof Category>;

One declaration produces four things you would otherwise write separately and let drift apart:

1. The runtime validator
2. The static type
3. The JSON Schema you send to the provider for constrained decoding — z.toJSONSchema(Classification)
4. The documentation of what the model is supposed to return

export async function classify(input: string): Promise<Classification | null> {
  const raw = await llm.complete({
    system: SYSTEM,
    messages: [{ role: "user", content: input }],
    responseSchema: z.toJSONSchema(Classification),
  });

  const parsed = Classification.safeParse(extractJson(raw));
  if (!parsed.success) {
    logger.warn({ raw, issues: parsed.error.issues }, "unparseable model output");
    return null;
  }
  return parsed.data;
}

safeParse rather than parse, and a Classification | null return rather than a throw. The caller has to handle the failure because the type says so — which is the type system doing exactly the job you want it to do.

Type-safe tool definitions#

The second place types earn their keep. A tool has three representations that must agree: the schema you send the model, the arguments you receive back, and the function that runs. Derive all three from one place.

import { z } from "zod";

function defineTool<S extends z.ZodType, R>(spec: {
  name: string;
  description: string;
  input: S;
  run: (args: z.infer<S>) => Promise<R>;
}) {
  return spec;
}

export const searchOrders = defineTool({
  name: "search_orders",
  description: "Find orders for a customer. Staging data only, max 50 results.",
  input: z.object({
    customerId: z.string().uuid(),
    status: z.enum(["open", "shipped", "cancelled"]).optional(),
    limit: z.number().int().min(1).max(50).default(20),
  }),
  run: async ({ customerId, status, limit }) => db.orders.find({ customerId, status, limit }),
});

run is fully typed from the schema. Add a field to input and the compiler tells you run needs updating. Change a name and every call site breaks. The classic bug in hand-written tool definitions — schema and handler disagreeing after a refactor — becomes impossible.

Dispatch stays type-safe with a discriminated lookup:

const TOOLS = { search_orders: searchOrders, cancel_order: cancelOrder } as const;

export async function runTool(name: string, rawArgs: unknown) {
  const tool = TOOLS[name as keyof typeof TOOLS];
  if (!tool) return { error: `Unknown tool: ${name}` };

  const parsed = tool.input.safeParse(rawArgs);      // the model's arguments are untrusted
  if (!parsed.success) return { error: z.prettifyError(parsed.error) };

  return { result: await tool.run(parsed.data as never) };
}

Note that the model's tool arguments get the same treatment as a network payload, because that is what they are. Returning the validation error to the model is deliberate — it corrects itself on the next turn, which is cheaper than failing the request.

Model state as a discriminated union#

Streaming and tool loops have several states, and generated code models them as a bag of optional fields where most combinations are nonsense.

// 32 possible states, about 5 of them valid
type Turn = {
  status: "idle" | "streaming" | "tool" | "done" | "error";
  text?: string;
  toolCall?: ToolCall;
  error?: Error;
  usage?: Usage;
};

// 5 states, all valid
type Turn =
  | { status: "idle" }
  | { status: "streaming"; text: string }
  | { status: "tool"; call: ToolCall; text: string }
  | { status: "done"; text: string; usage: Usage }
  | { status: "error"; error: Error; partial: string };

With the second, rendering the error branch cannot accidentally read usage, and adding a state is a compile error at every site that needs updating — provided you end your switches with an exhaustiveness check:

function assertNever(x: never): never {
  throw new Error(`Unhandled turn state: ${JSON.stringify(x)}`);
}

Testing#

The pyramid is the same as everywhere: pure logic, then contract tests against a fake, then cassettes, then nightly evals against a scored dataset. The JavaScript page covers the streaming, abort and cassette mechanics in detail and applies unchanged.

What TypeScript adds is that a large slice of layer 2 becomes a type test instead of a runtime one:

import { expectTypeOf, test } from "vitest";

test("tool handler args are inferred from the schema", () => {
  expectTypeOf(searchOrders.run).parameter(0).toMatchTypeOf<{
    customerId: string;
    status?: "open" | "shipped" | "cancelled";
    limit: number;
  }>();
});

test("classify forces the caller to handle failure", () => {
  expectTypeOf(classify).returns.toEqualTypeOf<Promise<Classification | null>>();
});

These run in milliseconds and catch drift between a schema and its handler — which is the failure mode most likely to survive review, because both halves look correct in isolation.

The one runtime contract test that still matters most:

test.each([
  ['{"category":"billing","confidence":0.9,"reasoning":"x"}', "billing"],
  ['```json\n{"category":"billing","confidence":0.9,"reasoning":"x"}\n```', "billing"],
  ['{"category":"billing","confidence":1.4,"reasoning":"x"}', null],   // out of range
  ['{"category":"refunds_and_returns"}', null],                        // not in the enum
  ["I think it is billing.", null],
  ['{"category":"billing"', null],                                     // truncated
])("parse(%j)", (raw, expected) => {
  expect(parseClassification(raw)?.category ?? null).toBe(expected);
});

Never throw, never invent. A parser that returns a plausible wrong answer is worse than one that returns null.

Evals with types#

Type the dataset too — the number of eval harnesses broken by a typo in a JSONL field name is not small.

const Case = z.object({ input: z.string(), expect: Category });
type Case = z.infer<typeof Case>;

const cases: Case[] = readFileSync("evals/dataset.jsonl", "utf8")
  .trim().split("\n").map((l, i) => {
    const r = Case.safeParse(JSON.parse(l));
    if (!r.success) throw new Error(`dataset line ${i + 1}: ${z.prettifyError(r.error)}`);
    return r.data;
  });

const results = await Promise.all(cases.map((c) => classify(c.input)));
const accuracy = results.filter((r, i) => r?.category === cases[i].expect).length / cases.length;

console.log(`accuracy ${(accuracy * 100).toFixed(1)}% on ${cases.length} cases`);
results.forEach((r, i) => {
  if (r?.category !== cases[i].expect) {
    console.log(`  MISS ${JSON.stringify(cases[i].input)}: got ${r?.category ?? "null"}, want ${cases[i].expect}`);
  }
});
process.exit(accuracy >= 0.9 ? 0 : 1);

Nightly, a threshold rather than an assertion, and read the misses.

Common questions#

Does structured output mode make validation unnecessary?#

No. Constrained decoding removes most malformed JSON and is worth using, but it does not remove provider outages, model changes, refusals, truncation at the token limit, or the day someone swaps the model. Validation at the boundary is what makes the type honest — and it costs one safeParse.

Zod, Valibot, ArkType or TypeBox?#

Any of them; the discipline matters more than the library. Zod has the largest ecosystem and the most direct JSON Schema story, which matters when you are feeding schemas to a provider. Valibot is dramatically smaller if you are shipping to a browser.

Should tool handlers validate arguments if the model was given a schema?#

Yes, always. The model's arguments are untrusted input in exactly the sense a form submission is — the schema is guidance to the model, not a guarantee. Validating and returning the error back to the model also gives it a chance to correct itself, which is cheaper than failing the request.

Can the type system help with cost as well as correctness?#

It can make the cost mistakes that come from drift impossible — an untagged call site, a tier with no price, a call with no budget. That is making token budgets a type error.

Are type tests worth the setup?#

For tool definitions and public API boundaries, yes — they catch schema/handler drift, which is the failure most likely to pass review. For ordinary application code the compiler already covers it and expectTypeOf is redundant.