Tools

With the network closed, programs reach your data through tool calls and through nothing else. providers declares what the model is told about. dispatch decides what a call gets.

Declaration and dispatch

One request carries both:

await executor.execute({
  code,
  providers: [{ name: 'store', tools: ['listUsers', 'findUser'] }],
  dispatch: async (provider, tool, argsJson) =>
    JSON.stringify({ result: await serve(provider, tool, argsJson) }),
})

Each declaration becomes a name in the sandbox. store is a Proxy over an empty object, and its get trap returns an async function for every property a program reads. Calling one sends three strings out of the isolate: the provider name, the tool name, and the arguments as one JSON string.

sequenceDiagram
    participant P as the program
    participant S as the store proxy
    participant D as your dispatch
    P->>S: store.findUser('ada')
    S->>D: 'store', 'findUser', '["ada"]'
    Note over S,D: the isolate boundary, and a socket in the daemon arrangement
    D-->>S: one JSON string, the envelope
    S-->>P: the value under result, or a throw carrying error

Your dispatch returns that envelope as a JSON string. {"result": value} becomes the value the call returns, and {"error": text} becomes a throw the program can catch. Your function never leaves your process. In the daemon arrangement the program runs on another machine, and every call it makes travels back over the same socket. So the tool still runs where your database credentials are, and the executor holds none of them.

The trap returns a function for every name a program reads, whether or not the declaration listed it. Two reads are exceptions. A property written onto the proxy wins over the trap, and provider source uses that to hang helpers on a namespace. The generated trap returns undefined for symbol keys and for the property then, so awaiting the provider itself yields the proxy and never invokes a tool that was never there:

await executor.execute({
  code: 'async () => { const v = await store; return typeof v }',
  providers: [{ name: 'store', tools: ['x'] }],
  dispatch,
})
// { result: 'object', logs: [] }

Values that cross a tool call

Arguments travel as one JSON string, and by default a call spreads them into an array. Models writing ordinary JavaScript produce exactly that:

await executor.execute({
  code: `async () => {
    await store.noArgs()
    await store.two('ada', 7)
    await store.obj({ id: 1, deep: { a: [1, 2] } })
    await store.bytes(new Uint8Array([1, 2, 3]))
    return 'sent'
  }`,
  providers: [{ name: 'store', tools: ['noArgs'] }],
  dispatch: async (provider, tool, argsJson) => {
    console.log({ provider, tool, argsJson })

    return JSON.stringify({ result: null })
  },
})

{ provider: 'store', tool: 'noArgs', argsJson: '[]' }
{ provider: 'store', tool: 'two', argsJson: '["ada",7]' }
{ provider: 'store', tool: 'obj', argsJson: '[{"id":1,"deep":{"a":[1,2]}}]' }
{ provider: 'store', tool: 'bytes', argsJson: '[{"__executor_binary_v1__":"Uint8Array","data":"AQID"}]' }

store.two('ada', 7) arrives as ["ada",7], and store.noArgs() arrives as []. Setting positionalArgs: false on the descriptor picks the other convention, where a tool takes one options object and a call with no argument sends {}:

providers: [{ name: 'store', tools: ['find'], positionalArgs: false }]
// store.find({ id: 7 })  ->  argsJson '{"id":7}'
// store.all()            ->  argsJson '{}'

Pick the one your model already writes for. Dialects that document their tools as taking a single object want false, and a prompt showing ordinary calls wants the default. Either way parseDispatchArguments gives your responder an array. A payload that is not one comes back as a single argument, so {"id":7} parses to [{ id: 7 }].

__executor_binary_v1__ on the last line of the spread example tags the binary envelope. JSON has no representation for bytes. A Uint8Array, an ArrayBuffer or any typed array crosses as a tagged base64 object, and the far side rebuilds it as bytes, in both directions. Never parse that shape by hand. parseDispatchArguments rebuilds it for you, and a Uint8Array returned from your tool reaches the program as a Uint8Array:

dispatch: createEnvelopeDispatch(state, async (provider, tool, argsJson) => {
  const [bytes] = parseDispatchArguments(argsJson) as [Uint8Array]

  return new Uint8Array([...bytes].reverse()) // arrives inside the sandbox as a Uint8Array
})

Node buffers do not cross as binary

A Buffer is a Uint8Array, and it still does not cross as binary. JSON.stringify calls toJSON before any replacer sees the value, and buf.toJSON() returns { type: 'Buffer', data: [...] }. So the codec’s replacer sees an ordinary object and adds no tag. stringifyForDispatch below is that codec, exported from @supolka/codemode-executor-protocol beside the two functions the next section uses:

stringifyForDispatch(Buffer.from([1, 2, 3]))
// '{"type":"Buffer","data":[1,2,3]}'

stringifyForDispatch(new Uint8Array([1, 2, 3]))
// '{"__executor_binary_v1__":"Uint8Array","data":"AQID"}'

No error is thrown anywhere along that path. Tools that return a Buffer hand the program the type and data shape above. A program written to read bytes reads undefined off it. Nested buffers behave the same way: an avatar column arriving out of a database has the type and data shape inside the sandbox whether or not you thought about it. Wrap every Buffer on the way out, once, wherever your tools return:

const asBytes = (value: unknown): unknown =>
  Buffer.isBuffer(value) ? new Uint8Array(value) : value

asBytes returns a plain Uint8Array, the codec tags it, and the program gets real bytes. Functions cross in neither direction. But a tool that needs to call back into the program takes a handle.

Dispatch worth copying

Building the envelope is the protocol package’s job, and the lookup and the work are yours:

import {
  createEnvelopeDispatch,
  parseDispatchArguments,
  ToolNotFoundError,
} from '@supolka/codemode-executor-protocol'

const tools = new Map<string, (args: Array<unknown>) => unknown>([
  ['listUsers', () => store.users()],
  ['findUser', ([id]) => store.user(String(id))],
])

const state = { active: true }

try {
  return await executor.execute({
    code,
    providers: [{ name: 'store', tools: [...tools.keys()] }],
    outbound: null,
    dispatch: createEnvelopeDispatch(state, async (provider, tool, argsJson) => {
      if (provider !== 'store') {
        throw new ToolNotFoundError(tool)
      }

      const handler = tools.get(tool)

      if (handler === undefined) {
        throw new ToolNotFoundError(tool)
      }

      return handler(parseDispatchArguments(argsJson))
    }),
  })
} finally {
  state.active = false
}

createEnvelopeDispatch does three things your own code would otherwise repeat. It serializes whatever the responder returns into {"result": value}, binary included. It turns a rejection into {"error": message}, so a throw inside a tool becomes a catchable error inside the program and not a fault on your side. And it checks the state latch before it calls your responder. A call arriving with the latch closed gets the pinned Execution has already completed. and never reaches your data. Programs are free to start a tool call and return without awaiting it:

await executor.execute({
  code: `async () => { store.fireAndForget(); return 'returned early' }`,
  providers: [{ name: 'store', tools: ['fireAndForget'] }],
  dispatch: createEnvelopeDispatch(state, respond),
})
// { result: 'returned early', logs: [] }

store.fireAndForget() is never awaited, and your responder still starts before execute resolves, because the call leaves the isolate before the arrow returns. Both arrangements behave that way. No part of the run waits for the answer, so a responder that takes a round trip to your database is still running when execute resolves. Models that start a write and return without awaiting it have started that write. Never read an unawaited call as one that did not happen.

But a call arriving genuinely late does not reach you at all. Tool calls from a handler of a retained execution meet the completion text inside the sandbox, and your dispatch never runs. Without retainHandles the isolate is disposed the moment the run ends, so a timer the program set for after it returned never fires. A retained run keeps its isolate, so its timers keep firing, and a tool call from one of those meets that same text. Set state.active to false in a finally, as the sample above does, and no code of yours can run on behalf of that execution again.

Undeclared names reach your dispatch

Programs may call a tool the declaration never listed, and the call arrives at your dispatch under that name. They may call a provider nobody declared at all, because __dispatch takes the provider name as an argument:

await executor.execute({
  code: `async () => await __dispatch('secrets', 'read', '[]')`,
  providers: [{ name: 'store', tools: ['listOrders'] }],
  dispatch: async (provider, tool, argsJson) =>
    JSON.stringify({ result: { provider, tool, argsJson } }),
})
// { result: '{"result":{"provider":"secrets","tool":"read","argsJson":"[]"}}', logs: [] }

secrets was never declared. Your dispatch ran under that name anyway, and @cloudflare/codemode routes an undeclared tool into the host’s own lookup the same way. The declaration says what the model was told about. Your function decides. Key your lookup by the provider and the tool, the way the sample above does, and turn an unknown name down with the declared text:

Tool "<tool>" not found

ToolNotFoundError composes that sentence for you. Every site in the executor that refuses a call composes it from the same class, so the model reads one sentence wherever the refusal came from. Route only the tools you meant to grant. A dispatch that routes whatever name arrives has granted the execution everything it can route. The executor cannot make that decision for you, and the threat model puts it on your dispatch function.

How a failure reads on both sides

Tools that fail are not exceptions on your side and never end the run. A failure becomes a rejection at the call site inside the program, where the model can catch it and work around it:

await executor.execute({
  code: `async () => {
    try {
      return await store.nope()
    } catch (failure) {
      return { name: failure.name, message: failure.message }
    }
  }`,
  providers: [{ name: 'store', tools: ['listUsers'] }],
  dispatch: createEnvelopeDispatch(state, async (provider, tool) => {
    throw new ToolNotFoundError(tool)
  }),
})
// { result: { name: 'Error', message: 'Tool "nope" not found' }, logs: [] }

Tool "nope" not found reaches the program as an ordinary Error, message and all. Three texts are pinned, so a program can recognize them and a host should not reword them:

What the program catches	What caused it
Tool "<tool>" not found	The request carried no dispatch at all, or your dispatch turned the name down.
Execution has already completed.	The call arrived after the run settled. The sandbox composes that text without entering your dispatch, and the latch composes it a second time.
The dispatch answered outside the result envelope.	Your dispatch returned something that is not an object at all.

The dispatch answered outside the result envelope. reports a defect in your code and not in the program’s. A dispatch that returns JSON.stringify('just a string') produces it, because the proxy finds a string where the envelope should be:

await executor.execute({
  code: `async () => { try { return await store.x() } catch (failure) { return failure.message } }`,
  providers: [{ name: 'store', tools: ['x'] }],
  dispatch: async () => JSON.stringify('just a string'),
})
// { result: 'The dispatch answered outside the result envelope.', logs: [] }

Whatever your tool throws keeps its own message, so a domain failure written for the model reaches it unchanged. The executor measures the provider name, the tool name and the arguments against the payload bound before the call leaves the isolate. Past that bound the call rejects at the call site with the payload text and is never delivered. Your dispatch’s answer is not measured against that bound, so a tool that returns more than the run can return moves the failure to the end of the run.

Provider source

Two optional fields on a descriptor shape what the model sees. Both are your source, not the model’s, and both run before the program does. prelude runs after every proxy is declared, so a provider can hang helpers on its own namespace. A property written this way wins over the get trap and never becomes a tool call:

await executor.execute({
  code: 'async () => store.helpers.twice(21)',
  providers: [
    { name: 'store', tools: ['listUsers'], prelude: 'store.helpers = { twice: (n) => n * 2 }' },
  ],
  dispatch,
})
// { result: 42, logs: [] }

proxySource replaces the generated proxy outright, the get trap included, so a host can reproduce a namespace shape its model already knows, nested names included. The trap’s guards go with it: a source that does not return undefined for then hands a program awaiting the provider a tool call named then. Your source runs in the entry scope and addresses __dispatch and the codec directly:

await executor.execute({
  code: 'async () => await api.v1.users.list()',
  providers: [
    {
      name: 'api',
      tools: ['v1.users.list'],
      proxySource: `const api = { v1: { users: { list: async () => {
        const json = await __dispatch('api', 'v1.users.list', __stringifyForDispatch([]))
        const data = __parseForDispatch(json)
        if (data.error) throw new Error(data.error)
        return data.result
      } } } };`,
    },
  ],
  dispatch: async (provider, tool) => JSON.stringify({ result: `${provider}.${tool} served` }),
})
// { result: 'api.v1.users.list served', logs: [] }

Names that scope provides are contract, and the protocol lists them: __dispatch, the codec pair __stringifyForDispatch and __parseForDispatch, and the codec’s own parts.

Names a provider may not take

Every provider name becomes a const declaration in the entry scope. It has to be an identifier the language allows, it has to be unique, and it may not shadow anything that scope already binds. Every name the generated source itself reads is therefore reserved. RESERVED_PROVIDER_NAMES is exported from the protocol package. The protocol says what the set covers and why each name is in it. Each rule reports its own text inside the envelope before the program runs:

await executor.execute({ code, providers: [{ name: 'console', tools: ['x'] }], dispatch })
// error: 'Provider name "console" is reserved. Give the provider a name the entry scope does not bind.'

await executor.execute({ code, providers: [{ name: 'my-store', tools: ['x'] }], dispatch })
// error: 'Provider name "my-store" must be ASCII letters, digits, underscore and dollar,
//         and must not start with a digit.'

await executor.execute({ code, providers: [{ name: 'class', tools: ['x'] }], dispatch })
// error: 'Provider name "class" is on the list this executor refuses: every ECMAScript
//         reserved word, the names strict mode reserves, and async. Pick a name outside it.'

await executor.execute({
  code,
  providers: [
    { name: 'store', tools: ['x'] },
    { name: 'store', tools: ['y'] },
  ],
  dispatch,
})
// error: 'Provider name "store" is declared twice. Give each provider its own name.'

The generated module is strict, so the third rule covers ECMA-262’s keywords and reserved words along with arguments and eval. Strict code may not bind either of those at all. async is on the list too: the language would accept it, but a name that also opens an async function head is too ambiguous to allow. That reserved set does not cover the rest of the standard library, because a provider that shadows fetch breaks no name the executor depends on. But it breaks the program, and silently:

await executor.execute({
  code: `async () => { try { const r = await fetch('https://example.com'); return 'reached ' + r.status } catch (failure) { return failure.message } }`,
  providers: [{ name: 'fetch', tools: ['x'] }],
  dispatch,
})
// { result: 'fetch is not a function', logs: [] }

typeof fetch reads object there, so even a program that checks first takes the wrong branch. Where provider names come from a model or from user input, turn down any name that shadows a global your programs need. The protocol says which names are checked for you.

Next

Tool calls are one direction a program can reach outward, and the network is the other. Decide that one in the network guide. The failures guide says what a caller sees when a call fails, and the pool chapter follows the round trip a call makes between the isolate and your process.