AgentClash Docs

Product, reference, and contributor docs in one place.

ProductGitHub

Concepts

Tools, Network, and Secrets

Learn the difference between workspace tools, pack-defined tools, and engine primitives, and how network and secret handling are constrained.

AgentClash has more than one “tool” layer. If you do not separate them mentally, the rest of the runtime model gets confusing fast.

There are three different layers to know

1. Workspace tool resources

The workspace API already exposes first-class tools resources with fields like:

  • name
  • tool_kind
  • capability_key
  • definition
  • lifecycle_status

These are infrastructure resources that live alongside runtime profiles, provider accounts, and model aliases.

2. Pack-defined composed tools

Inside a challenge pack, the optional top-level tools block lets a pack author define custom tool interfaces that the evaluated agent can see.

Those definitions are pack-local. They are part of the authored benchmark bundle.

3. Engine primitives

At the bottom are the built-in executor primitives, like http_request. These are the concrete operations the runtime knows how to execute safely.

A pack-defined tool can delegate to a primitive. That is the key distinction.

Primitive versus composed tool

The current validation code expects composed tools to look roughly like this:

tools:
  custom:
    - name: check_inventory
      description: Check inventory by SKU
      parameters:
        type: object
        properties:
          sku:
            type: string
      implementation:
        primitive: http_request
        args:
          method: GET
          url: https://api.example.com/inventory/${sku}
          headers:
            Authorization: Bearer ${secrets.INVENTORY_API_KEY}

What this means:

  • check_inventory is the tool name the agent sees
  • http_request is the engine primitive that actually runs
  • args is the templated mapping from tool parameters to primitive inputs

So when people ask “primitive tools vs actual tools,” the clean answer is:

  • primitives are built-in executor operations
  • composed tools are the author-defined tool contracts that delegate to those primitives
  • workspace tool resources are a separate infrastructure surface

Validation is strict on purpose

The current parser and tests already reject several dangerous or ambiguous cases:

  • unknown template placeholders like ${missing}
  • self-referencing tools where a tool delegates to itself
  • delegation cycles across composed tools
  • invalid JSON-schema parameter definitions
  • missing primitive names or missing args blocks

That strictness is good. A benchmark bundle should fail at publish time rather than fail mysteriously at run time.

Tool kinds are a separate gate from tool names

The sandbox policy also carries allowed_tool_kinds. That means the pack can say which broad categories are available, for example:

  • file
  • shell
  • network

This is different from a specific composed-tool name. A pack might define check_inventory, but the runtime still checks whether the underlying kind is allowed.

Internet access is not automatic

The current runtime does not treat network as free ambient capability.

There are at least three control points visible in the repo:

  • the sandbox/tool policy starts with network disabled by default
  • the pack can enable outbound networking through sandbox.network_access and related policy toggles
  • the http_request primitive validates the target URL and CIDR allowlist before making a request

The current http_request.py helper does all of this:

  • allows only http and https
  • rejects missing hosts
  • resolves DNS and checks resolved addresses
  • blocks private, loopback, link-local, reserved, and multicast addresses unless explicitly allowlisted
  • enforces request and response body limits
  • sanitizes error handling so secret-bearing values do not leak back to the agent

So the current answer to “how can you call the external internet?” is:

  • use a tool path that ultimately delegates to a network-capable primitive like http_request
  • enable network access in the pack/runtime policy
  • keep the destination within the permitted network rules

Secrets live outside the pack

The product already exposes workspace-scoped secrets as a first-class surface.

You can:

  • list secret keys
  • set a secret value
  • delete a secret

The list endpoint intentionally returns metadata only. Secret values never come back out.

The CLI surface is:

agentclash secret list
agentclash secret set <KEY>
agentclash secret delete <KEY>

Where secret references resolve

There are two distinct secret-reference patterns in the current code:

  • workspace-secret://KEY for provider credential resolution
  • ${secrets.KEY} inside composed-tool argument templates

These are not interchangeable.

workspace-secret://KEY is used when the provider layer resolves account credentials.

${secrets.KEY} is used during composed-tool argument substitution. The engine then decides whether the target primitive is allowed to receive secret-bearing args.

Only hardened primitives can accept ${secrets.*}

This is a security boundary, not a convenience feature.

The current primitive_secrets.go file says only secret-safe primitives may receive ${secrets.*} substitutions, and today that allowlist intentionally includes only http_request.

The reason is straightforward:

  • secrets must not end up in argv
  • secrets must not land in readable sandbox files
  • secrets must not come back in response headers or stderr
  • secrets must not be echoed into the agent context accidentally

That is also why sandbox env_vars are literal-only. The executor explicitly rejects ${...} placeholders there, and the code comment tells pack authors to use http_request headers instead when remote authentication is needed.

See also

PreviousReplay and ScorecardsNextArtifacts