All pages

Agentic pipelines · Lesson 1.3 · 12 min

Extend the toolbox with plugins

Add a project-specific tool every agent can call.

After this lesson you can spot the project knowledge your agents keep rediscovering, wrap it in one tool, and ship that tool as a TRMNLY plugin that the sidebar agent and Claude Code can both call.

Time: 12 minutes · Works on: macOS app.

Agents rediscover your project every session

Watch an agent start a task in a project it doesn't know. It lists the folder, reads package.json, tries npm test, finds the tests need a database, reads the README, finds the script that starts one, and only then begins. The next session does the same. So does the agent in the next pane.

Some of that exploration is fine. A lot of it is the agent reconstructing something you could have told it in one line: "run ./scripts/check before you say you're done; it lints, type-checks and runs the fast tests." You can write that line in CLAUDE.md or AGENTS.md, and you should. But an instruction is something the agent has to read, remember and follow by hand. A tool is something it can call, and that gives you a few things an instruction doesn't:

  • One name for the right way. preflight means the same thing in every session and every agent, and it runs the exact command you chose.
  • Output shaped for a model. Your check script may print 3,000 lines. The tool can return the verdict and the 40 lines that matter.
  • A safety setting of its own. A tool that only reads can run without asking. One that changes things asks every time, whatever the agent is doing.

What deserves a tool

Make a tool when all three are true:

  1. Agents need it in most sessions on this project.
  2. Getting it wrong is costly or slow: the wrong test command, a migration run against the wrong database, a deploy from the wrong branch.
  3. It's short. A plugin's shell call is capped at 30 seconds and 64 KB of output, so a plugin tool is for quick checks and lookups. A 20-minute test suite belongs in the agent's own test tools or a background job.

Good first tools are almost always read-only: "is this branch ready to push", "which migrations haven't run", "what port is the dev server on in this worktree", "which feature flags does this file read". Leave tools that change things until you have a read-only tool you trust.

A worked example: preflight

The project has a script, scripts/preflight.sh, that lints, type-checks and runs the unit tests in under 30 seconds. Every agent should run it before it calls a task done. The plugin wraps it:

trmnly.manifest({
  name: 'preflight',
  description: 'Runs this project\'s preflight checks.',
  permissions: { shell: true },
  tools: { preflight: 'never' },
});

trmnly.tools.register({
  name: 'preflight',
  description:
    'Run the project\'s preflight checks (lint, types, unit tests). ' +
    'Call this before saying a task is done. Returns PASS or FAIL ' +
    'and the last 40 lines of output.',
  inputSchema: { type: 'object', properties: {} },
  execute: () => {
    const probe = trmnly.bash('test -x scripts/preflight.sh');
    if (probe.exit !== 0) {
      return 'This project has no scripts/preflight.sh. Run its tests directly.';
    }
    const r = trmnly.bash('scripts/preflight.sh 2>&1 | tail -n 40; exit ${pipestatus[1]}');
    return (r.exit === 0 ? 'PASS' : 'FAIL (exit ' + r.exit + ')') + '\n' + r.stdout;
  },
});

A few decisions in that file are worth copying:

  • The description says when to call it. Agents choose tools by their descriptions. "Call this before saying a task is done" does more than any instruction elsewhere.
  • It answers in every project. Plugins are installed for the whole Mac, not one repository. trmnly.bash runs in the project's folder (or the active worktree's) with the project's .env loaded, so the tool can check for the script and say clearly when a project doesn't have one, rather than failing with a confusing error.
  • The verdict comes first. PASS or FAIL on the first line, then the tail. The agent doesn't have to parse a log to know what happened. The command runs through zsh, which is why it reads the script's own exit status from pipestatus.
  • 'never' is earned, not assumed. The tool only reads and reports, so it runs without an approval card. If your preflight writes files, such as a formatter that fixes in place, leave it on the default and approve each call.

Teach agents to use it

The tool's description reaches every agent that lists TRMNLY's tools. For the sidebar agent you can also add a line to its system prompt from the plugin with trmnly.systemPrompt.add("Before saying a task is done, call preflight."). That fragment is for TRMNLY's own agent. For Claude Code, put the same sentence in the project's CLAUDE.md, because Claude Code reads its instructions from there.

Do it in TRMNLY

  1. Show the right sidebar with ⌘⇧\, open More ▾ and choose Plugins. If you haven't written a plugin before, do Plugins first. It covers the tab, example.js.template and the permission review.
  2. Write preflight.js in your editor. Keep the file in your repository, for example under tools/trmnly/, so it's reviewed and versioned with the code it wraps.
  3. Press Install… in the Plugins tab and pick the file. TRMNLY copies it into the plugins folder and loads it. Because it's a copy, re-run Install… after you change the repository version. Install… asks before it replaces a file with the same name.
  4. The row shows Permissions need review, because the manifest asks for shell. Press Review…, read the request and press Allow requested. The grant is tied to a hash of the file, so any later edit asks again. A plugin can't change what it does after you approved it without you noticing.
  5. Check the tool's badge. Because the manifest says 'never', the row badges preflight as auto. A tool you leave on the default shows ask.
  6. In the Agent tab, ask: "Add a comment to the top of the README, then check your work." The agent should call preflight without being told the name.
  7. Break the build on purpose and ask again. The tool should come back with FAIL and the lines that explain it. If the tool throws instead, the agent gets a failed call, and the plugin's row shows the error in red.
  8. With the MCP server on, the same tool is in the list Claude Code sees. Plugin tools travel over the MCP server with TRMNLY's built-in ones. In a claude session connected to TRMNLY, ask it to run preflight. Drive TRMNLY from Claude Code sets up the connection.
  9. While you iterate, press Reload on the plugin's row instead of restarting the app. A plugin that keeps failing is marked Quarantined and stops reloading by itself until you fix it and press Reload.

Other things a plugin can add

The same file can add a status widget to the strip above the right sidebar, such as the preflight verdict from the last run, so you see the state of the branch at a glance. It can add a persona, an alternate prompt with a tool allowlist, for a reviewer that may only read. And with the notifications permission, trmnly.notify can tell you when something the plugin watches changes. Start with the one tool. Add the rest once it has earned its place.

Check yourself

  • preflight appears under the plugin's row badged auto.
  • The sidebar agent called it on its own before saying a task was done.
  • A deliberately broken build came back as FAIL with the useful lines.
  • Editing the plugin file brought the permission review back.
  • You can say why a slow test suite shouldn't be a plugin tool.

Go further