External AI tool evaluations — OpenWolf, Headroom, browser-use, Sakana Fugu
Context
Four external AI tools were evaluated 2026-06-22 for adoption into the Aspire stack, each tested against the locked architectural invariants (single-egress LLM Gateway · Hub vault · Knowledge OS · API-first · flat-rate economics) using ai-tool-evaluation-rubric. Outcome: all four = pass (do not adopt now). Recorded so they are not re-pitched cold.
Detail
| Tool | Verdict | Why | Revisit when |
|---|---|---|---|
| OpenWolf — Claude Code token middleware | Pass | Redundant with KO (project map), the recall/user-model MCPs (memory), and the gateway's usage ledger. | A measured code-navigation token problem in dev sessions. |
| Headroom — context-compression layer | Pass | Its value is cutting per-token spend; our LLM is flat-rate subscriptions (Claude Max / ChatGPT Pro) + zero-marginal on-prem Qwen, so savings ≈ $0. Adds a lossy layer + redundant cross-agent memory. | The fleet hits Claude Max usage caps / throttling (compression buys headroom under the cap). |
| browser-use — LLM browser-automation agent | Pass | Redundant with the 4 wired browser MCPs (Claude in Chrome · chrome-devtools · Playwright · playwright-parallel) + Apify/Firecrawl. Cuts against the API-first / browser-last rule; runs its own non-OpenAI-compatible LLM loop (a second egress). | A recurring no-API web portal that must run unattended at scale — self-host the OSS lib behind the gateway, never the CAPTCHA/stealth cloud. |
| Sakana Fugu — multi-agent orchestration model | Pass · watch | Overlaps the Gateway (one OpenAI-compatible endpoint over a model pool) AND OpenClaw + AIOS orchestration. Paid/metered hosted API (fights flat-rate economics; we can't point it at our own pool). Launched 2026-06-22 — day-one. The only one offering a capability we lack (learned multi-model orchestration). | Single-model + hand-orchestration becomes a proven bottleneck on hard multi-step tasks. Trial behind the gateway (it is OpenAI-compatible). See ai-tooling-watchlist. |
Security note — counterfeit Fugu repo
Sakana Fugu is a real Sakana AI product — official sources are github.com/SakanaAI/fugu, sakana.ai/fugu, console.sakana.ai (an OpenAI-compatible API/model). A counterfeit repo github.com/Sakana-AI-labs/Sakana-Fugu (wrong org, ~36 stars / 7 commits, distributes .exe/.dmg binaries) impersonates it. Do not download or run anything from the lookalike — wrong-org + claims-affiliation + unverified-binaries is a classic malware-delivery pattern. Use only the official source.
Actions
- Treat all four as decided; do not re-evaluate without a new trigger (the "Revisit when" column).
- If a tool is pitched again, point to this page + ai-tool-evaluation-rubric.
- Fugu only: monitor maturity + the orchestration-bottleneck trigger in ai-tooling-watchlist.
Provenance
Evaluated 2026-06-22 by Kom + Claude Code against the rubric, from the linked repos + official launch coverage. Verdicts reflect the Aspire stack as of that date (flat-rate subscription gateway + on-prem Qwen, 4 browser MCPs, OpenClaw fleet, Knowledge OS).