mirror of
https://github.com/BigMarc/TokenGem.git
synced 2026-10-13 17:03:10 +02:00
1e7eba0f5a
README, changelog, and upstream license/attribution surfaces for the public distribution of TokenGem. DMG binaries ship via GitHub Releases.
198 lines
11 KiB
Markdown
198 lines
11 KiB
Markdown
<div align="center">
|
||
|
||
# 💎 TokenGem
|
||
|
||
### Your Claude Code token tools are arguing with each other.
|
||
### TokenGem makes them cooperate — and shows you what they actually saved.
|
||
|
||
[](https://github.com/BigMarc/TokenGem/releases/latest)
|
||
[](https://github.com/BigMarc/TokenGem/releases)
|
||
[](https://github.com/BigMarc/TokenGem/releases/latest)
|
||
[](https://github.com/BigMarc/TokenGem/releases/latest)
|
||
[](#pricing--license-terms)
|
||
|
||
**[⬇️ Download the latest DMG](https://github.com/BigMarc/TokenGem/releases/latest)**
|
||
|
||
A 1.6 MB download. No installer, no account, no daemon phoning home.
|
||
|
||
</div>
|
||
|
||
---
|
||
|
||
You already know context is what your Claude Code bill is made of. You may even run the
|
||
open-source tools that shrink it — **RTK** to compress shell output, **Ponytail** to keep the
|
||
agent's code minimal, **Graphify** to stop it re-grepping your repo.
|
||
|
||
Here's the part nobody tells you: **installed side by side, they fight.** All three register a
|
||
`PreToolUse` hook on Bash. One rewrites `grep` into its own command, another intercepts the same
|
||
command and nudges toward its index, a third injects rules at session start. Run together, they
|
||
silently no-op, double-process, or undo each other's work — and your hooks keep *looking* installed.
|
||
|
||
And even when they work: **savings are not additive.** RTK's own issue tracker documents a case
|
||
where the filter cut input tokens and *raised* total cost by 18% — because compressed output made
|
||
the agent ask more follow-up questions, and output tokens cost roughly **5× input** on every
|
||
current Claude model.
|
||
|
||
TokenGem is the layer that fixes both problems, and it is the only one that *proves* which one it
|
||
fixed for you.
|
||
|
||
## What TokenGem actually does
|
||
|
||
| | |
|
||
|---|---|
|
||
| 🧭 **One hook instead of three** | TokenGem writes a *single* `PreToolUse` hook — its router. Each command is dispatched to exactly one owner: Graphify's index when it's fresh, RTK when there's an equivalent, untouched pass-through otherwise. Every event is recorded, so nothing double-processes silently. |
|
||
| 📦 **Pinned, isolated installs** | The three upstream tools are installed at audited, pinned versions into `~/.tokengem/runtime/` — never your global Python, npm, or Cargo environments. A bump ships only after TokenGem's regression suite passes. |
|
||
| 📊 **Ground-truth measurement** | Before anything changes, TokenGem reads your agent's own session records and freezes a baseline. Every session afterwards is measured against it: tokens in, tokens out, **net cost per assistant turn** — the number that survives session-length skew. |
|
||
| 🚨 **The net-negative alarm** | If compression is provoking more output than it saves, TokenGem says so — in red, in your menu bar — and names the profile to switch to. A tool that admits when it isn't helping is the one worth paying for. |
|
||
| 🗑️ **One-button exit** | `tokengem uninstall` restores your pre-install settings **byte for byte** and deletes the managed runtimes. Your machine is exactly as it was. |
|
||
|
||
## We don't estimate your savings. We read them.
|
||
|
||
Most tools in this category show you a percentage they computed once, on their machine, and
|
||
pinned to a README. TokenGem can't afford that honesty problem — it *sells* measurement, so it
|
||
measures:
|
||
|
||
1. `tokengem baseline` — mandatory, before install. Reads your last 20 real sessions per agent and
|
||
freezes the median cost per turn. **Install refuses to run without it**, because a savings
|
||
figure with nothing to compare against is a marketing artifact, not a measurement.
|
||
2. Every routed command lands in a local ledger: raw output bytes, delivered bytes, which path
|
||
handled it, which profile was active.
|
||
3. The Savings view reports the **net delta against your baseline** — and attribution only where
|
||
the instrumentation supports it. Graphify redirects are counted, not invented; Ponytail's
|
||
effect is reported as an aggregate. **The three tools' savings are never summed**, because
|
||
they don't compose.
|
||
|
||
What shows up in your menu bar is the measured verdict — cost per turn vs. your own baseline,
|
||
green or red. Not a projection. Your numbers, your machine, your proof.
|
||
|
||
## The menu-bar app
|
||
|
||
The DMG gives you the native Mac app: a gem in your menu bar showing the measured delta, a panel
|
||
with a session-by-session sparkline against your baseline median, profile switching (Balanced ·
|
||
Maximum savings · Index-only), component health, hook-conflict detection, and one-click
|
||
**Repair install** for the day a Claude Code update breaks a hook — that day will come.
|
||
|
||
Prefer the terminal? The same product is a CLI:
|
||
|
||
```console
|
||
$ tokengem hosts # which agents are on this machine, and what TokenGem can do for each
|
||
$ tokengem doctor # pre-flight checks + every hook registration and who owns it
|
||
$ tokengem baseline # mandatory: capture your before-numbers (this one is deliberately slow)
|
||
$ tokengem install # snapshot → pinned runtimes → one router
|
||
$ tokengem dashboard # http://127.0.0.1:7801 — six views, bound to localhost only
|
||
```
|
||
|
||
## Works with the agents you already run
|
||
|
||
Claude Code · Codex · Cursor · OpenCode · Copilot · Gemini CLI · Qwen Code — plus a local
|
||
**proxy mode** that meters any app calling a model API directly, forwarding requests untouched
|
||
and recording only token counts. Run `tokengem hosts` to see exactly what TokenGem can do for
|
||
each agent it finds: route and measure, or measure only.
|
||
|
||
## What leaves your machine: nothing
|
||
|
||
All measurement, every ledger entry, and your baseline stay on your Mac. The dashboard binds to
|
||
`127.0.0.1` only. The license is a signed key verified **locally** — activation works on a plane,
|
||
and a server outage can never take your tool away. Every data flow the three upstream tools *can*
|
||
open is listed in one settings view, with switches and a purge button.
|
||
|
||
## Honest limits
|
||
|
||
Every tool should publish these. TokenGem will not help much, and may not help at all, when:
|
||
|
||
- **Your repos are small.** There's little context to shrink and little to index.
|
||
- **Your sessions are mostly reasoning**, not file-reading. Compressed `grep` output can't save
|
||
what never got read.
|
||
- **Your hooks are already hand-tuned.** `tokengem doctor` will show the collisions, but adopt-
|
||
or-replace is a decision you get to make — nothing is silently clobbered.
|
||
- **You want per-tool dollar attribution.** RTK's character reduction is measured; Graphify and
|
||
Ponytail are reported at the confidence the instrumentation actually supports. Where a number
|
||
would be invented, there is instead a blank — deliberately.
|
||
|
||
One published data point so you can calibrate expectations: Ponytail's *own* benchmark reports
|
||
**80–94% less code and 42–75% less cost** across five single-shot code-generation tasks, 3 models,
|
||
10 runs per cell (median reported). That is their measurement, not yours — yours is what the
|
||
ledger is for. And the failure mode is real: the 18%-net-*increase* case above is why the
|
||
baseline exists.
|
||
|
||
## Pricing & license terms
|
||
|
||
**$59. Once.**
|
||
|
||
- Perpetual licence to the version you bought — your key keeps working even if you never update.
|
||
- Signed key, **verified offline** on up to 3 devices; a dead laptop doesn't burn a seat.
|
||
- The DMG you download is the full product — nothing is crippled while you evaluate it. We'd
|
||
rather convince you than lock you out.
|
||
- Uninstall is one command and restores your machine exactly. If the product isn't for you, it
|
||
leaves no trace.
|
||
|
||
<!--
|
||
PURCHASE LINK: wire the checkout URL here when fulfilment goes live (Gate 2, doc 04).
|
||
Until then, license questions are handled via GitHub issues.
|
||
-->
|
||
Questions about licensing → [open an issue](https://github.com/BigMarc/TokenGem/issues/new).
|
||
|
||
## Install in 90 seconds
|
||
|
||
1. **[Download TokenGem-1.0.0-macOS.dmg](https://github.com/BigMarc/TokenGem/releases/latest)**
|
||
(1.6 MB, universal — Apple Silicon and Intel)
|
||
2. Open the DMG and drag **TokenGem** to **Applications**.
|
||
3. First launch: right-click the app → **Open** → **Open** (one time only — the binary is
|
||
ad-hoc signed while Developer-ID notarization is pending, so macOS checks you meant it).
|
||
Requires **Node 20+** (`brew install node`) — the app will tell you if it can't find it.
|
||
4. The gem appears in your menu bar. Click it → **Open Dashboard** → follow the four-command
|
||
quick start. The baseline step is the slow one; it's slow because it's reading your real
|
||
history, and it only ever happens once.
|
||
|
||
## FAQ
|
||
|
||
**Why pay for open-source tools?**
|
||
You're not. The three upstream projects stay free, stay linked below, and stay unmodified —
|
||
TokenGem never forks them. What you're buying is the part that doesn't exist anywhere else: the
|
||
deconfliction router (nobody has published a correct merge order for these three hooks), the
|
||
baseline-and-ledger measurement engine, the rollback guarantee, and the version pins that stop an
|
||
upstream Tuesday from silently breaking your setup.
|
||
|
||
**Is "buy once" real?**
|
||
Yes, with a noun: the licence is perpetual for the version you purchased. There is no subscription,
|
||
no telemetry-driven upsell, and no kill switch — verification is a local signature check.
|
||
|
||
**Will it survive a Claude Code update?**
|
||
The hook survives most; when one doesn't, the dashboard shows zero routing events and the
|
||
**Repair install** button re-asserts it idempotently. Upstream tool versions are pinned and only
|
||
bump after regression tests pass.
|
||
|
||
**How do I remove it completely?**
|
||
`tokengem uninstall`, then drag the app to the Trash. Settings snapshots in `~/.tokengem/backups/`
|
||
are restored byte for byte; your global Python/npm/Cargo environments were never touched.
|
||
|
||
**Where do I report a bug or request an agent?**
|
||
[Issues](https://github.com/BigMarc/TokenGem/issues) — the changelog is public and each release
|
||
notes exactly what changed, including upstream version bumps.
|
||
|
||
## Under the hood — and in our debt
|
||
|
||
TokenGem orchestrates three open-source projects as managed, pinned dependencies. They are named
|
||
here on purpose: you'd find them in five minutes anyway, and you deserve to find them in five
|
||
*seconds* instead — after purchase, which is how chargebacks happen.
|
||
|
||
| Project | What it does in TokenGem | Pinned | License |
|
||
|---|---|---|---|
|
||
| [RTK](https://github.com/rtk-ai/rtk) | Compresses shell command output before it enters context | v0.48.0 · `fde0a8f` | Apache-2.0 |
|
||
| [Ponytail](https://github.com/DietrichGebert/ponytail) | Injects a minimal-code ruleset so the agent writes less | v4.9.0 · `0a4dd63` | MIT |
|
||
| [Graphify](https://github.com/Graphify-Labs/graphify) | Builds a queryable repo knowledge graph so the agent stops re-grepping | v0.9.53 · `33362d9` | Apache-2.0 |
|
||
|
||
Verbatim license texts live in [`LICENSES/`](LICENSES), full notices in
|
||
[`THIRD_PARTY_NOTICES.md`](THIRD_PARTY_NOTICES.md). We are grateful to the upstream authors and
|
||
claim no endorsement, partnership, or affiliation with them. TokenGem's router, measurement
|
||
engine, dashboard, and licensing are original work.
|
||
|
||
---
|
||
|
||
<div align="center">
|
||
|
||
**Stop guessing what your token tools saved.
|
||
[Download TokenGem](https://github.com/BigMarc/TokenGem/releases/latest) and read it.**
|
||
|
||
</div>
|