gitadmin/TokenGem
1
0
Fork 0
mirror of https://github.com/BigMarc/TokenGem.git synced 2026-10-13 17:03:10 +02:00
Code Issues Packages Projects Releases Wiki Activity

Restructure README in the VoiceGem house style

Browse Source 
Centered hero with tagline and for-the-badge row, tighter narrative,
features and pricing tables (Solo/Duo/Studio device ladder), condensed
FAQ — same facts, half the words.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
BigMarc 2026-09-05 21:44:24 +02:00
parent 1e7eba0f5a
commit 09abb5b0f0
1 changed files with 109 additions and 119 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

228
README.md
View File

@ -2,129 +2,102 @@
# 💎 TokenGem # 💎 TokenGem
### Your Claude Code token tools are arguing with each other. ### Make your Claude Code token tools cooperate — and prove what they saved
### TokenGem makes them cooperate — and shows you what they actually saved.
[![Latest release](https://img.shields.io/github/v/release/BigMarc/TokenGem?color=7c3aed&label=download&sort=semver)](https://github.com/BigMarc/TokenGem/releases/latest) **TokenGem is the orchestration + measurement layer for AI coding-agent token tools.**
[![Downloads](https://img.shields.io/github/downloads/BigMarc/TokenGem/total?color=7c3aed)](https://github.com/BigMarc/TokenGem/releases) One router instead of three fighting hooks. A baseline frozen from **your own sessions**.
[![macOS 13+](https://img.shields.io/badge/macOS-13%2B-black)](https://github.com/BigMarc/TokenGem/releases/latest) A verdict in your menu bar — green or red, measured, never estimated.
[![Universal binary](https://img.shields.io/badge/Apple%20Silicon-%20%26%20Intel-blue)](https://github.com/BigMarc/TokenGem/releases/latest)
[![Price](https://img.shields.io/badge/price-%2459%20once-success)](#pricing--license-terms)
**[⬇️ Download the latest DMG](https://github.com/BigMarc/TokenGem/releases/latest)** [![Download for macOS](https://img.shields.io/badge/Download-macOS%2013%2B-7c3aed?style=for-the-badge&logo=apple&logoColor=white)](https://github.com/BigMarc/TokenGem/releases/latest)
[![Latest release](https://img.shields.io/github/v/release/BigMarc/TokenGem?style=for-the-badge&color=1a1500&label=version&sort=semver)](https://github.com/BigMarc/TokenGem/releases/latest)
[![Price](https://img.shields.io/badge/%2459-once%2C%20no%20subscription-2d4a3e?style=for-the-badge)](#pricing)
A 1.6 MB download. No installer, no account, no daemon phoning home. <sub>Native macOS menu‑bar app · Node CLI · localhost‑only dashboard — a 1.6 MB DMG, no installer, no account, no daemon phoning home</sub>
</div> </div>
--- ---
You already know context is what your Claude Code bill is made of. You may even run the ## Why TokenGem?
open-source tools that shrink it — **RTK** to compress shell output, **Ponytail** to keep the
agent's code minimal, **Graphify** to stop it re-grepping your repo.
Here's the part nobody tells you: **installed side by side, they fight.** All three register a Context is what your Claude Code bill is made of, and the open‑source tools that shrink it are
`PreToolUse` hook on Bash. One rewrites `grep` into its own command, another intercepts the same excellent — **RTK** compresses shell output, **Ponytail** keeps generated code minimal, **Graphify**
command and nudges toward its index, a third injects rules at session start. Run together, they stops the agent re‑grepping your repo. Here's the part nobody tells you: **installed side by side,
silently no-op, double-process, or undo each other's work — and your hooks keep *looking* installed. they fight.** All three register a `PreToolUse` hook on Bash, so together they silently no‑op,
double‑process, or undo each other's work — while your hooks keep *looking* installed.
And even when they work: **savings are not additive.** RTK's own issue tracker documents a case And even when they work, **savings aren't additive**. RTK's own issue tracker documents a case
where the filter cut input tokens and *raised* total cost by 18% — because compressed output made where compression cut input tokens and *raised* total cost 18% — shorter input provoked longer
the agent ask more follow-up questions, and output tokens cost roughly **5× input** on every answers, and output tokens cost roughly **5× input** on every current Claude model. TokenGem fixes
current Claude model. the conflict, then measures the result on your machine, against your own before‑numbers.
TokenGem is the layer that fixes both problems, and it is the only one that *proves* which one it > 🧭 **One router, one hook** · 📊 **Measured vs your baseline** · 🚨 **Red when net‑negative** · 🍎 **Native macOS app**
fixed for you.
## What TokenGem actually does ## Features
| | | | | |
|---|---| |---|---|
| 🧭 **One hook instead of three** | TokenGem writes a *single* `PreToolUse` hook — its router. Each command is dispatched to exactly one owner: Graphify's index when it's fresh, RTK when there's an equivalent, untouched pass-through otherwise. Every event is recorded, so nothing double-processes silently. | | 🧭 **One hook instead of three** | A single `PreToolUse` router dispatches each command to exactly one owner — Graphify's index when fresh, RTK where an equivalent exists, untouched pass‑through otherwise. Nothing double‑processes silently. |
| 📦 **Pinned, isolated installs** | The three upstream tools are installed at audited, pinned versions into `~/.tokengem/runtime/` — never your global Python, npm, or Cargo environments. A bump ships only after TokenGem's regression suite passes. | | 📊 **Ground‑truth measurement** | A mandatory baseline from your last 20 real sessions, then a local ledger of every routed command: tokens in, tokens out, **net cost per assistant turn**. |
| 📊 **Ground-truth measurement** | Before anything changes, TokenGem reads your agent's own session records and freezes a baseline. Every session afterwards is measured against it: tokens in, tokens out, **net cost per assistant turn** — the number that survives session-length skew. | | 🚨 **The net‑negative alarm** | If compression is provoking more output than it saves, the menu bar turns red and names the profile to switch to. A tool that admits when it isn't helping is the one worth paying for. |
| 🚨 **The net-negative alarm** | If compression is provoking more output than it saves, TokenGem says so — in red, in your menu bar — and names the profile to switch to. A tool that admits when it isn't helping is the one worth paying for. | | 📦 **Pinned, isolated installs** | Upstream tools run at audited, pinned versions inside `~/.tokengem/runtime/` — never your global Python, npm, or Cargo. Bumps ship only after regression tests pass. |
| 🗑️ **One-button exit** | `tokengem uninstall` restores your pre-install settings **byte for byte** and deletes the managed runtimes. Your machine is exactly as it was. | | 🪄 **Profiles** | Balanced · Maximum savings · Index‑only — switch from the menu bar; the ledger records which was active for every event. |
| 🩺 **Doctor & repair** | `tokengem doctor` shows every hook registration and who owns it; one‑click **Repair install** for the day a Claude Code update breaks a hook. |
| 🗑️ **One‑button exit** | `tokengem uninstall` restores your pre‑install settings **byte for byte** and deletes the managed runtimes. No trace. |
## We don't estimate your savings. We read them. ## We don't estimate your savings. We read them.
Most tools in this category show you a percentage they computed once, on their machine, and Most tools in this category show a percentage computed once, on someone else's machine, and pinned
pinned to a README. TokenGem can't afford that honesty problem — it *sells* measurement, so it to a README. TokenGem *sells* measurement, so it measures:
measures:
1. `tokengem baseline` — mandatory, before install. Reads your last 20 real sessions per agent and 1. **`tokengem baseline`** — mandatory, before install. Reads your real session history and freezes
freezes the median cost per turn. **Install refuses to run without it**, because a savings the median cost per turn. Install refuses to run without it.
figure with nothing to compare against is a marketing artifact, not a measurement. 2. **Every routed command lands in a local ledger** — raw bytes, delivered bytes, which path
2. Every routed command lands in a local ledger: raw output bytes, delivered bytes, which path
handled it, which profile was active. handled it, which profile was active.
3. The Savings view reports the **net delta against your baseline** — and attribution only where 3. **The Savings view reports the net delta against your baseline** — and attribution only where
the instrumentation supports it. Graphify redirects are counted, not invented; Ponytail's the instrumentation supports it. The three tools' savings are never summed, because they don't
effect is reported as an aggregate. **The three tools' savings are never summed**, because compose. Where a number would be invented, there is a blank — deliberately.
they don't compose.
What shows up in your menu bar is the measured verdict — cost per turn vs. your own baseline, What shows up in your menu bar is the measured verdict. Not a projection. Your numbers, your
green or red. Not a projection. Your numbers, your machine, your proof. machine, your proof.
## The menu-bar app ## Menu bar or terminal — same product
The DMG gives you the native Mac app: a gem in your menu bar showing the measured delta, a panel The DMG gives you the native Mac app: a gem showing the measured delta, a session‑by‑session
with a session-by-session sparkline against your baseline median, profile switching (Balanced · sparkline against your baseline, profile switching, component health, hook‑conflict detection, and
Maximum savings · Index-only), component health, hook-conflict detection, and one-click one‑click repair. Prefer the terminal? It's all a CLI:
**Repair install** for the day a Claude Code update breaks a hook — that day will come.
Prefer the terminal? The same product is a CLI:
```console ```console
$ tokengem hosts # which agents are on this machine, and what TokenGem can do for each $ tokengem hosts # which agents are on this machine, and what TokenGem can do for each
$ tokengem doctor # pre-flight checks + every hook registration and who owns it $ tokengem doctor # pre-flight checks + every hook registration and who owns it
$ tokengem baseline # mandatory: capture your before-numbers (this one is deliberately slow) $ tokengem baseline # mandatory: capture your before-numbers (deliberately slow, runs once)
$ tokengem install # snapshot → pinned runtimes → one router $ tokengem install # snapshot → pinned runtimes → one router
$ tokengem dashboard # http://127.0.0.1:7801 — six views, bound to localhost only $ tokengem dashboard # http://127.0.0.1:7801 — six views, bound to localhost only
``` ```
## Works with the agents you already run **Works with the agents you already run:** Claude Code · Codex · Cursor · OpenCode · Copilot ·
Gemini CLI · Qwen Code — plus a local **proxy mode** that meters any app calling a model API
directly, forwarding requests untouched and recording only token counts.
Claude Code · Codex · Cursor · OpenCode · Copilot · Gemini CLI · Qwen Code — plus a local ## Pricing
**proxy mode** that meters any app calling a model API directly, forwarding requests untouched
and recording only token counts. Run `tokengem hosts` to see exactly what TokenGem can do for
each agent it finds: route and measure, or measure only.
## What leaves your machine: nothing TokenGem is a **one‑time purchase** — no subscription, no telemetry‑driven upsell, no kill switch.
The ladder is devices, not features: every tier gets the whole product.
All measurement, every ledger entry, and your baseline stay on your Mac. The dashboard binds to | | **Solo** | **Duo** | **Studio** |
`127.0.0.1` only. The license is a signed key verified **locally** — activation works on a plane, |---|---|---|---|
and a server outage can never take your tool away. Every data flow the three upstream tools *can* | One‑time price | **$59** | **$89** | **$239** |
open is listed in one settings view, with switches and a purge button. | Devices | 1 | 3 | 10 |
| Every feature | ✅ | ✅ | ✅ |
| Updates included | 1 year | 1 year | 1 year |
| Licence | **Perpetual** | **Perpetual** | **Perpetual** |
## Honest limits - Your key is verified **offline** — a signed payload checked locally, so activation works on a
plane and a server outage can never take your tool away. A dead laptop doesn't burn a seat.
Every tool should publish these. TokenGem will not help much, and may not help at all, when: - Perpetual means perpetual: the key keeps working on the version you bought, even if you never
update again.
- **Your repos are small.** There's little context to shrink and little to index. - The DMG you download is the **full product** — nothing is crippled while you evaluate it. We'd
- **Your sessions are mostly reasoning**, not file-reading. Compressed `grep` output can't save
what never got read.
- **Your hooks are already hand-tuned.** `tokengem doctor` will show the collisions, but adopt-
or-replace is a decision you get to make — nothing is silently clobbered.
- **You want per-tool dollar attribution.** RTK's character reduction is measured; Graphify and
Ponytail are reported at the confidence the instrumentation actually supports. Where a number
would be invented, there is instead a blank — deliberately.
One published data point so you can calibrate expectations: Ponytail's *own* benchmark reports
**80–94% less code and 42–75% less cost** across five single-shot code-generation tasks, 3 models,
10 runs per cell (median reported). That is their measurement, not yours — yours is what the
ledger is for. And the failure mode is real: the 18%-net-*increase* case above is why the
baseline exists.
## Pricing & license terms
**$59. Once.**
- Perpetual licence to the version you bought — your key keeps working even if you never update.
- Signed key, **verified offline** on up to 3 devices; a dead laptop doesn't burn a seat.
- The DMG you download is the full product — nothing is crippled while you evaluate it. We'd
rather convince you than lock you out. rather convince you than lock you out.
- Uninstall is one command and restores your machine exactly. If the product isn't for you, it
leaves no trace.
<!-- <!--
PURCHASE LINK: wire the checkout URL here when fulfilment goes live (Gate 2, doc 04). PURCHASE LINK: wire the checkout URL here when fulfilment goes live (Gate 2, doc 04).
@ -134,58 +107,73 @@ Questions about licensing → [open an issue](https://github.com/BigMarc/TokenGe
## Install in 90 seconds ## Install in 90 seconds
1. **[Download TokenGem-1.0.0-macOS.dmg](https://github.com/BigMarc/TokenGem/releases/latest)** 1. **[Download TokenGem‑1.0.0‑macOS.dmg](https://github.com/BigMarc/TokenGem/releases/latest)**
(1.6 MB, universal — Apple Silicon and Intel) (1.6 MB, universal — Apple Silicon & Intel, macOS 13+)
2. Open the DMG and drag **TokenGem** to **Applications**. 2. Open the DMG and drag **TokenGem** to **Applications**.
3. First launch: right-click the app → **Open** → **Open** (one time only — the binary is 3. First launch: right‑click → **Open** → **Open** (one time — the binary is ad‑hoc signed while
ad-hoc signed while Developer-ID notarization is pending, so macOS checks you meant it). Developer‑ID notarization is pending). Requires **Node 20+** (`brew install node`); the app
Requires **Node 20+** (`brew install node`) — the app will tell you if it can't find it. tells you if it can't find it.
4. The gem appears in your menu bar. Click it → **Open Dashboard** → follow the four-command 4. Click the gem in your menu bar → **Open Dashboard** → follow the four‑command quick start. The
quick start. The baseline step is the slow one; it's slow because it's reading your real baseline step is the slow one — it's reading your real history, and it only happens once.
history, and it only ever happens once.
## Privacy: nothing leaves your machine
All measurement, every ledger entry, and your baseline stay on your Mac. The dashboard binds to
`127.0.0.1` only. The licence check is a local signature verification — no account, no telemetry.
Every data flow the upstream tools *can* open is listed in one settings view, with switches and a
purge button.
## Honest limits
TokenGem will not help much, and may not help at all, when:
- **Your repos are small** — little context to shrink, little to index.
- **Your sessions are mostly reasoning**, not file‑reading. Compressed `grep` output can't save
what never got read.
- **Your hooks are already hand‑tuned.** `tokengem doctor` shows the collisions, but
adopt‑or‑replace is your decision — nothing is silently clobbered.
- **You want per‑tool dollar attribution.** Numbers are reported at the confidence the
instrumentation supports, and no further.
One published data point to calibrate: Ponytail's *own* benchmark reports **80–94% less code and
42–75% less cost** on single‑shot code‑generation tasks. That's their measurement, not yours —
yours is what the ledger is for. And the failure mode is real: the 18% net‑*increase* case above
is why the baseline exists.
## FAQ ## FAQ
**Why pay for open-source tools?** **Why pay for open‑source tools?**
You're not. The three upstream projects stay free, stay linked below, and stay unmodified — You're not. The upstream projects stay free, linked below, and unmodified — TokenGem never forks
TokenGem never forks them. What you're buying is the part that doesn't exist anywhere else: the them. You're buying the part that doesn't exist anywhere else: the deconfliction router, the
deconfliction router (nobody has published a correct merge order for these three hooks), the baseline‑and‑ledger measurement engine, the rollback guarantee, and the version pins that stop an
baseline-and-ledger measurement engine, the rollback guarantee, and the version pins that stop an
upstream Tuesday from silently breaking your setup. upstream Tuesday from silently breaking your setup.
**Is "buy once" real?**
Yes, with a noun: the licence is perpetual for the version you purchased. There is no subscription,
no telemetry-driven upsell, and no kill switch — verification is a local signature check.
**Will it survive a Claude Code update?** **Will it survive a Claude Code update?**
The hook survives most; when one doesn't, the dashboard shows zero routing events and the The hook survives most. When one doesn't, the dashboard shows zero routing events and **Repair
**Repair install** button re-asserts it idempotently. Upstream tool versions are pinned and only install** re‑asserts it idempotently.
bump after regression tests pass.
**How do I remove it completely?** **How do I remove it completely?**
`tokengem uninstall`, then drag the app to the Trash. Settings snapshots in `~/.tokengem/backups/` `tokengem uninstall`, then drag the app to the Trash. Settings snapshots in `~/.tokengem/backups/`
are restored byte for byte; your global Python/npm/Cargo environments were never touched. are restored byte for byte; your global environments were never touched.
**Where do I report a bug or request an agent?** **Where do I report a bug or request an agent?**
[Issues](https://github.com/BigMarc/TokenGem/issues) — the changelog is public and each release [Issues](https://github.com/BigMarc/TokenGem/issues) — the [changelog](CHANGELOG.md) is public and
notes exactly what changed, including upstream version bumps. each release notes exactly what changed, including upstream version bumps.
## Under the hood — and in our debt ## Under the hood — and in our debt
TokenGem orchestrates three open-source projects as managed, pinned dependencies. They are named TokenGem orchestrates three open‑source projects as managed, pinned dependencies:
here on purpose: you'd find them in five minutes anyway, and you deserve to find them in five
*seconds* instead — after purchase, which is how chargebacks happen.
| Project | What it does in TokenGem | Pinned | License | | Project | What it does in TokenGem | Pinned | License |
|---|---|---|---| |---|---|---|---|
| [RTK](https://github.com/rtk-ai/rtk) | Compresses shell command output before it enters context | v0.48.0 · `fde0a8f` | Apache-2.0 | | [RTK](https://github.com/rtk-ai/rtk) | Compresses shell command output before it enters context | v0.48.0 · `fde0a8f` | Apache‑2.0 |
| [Ponytail](https://github.com/DietrichGebert/ponytail) | Injects a minimal-code ruleset so the agent writes less | v4.9.0 · `0a4dd63` | MIT | | [Ponytail](https://github.com/DietrichGebert/ponytail) | Injects a minimal‑code ruleset so the agent writes less | v4.9.0 · `0a4dd63` | MIT |
| [Graphify](https://github.com/Graphify-Labs/graphify) | Builds a queryable repo knowledge graph so the agent stops re-grepping | v0.9.53 · `33362d9` | Apache-2.0 | | [Graphify](https://github.com/Graphify-Labs/graphify) | Builds a queryable repo knowledge graph so the agent stops re‑grepping | v0.9.53 · `33362d9` | Apache‑2.0 |
Verbatim license texts live in [`LICENSES/`](LICENSES), full notices in Verbatim license texts live in [`LICENSES/`](LICENSES), full notices in
[`THIRD_PARTY_NOTICES.md`](THIRD_PARTY_NOTICES.md). We are grateful to the upstream authors and [`THIRD_PARTY_NOTICES.md`](THIRD_PARTY_NOTICES.md). We're grateful to the upstream authors and
claim no endorsement, partnership, or affiliation with them. TokenGem's router, measurement claim no endorsement, partnership, or affiliation. TokenGem's router, measurement engine,
engine, dashboard, and licensing are original work. dashboard, and licensing are original work.
--- ---
@ -194,4 +182,6 @@ engine, dashboard, and licensing are original work.
**Stop guessing what your token tools saved. **Stop guessing what your token tools saved.
[Download TokenGem](https://github.com/BigMarc/TokenGem/releases/latest) and read it.** [Download TokenGem](https://github.com/BigMarc/TokenGem/releases/latest) and read it.**
<sub>Made with care by <a href="https://github.com/BigMarc">BigMarc</a></sub>
</div> </div>