mirror of
https://github.com/BigMarc/TokenGem.git
synced 2026-10-13 17:03:10 +02:00
Compare commits
No commits in common. "main" and "v1.0.0" have entirely different histories.
228
README.md
228
README.md
@ -2,102 +2,129 @@
|
||||
|
||||
# 💎 TokenGem
|
||||
|
||||
### Make your Claude Code token tools cooperate — and prove what they saved
|
||||
### Your Claude Code token tools are arguing with each other.
|
||||
### TokenGem makes them cooperate — and shows you what they actually saved.
|
||||
|
||||
**TokenGem is the orchestration + measurement layer for AI coding-agent token tools.**
|
||||
One router instead of three fighting hooks. A baseline frozen from **your own sessions**.
|
||||
A verdict in your menu bar — green or red, measured, never estimated.
|
||||
[](https://github.com/BigMarc/TokenGem/releases/latest)
|
||||
[](https://github.com/BigMarc/TokenGem/releases)
|
||||
[](https://github.com/BigMarc/TokenGem/releases/latest)
|
||||
[](https://github.com/BigMarc/TokenGem/releases/latest)
|
||||
[](#pricing--license-terms)
|
||||
|
||||
[](https://github.com/BigMarc/TokenGem/releases/latest)
|
||||
[](https://github.com/BigMarc/TokenGem/releases/latest)
|
||||
[](#pricing)
|
||||
**[⬇️ Download the latest DMG](https://github.com/BigMarc/TokenGem/releases/latest)**
|
||||
|
||||
<sub>Native macOS menu‑bar app · Node CLI · localhost‑only dashboard — a 1.6 MB DMG, no installer, no account, no daemon phoning home</sub>
|
||||
A 1.6 MB download. No installer, no account, no daemon phoning home.
|
||||
|
||||
</div>
|
||||
|
||||
---
|
||||
|
||||
## Why TokenGem?
|
||||
You already know context is what your Claude Code bill is made of. You may even run the
|
||||
open-source tools that shrink it — **RTK** to compress shell output, **Ponytail** to keep the
|
||||
agent's code minimal, **Graphify** to stop it re-grepping your repo.
|
||||
|
||||
Context is what your Claude Code bill is made of, and the open‑source tools that shrink it are
|
||||
excellent — **RTK** compresses shell output, **Ponytail** keeps generated code minimal, **Graphify**
|
||||
stops the agent re‑grepping your repo. Here's the part nobody tells you: **installed side by side,
|
||||
they fight.** All three register a `PreToolUse` hook on Bash, so together they silently no‑op,
|
||||
double‑process, or undo each other's work — while your hooks keep *looking* installed.
|
||||
Here's the part nobody tells you: **installed side by side, they fight.** All three register a
|
||||
`PreToolUse` hook on Bash. One rewrites `grep` into its own command, another intercepts the same
|
||||
command and nudges toward its index, a third injects rules at session start. Run together, they
|
||||
silently no-op, double-process, or undo each other's work — and your hooks keep *looking* installed.
|
||||
|
||||
And even when they work, **savings aren't additive**. RTK's own issue tracker documents a case
|
||||
where compression cut input tokens and *raised* total cost 18% — shorter input provoked longer
|
||||
answers, and output tokens cost roughly **5× input** on every current Claude model. TokenGem fixes
|
||||
the conflict, then measures the result on your machine, against your own before‑numbers.
|
||||
And even when they work: **savings are not additive.** RTK's own issue tracker documents a case
|
||||
where the filter cut input tokens and *raised* total cost by 18% — because compressed output made
|
||||
the agent ask more follow-up questions, and output tokens cost roughly **5× input** on every
|
||||
current Claude model.
|
||||
|
||||
> 🧭 **One router, one hook** · 📊 **Measured vs your baseline** · 🚨 **Red when net‑negative** · 🍎 **Native macOS app**
|
||||
TokenGem is the layer that fixes both problems, and it is the only one that *proves* which one it
|
||||
fixed for you.
|
||||
|
||||
## Features
|
||||
## What TokenGem actually does
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| 🧭 **One hook instead of three** | A single `PreToolUse` router dispatches each command to exactly one owner — Graphify's index when fresh, RTK where an equivalent exists, untouched pass‑through otherwise. Nothing double‑processes silently. |
|
||||
| 📊 **Ground‑truth measurement** | A mandatory baseline from your last 20 real sessions, then a local ledger of every routed command: tokens in, tokens out, **net cost per assistant turn**. |
|
||||
| 🚨 **The net‑negative alarm** | If compression is provoking more output than it saves, the menu bar turns red and names the profile to switch to. A tool that admits when it isn't helping is the one worth paying for. |
|
||||
| 📦 **Pinned, isolated installs** | Upstream tools run at audited, pinned versions inside `~/.tokengem/runtime/` — never your global Python, npm, or Cargo. Bumps ship only after regression tests pass. |
|
||||
| 🪄 **Profiles** | Balanced · Maximum savings · Index‑only — switch from the menu bar; the ledger records which was active for every event. |
|
||||
| 🩺 **Doctor & repair** | `tokengem doctor` shows every hook registration and who owns it; one‑click **Repair install** for the day a Claude Code update breaks a hook. |
|
||||
| 🗑️ **One‑button exit** | `tokengem uninstall` restores your pre‑install settings **byte for byte** and deletes the managed runtimes. No trace. |
|
||||
| 🧭 **One hook instead of three** | TokenGem writes a *single* `PreToolUse` hook — its router. Each command is dispatched to exactly one owner: Graphify's index when it's fresh, RTK when there's an equivalent, untouched pass-through otherwise. Every event is recorded, so nothing double-processes silently. |
|
||||
| 📦 **Pinned, isolated installs** | The three upstream tools are installed at audited, pinned versions into `~/.tokengem/runtime/` — never your global Python, npm, or Cargo environments. A bump ships only after TokenGem's regression suite passes. |
|
||||
| 📊 **Ground-truth measurement** | Before anything changes, TokenGem reads your agent's own session records and freezes a baseline. Every session afterwards is measured against it: tokens in, tokens out, **net cost per assistant turn** — the number that survives session-length skew. |
|
||||
| 🚨 **The net-negative alarm** | If compression is provoking more output than it saves, TokenGem says so — in red, in your menu bar — and names the profile to switch to. A tool that admits when it isn't helping is the one worth paying for. |
|
||||
| 🗑️ **One-button exit** | `tokengem uninstall` restores your pre-install settings **byte for byte** and deletes the managed runtimes. Your machine is exactly as it was. |
|
||||
|
||||
## We don't estimate your savings. We read them.
|
||||
|
||||
Most tools in this category show a percentage computed once, on someone else's machine, and pinned
|
||||
to a README. TokenGem *sells* measurement, so it measures:
|
||||
Most tools in this category show you a percentage they computed once, on their machine, and
|
||||
pinned to a README. TokenGem can't afford that honesty problem — it *sells* measurement, so it
|
||||
measures:
|
||||
|
||||
1. **`tokengem baseline`** — mandatory, before install. Reads your real session history and freezes
|
||||
the median cost per turn. Install refuses to run without it.
|
||||
2. **Every routed command lands in a local ledger** — raw bytes, delivered bytes, which path
|
||||
1. `tokengem baseline` — mandatory, before install. Reads your last 20 real sessions per agent and
|
||||
freezes the median cost per turn. **Install refuses to run without it**, because a savings
|
||||
figure with nothing to compare against is a marketing artifact, not a measurement.
|
||||
2. Every routed command lands in a local ledger: raw output bytes, delivered bytes, which path
|
||||
handled it, which profile was active.
|
||||
3. **The Savings view reports the net delta against your baseline** — and attribution only where
|
||||
the instrumentation supports it. The three tools' savings are never summed, because they don't
|
||||
compose. Where a number would be invented, there is a blank — deliberately.
|
||||
3. The Savings view reports the **net delta against your baseline** — and attribution only where
|
||||
the instrumentation supports it. Graphify redirects are counted, not invented; Ponytail's
|
||||
effect is reported as an aggregate. **The three tools' savings are never summed**, because
|
||||
they don't compose.
|
||||
|
||||
What shows up in your menu bar is the measured verdict. Not a projection. Your numbers, your
|
||||
machine, your proof.
|
||||
What shows up in your menu bar is the measured verdict — cost per turn vs. your own baseline,
|
||||
green or red. Not a projection. Your numbers, your machine, your proof.
|
||||
|
||||
## Menu bar or terminal — same product
|
||||
## The menu-bar app
|
||||
|
||||
The DMG gives you the native Mac app: a gem showing the measured delta, a session‑by‑session
|
||||
sparkline against your baseline, profile switching, component health, hook‑conflict detection, and
|
||||
one‑click repair. Prefer the terminal? It's all a CLI:
|
||||
The DMG gives you the native Mac app: a gem in your menu bar showing the measured delta, a panel
|
||||
with a session-by-session sparkline against your baseline median, profile switching (Balanced ·
|
||||
Maximum savings · Index-only), component health, hook-conflict detection, and one-click
|
||||
**Repair install** for the day a Claude Code update breaks a hook — that day will come.
|
||||
|
||||
Prefer the terminal? The same product is a CLI:
|
||||
|
||||
```console
|
||||
$ tokengem hosts # which agents are on this machine, and what TokenGem can do for each
|
||||
$ tokengem doctor # pre-flight checks + every hook registration and who owns it
|
||||
$ tokengem baseline # mandatory: capture your before-numbers (deliberately slow, runs once)
|
||||
$ tokengem baseline # mandatory: capture your before-numbers (this one is deliberately slow)
|
||||
$ tokengem install # snapshot → pinned runtimes → one router
|
||||
$ tokengem dashboard # http://127.0.0.1:7801 — six views, bound to localhost only
|
||||
```
|
||||
|
||||
**Works with the agents you already run:** Claude Code · Codex · Cursor · OpenCode · Copilot ·
|
||||
Gemini CLI · Qwen Code — plus a local **proxy mode** that meters any app calling a model API
|
||||
directly, forwarding requests untouched and recording only token counts.
|
||||
## Works with the agents you already run
|
||||
|
||||
## Pricing
|
||||
Claude Code · Codex · Cursor · OpenCode · Copilot · Gemini CLI · Qwen Code — plus a local
|
||||
**proxy mode** that meters any app calling a model API directly, forwarding requests untouched
|
||||
and recording only token counts. Run `tokengem hosts` to see exactly what TokenGem can do for
|
||||
each agent it finds: route and measure, or measure only.
|
||||
|
||||
TokenGem is a **one‑time purchase** — no subscription, no telemetry‑driven upsell, no kill switch.
|
||||
The ladder is devices, not features: every tier gets the whole product.
|
||||
## What leaves your machine: nothing
|
||||
|
||||
| | **Solo** | **Duo** | **Studio** |
|
||||
|---|---|---|---|
|
||||
| One‑time price | **$59** | **$89** | **$239** |
|
||||
| Devices | 1 | 3 | 10 |
|
||||
| Every feature | ✅ | ✅ | ✅ |
|
||||
| Updates included | 1 year | 1 year | 1 year |
|
||||
| Licence | **Perpetual** | **Perpetual** | **Perpetual** |
|
||||
All measurement, every ledger entry, and your baseline stay on your Mac. The dashboard binds to
|
||||
`127.0.0.1` only. The license is a signed key verified **locally** — activation works on a plane,
|
||||
and a server outage can never take your tool away. Every data flow the three upstream tools *can*
|
||||
open is listed in one settings view, with switches and a purge button.
|
||||
|
||||
- Your key is verified **offline** — a signed payload checked locally, so activation works on a
|
||||
plane and a server outage can never take your tool away. A dead laptop doesn't burn a seat.
|
||||
- Perpetual means perpetual: the key keeps working on the version you bought, even if you never
|
||||
update again.
|
||||
- The DMG you download is the **full product** — nothing is crippled while you evaluate it. We'd
|
||||
## Honest limits
|
||||
|
||||
Every tool should publish these. TokenGem will not help much, and may not help at all, when:
|
||||
|
||||
- **Your repos are small.** There's little context to shrink and little to index.
|
||||
- **Your sessions are mostly reasoning**, not file-reading. Compressed `grep` output can't save
|
||||
what never got read.
|
||||
- **Your hooks are already hand-tuned.** `tokengem doctor` will show the collisions, but adopt-
|
||||
or-replace is a decision you get to make — nothing is silently clobbered.
|
||||
- **You want per-tool dollar attribution.** RTK's character reduction is measured; Graphify and
|
||||
Ponytail are reported at the confidence the instrumentation actually supports. Where a number
|
||||
would be invented, there is instead a blank — deliberately.
|
||||
|
||||
One published data point so you can calibrate expectations: Ponytail's *own* benchmark reports
|
||||
**80–94% less code and 42–75% less cost** across five single-shot code-generation tasks, 3 models,
|
||||
10 runs per cell (median reported). That is their measurement, not yours — yours is what the
|
||||
ledger is for. And the failure mode is real: the 18%-net-*increase* case above is why the
|
||||
baseline exists.
|
||||
|
||||
## Pricing & license terms
|
||||
|
||||
**$59. Once.**
|
||||
|
||||
- Perpetual licence to the version you bought — your key keeps working even if you never update.
|
||||
- Signed key, **verified offline** on up to 3 devices; a dead laptop doesn't burn a seat.
|
||||
- The DMG you download is the full product — nothing is crippled while you evaluate it. We'd
|
||||
rather convince you than lock you out.
|
||||
- Uninstall is one command and restores your machine exactly. If the product isn't for you, it
|
||||
leaves no trace.
|
||||
|
||||
<!--
|
||||
PURCHASE LINK: wire the checkout URL here when fulfilment goes live (Gate 2, doc 04).
|
||||
@ -107,73 +134,58 @@ Questions about licensing → [open an issue](https://github.com/BigMarc/TokenGe
|
||||
|
||||
## Install in 90 seconds
|
||||
|
||||
1. **[Download TokenGem‑1.0.0‑macOS.dmg](https://github.com/BigMarc/TokenGem/releases/latest)**
|
||||
(1.6 MB, universal — Apple Silicon & Intel, macOS 13+)
|
||||
1. **[Download TokenGem-1.0.0-macOS.dmg](https://github.com/BigMarc/TokenGem/releases/latest)**
|
||||
(1.6 MB, universal — Apple Silicon and Intel)
|
||||
2. Open the DMG and drag **TokenGem** to **Applications**.
|
||||
3. First launch: right‑click → **Open** → **Open** (one time — the binary is ad‑hoc signed while
|
||||
Developer‑ID notarization is pending). Requires **Node 20+** (`brew install node`); the app
|
||||
tells you if it can't find it.
|
||||
4. Click the gem in your menu bar → **Open Dashboard** → follow the four‑command quick start. The
|
||||
baseline step is the slow one — it's reading your real history, and it only happens once.
|
||||
|
||||
## Privacy: nothing leaves your machine
|
||||
|
||||
All measurement, every ledger entry, and your baseline stay on your Mac. The dashboard binds to
|
||||
`127.0.0.1` only. The licence check is a local signature verification — no account, no telemetry.
|
||||
Every data flow the upstream tools *can* open is listed in one settings view, with switches and a
|
||||
purge button.
|
||||
|
||||
## Honest limits
|
||||
|
||||
TokenGem will not help much, and may not help at all, when:
|
||||
|
||||
- **Your repos are small** — little context to shrink, little to index.
|
||||
- **Your sessions are mostly reasoning**, not file‑reading. Compressed `grep` output can't save
|
||||
what never got read.
|
||||
- **Your hooks are already hand‑tuned.** `tokengem doctor` shows the collisions, but
|
||||
adopt‑or‑replace is your decision — nothing is silently clobbered.
|
||||
- **You want per‑tool dollar attribution.** Numbers are reported at the confidence the
|
||||
instrumentation supports, and no further.
|
||||
|
||||
One published data point to calibrate: Ponytail's *own* benchmark reports **80–94% less code and
|
||||
42–75% less cost** on single‑shot code‑generation tasks. That's their measurement, not yours —
|
||||
yours is what the ledger is for. And the failure mode is real: the 18% net‑*increase* case above
|
||||
is why the baseline exists.
|
||||
3. First launch: right-click the app → **Open** → **Open** (one time only — the binary is
|
||||
ad-hoc signed while Developer-ID notarization is pending, so macOS checks you meant it).
|
||||
Requires **Node 20+** (`brew install node`) — the app will tell you if it can't find it.
|
||||
4. The gem appears in your menu bar. Click it → **Open Dashboard** → follow the four-command
|
||||
quick start. The baseline step is the slow one; it's slow because it's reading your real
|
||||
history, and it only ever happens once.
|
||||
|
||||
## FAQ
|
||||
|
||||
**Why pay for open‑source tools?**
|
||||
You're not. The upstream projects stay free, linked below, and unmodified — TokenGem never forks
|
||||
them. You're buying the part that doesn't exist anywhere else: the deconfliction router, the
|
||||
baseline‑and‑ledger measurement engine, the rollback guarantee, and the version pins that stop an
|
||||
**Why pay for open-source tools?**
|
||||
You're not. The three upstream projects stay free, stay linked below, and stay unmodified —
|
||||
TokenGem never forks them. What you're buying is the part that doesn't exist anywhere else: the
|
||||
deconfliction router (nobody has published a correct merge order for these three hooks), the
|
||||
baseline-and-ledger measurement engine, the rollback guarantee, and the version pins that stop an
|
||||
upstream Tuesday from silently breaking your setup.
|
||||
|
||||
**Is "buy once" real?**
|
||||
Yes, with a noun: the licence is perpetual for the version you purchased. There is no subscription,
|
||||
no telemetry-driven upsell, and no kill switch — verification is a local signature check.
|
||||
|
||||
**Will it survive a Claude Code update?**
|
||||
The hook survives most. When one doesn't, the dashboard shows zero routing events and **Repair
|
||||
install** re‑asserts it idempotently.
|
||||
The hook survives most; when one doesn't, the dashboard shows zero routing events and the
|
||||
**Repair install** button re-asserts it idempotently. Upstream tool versions are pinned and only
|
||||
bump after regression tests pass.
|
||||
|
||||
**How do I remove it completely?**
|
||||
`tokengem uninstall`, then drag the app to the Trash. Settings snapshots in `~/.tokengem/backups/`
|
||||
are restored byte for byte; your global environments were never touched.
|
||||
are restored byte for byte; your global Python/npm/Cargo environments were never touched.
|
||||
|
||||
**Where do I report a bug or request an agent?**
|
||||
[Issues](https://github.com/BigMarc/TokenGem/issues) — the [changelog](CHANGELOG.md) is public and
|
||||
each release notes exactly what changed, including upstream version bumps.
|
||||
[Issues](https://github.com/BigMarc/TokenGem/issues) — the changelog is public and each release
|
||||
notes exactly what changed, including upstream version bumps.
|
||||
|
||||
## Under the hood — and in our debt
|
||||
|
||||
TokenGem orchestrates three open‑source projects as managed, pinned dependencies:
|
||||
TokenGem orchestrates three open-source projects as managed, pinned dependencies. They are named
|
||||
here on purpose: you'd find them in five minutes anyway, and you deserve to find them in five
|
||||
*seconds* instead — after purchase, which is how chargebacks happen.
|
||||
|
||||
| Project | What it does in TokenGem | Pinned | License |
|
||||
|---|---|---|---|
|
||||
| [RTK](https://github.com/rtk-ai/rtk) | Compresses shell command output before it enters context | v0.48.0 · `fde0a8f` | Apache‑2.0 |
|
||||
| [Ponytail](https://github.com/DietrichGebert/ponytail) | Injects a minimal‑code ruleset so the agent writes less | v4.9.0 · `0a4dd63` | MIT |
|
||||
| [Graphify](https://github.com/Graphify-Labs/graphify) | Builds a queryable repo knowledge graph so the agent stops re‑grepping | v0.9.53 · `33362d9` | Apache‑2.0 |
|
||||
| [RTK](https://github.com/rtk-ai/rtk) | Compresses shell command output before it enters context | v0.48.0 · `fde0a8f` | Apache-2.0 |
|
||||
| [Ponytail](https://github.com/DietrichGebert/ponytail) | Injects a minimal-code ruleset so the agent writes less | v4.9.0 · `0a4dd63` | MIT |
|
||||
| [Graphify](https://github.com/Graphify-Labs/graphify) | Builds a queryable repo knowledge graph so the agent stops re-grepping | v0.9.53 · `33362d9` | Apache-2.0 |
|
||||
|
||||
Verbatim license texts live in [`LICENSES/`](LICENSES), full notices in
|
||||
[`THIRD_PARTY_NOTICES.md`](THIRD_PARTY_NOTICES.md). We're grateful to the upstream authors and
|
||||
claim no endorsement, partnership, or affiliation. TokenGem's router, measurement engine,
|
||||
dashboard, and licensing are original work.
|
||||
[`THIRD_PARTY_NOTICES.md`](THIRD_PARTY_NOTICES.md). We are grateful to the upstream authors and
|
||||
claim no endorsement, partnership, or affiliation with them. TokenGem's router, measurement
|
||||
engine, dashboard, and licensing are original work.
|
||||
|
||||
---
|
||||
|
||||
@ -182,6 +194,4 @@ dashboard, and licensing are original work.
|
||||
**Stop guessing what your token tools saved.
|
||||
[Download TokenGem](https://github.com/BigMarc/TokenGem/releases/latest) and read it.**
|
||||
|
||||
<sub>Made with care by <a href="https://github.com/BigMarc">BigMarc</a></sub>
|
||||
|
||||
</div>
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user