BigMarc 8a9ffb55af Add real on-device LLM (MLX) for rewrite / tone / Ask ...

Replace the MLXEngine stub with real Qwen2.5-3B-4bit inference via
mlx-swift-examples (2.25.9): loads the local model snapshot on disk (no
network), chat-templated generation per AIAction, and tone classification.
Wired behind the existing AIEngine seam; AppState resolves the model dir
through ModelManager.

Build notes:
- arm64-only now (MLX is Metal / Apple-Silicon only; matches the spec).
- swift build (CLI) cannot compile mlx-swift's .metal kernels, so the
  metallib is generated once via scripts/build-metallib.sh (xcodebuild) and
  vendored at AppSupport/mlx.metallib; scripts/build.sh ships it as
  Contents/Resources/mlx-swift_Cmlx.bundle/Contents/Resources/default.metallib
  (the path mlx resolves, and it codesigns cleanly).
- A hidden --mlx-selftest entrypoint runs one headless generation to verify
  the runtime end-to-end.

Verified: the signed app's self-test produces a real rewrite; full test
suite green (the MLX integration test is gated behind GRAMMAGEM_MLX_TEST).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

2026-06-18 07:34:02 +02:00

2.7 MiB

Raw Permalink History

View Raw