EnZIMErgent

ZIM Reader — Neural Archive Interface. A privacy-respecting, offline-first reader for ZIM files (Wikipedia, Stack Exchange, Project Gutenberg, Wiktionary, and ~hundreds of other human-knowledge corpora available as static archives) with cross-document annotations, on-device AI inference, and an explicit cloud-opt-in escape hatch. Cyberpunk Neon visual language; the user is framed as a Netrunner working a neural archive.

• Namespace — mba.robin.enzimergent
• Stack — FastAPI + Motor/MongoDB backend, Expo React Native (SDK 54) frontend
• Status — pre-release; 65 tasks tracked dependency-ordered in CHECKLIST.md
• Version — 1.0.20311 (MAJOR.MINOR.BUILD, stamped via scripts/stamp-version.sh)

---

Contents

1. What it is, and why
2. What makes EnZIMErgent unique (USPs)
   • Dynamic storage-aware predownload planning
   • W3C-range annotation and highlighting
   • Shareable annotation sidecars
   • Drawing on the page
   • Voice notes
   • Chat-with-ZIM (AI-mediated reading)
3. Ethos
4. Underlying theory
5. Build & run
   • Backend
   • Frontend
   • Verification
   • Versioning
   • CI
   • Git
6. Repo map
7. Working in this repo with an AI agent
8. License

---

What it is, and why

ZIM files are a beautiful artifact: an entire encyclopedia, a stack-exchange dump, or a Project-Gutenberg shelf — compressed, indexed, and self-contained in a single file you can sideload onto a phone. They are how human knowledge travels offline. The existing reader ecosystem (Kiwix, etc.) treats them mostly as static lookup. EnZIMErgent treats them as the substrate for an active reading practice — annotation, cross-referencing, AI-mediated exploration — without giving up any of the offline guarantees that make ZIM worth using in the first place.

---

What makes EnZIMErgent unique (USPs)

Every other ZIM reader the project has surveyed treats archives as either "download whole or not at all" and treats articles as read-only. EnZIMErgent breaks both of those constraints.

1. Dynamic storage-aware predownload planning

The hard blocker EnZIMErgent removes. Full English Wikipedia (wikipedia_en_all_maxi) is ~100 GB. Full Stack Exchange dumps run in similar territory. On a 64 GB phone with 12 GB free, every other reader the project has tested forces a binary choice: download the entire archive (impossible) or do nothing.

EnZIMErgent's predownload planner asks two questions before any bytes are fetched:

1. What's the storage budget? (default: read free disk space, leave a safety margin; user can override.)
2. What's the user's interest profile? (categories, language depth, media inclusion, article-length cap.)

It then computes a depth-breadth tradeoff against the budget and shows the user a plan: "With 12 GB you can have all of Mathematics + Computer Science + Physics down to depth-3 sub-categories with media stripped, OR all of History + Geography to depth-2 with media included, OR a thin slice of everything to depth-1." The user picks; the filtered download fetches only what's planned. Backend implementation: POST /api/zim/plan (calls calculate_storage_plan), then POST /api/zim/download-filtered (_do_filtered_download orchestrates the streamed chunked fetch using ZimStreamReader's HTTP-Range pattern).

Result: a 100 GB archive is usable on a 12 GB free device. No other ZIM reader the project has surveyed exposes this.

2. W3C-range annotation and highlighting

Section-based highlighting (the legacy approach in most readers) cannot represent "highlight from the third sentence of paragraph 4 to the second sentence of paragraph 6" — yet that's exactly the granularity a reader wants. EnZIMErgent's annotation layer (AnnotationCanvas, G6.5) anchors each annotation to a W3C Range over the article's normalised DOM:

• Highlights land on arbitrary text spans, including inside a single paragraph and across paragraph boundaries.
• A robust resolver (RobustAnchorResolver) survives ZIM article re-renders by tolerating whitespace and minor markup changes; if anchoring genuinely fails, the annotation is preserved with a "needs reattach" badge rather than silently lost.
• migrateV1Anchor upgrades section-based annotations from the legacy schema (annotations created before G6.5 are not orphaned).

3. Shareable annotation sidecars

Annotations live as sidecar documents keyed by (zim_id, user_id, anchor) — not embedded in the ZIM. That makes them:

• Exportable: GET /api/annotations/{zim_id}/export returns the full annotation set for an archive as a portable JSON document.
• Shareable: a user's annotations on Wikipedia can be sent to another user with the same ZIM file; the receiver imports the sidecar and sees the highlights, drawings, and voice notes anchored on the same passages.
• Composable: a study group can build a shared annotation layer on a textbook ZIM without modifying the source archive.

4. Drawing on the page

The annotation layer includes a Skia-backed drawing canvas (DrawingLayer, BezierStroke) synchronised to the same range coordinate space as highlights. A drawing made over a passage stays anchored to that passage through scrolls, re-renders, and font-size changes. The legacy PanResponder + SVG implementation is being replaced (G6.5.6) because it drops touches inside scrollable parents — a structural failure of PanResponder that React Native Gesture Handler resolves.

5. Voice notes

VoiceRecorder (using expo-audio, the SDK 54 successor to the deprecated expo-av) attaches a recorded audio clip to a W3C range like any other annotation type. Voice notes ride the same sidecar export/share path as text and drawing annotations — a study group can leave spoken margin-comments on a textbook for each other.

6. Chat-with-ZIM (AI-mediated reading)

POST /api/inference/chat accepts a passage + a question and routes the query to:

1. The on-device model by default (rung 1–4 of the AI ladder, see theory section), so the conversation never leaves the device.
2. A cloud frontier model (via OpenRouter, openrouter.ts) only if the user has explicitly enabled cloud inference (AD-09) and explicitly sent this query to it. There is no implicit fallback.

The chat has access to the loaded ZIM article as context, so questions like "summarise the dispute between section 3 and the references in section 5" get answered against the actual archive content, not the model's training data alone.

---

Ethos

• Offline-first is not a fallback. Every primary capability — read, annotate, query an LLM about a passage, transcribe voice, hear a passage read back — works without network. Cloud is a terminal-rung escape for tasks the device genuinely cannot do, not the default path.
• User agency over engagement. No telemetry without explicit opt-in. No dark patterns nudging cloud usage. No lock-in: ZIM library, annotations, and on-device models are user-owned files, exportable.
• Quality over velocity. The project is built without runway pressure; "ship now, clean up later" is rejected. Architecture (DOCS/ARCHITECTURE.md) and execution contract (CHECKLIST.md) are pre-committed; coders work from them, not from improvised cleanup later.
• The annotation system is the USP. Section-based highlighting is a failure mode the project explicitly rejects — see USP §2.

---

Underlying theory

On-device AI ladder (AD-07). Devices vary by ~1000× in inference capability. The reader runs a probe at first launch, picks the largest model variant the device can sustain at acceptable latency, and falls through a 5-rung ladder if the chosen rung underperforms:

Rung	Variant	Approx. size	Target
1	gemma-4-e4b	2–3 GB int4	Flagship (Snapdragon 8 Gen 3+, Tensor G3+, iPhone Pro recent)
2	gemma-4-e2b	1.5–3 GB int4	Mid-range / 2–3yr flagships
3	qwen-3_5-1_5b or lfm-1_3b	600–800 MB int4	Older / constrained
4	bonsai-1bit (bundled)	125–250 MB	Very constrained; bundled emergency fallback
5	cloud-only	n/a	Terminal — requires explicit opt-in (AD-09)

The bundled rung-4 model means the app ships with a working AI even on a device that can't reach the network. The cloud rung is reachable only after the user explicitly enables it; no implicit fallback.

Quantization policy (AD-12). Only int4, int8, and 1bit variants are accepted by the model manifest schema. fp16, bf16, and fp32 are rejected at validation time. This is a deliberate constraint — full-precision weights are storage-prohibitive on the target hardware, and the test matrix gets unmanageable if every device must work with every quantization.

Storage planning (AD-08). ZIM files are large (English Wikipedia ≈ 100 GB). The HTTP-Range pattern in zim_parser.py is reused by the model downloader so that on-device model downloads benefit from the same resumable, partial, budget-aware mechanics that drive the predownload planner (USP §1).

---

Build & run

Backend (FastAPI on :8000, Motor → LAN MongoDB)

cd backend
pip install -r requirements.txt
uvicorn server:app --reload --port 8000

Backend depends on a reachable MongoDB. Project default is the LAN-hosted instance from ~/.claude/CLAUDE.md. Override via env vars (MONGO_URL, etc.) if running standalone.

Frontend (Expo dev server, runs on phone via Expo Go or a dev build)

cd frontend
yarn install
yarn start

Verification

cd backend && pytest tests/ -v          # 18 backend tests; 26 once G6 lands
cd frontend && yarn tsc --noEmit         # TypeScript type-check
cd frontend && yarn lint                 # eslint

UI changes additionally require in-situ verification — the project uses the mcp__claude-in-chrome__* MCP tools, not Playwright (per AD-16). "It should work" is not a result; observed runtime behaviour is.

Versioning

Version stamp lives in version.txt as MAJOR.MINOR.BUILD (currently 1.0.20311). Stamping is automatic via scripts/stamp-version.sh:

• MAJOR is judgment-based — bumped at the start of a major undertaking.
• MINOR auto-increments on every coherent codebase change unit (pre-CI, runs on every change whether CI fires or not).
• BUILD = epoch_minutes % 100000, computed once per CI run.

Android versionCode = MAJOR × 100_000 + MINOR. BUILD is deliberately excluded from versionCode because Play Store requires monotonic versionCode and BUILD wraps every ~69 days. Full conventions in ~/.claude/BUILD_CONVENTIONS.md.

CI

Forgejo Actions, self-hosted runners. Workflow target labels: linux-amd64 (verify jobs) and android (Android build), per AD-15. Never bare self-hosted. Workflow lives at .forgejo/workflows/build.yml once G7.3 lands.

Git

git clone git@git.robin.mba:rcheung/EnZIMErgent.git

SSH only — HTTPS (https://git.robin.mba/...) routes through nginx-ui and 504s on large pushes.

---

Repo map

EnZIMErgent/
├── backend/                FastAPI + Motor/MongoDB Python service
│   ├── server.py             Main app — 36+ routes, ~1379 lines
│   ├── zim_parser.py         Clean-room ZIM stream parser
│   ├── tests/                Pytest suite
│   ├── config/               model_manifest.json (G5.5)
│   └── requirements.txt
├── frontend/               Expo React Native SDK 54, TypeScript
│   ├── app/                  Expo Router — (tabs)/, chat.tsx, reader.tsx
│   ├── src/
│   │   ├── api/              apiCall / apiCallRaw fetch wrappers
│   │   ├── components/       UI components incl. AnnotationCanvas (G6.5)
│   │   ├── contexts/         AuthContext, PurchasesContext, SettingsContext
│   │   ├── services/         openrouter (cloud LLM), onDeviceAi/ (G5.2)
│   │   └── theme/            Cyberpunk Neon palette + ThemeContext
│   └── package.json
├── DOCS/
│   ├── ARCHITECTURE.md     Authoritative release-grade spec — entity
│   │                       table, ADs, data flows. The vocabulary.
│   └── architecture/       PlantUML + Mermaid diagram sources
├── CHECKLIST.md            Executable contract — every task with file
│                           paths, acceptance criteria, verify commands.
│                           The work.
├── PROJECT_INDEX.md/.json  Machine-readable codebase map. Read first.
├── CLAUDE.md               Project-specific agent rules (index-first
│                           gate, browser tooling, runner labels)
├── memory/PRD.md           Original product requirements
├── design_guidelines.json  Cyberpunk Neon design system tokens
├── scripts/                Build/version stamping utilities
├── version.txt             Canonical MAJOR.MINOR.BUILD source of truth
└── .emergent/              Emergent.sh preview config

Key file roles

File	Role
DOCS/ARCHITECTURE.md	The authoritative spec. Entity table is the project's vocabulary; every CHECKLIST task draws names from it verbatim.
CHECKLIST.md	The contract. CODER mode reads only this file. Each task carries its own verification command.
PROJECT_INDEX.md/.json	The map. Read before any source file (CLAUDE.md I-1). Regenerated only via /sc:index-repo.
CLAUDE.md	Project-scoped agent rules. Inherits from ~/.claude/CLAUDE.md (global).
version.txt	The single source of truth for the version triple — every platform manifest reads from this.

---

Working in this repo with an AI agent

This repo is set up for the four-phase workflow (MAP → ARCHITECT → PLAN → CODE) documented in ~/.claude/CLAUDE.md. The four phases never overlap:

1. MAP — read PROJECT_INDEX.md/.json. No source files.
2. ARCHITECT — write/update DOCS/ARCHITECTURE.md (entity table is mandatory). No code editing.
3. PLAN — translate the architecture into CHECKLIST.md tasks. Each task cites entities by exact name from the entity table. No code editing.
4. CODE — read CHECKLIST.md only. Execute tasks in order. Verify each. If a task lacks information, halt and return to PHASE 2 — never improvise.

Task progress markers: [ ] not started, [/] in progress, [X] code written, ✅ verified by running the command and observing acceptance. "It should work" is [X] at most; ✅ requires observed output.

The PROJECT_INDEX outranks this README by design — if any fact in here disagrees with the index, the index wins.

---

License

Private. Not currently distributed.