Here is a project with similar goals (I am the main author), but using late interaction semantic search: https://github.com/dropbox/witchcraft

Hi Jacob, thanks for sharing! Witchcraft looks super interesting—starring the repo now.

The late interaction (ColBERT / MaxSim token-level) approach is fascinating for local search. Retaining per-token interaction matrices gives incredible semantic retrieval precision without losing fine-grained token context.

From an architectural standpoint, the main trade-off I explored with Hillock was moving away from dense token-embedding matrices entirely toward an explicit neuro-symbolic split:

1. Symbolic Grounding: Hard Subject-Predicate-Object (SPO) triples in relational SQLite tables to eliminate vector drift for exact factual memory. 2. Subsymbolic VSA Gating: A 10,000-dimensional Vector Symbolic Architecture (HDC) hypervector space on CPU using subword n-grams and GloVe SimHash projections for <1ms gating and pronoun resolution. 3. Edge VRAM Footprint: Keeping the whole engine under 1.2 GB VRAM on a GTX 1070 by using a CUDA bi-encoder pipeline (Fastcoref + MiniLM + GLiREL) for non-generative document parsing.

I'm curious—how do you manage the index footprint and per-token memory overhead in Witchcraft when scaling to larger local document collections?

Excited to dig deeper into your codebase!

There is no memory overhead, in that everything resides on disk/in the sqlite database, and is read from there on demand. That said, sqlite for blob storage is a bit inefficient, so the next version will use tightly packed structs on disk. I also managed to finetune an embedding down to 96d, so in the coming version the storage for the index is getting close to parity with the size of the text being indexed.

Hi HN,

I built Hillock (https://github.com/roandejager/Hillock), a 100% local, offline neuro-symbolic memory engine designed to replace heavy Vector DBs and token-hungry LLM document ingestion pipelines on edge hardware.

### The Problem with Local RAG Standard local RAG is surprisingly heavy. Running dense vector databases alongside an 8B+ generative LLM just to parse documents and maintain conversational context burns VRAM, chokes mid-range GPUs (like a GTX 1070), and still hallucinates when asked about missing or unverified information.

### The Architecture Instead of using autoregressive LLMs to generate JSON extractions, Hillock decouples memory into three lightweight tiers:

1. SQLite Knowledge Graph (database.py): Ground-truth Subject-Predicate-Object (SPO) triples stored in relational tables. Zero vector drift for factual data. 2. Hebbian Plasticity Engine (plasticity.py): Gradient-free co-activation associative learning that updates synaptic weights between entity nodes across chat turns. 3. 10,000-D VSA/HDC Reservoir (reservoir.py): A Vector Symbolic Architecture hypervector space using Subword N-Grams (3,4,5-grams) and Locality-Sensitive Sign Random Projections (SimHash) over a static 10MB GloVe dictionary. It handles pronoun resolution and context similarity gating on CPU in under 1 millisecond.

### Document Ingestion (TALON Engine) Document parsing uses TALON—a 3-stage CUDA pipeline: - Stage 1: Document coreference preprocessing (Fastcoref) - Stage 2: Bi-Encoder candidate predicate routing (all-MiniLM-L6-v2 in <2ms) - Stage 3: Zero-shot span relation extraction (GLiREL Large)

Ingestion completes in ~3.9–5.0 seconds for a 32-sentence academic document (6.3–8.0 sentences/sec pure GPU rate) using <1.2 GB VRAM on a GTX 1070. Generative LLMs (via local Ollama) are invoked ONLY for final response rendering once a query passes the HDC similarity gate.

### What's New in v0.4 In the v0.4 release series, we tackled zero-shot extraction noise and inverted facts: - O(1) Type-Constrained Schema Validation: Set-based domain/codomain filtering over GLiREL outputs. - Automatic Direction Correction: Detects and swaps inverted extractions (e.g., turning [Budapest born_in John von Neumann] into [John von Neumann born_in Budapest]). - Precompiled Entity Sanitization: Strips possessive artifacts ('s), trailing action verbs, and prepositional tails. - Raw extraction precision improved from 11.5% to 15.5% (+35% cleaner graph).

### Current Benchmark Status (Honest Baseline) On a fixed 32-sentence benchmark (20 answerable queries, 10 hard-negative trick queries): - Retrieval Accuracy: 55.0% - Extraction Recall: 50.0% - Extraction Precision: 15.5% - Fast-Eval Retrieval Duration: ~1.27s for 30 queries (~0.04s per query) - Unanswerable Queries: 0 GPU tokens wasted (CPU similarity gate shuts down LLM calls instantly).

Repo: https://github.com/roandejager/Hillock (AGPL-3.0)

I'd love to hear feedback, criticism, or ideas from the community on hypervector binding, schema constraints, or low-power edge memory architectures!