usefoyer

A private AI second brain with a knowledge graph at the center.

Most personal RAG tools do this:

Documents → Embeddings → LLM

usefoyer does this:

Documents
    ↓
Entities
    ↓
Relationships
    ↓
Knowledge Graph
    ↓
Vector Search
    ↓
LLM

You upload PDFs, lecture slides, notes, articles, and bookmarks. usefoyer extracts the ideas in them — IPv6, 6LoWPAN, IEEE 802.15.4 — and how those ideas connect. Then you ask questions in your own language:

• What did I learn about IPv6 last month?
• Explain the difference between the networking concepts I studied.
• How does 6LoWPAN relate to IEEE 802.15.4?

Answers come from your material, with citations and the slice of the graph that was used.

Everything lives on disk in data/. Nothing is sent anywhere unless you add an OpenAI key or point usefoyer at a local Ollama server.

---

Why a graph

Vector search is good at “find me the paragraph that sounds like this question.” It is weak at structure.

A lecture that covers IPv6 is not a bag of similar sentences. It is a tree:

IPv6
 ├── Header
 ├── Fragmentation
 ├── Addressing
 └── 6LoWPAN
       └── IEEE 802.15.4

usefoyer stores that tree. When you ask about IPv6, retrieval can walk to Header, Fragmentation, Addressing, and 6LoWPAN even if those words never appear in the question. That is the difference between a search box and a second brain.

---

Quick start

Requires Python 3.12+ and Node 18+.

.\start.ps1

The script creates a virtualenv, installs dependencies, and opens two terminals:

Service	URL
App	http://127.0.0.1:6173
API	http://127.0.0.1:6000

First run takes a few minutes while packages install.

Deploy (single process)

Build the UI, then serve it from the API:

.\start-prod.ps1

Linux / macOS:

chmod +x start-prod.sh
./start-prod.sh

That runs npm run build in frontend/ and starts uvicorn on http://127.0.0.1:6000 (0.0.0.0, so it is reachable on the LAN). Open that URL for the landing page; Open library goes to the app.

Manual equivalent:

cd frontend
npm install
npm run build
cd ..
.\.venv\Scripts\python -m uvicorn app.main:app --app-dir backend --host 0.0.0.0 --port 6000

Optional CORS for a separate UI origin: set LUMEN_CORS_ORIGINS to a comma-separated list.

Deploy on Railway (recommended)

The repo includes a production Dockerfile and railway.toml so Railway builds the React UI and runs the FastAPI server as one always-on service.

1. Push this repo to GitHub.
2. In Railway: New Project → Deploy from GitHub repo.
3. Attach a Volume mounted at /data (persists SQLite + uploads).
4. Set variables (optional):
   • LUMEN_DATA_DIR=/data (already the Docker default)
   • PORT is set by Railway automatically
5. Generate a public domain under Settings → Networking.

Health check: GET /api/health.

Local Docker check:

docker build -t usefoyer .
docker run --rm -p 6000:6000 -v usefoyer-data:/data usefoyer

This is a shared demo brain (one database, no per-user accounts). Do not put a personal OpenAI key in Settings on a public instance unless you accept the cost risk.

Manual start

python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r backend\requirements.txt

cd frontend
npm install
cd ..

Terminal 1 — API:

.\.venv\Scripts\python -m uvicorn app.main:app --reload --app-dir backend --port 6000

Terminal 2 — UI:

cd frontend
npm run dev

---

Five-minute tour

1. Open the landing page, then Open library, and click Load IPv6 lecture.
   That ingests sample/ipv6-wireless-networks.md and stamps it in the previous calendar month, so time-aware questions work immediately.
2. Open Knowledge graph.
   Drag nodes, scroll to zoom, click a concept to see relationships, quotes, and source documents. Double-click the canvas to clear focus.
3. Open Ask and try:
   • What did I learn about IPv6 last month?
   • Explain the difference between the networking concepts I studied.
   • How does 6LoWPAN relate to IEEE 802.15.4?
4. Drop your own PDFs, .pptx slides, Word files, or Markdown onto the library. Paste a URL to save a bookmark. Write a note in place.

Without a language-model key, usefoyer still builds a graph from headings, bold terms, and technical patterns, and answers from retrieved passages plus graph facts. Add a model in Settings when you want synthesized writing and richer entity extraction.

---

What you can ingest

Source	How	Notes
PDF	Drag onto Library	Text extracted with PyMuPDF, page-aware chunks
Lecture slides (.pptx)	Drag onto Library	Slide-by-slide text, including tables
Word (.docx)	Drag onto Library	Paragraphs and tables
Markdown / plain text	Drag, or New note	Best input for a clean graph — use headings and bold terms
HTML	Drag	Article text extracted from the page
Bookmark	Paste a URL	Fetches the page (or PDF) and stores the readable text

Maximum file size is 40 MB.

Well-structured notes produce a better graph. Headings become hierarchy (has_part). Sentences like “6LoWPAN depends on IEEE 802.15.4” become typed edges.

---

How a document becomes knowledge

file / URL / note
        │
        ▼
   text extraction
        │
        ▼
   chunking (~1400 chars, overlap)
        │
        ├──► embeddings  ──► chunk index
        │
        └──► entity & relation extraction
                    │
                    ▼
              knowledge graph
              (entities + typed edges)
                    │
                    └──► entity embeddings

Entities are the named ideas in your notes: protocols, technologies, methods, people, places.
Relationships are typed: has_part, part_of, depends_on, is_a, implements, used_by, evolved_from, and similar.

New documents merge into the existing graph. “IPv6” from a lecture and “IPv6” from a later article become one node, with more mentions and more edges.

---

How a question is answered

1. Embed the question.

2. Vector-search passage chunks (with a small lexical boost for matching terms).

3. Vector-search entities, and boost exact name hits.

4. Expand the graph one hop from those entities, so neighbors and typed links come along.

5. Time filter, if the question asks for one:

   Phrase	Window
   yesterday / today	that day
   this week / last week	ISO week
   this month / last month	calendar month
   last year / in 2024	calendar year
   2024-03	that month

6. Generate an answer from graph facts + passages, with [1], [2] citations.
   If no model is configured, usefoyer returns an extractive answer: the graph connections and the quoted chunks.

The Ask sidebar shows the retrieved subgraph, the entities that fired, and links back into the source documents.

---

Models (optional)

Open Settings.

Mode	What you get
No key (default)	Local hashed embeddings, heuristic graph from headings / bold / patterns, extractive answers. Fully offline.
OpenAI	gpt-4o-mini + text-embedding-3-small by default. Stronger extraction and written answers. Optional custom base URL for a compatible proxy.
Ollama	Fully local. Default chat llama3.2, embeddings nomic-embed-text, URL http://127.0.0.1:11434.

The API key is stored in the local SQLite database, not in the repo. Changing models applies to new ingest and new questions; already-embedded chunks keep their existing vectors until you re-ingest.

---

Privacy

• Database: data/lumen.db
• Uploaded files: data/uploads/
• Both are gitignored.

usefoyer does not phone home. Network is used only to:

• fetch a bookmark URL you pasted
• call OpenAI, if you saved a key
• call Ollama, if you selected it

---

Project layout

Personal brain agent/
├── backend/app/          FastAPI knowledge engine
│   ├── main.py           HTTP API
│   ├── ingest.py         upload → parse → chunk → embed → graph
│   ├── extract.py        entity / relationship extraction
│   ├── retrieve.py       hybrid retrieval + answers
│   ├── embeddings.py     OpenAI, Ollama, or local hashing
│   ├── llm.py            chat completions
│   ├── text_extract.py   PDF, PPTX, DOCX, HTML, URLs
│   └── database.py       SQLite schema
├── frontend/             Vite + React UI
│   └── src/pages/        Landing, Library, Graph, Ask, Settings
├── sample/               demo IPv6 lecture
├── data/                 created at runtime (not in git)
├── Dockerfile            production image (UI build + API)
├── railway.toml          Railway deploy settings
├── start.ps1             one-command launch on Windows
├── start-prod.ps1        build UI and serve on :6000 (Windows)
├── start-prod.sh         build UI and serve on :6000 (Linux/macOS)
└── README.md

API surface

Method	Path	Purpose
GET	/api/health	liveness
GET	/api/stats	document / entity / relation counts
GET/PUT	/api/settings	provider and models
GET/POST/DELETE	/api/documents…	library, upload, note, bookmark, demo
GET	/api/graph	nodes and edges (?q= to focus)
GET	/api/entities/{id}	inspector payload
POST	/api/ask	question → answer + sources + subgraph

---

Troubleshooting

The graph is empty.
Ingest something from Library. Status on each card should become ready. If it stays error, open the document — the message is stored on the record.

Answers feel thin.
That is expected without a model: you are seeing graph facts and quoted passages. Add OpenAI or Ollama in Settings, then ask again. Re-ingest a document if you want the LLM extractor (not just the heuristic) to rebuild its entities.

“Last month” returned nothing.
Time filters use the document’s ingest date, not dates mentioned inside the text. The demo lecture is backdated on purpose; your own uploads are dated now.

Bookmark fetch failed.
Some sites block unnamed clients. Download the page or PDF and drop the file instead.

Port already in use.
API expects 6000, UI expects 6173. Stop the old uvicorn / vite processes and run .\start.ps1 again.

Want a clean slate.
Delete data/lumen.db and data/uploads/, then restart the API. The next ingest starts a new graph.

---

Design choices

• SQLite, not a hosted vector DB. Personal scale is thousands of chunks. Cosine search over stored float32 embeddings is enough, and the whole brain is one folder you can copy.
• Graph first, embeddings second. The index is how you find; the graph is how you understand.
• Works without an API key. A second brain that only functions with a cloud bill is a demo. Heuristic extraction and hashed embeddings keep Library, Graph, and Ask usable offline. A model makes them sharper, not possible.
• Citations over vibes. Every answer is grounded in chunks and typed edges from documents you actually uploaded.