Scaffold project: pyproject, README, license, CI, smoke test

Author: carloscae

.github/workflows/test.yml

@@ -0,0 +1,24 @@
+name: test
+on:
+  push:
+    branches: [main]
+  pull_request:
+    branches: [main]
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - name: Install uv
+        uses: astral-sh/setup-uv@v3
+        with:
+          version: "latest"
+      - name: Set up Python
+        run: uv python install 3.11
+      - name: Install deps
+        run: uv sync --all-extras
+      - name: Lint
+        run: uv run ruff check src tests
+      - name: Test
+        run: uv run pytest --cov=sift --cov-report=term

.gitignore

@@ -216,3 +216,12 @@ __marimo__/
 
 # Streamlit
 .streamlit/secrets.toml
+
+# sift
+.venv/
+.vault-ingest/
+*.egg-info/
+dist/
+.coverage
+.pytest_cache/
+.ruff_cache/

.python-version

@@ -0,0 +1 @@
+3.11

README.md

@@ -0,0 +1,38 @@
+# sift
+
+[![Build](https://github.com/carloscae/sift/actions/workflows/test.yml/badge.svg)](https://github.com/carloscae/sift/actions/workflows/test.yml)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
+[![Python 3.11+](https://img.shields.io/badge/python-3.11+-blue.svg)](https://python.org)
+
+**The working bridge between social media and your second brain.**
+
+Drop a link to a TikTok, an X thread, a YouTube short, an Instagram reel, a Reddit comment — or a voice note, screenshot, PDF — into your Obsidian vault. Get a clean, transcribed, summarized markdown note back. URL in, note out. Local-first, your own AI keys, opinionated.
+
+We maintain extractors for the platforms that fight scraping the hardest, so you don't have to.
+
+## Status
+
+Pre-alpha. v0.1.0 currently supports YouTube, TikTok, and generic article URLs, with OpenRouter as the AI backend. Roadmap → Instagram, X (video + text threads), Reddit, BlueSky, plus a local-model backend that runs fully offline.
+
+## Quickstart
+
+```bash
+pip install sift
+sift init ~/Documents/MyVault
+export OPENROUTER_API_KEY=sk-or-...
+sift add https://tiktok.com/@user/video/123 --now
+```
+
+Open `~/Documents/MyVault/captures/` — your note is there.
+
+## Why not just use \<obvious alternative\>?
+
+Most "save to second brain" tools either stop at clipping the URL (and leave you to actually read/watch the thing later) or work only for clean articles, breaking entirely on video platforms. Sift commits to maintaining the gnarly per-platform extraction layer — the part that breaks every few weeks when TikTok rotates its anti-scrape defenses. That maintenance commitment is the differentiator.
+
+## Configuration
+
+See `vault-ingest.yaml.example` for a full config. Defaults work for most users.
+
+## License
+
+MIT

pyproject.toml

@@ -0,0 +1,48 @@
+[project]
+name = "sift"
+version = "0.1.0"
+description = "The working bridge between social media and your second brain. URL in, clean vault note out."
+readme = "README.md"
+license = { text = "MIT" }
+authors = [{ name = "Carlos Vargas", email = "carlos@carloscae.com" }]
+requires-python = ">=3.11"
+dependencies = [
+    "click>=8.1",
+    "pydantic>=2.6",
+    "pyyaml>=6.0",
+    "httpx>=0.27",
+    "structlog>=24.1",
+    "yt-dlp",
+    "readability-lxml>=0.8.1",
+    "lxml>=5.0",
+]
+
+[project.scripts]
+sift = "sift.cli:main"
+
+[project.urls]
+Homepage = "https://github.com/carloscae/sift"
+Issues = "https://github.com/carloscae/sift/issues"
+
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[tool.uv]
+dev-dependencies = [
+    "pytest>=8.0",
+    "pytest-cov>=5.0",
+    "respx>=0.21",
+    "ruff>=0.5",
+]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+addopts = "-v --tb=short"
+
+[tool.ruff]
+target-version = "py311"
+line-length = 100
+
+[tool.ruff.lint]
+select = ["E", "F", "I", "B", "UP", "SIM"]

src/sift/__init__.py

@@ -0,0 +1,3 @@
+from sift.version import __version__
+
+__all__ = ["__version__"]

src/sift/version.py

@@ -0,0 +1 @@
+__version__ = "0.1.0"

tests/__init__.py

@@ -0,0 +1,25 @@
+from pathlib import Path
+
+import pytest
+
+
+@pytest.fixture
+def tmp_vault(tmp_path: Path) -> Path:
+    """A scratch vault dir with raw/, captures/, .vault-ingest/ created."""
+    (tmp_path / "raw").mkdir()
+    (tmp_path / "captures").mkdir()
+    (tmp_path / ".vault-ingest").mkdir()
+    return tmp_path
+
+
+@pytest.fixture(autouse=True)
+def _clear_extractor_registry():
+    """Drop any registered extractors before each test so module-level state
+    from `sift.extractors` side-effect imports doesn't leak between tests."""
+    try:
+        from sift.extractors.base import clear_registry
+        clear_registry()
+    except ImportError:
+        # base.py doesn't exist yet (Phase 0–1 tests run before Phase 2).
+        pass
+    yield