Skip to content

Commit 9d27515

Browse files
committed
feat(deploy): /get-feeds/ list of the active release's feeds
1 parent fee55e0 commit 9d27515

8 files changed

Lines changed: 223 additions & 9 deletions

File tree

‎Dockerfile‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -21,9 +21,9 @@ COPY src ./src
2121
RUN uv sync --locked --no-dev
2222
COPY deploy ./deploy
2323

24-
# data/ holds fetched releases, the raw archive and the GTFS-RT output (one volume).
24+
# data/ holds fetched releases, the raw archive, the GTFS-RT output and the feed list (one volume).
2525
RUN useradd --system --uid 10001 --home-dir /app obehy \
26-
&& mkdir -p data/releases data/rt-raw data/gtfs-rt \
26+
&& mkdir -p data/releases data/rt-raw data/gtfs-rt data/public \
2727
&& chown -R obehy:obehy data
2828
USER obehy
2929
ENTRYPOINT ["obehy"]

‎PROGRESS.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -227,6 +227,10 @@ rest of §2 and §3 and static acceptance wait on the first GitHub Actions build
227227

228228
## Recent log
229229

230+
- **2026-10-10** — `/get-feeds/` on the server stack: the active release's JDF, filtered JDF and
231+
CZPTT GTFS plus the live GTFS-RT under stable names (those of the MOTIS node's old directory),
232+
with a list page and `feeds.json` written by `obehy release feed-list` after each release
233+
update. Unit-tested; the Caddy routes are not run here (no Docker).
230234
- **2026-10-10** — DPmÚL gaps from the first live night documented and parked: Friday-night
231235
weekend numbers on night lines (DUK-Q16), trip numbers missing from the release (DUK-Q17).
232236
- **2026-10-10** — First live day exposed DUK-Q14/Q15: a bus in the depot in State 3 with its

‎README.md‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -224,9 +224,11 @@ and points `data/releases/active` at what it activated.
224224

225225
CI publishes `ghcr.io/sap1k/obehy:latest` (and `:sha-<commit>`) from the `Dockerfile` for
226226
every green push to main. `deploy/compose.yaml` runs the server from it: PostGIS, migrations,
227-
an hourly release update, the realtime worker, the nightly jobs and Caddy serving
228-
`/gtfs-rt/jdf.pb` and the active release's `/gtfs/jdf.zip` over plain HTTP to your reverse
229-
proxy, which owns TLS and the hostname. Copy `deploy/` to the server and next to it write
227+
an hourly release update, the realtime worker, the nightly jobs and Caddy serving over plain
228+
HTTP to your reverse proxy, which owns TLS and the hostname. `/get-feeds/` lists every feed
229+
under a stable name (`cz-jdf-gtfs.zip`, `cz-jdf-filtered-gtfs.zip`, `cz-czptt-gtfs.zip` from
230+
the active release, `cz-jdf-gtfs-rt.pb` live; `feeds.json` for scripts); the release update
231+
rewrites the list (`obehy release feed-list`). `/gtfs-rt/jdf.pb` is served too. Copy `deploy/` to the server and next to it write
230232
`.env`:
231233

232234
```bash

‎deploy/Caddyfile‎

Lines changed: 20 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -13,10 +13,27 @@
1313
header Cache-Control "no-cache"
1414
file_server
1515
}
16-
handle /gtfs/jdf.zip {
16+
# Every feed under one stable name, served from the active release (or the worker's
17+
# output), and their list from `obehy release feed-list`.
18+
redir /get-feeds /get-feeds/
19+
handle_path /get-feeds/* {
1720
root * /srv
18-
rewrite * /releases/active/jdf/gtfs.zip
19-
file_server
21+
route {
22+
@list path / /feeds.json
23+
header @list Cache-Control "no-cache"
24+
@realtime path /cz-jdf-gtfs-rt.pb
25+
header @realtime Content-Type application/x-protobuf
26+
header @realtime Cache-Control "no-cache"
27+
rewrite /cz-jdf-gtfs.zip /releases/active/jdf/gtfs.zip
28+
rewrite /cz-jdf-filtered-gtfs.zip /releases/active/jdf-filtered/gtfs.zip
29+
rewrite /cz-czptt-gtfs.zip /releases/active/czptt/gtfs.zip
30+
rewrite /cz-jdf-gtfs-rt.pb /gtfs-rt/jdf.pb
31+
rewrite / /public/index.html
32+
rewrite /feeds.json /public/feeds.json
33+
@served path /releases/active/* /gtfs-rt/* /public/*
34+
file_server @served
35+
respond 404
36+
}
2037
}
2138
handle {
2239
respond 404

‎deploy/obehy-release-update‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,8 @@
33
# Every step is idempotent: a fetched release is not downloaded again, a loaded package is not
44
# loaded again, and a release no newer than those activated before is left alone, so this can
55
# run often and a failed step is simply retried on the next run. After an activation,
6-
# data/releases/active points at the release, so its gtfs.zip is served beside the GTFS-RT.
6+
# data/releases/active points at the release, whose feeds Caddy serves under /get-feeds/, and
7+
# the feed list there is rewritten.
78
set -eu
89

910
obehy() {
@@ -16,3 +17,6 @@ obehy release load "$release_dir"
1617
if obehy release activate --if-newer "$run_id" | tee /dev/stderr | grep -q '^activated '; then
1718
ln -sfn "$run_id" "$(dirname "$release_dir")/active"
1819
fi
20+
if [ -e "$(dirname "$release_dir")/active" ]; then
21+
obehy release feed-list
22+
fi

‎src/obehy/release/commands.py‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@
1313
from obehy.release import activate as activation
1414
from obehy.release import fetch as fetching
1515
from obehy.release.contract import load_contract
16+
from obehy.release.feed_list import write_feed_list
1617
from obehy.release.load import PACKAGES, LoadError, load_release
1718
from obehy.release.migrate import MigrationError, migrate
1819
from obehy.runtime_config import ConfigurationError, default_config_path, load_database_url
@@ -40,6 +41,13 @@ def add_parsers(commands: Any) -> None:
4041
help="directory of fetched releases (default: data/releases)",
4142
)
4243
fetcher.add_argument("--repository", default=fetching.DEFAULT_REPOSITORY)
44+
lister = release_commands.add_parser(
45+
"feed-list", help="write the public feed list (index.html, feeds.json) of releases/active"
46+
)
47+
lister.add_argument("--into", type=Path, default=DEFAULT_RELEASES, help="fetched releases")
48+
lister.add_argument(
49+
"--out", type=Path, default=DEFAULT_RELEASES.parent / "public", help="default: data/public"
50+
)
4351
fetcher.add_argument(
4452
"--keep",
4553
type=int,
@@ -96,6 +104,16 @@ def _fetch(args: argparse.Namespace) -> int:
96104
return 0
97105

98106

107+
def _feed_list(args: argparse.Namespace) -> int:
108+
active = cast(Path, args.into) / "active"
109+
if not (active / "release.json").is_file():
110+
print(f"error: {active} names no fetched release", file=sys.stderr)
111+
return 1
112+
listing = write_feed_list(active, cast(Path, args.out))
113+
print(f"listed {len(listing['feeds'])} feeds of {listing['run_id']} in {args.out}")
114+
return 0
115+
116+
99117
def _progress(message: str) -> None:
100118
print(message, file=sys.stderr, flush=True)
101119

@@ -117,6 +135,8 @@ def _report(message: str) -> None:
117135
def run(args: argparse.Namespace) -> int:
118136
if args.command == "release" and args.release_command == "fetch":
119137
return _fetch(args)
138+
if args.command == "release" and args.release_command == "feed-list":
139+
return _feed_list(args)
120140
try:
121141
with _connect(args) as connection:
122142
if args.command == "db":

‎src/obehy/release/feed_list.py‎

Lines changed: 136 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,136 @@
1+
"""``obehy release feed-list``: the public list of the active release's feeds.
2+
3+
Caddy serves each static feed under a stable name straight from ``releases/active``
4+
(``deploy/Caddyfile``), so the URLs never change; this writes the page that lists them, with
5+
sizes and the release they come from, as ``index.html`` and ``feeds.json``.
6+
"""
7+
8+
from __future__ import annotations
9+
10+
import html
11+
import json
12+
from dataclasses import dataclass
13+
from datetime import UTC, datetime
14+
from pathlib import Path
15+
from typing import Any, cast
16+
from zoneinfo import ZoneInfo
17+
18+
from obehy.pipeline.files import atomic_output_path
19+
20+
21+
@dataclass(frozen=True)
22+
class Feed:
23+
name: str # the published file name, served next to the list
24+
source: str # path inside the release directory
25+
title: str
26+
27+
28+
# The names the MOTIS node's /get-feeds/ directory used, so its URLs keep working when it points
29+
# here.
30+
STATIC_FEEDS = (
31+
Feed("cz-jdf-gtfs.zip", "jdf/gtfs.zip", "Buses (JDF), national"),
32+
Feed("cz-jdf-filtered-gtfs.zip", "jdf-filtered/gtfs.zip", "Buses (JDF), filtered"),
33+
Feed("cz-czptt-gtfs.zip", "czptt/gtfs.zip", "Rail (CZPTT), national"),
34+
)
35+
REALTIME_FEEDS = (Feed("cz-jdf-gtfs-rt.pb", "", "Buses (JDF), GTFS-RT, updated every 10 s"),)
36+
37+
38+
def _built(completed_at: object) -> str:
39+
"""The build time in Prague, as people read it; the raw value if it is not a timestamp."""
40+
41+
try:
42+
when = datetime.fromisoformat(str(completed_at))
43+
except ValueError:
44+
return str(completed_at)
45+
local = when.astimezone(ZoneInfo("Europe/Prague"))
46+
return f"{local.day}. {local.month}. {local.year} {local:%H:%M}"
47+
48+
49+
def _size(size: int) -> str:
50+
value = float(size)
51+
for unit in ("B", "kB", "MB", "GB"):
52+
if value < 1000 or unit == "GB":
53+
return f"{value:.0f} {unit}" if unit == "B" else f"{value:.1f} {unit}"
54+
value /= 1000
55+
raise AssertionError("unreachable")
56+
57+
58+
def feed_list(release_dir: Path) -> dict[str, Any]:
59+
release = cast(
60+
dict[str, Any], json.loads((release_dir / "release.json").read_text(encoding="utf-8"))
61+
)
62+
feeds: list[dict[str, Any]] = []
63+
for feed in STATIC_FEEDS:
64+
path = release_dir / feed.source
65+
if path.is_file():
66+
stat = path.stat()
67+
feeds.append(
68+
{
69+
"name": feed.name,
70+
"title": feed.title,
71+
"size_bytes": stat.st_size,
72+
"modified": datetime.fromtimestamp(stat.st_mtime, UTC).isoformat(
73+
timespec="seconds"
74+
),
75+
}
76+
)
77+
feeds.extend({"name": feed.name, "title": feed.title} for feed in REALTIME_FEEDS)
78+
return {
79+
"run_id": release["run_id"],
80+
"completed_at": release.get("completed_at"),
81+
"feeds": feeds,
82+
}
83+
84+
85+
def _html(listing: dict[str, Any]) -> str:
86+
rows = "\n".join(
87+
'<tr><td><a href="{href}">{name}</a></td><td>{title}</td><td>{size}</td></tr>'.format(
88+
href=html.escape(feed["name"], quote=True),
89+
name=html.escape(feed["name"]),
90+
title=html.escape(feed["title"]),
91+
size=_size(feed["size_bytes"]) if "size_bytes" in feed else "live",
92+
)
93+
for feed in listing["feeds"]
94+
)
95+
return f"""<!doctype html>
96+
<html lang="en">
97+
<head>
98+
<meta charset="utf-8">
99+
<meta name="viewport" content="width=device-width, initial-scale=1">
100+
<title>Oběhy feeds</title>
101+
<style>
102+
:root {{ color-scheme: light dark; font-family: system-ui, sans-serif; }}
103+
body {{ max-width: 44rem; margin: 2rem auto; padding: 0 1rem; line-height: 1.5; }}
104+
table {{ border-collapse: collapse; width: 100%; }}
105+
td, th {{ padding: .35rem .6rem; border-bottom: 1px solid #8884; text-align: left; }}
106+
td:last-child, th:last-child {{ text-align: right; white-space: nowrap; }}
107+
small {{ opacity: .7; }}
108+
</style>
109+
</head>
110+
<body>
111+
<h1>Oběhy feeds</h1>
112+
<p><small>Release {html.escape(listing["run_id"])}, built {
113+
html.escape(_built(listing["completed_at"]))
114+
}. File names stay the same across releases. Machine-readable:
115+
<a href="feeds.json">feeds.json</a>.</small></p>
116+
<table>
117+
<thead><tr><th>File</th><th>Feed</th><th>Size</th></tr></thead>
118+
<tbody>
119+
{rows}
120+
</tbody>
121+
</table>
122+
</body>
123+
</html>
124+
"""
125+
126+
127+
def write_feed_list(release_dir: Path, out_dir: Path) -> dict[str, Any]:
128+
"""Write ``index.html`` and ``feeds.json`` for the release into ``out_dir``."""
129+
130+
listing = feed_list(release_dir)
131+
out_dir.mkdir(parents=True, exist_ok=True)
132+
with atomic_output_path(out_dir / "feeds.json") as path:
133+
path.write_text(json.dumps(listing, indent=2) + "\n", encoding="utf-8")
134+
with atomic_output_path(out_dir / "index.html") as path:
135+
path.write_text(_html(listing), encoding="utf-8")
136+
return listing

‎tests/unit/test_feed_list.py‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
from __future__ import annotations
2+
3+
import json
4+
from pathlib import Path
5+
6+
from obehy.release.feed_list import write_feed_list
7+
8+
9+
def test_feed_list_names_every_feed_of_the_release(tmp_path: Path) -> None:
10+
release = tmp_path / "20261009T030215Z-8cf199243726"
11+
for package in ("jdf", "czptt"): # no jdf-filtered in this release
12+
(release / package).mkdir(parents=True)
13+
(release / package / "gtfs.zip").write_bytes(b"x" * 2_500_000)
14+
(release / "release.json").write_text(
15+
json.dumps({"run_id": release.name, "completed_at": "2026-10-09T03:51:16+00:00"}),
16+
encoding="utf-8",
17+
)
18+
19+
listing = write_feed_list(release, tmp_path / "public")
20+
21+
assert [feed["name"] for feed in listing["feeds"]] == [
22+
"cz-jdf-gtfs.zip",
23+
"cz-czptt-gtfs.zip",
24+
"cz-jdf-gtfs-rt.pb",
25+
]
26+
stored = json.loads((tmp_path / "public" / "feeds.json").read_text(encoding="utf-8"))
27+
assert stored["run_id"] == release.name and stored["feeds"][0]["size_bytes"] == 2_500_000
28+
page = (tmp_path / "public" / "index.html").read_text(encoding="utf-8")
29+
assert '<a href="cz-czptt-gtfs.zip">cz-czptt-gtfs.zip</a>' in page
30+
assert "2.5 MB" in page and release.name in page
31+
assert "built 9. 10. 2026 05:51" in page

0 commit comments

Comments
 (0)