Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ jobs:
cmake --build build --parallel --target
test_packets test_json test_frames test_quad test_world test_collision
test_endpoint test_manager test_api test_radio test_radio_api test_thread_pool
test_streamer test_still
test_streamer test_still test_cpus
- name: Run unit tests
run: ctest --test-dir build --label-regex '^unit$' --output-on-failure

Expand Down
9 changes: 7 additions & 2 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ target_link_libraries(skysim_api PUBLIC skysim_vehicle skysim_radio Threads::Thr
target_compile_options(skysim_api PRIVATE -Wall -Wextra -Wshadow)

# --- core: Jolt world (the only Jolt consumer above tools/) ----------------------------------
add_library(skysim_core STATIC src/core/world.cpp)
add_library(skysim_core STATIC src/core/world.cpp src/core/cpus.cpp)
target_include_directories(skysim_core PUBLIC src)
target_link_libraries(skysim_core PRIVATE Jolt)
target_compile_options(skysim_core PRIVATE -Wall -Wextra -Wshadow)
Expand Down Expand Up @@ -227,13 +227,18 @@ target_link_libraries(test_thread_pool PRIVATE Threads::Threads)
target_compile_options(test_thread_pool PRIVATE -Wall -Wextra -Wshadow)
add_test(NAME thread_pool COMMAND test_thread_pool)

add_executable(test_cpus tests/test_cpus.cpp)
target_link_libraries(test_cpus PRIVATE skysim_core)
target_compile_options(test_cpus PRIVATE -Wall -Wextra -Wshadow)
add_test(NAME cpus COMMAND test_cpus)

add_executable(test_streamer tests/test_streamer.cpp)
target_link_libraries(test_streamer PRIVATE skysim_terrain skysim_core)
target_compile_options(test_streamer PRIVATE -Wall -Wextra -Wshadow)
add_test(NAME streamer COMMAND test_streamer)

set_tests_properties(
packets json frames quad world collision endpoint manager api radio radio_api thread_pool streamer still
packets json frames quad world collision endpoint manager api radio radio_api thread_pool streamer still cpus
PROPERTIES LABELS "unit"
)

Expand Down
13 changes: 11 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -202,7 +202,16 @@ deadline, and while that no longer corrupts the flight it does cost wall-clock p
**`SKYSIM_CAMERA_FPS` is the camera switch.** Without it no frames are rendered and
the camera endpoints serve nothing, whatever else is set. With it, `SKYSIM_CAMERA_SIZE`
(default `256x144`), `SKYSIM_CAMERA_QUALITY`, `SKYSIM_CAMERA_THREADS`,
`SKYSIM_CAMERA_FOV`, `SKYSIM_CAMERA_PITCH` and `SKYSIM_CAMERA_RANGE` apply:
`SKYSIM_CAMERA_FOV`, `SKYSIM_CAMERA_PITCH` and `SKYSIM_CAMERA_RANGE` apply.
A vehicle is only drawn while somebody is asking for its picture — an open `camera.mjpg`,
or a `camera.jpg` within the last 3 s — so a camera nobody watches costs nothing, however
many vehicles there are, and a parked vehicle's frame is re-sent rather than redrawn.
`SKYSIM_CAMERA_THREADS` is the number of threads drawing a frame (at background priority);
by default it is the number of CPUs the container may use — its cgroup quota and affinity,
not the host's core count — at most 4. A textured 640x360 frame of a flying vehicle is about
75 ms of CPU in the image, so a 2-vCPU task streams one watched flying vehicle at 15 fps and
three at about 9 fps each. `GET /metrics` reports what the camera achieves (`camera.fps`,
`camera.render_ms`) and the log carries the same every 10 s while frames are drawn:

```bash
docker run --rm -p 8642:8642 -p 9002-9202:9002-9202/udp \
Expand All @@ -226,7 +235,7 @@ camera feed alike — see [Survey stills](#survey-stills-and-the-reconstruction-
| `POST /vehicles` `{"instance":N, "launch_process":true}` (both fields optional) | reserve a requested or next-free ArduPilot instance (optionally forks arducopter) → `{id, instance, json_port, mavlink_tcp}` |
| `DELETE /vehicles/{id}` | despawn, release instance, kill managed process |
| `GET /vehicles` | per-vehicle: connected, frozen, silent_ticks, pos_ned, `midair_collisions`, `building_contacts`; `radio` when a radio field is set |
| `GET /metrics` | tick p50/p99 µs, straggler_events, freezes, resident_tiles |
| `GET /metrics` | tick p50/p99 µs, straggler_events, freezes, resident_tiles; `camera` (`null` when off): `target_fps`, achieved `fps` per watched vehicle, `watched`, `render_ms` / `render_ms_max` / `encode_ms` per frame, `threads`, `frames_drawn`, `frames_reused` |
| `POST /mission/check` `{"waypoints_ned":[[n,e,d],...], "clearance_m":1.0}` | sweep a planned route through loaded building geometry → `{checked, clear, hits}` |
| `PUT` / `GET` / `DELETE /radio` | set, read or clear the simulated 4G radio field (carriers, towers, coverage holes) |
| `GET /radio/sample?lat&lon&alt_amsl` | RSRP / RSRQ / SINR / RSSI, serving cell and link quality per carrier at a point |
Expand Down
23 changes: 22 additions & 1 deletion docs/DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,19 @@ core. Beyond that, shard. Re-measure with `world_bench` / `proto_bench` before t
Two cameras draw the same static world, off the tick thread:

- **Pilot camera** (`--camera-fps`, `RenderService`): a small frame (256x144) per vehicle at a
fixed rate on its own thread and pool, served as `camera.jpg` / `camera.mjpg`.
fixed rate on its own thread and pool, served as `camera.jpg` / `camera.mjpg`. Drawn on
demand: a vehicle is on the draw list only while somebody has asked for its picture within
the last 3 s (an open MJPEG stream asks continuously, each `camera.jpg` once), and with
nobody asking the render thread waits on a condition variable, not a timer. A vehicle whose
pose has not moved since its last frame (to within a nanometre: a parked quad is never put
to sleep and the contact solver jiggles it by ~1e-13 m) has that JPEG re-sent instead of
drawn again. `--camera-threads` is the total number of threads drawing a frame, the render
thread included, all at nice +10; its default is `core::available_cpus()` — the smallest
of the online CPUs, the affinity mask and the cgroup CPU quota — clamped to 1..4, because
an ECS task's `hardware_concurrency()` is the host's. The old fixed pool was five
un-niced threads on a 2-vCPU task; with three watched vehicles flying it took the tick
from 400 to 378 Hz, where two niced threads saturating the same quota leave it at 400. MJPEG routes block on the
next frame's sequence number (`FrameStore::wait_newer`) instead of polling the slot.
- **Survey still** (`GET .../still.jpg`, `StillService`): one large frame on request. The HTTP
route copies the vehicle out of the per-tick snapshot (position, attitude and sim time from
the same tick — `VehicleInfo` carries all three) and hands it to the still service, which
Expand Down Expand Up @@ -169,3 +181,12 @@ Under a sustained burst of stills (two rendering, the rest turned away with 503)
from 12.8 to 15.0 µs and the world kept 800 Hz. The pilot camera's 256x144 frame went from
0.3 ms to about 2.5 ms on its four-thread pool with texture and shadows, which still holds
10 fps for some 40 vehicles.

The 640x360 pilot frame is ~55 ms on one core (Ryzen 7 5700X, demo city, 40 m up, pitch
-15°), ~19 ms on two threads, plus ~3 ms of JPEG encode. Measured per stage: the procedural
texture is ~60% of it (value noise and fbm, Voronoi land cover, windows), shadow rays ~15%,
ray and footprint set-up ~10%. Cheap cuts that keep the picture as it is were tried and do
not pay: a 2x coarser texture footprint saves 4%, `-O3` 4%, precomputing the shadow ray's
direction-only Moller-Trumbore terms per triangle 1% (for 32 bytes a triangle). What does pay
is not drawing: before on-demand rendering, an idle dev server with one never-connected
vehicle slot spent ~1.2 cores here (95% of its 2-vCPU task) drawing frames for nobody.
82 changes: 55 additions & 27 deletions src/api/control_server.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,20 @@ namespace {
// plane, which must stay answerable while they run.
constexpr size_t kWorkerThreads = 64;

// How often an MJPEG stream looks for a newly drawn frame. Short relative to any render rate
// worth watching, because it bounds how long a finished frame waits before it goes out; it does
// not set the rate, since a frame is only written once (see write_mjpeg_part callers).
constexpr auto kFramePoll = std::chrono::milliseconds(5);
// How long an MJPEG stream blocks waiting for the next frame before it gives the server a turn
// (a shutdown is noticed between waits). A frame goes out the moment it is published, so this
// does not set the rate or the latency — the 5 ms poll it replaces did neither either, and it
// copied the whole JPEG out of the slot two hundred times a second per viewer to find that out.
constexpr auto kFrameWait = std::chrono::milliseconds(250);

// When the wait came back with nothing new straight away — the vehicle is not in the fleet, or
// the camera does not block — the stream sleeps this long before asking again.
constexpr auto kFrameRetry = std::chrono::milliseconds(20);

// How long camera.jpg waits for a first frame. Rendering is on demand, so the first request for
// a vehicle nobody is watching finds no frame yet: it is what starts the drawing, and one frame
// is tens of milliseconds away. Bounded so a camera that cannot keep up still answers.
constexpr auto kFirstFrameWait = std::chrono::milliseconds(2000);

// One multipart part: headers plus the JPEG.
//
Expand All @@ -52,6 +62,24 @@ void write_mjpeg_part(httplib::DataSink &sink, const std::vector<uint8_t> &jpeg)
sink.write("\r\n", 2);
}

// One turn of an MJPEG stream: wait for the vehicle's next frame and write it, once. The wait is
// also the stream's standing request for frames — the camera draws only what is being asked for.
void stream_next_frame(const ControlServer::Snapshots &snapshots, uint32_t id, uint64_t &last_seq,
httplib::DataSink &sink) {
const auto asked = std::chrono::steady_clock::now();
const auto frame =
id != 0 ? snapshots.camera_frame(id, last_seq, kFrameWait) : skysim::render::FrameStore::Frame{};
if (!frame.jpeg.empty() && frame.seq != last_seq) {
last_seq = frame.seq;
write_mjpeg_part(sink, frame.jpeg);
return;
}
const auto waited = std::chrono::steady_clock::now() - asked;
if (waited < kFrameRetry) {
std::this_thread::sleep_for(kFrameRetry - waited);
}
}

// {"launch_process":true} — absent key means false.
bool parse_launch_process(const std::string &body) {
const char *p = std::strstr(body.c_str(), "\"launch_process\"");
Expand Down Expand Up @@ -673,7 +701,7 @@ ControlServer::ControlServer(const std::string &bind_addr, int port, CommandQueu
res.set_content("{\"error\":\"no such instance\"}", "application/json");
return;
}
std::vector<uint8_t> jpeg = snapshots.camera_frame(id).jpeg;
std::vector<uint8_t> jpeg = snapshots.camera_frame(id, 0, kFirstFrameWait).jpeg;
if (jpeg.empty()) {
res.status = 404;
res.set_content("{\"error\":\"no frame\"}", "application/json");
Expand All @@ -692,17 +720,11 @@ ControlServer::ControlServer(const std::string &bind_addr, int port, CommandQueu
}
res.set_chunked_content_provider(
"multipart/x-mixed-replace; boundary=skysimframe",
[snapshots, resolve_instance, instance, last_sent = -1.0](size_t,
httplib::DataSink &sink) mutable {
[snapshots, resolve_instance, instance, last_seq = uint64_t{0}](size_t,
httplib::DataSink &sink) mutable {
// Re-resolved per frame, so a skysim restart mid-recording picks
// the vehicle back up instead of streaming nothing forever.
const uint32_t id = resolve_instance(instance);
auto frame = id != 0 ? snapshots.camera_frame(id) : skysim::render::FrameStore::Frame{};
if (!frame.jpeg.empty() && frame.sim_time_s != last_sent) {
last_sent = frame.sim_time_s;
write_mjpeg_part(sink, frame.jpeg);
}
std::this_thread::sleep_for(kFramePoll);
stream_next_frame(snapshots, resolve_instance(instance), last_seq, sink);
return true;
});
});
Expand All @@ -717,9 +739,8 @@ ControlServer::ControlServer(const std::string &bind_addr, int port, CommandQueu
// strtoul, not stoul: the route regex accepts any number of digits, and
// stoul throws on a value too big for the type. Saturating is the same
// answer as "no such vehicle" without the exception.
std::vector<uint8_t> jpeg =
snapshots.camera_frame(static_cast<uint32_t>(std::strtoul(req.matches[1].str().c_str(), nullptr, 10)))
.jpeg;
const auto id = static_cast<uint32_t>(std::strtoul(req.matches[1].str().c_str(), nullptr, 10));
std::vector<uint8_t> jpeg = snapshots.camera_frame(id, 0, kFirstFrameWait).jpeg;
if (jpeg.empty()) {
res.status = 404;
res.set_content("{\"error\":\"no frame\"}", "application/json");
Expand All @@ -742,13 +763,8 @@ ControlServer::ControlServer(const std::string &bind_addr, int port, CommandQueu
const auto id = static_cast<uint32_t>(std::strtoul(req.matches[1].str().c_str(), nullptr, 10));
res.set_chunked_content_provider(
"multipart/x-mixed-replace; boundary=skysimframe",
[snapshots, id, last_sent = -1.0](size_t /*offset*/, httplib::DataSink &sink) mutable {
auto frame = snapshots.camera_frame(id);
if (!frame.jpeg.empty() && frame.sim_time_s != last_sent) {
last_sent = frame.sim_time_s;
write_mjpeg_part(sink, frame.jpeg);
}
std::this_thread::sleep_for(kFramePoll);
[snapshots, id, last_seq = uint64_t{0}](size_t /*offset*/, httplib::DataSink &sink) mutable {
stream_next_frame(snapshots, id, last_seq, sink);
return true;
});
});
Expand Down Expand Up @@ -796,14 +812,26 @@ ControlServer::ControlServer(const std::string &bind_addr, int port, CommandQueu

s.Get("/metrics", [snapshots](const httplib::Request &, httplib::Response &res) {
const MetricsInfo m = snapshots.metrics();
char buf[512];
// The pilot camera's achieved rate and per-frame cost, or null when it is off.
char camera[384] = "null";
if (m.camera.enabled) {
const CameraMetrics &c = m.camera;
std::snprintf(camera, sizeof(camera),
"{\"target_fps\":%.1f,\"fps\":%.2f,\"watched\":%zu,\"render_ms\":%.2f,"
"\"render_ms_max\":%.2f,\"encode_ms\":%.2f,\"threads\":%d,\"frames_drawn\":%llu,"
"\"frames_reused\":%llu}",
c.target_fps, c.fps, c.watched, c.render_ms, c.render_ms_max, c.encode_ms, c.threads,
static_cast<unsigned long long>(c.frames_drawn),
static_cast<unsigned long long>(c.frames_reused));
}
char buf[1024];
std::snprintf(buf, sizeof(buf),
"{\"tick\":%llu,\"sim_time_s\":%.3f,\"tick_p50_us\":%.1f,"
"\"tick_p99_us\":%.1f,\"straggler_events\":%llu,\"freezes\":%llu,"
"\"vehicles\":%zu,\"resident_tiles\":%zu}",
"\"vehicles\":%zu,\"resident_tiles\":%zu,\"camera\":%s}",
static_cast<unsigned long long>(m.tick), m.sim_time_s, m.tick_p50_us, m.tick_p99_us,
static_cast<unsigned long long>(m.straggler_events),
static_cast<unsigned long long>(m.freezes), m.vehicles, m.resident_tiles);
static_cast<unsigned long long>(m.freezes), m.vehicles, m.resident_tiles, camera);
res.set_content(buf, "application/json");
});

Expand Down
31 changes: 26 additions & 5 deletions src/api/control_server.h
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
// thread never touches world state: writes go through the tick-boundary CommandQueue (promises
// for replies), reads come from snapshots published once per tick (CLAUDE.md invariant 5).
// WS /state lands at M6.
#include <chrono>
#include <cstdint>
#include <functional>
#include <future>
Expand Down Expand Up @@ -38,6 +39,21 @@ struct VehicleInfo {
double sim_time_s = 0.0;
};

// The pilot camera's achieved rate and cost (RenderService::Stats), so a laggy stream can be
// read off /metrics instead of guessed at.
struct CameraMetrics {
bool enabled = false; // --camera-fps given; the rest is meaningless without it
double target_fps = 0.0;
double fps = 0.0; // frames per second each watched vehicle is getting
size_t watched = 0; // vehicles being drawn because somebody is asking for them
double render_ms = 0.0;
double render_ms_max = 0.0;
double encode_ms = 0.0;
int threads = 0;
uint64_t frames_drawn = 0;
uint64_t frames_reused = 0;
};

struct MetricsInfo {
double tick_p50_us = 0.0;
double tick_p99_us = 0.0;
Expand All @@ -47,6 +63,7 @@ struct MetricsInfo {
double sim_time_s = 0.0;
size_t vehicles = 0;
size_t resident_tiles = 0; // streamed-in city tiles (M5 memory bound observable)
CameraMetrics camera;
};

struct SpawnCommand {
Expand Down Expand Up @@ -111,11 +128,15 @@ class ControlServer {
struct Snapshots {
std::function<std::vector<VehicleInfo>()> vehicles;
std::function<MetricsInfo()> metrics;
// Latest camera frame for a vehicle as encoded JPEG, with the sim time it was drawn
// at; empty when the camera is off or that vehicle has not been rendered yet. The
// timestamp is what lets the MJPEG routes send a frame once instead of re-sending
// whatever is in the slot. Published by the tick thread like everything else here.
std::function<skysim::render::FrameStore::Frame(uint32_t)> camera_frame;
// Latest camera frame for a vehicle as encoded JPEG, with the sim time it was drawn at
// and its sequence number; empty when the camera is off or that vehicle has not been
// rendered yet. Asking is also what gets a vehicle drawn: the camera renders only the
// vehicles somebody is asking for. With a wait, blocks up to that long for a frame
// newer than after_seq — how an MJPEG route sends each frame once, as it lands, without
// polling the slot on a timer.
std::function<skysim::render::FrameStore::Frame(uint32_t id, uint64_t after_seq,
std::chrono::milliseconds wait)>
camera_frame;
// Survey still camera: render one frame from this vehicle's snapshot pose. Null when
// the still camera is off, and the still routes answer 404. Called on an HTTP thread;
// it must not touch world state (it gets everything it needs in the VehicleInfo).
Expand Down
Loading