Skip to content

Commit fc7b1f3

Browse files
committed
lws_spawn: add cgroup support if on linux
1 parent 5d00d40 commit fc7b1f3

7 files changed

Lines changed: 534 additions & 1 deletion

File tree

READMEs/lws_spawn.md

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
# Spawning and Managing Child Processes with `lws_spawn`
2+
3+
The `lws_spawn` API provides a robust, platform-agnostic way to create and manage child processes directly from your C application, fully integrated with the libwebsockets event loop.
4+
5+
## Overview
6+
7+
The `lws_spawn` API is designed to securely run external executables as child processes. Its key features include:
8+
9+
* **Event-Loop Integration**: The child process lifecycle is managed without blocking the main lws event loop.
10+
* **Standard I/O Redirection**: The child's `stdin`, `stdout`, and `stderr` are automatically redirected to pipes. These pipes are represented as standard `struct lws` connection instances (`wsi`), allowing you to interact with the child process using familiar lws protocol callbacks.
11+
* **Automatic Lifecycle Management**: Lws handles waiting for and "reaping" the terminated child process, providing a callback with its exit status and resource accounting information.
12+
* **Security**: Provides options for setting a `chroot()` jail and the working directory for the child process.
13+
* **Timeout Management**: An optional timeout can be specified to automatically kill a child process that runs for too long.
14+
* **Linux Cgroup Containment**: On Linux, spawned processes can be automatically placed into a new, dedicated cgroup for resource control and isolation. The cgroup is automatically removed when the process is reaped.
15+
16+
## Core API Usage
17+
18+
The primary entrypoint for this API is `lws_spawn_piped()`. It takes a single argument: a pointer to a `struct lws_spawn_piped_info` which you populate to describe the child process you want to create.
19+
20+
Key members of `struct lws_spawn_piped_info`:
21+
22+
- `exec_array`: A `NULL`-terminated array of strings for the executable path and its arguments (equivalent to `argv`).
23+
- `env_array`: A `NULL`-terminated array of strings for the child's environment variables (e.g., `{"VAR1=VALUE1", "VAR2=VALUE2", NULL}`).
24+
- `vh`: The `lws_vhost` the new stdio `wsi` should be associated with.
25+
- `protocol_name`: The name of the lws protocol that will handle events for the stdio `wsi`. This is **mandatory** for proper cleanup.
26+
- `reap_cb`: A **mandatory** callback function of type `lsp_cb_t` that lws will call after the child process has terminated and been reaped.
27+
- `opaque`: A user pointer that will be passed to your `reap_cb` and will also be available on the stdio `wsi` via `lws_get_opaque_user_data()`.
28+
- `timeout_us`: Optional timeout in microseconds. If the process runs longer than this, it will be sent a `SIGTERM`.
29+
30+
### Lifecycle and Cleanup
31+
32+
Proper cleanup is essential. When the child process exits, its stdio pipes are closed by the operating system. This generates a `LWS_CALLBACK_RAW_FILE_CLOSE` event on each of the three stdio `wsi`.
33+
34+
Your protocol handler **must** implement a case for this reason and call `lws_spawn_stdwsi_closed()`:
35+
36+
```c
37+
static int my_spawn_protocol_cb(struct lws *wsi, enum lws_callback_reasons reason, ...)
38+
{
39+
struct my_spawn_state *st = (struct my_spawn_state *)
40+
lws_get_opaque_user_data(wsi);
41+
42+
switch (reason) {
43+
case LWS_CALLBACK_RAW_FILE_CLOSE:
44+
if (st && st->lsp)
45+
lws_spawn_stdwsi_closed(st->lsp, wsi);
46+
break;
47+
/* ... other cases ... */
48+
}
49+
return 0;
50+
}
51+
```
52+
53+
When lws has been notified that all three stdio `wsi` have closed, it will proceed to reap the child process and invoke your `reap_cb`.
54+
55+
## Linux Cgroup Support
56+
57+
On Linux, `lws_spawn` can automatically create a transient cgroup v2 for each spawned process, providing resource isolation. The cgroup is automatically removed when the process is reaped.
58+
59+
### Permissions and One-Time Setup
60+
61+
By default, creating new cgroups in `/sys/fs/cgroup/` requires `root` privileges. A non-root user will get a "Permission Denied" error.
62+
63+
The recommended solution is to have an administrator (or a CI setup script) perform a **one-time setup** to delegate control of a subdirectory to the user running the lws application:
64+
65+
```sh
66+
# Run these commands as root once
67+
sudo mkdir -p /sys/fs/cgroup/lws
68+
sudo chown myuser:mygroup /sys/fs/cgroup/lws
69+
echo "+cpu +memory +pids +io" | sudo tee /sys/fs/cgroup/lws/cgroup.subtree_control
70+
```
71+
72+
For applications that start as `root` and then drop privileges, lws provides a helper function to perform this setup programmatically: `int lws_spawn_cgroup_admin_init(const char *toplevel_name);`. Call this function once at startup while your process still has root privileges. If `toplevel_name` is NULL, it defaults to the builtin one "lws" as the toplevel token. It returns success (0) if the toplevel entry already existed as well as if successfully created.
73+
74+
### API Usage
75+
76+
To enable cgroup containment for a spawned process, set the following members in your `struct lws_spawn_piped_info`:
77+
78+
- `cgroup_name_suffix`: A string that will be used to name the cgroup directory. For example, a suffix of `"my-task"` will create a cgroup at `/sys/fs/cgroup/lws/my-task`. This must be unique for concurrent spawns.
79+
80+
- `p_cgroup_ret`: An optional pointer to an `int`. If provided, lws will write `0` to this integer on successful cgroup creation, and `1` on failure.
81+
82+
If cgroup creation fails (e.g., due to permissions), `lws_spawn_piped()` will **not** fail. It will log a warning and proceed to spawn the process without cgroup containment. Use `p_cgroup_ret` to confirm the outcome.
83+
84+
## Example
85+
86+
For a complete, working example that demonstrates these concepts, including cgroup setup and verification, please refer to the API test located at:
87+
88+
`minimal-examples-lowlevel/api-tests/api-test-lws_spawn/`

include/libwebsockets/lws-misc.h

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1053,6 +1053,8 @@ typedef void (*lsp_cb_t)(void *opaque, lws_usec_t *accounting, siginfo_t *si,
10531053
* \p timeout: optional us-resolution timeout, or zero
10541054
* \p reap_cb: callback when child process has been reaped and the lsp destroyed
10551055
* \p tsi: tsi to bind stdwsi to... from opt_parent if given
1056+
* \p cgroup_name_suffix: for Linux, encapsulate spawn into this new cgroup
1057+
* \p p_cgroup_ret: NULL, or pointer to int to show if cgroups applied OK (0 = OK)
10561058
*/
10571059
struct lws_spawn_piped_info {
10581060
struct lws_dll2_owner *owner;
@@ -1078,6 +1080,9 @@ struct lws_spawn_piped_info {
10781080
const struct lws_role_ops *ops; /* NULL is raw file */
10791081

10801082
uint8_t disable_ctrlc;
1083+
1084+
const char *cgroup_name_suffix;
1085+
int *p_cgroup_ret;
10811086
};
10821087

10831088
/**
@@ -1154,6 +1159,26 @@ lws_spawn_get_stdfd(struct lws *wsi);
11541159
*/
11551160
LWS_VISIBLE LWS_EXTERN int
11561161
lws_spawn_get_fd_stdxxx(struct lws_spawn_piped *lsp, int std_idx);
1162+
1163+
/**
1164+
* lws_spawn_cgroup_admin_init() - Create lws parent cgroup
1165+
*
1166+
* \p toplevel_name: NULL (chooses name 'lws') or the name fragment to
1167+
* try to create.
1168+
*
1169+
* This helper should be called once at startup by a process that has root
1170+
* privileges. It will create and configure the master cgroup directory
1171+
* `/sys/fs/cgroup/<toplevel_name>`.
1172+
*
1173+
* After this has been called successfully, the process can drop privileges
1174+
* to a non-root user, and subsequent calls to lws_spawn_piped() with a
1175+
* cgroup_name_suffix will succeed as long as that user has write permission
1176+
* in the master cgroup directory (which can be arranged via chown).
1177+
*
1178+
* Returns 0 on success. On non-Linux platforms, it's a no-op that returns 1.
1179+
*/
1180+
LWS_VISIBLE LWS_EXTERN int
1181+
lws_spawn_cgroup_admin_init(const char *topelevel_name);
11571182
#endif
11581183

11591184
struct lws_fsmount {

lib/core-net/private-lib-core-net.h

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -926,6 +926,10 @@ struct lws_spawn_piped {
926926
lws_sorted_usec_list_t sul;
927927
lws_sorted_usec_list_t sul_reap;
928928

929+
#if defined(__linux__)
930+
char cgroup_path[256];
931+
#endif
932+
929933
struct lws_context *context;
930934
struct lws *stdwsi[3];
931935
lws_filefd_type pipe_fds[3][2];

lib/plat/unix/unix-spawn.c

Lines changed: 142 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,10 @@
2929
#include "private-lib-core.h"
3030
#include <unistd.h>
3131

32+
#if defined(__linux__)
33+
#include <sys/stat.h>
34+
#endif
35+
3236
#if defined(__OpenBSD__) || defined(__NetBSD__)
3337
#include <sys/resource.h>
3438
#include <sys/wait.h>
@@ -51,7 +55,7 @@ lws_spawn_sul_reap(struct lws_sorted_usec_list *sul)
5155
struct lws_spawn_piped *lsp = lws_container_of(sul,
5256
struct lws_spawn_piped, sul_reap);
5357

54-
lwsl_notice("%s: reaping spawn after last stdpipe, tries left %d\n",
58+
lwsl_info("%s: reaping spawn after last stdpipe, tries left %d\n",
5559
__func__, lsp->reap_retry_budget);
5660
if (!lws_spawn_reap(lsp) && !lsp->pipes_alive) {
5761
if (--lsp->reap_retry_budget) {
@@ -212,6 +216,21 @@ lws_spawn_reap(struct lws_spawn_piped *lsp)
212216

213217
lws_sul_cancel(&lsp->sul);
214218

219+
#if defined(__linux__)
220+
if (lsp->cgroup_path[0]) {
221+
/*
222+
* The child has been reaped, we can remove the cgroup dir.
223+
* This will only work if the cgroup is empty, which it should
224+
* be now.
225+
*/
226+
if (rmdir(lsp->cgroup_path))
227+
lwsl_warn("%s: unable to rmdir cgroup %s, errno %d\n",
228+
__func__, lsp->cgroup_path, errno);
229+
else
230+
lwsl_info("%s: reaped cgroup %s\n", __func__, lsp->cgroup_path);
231+
}
232+
#endif
233+
215234
/*
216235
* All the stdwsi went down, nothing more is coming... it's over
217236
* Collect the final information and then reap the dead process
@@ -350,6 +369,13 @@ lws_spawn_piped(const struct lws_spawn_piped_info *i)
350369
lsp->info = *i;
351370
lsp->reap_retry_budget = 20;
352371

372+
#if defined(__linux__)
373+
lsp->cgroup_path[0] = '\0';
374+
#endif
375+
376+
if (i->p_cgroup_ret)
377+
*i->p_cgroup_ret = 1; /* Default to cgroup failed */
378+
353379
/*
354380
* Prepare the stdin / out / err pipes
355381
*/
@@ -445,6 +471,31 @@ lws_spawn_piped(const struct lws_spawn_piped_info *i)
445471
lsp->stdwsi[LWS_STDIN]->desc.sockfd,
446472
lsp->stdwsi[LWS_STDOUT]->desc.sockfd,
447473
lsp->stdwsi[LWS_STDERR]->desc.sockfd);
474+
475+
#if defined(__linux__)
476+
if (i->cgroup_name_suffix && i->cgroup_name_suffix[0]) {
477+
lws_snprintf(lsp->cgroup_path, sizeof(lsp->cgroup_path),
478+
"/sys/fs/cgroup/lws/%s", i->cgroup_name_suffix);
479+
480+
/*
481+
* This is the step that requires the process to either be root,
482+
* or for an admin to have delegated control of /sys/fs/cgroup/lws
483+
* to the user running the process.
484+
*/
485+
if (mkdir(lsp->cgroup_path, 0755)) {
486+
lwsl_warn("%s: Failed to create cgroup %s, errno %d. "
487+
"Continuing without cgroup.\n",
488+
__func__, lsp->cgroup_path, errno);
489+
lsp->cgroup_path[0] = '\0';
490+
/* Do not abort, just clear the path and continue */
491+
} else {
492+
lwsl_info("%s: created cgroup %s\n", __func__, lsp->cgroup_path);
493+
if (i->p_cgroup_ret)
494+
/* Report cgroup success to caller */
495+
*i->p_cgroup_ret = 0;
496+
}
497+
}
498+
#endif
448499

449500
/* we are ready with the redirection pipes... do the (v)fork */
450501
#if defined(__sun) || !defined(LWS_HAVE_VFORK) || !defined(LWS_HAVE_EXECVPE)
@@ -510,6 +561,32 @@ lws_spawn_piped(const struct lws_spawn_piped_info *i)
510561
* process is OK. Stuff that happens after the execvpe() is OK.
511562
*/
512563

564+
#if defined(__linux__)
565+
if (lsp->cgroup_path[0]) {
566+
char path[300], pid_str[20];
567+
int fd, len;
568+
569+
/*
570+
* We are the new child process. We must move ourselves into
571+
* the cgroup created for us by the parent.
572+
*/
573+
lws_snprintf(path, sizeof(path) - 1, "%s/cgroup.procs", lsp->cgroup_path);
574+
fd = open(path, O_WRONLY);
575+
if (fd >= 0) {
576+
len = lws_snprintf(pid_str, sizeof(pid_str) - 1, "%d", (int)getpid());
577+
if (write(fd, pid_str, (size_t)len) != (ssize_t)len) {
578+
/*
579+
* using lwsl_err here is unsafe in vfork()
580+
* child, just exit with a special code
581+
*/
582+
_exit(121);
583+
}
584+
close(fd);
585+
} else
586+
_exit(122);
587+
}
588+
#endif
589+
513590
if (i->chroot_path && chroot(i->chroot_path)) {
514591
lwsl_err("%s: child chroot %s failed, errno %d\n",
515592
__func__, i->chroot_path, errno);
@@ -603,6 +680,21 @@ lws_spawn_stdwsi_closed(struct lws_spawn_piped *lsp, struct lws *wsi)
603680
{
604681
int n;
605682

683+
/*
684+
* This is part of the normal cleanup path, check if the lsp has already
685+
* been destroyed by a timeout or other error path. If the stdwsi that
686+
* is closing has already been nulled out, we have already been through
687+
* destroy.
688+
*/
689+
for (n = 0; n < 3; n++)
690+
if (lsp->stdwsi[n] == wsi)
691+
goto found;
692+
693+
/* Not found, so must have been destroyed already */
694+
return;
695+
696+
found:
697+
606698
assert(lsp);
607699
lsp->pipes_alive--;
608700
lwsl_debug("%s: pipes alive %d\n", __func__, lsp->pipes_alive);
@@ -621,6 +713,55 @@ lws_spawn_get_stdfd(struct lws *wsi)
621713
return wsi->lsp_channel;
622714
}
623715

716+
int
717+
lws_spawn_cgroup_admin_init(const char *toplevel_name)
718+
{
719+
#if defined(__linux__)
720+
char path[128];
721+
int cfd;
722+
723+
if (!toplevel_name)
724+
toplevel_name = "lws";
725+
726+
lws_snprintf(path, sizeof(path), "/sys/fs/cgroup/%s", toplevel_name);
727+
728+
/*
729+
* Create the lws parent cgroup directory. This will only succeed if
730+
* the process has sufficient privileges (e.g., root).
731+
*/
732+
if (mkdir(path, 0755) && errno != EEXIST) {
733+
lwsl_warn("%s: Failed to mkdir %s: %s\n",
734+
__func__, path, strerror(errno));
735+
return 1;
736+
}
737+
738+
lws_snprintf(path, sizeof(path), "/sys/fs/cgroup/%s/cgroup.subtree_control",
739+
toplevel_name);
740+
741+
/*
742+
* We must enable controllers for the parent before they can be
743+
* used by children. This makes them available for delegation.
744+
* This step might require root.
745+
*/
746+
cfd = lws_open(path, LWS_O_WRONLY);
747+
if (cfd < 0) {
748+
/* May fail if user doesn't own the file, that's okay */
749+
lwsl_info("%s: cannot open subtree_control: %s\n",
750+
__func__, strerror(errno));
751+
return 0; /* Still a success if dir exists */
752+
}
753+
754+
if (write(cfd, "+cpu +memory +pids +io", 22) != 22)
755+
/* ignore, may be there already or fail due to perms */
756+
lwsl_debug("%s: setting admin cgroup options failed\n", __func__);
757+
close(cfd);
758+
lwsl_notice("%s: lws cgroup parent configured\n", __func__);
759+
760+
return 0;
761+
#endif
762+
return 1; /* Not supported on this platform */
763+
}
764+
624765
int
625766
lws_spawn_get_fd_stdxxx(struct lws_spawn_piped *lsp, int std_idx)
626767
{

lib/plat/windows/windows-spawn.c

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -570,6 +570,12 @@ lws_spawn_stdwsi_closed(struct lws_spawn_piped *lsp, struct lws *wsi)
570570
lsp->stdwsi[n] = NULL;
571571
}
572572

573+
int
574+
lws_spawn_cgroup_admin_init(const char *toplevel_name)
575+
{
576+
return 1; /* Not supported on this platform */
577+
}
578+
573579
int
574580
lws_spawn_get_stdfd(struct lws *wsi)
575581
{
Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
project(lws-api-test-spawn C)
2+
cmake_minimum_required(VERSION 3.10)
3+
find_package(libwebsockets CONFIG REQUIRED)
4+
list(APPEND CMAKE_MODULE_PATH ${LWS_CMAKE_DIR})
5+
include(CheckCSourceCompiles)
6+
include(LwsCheckRequirements)
7+
8+
set(SAMP lws-api-test-spawn)
9+
set(SRCS main.c)
10+
11+
set(requirements 1)
12+
# This API test requires the spawn feature to be enabled in lws
13+
require_lws_config(LWS_WITH_SPAWN 1 requirements)
14+
15+
if (requirements)
16+
17+
add_executable(${SAMP} ${SRCS})
18+
19+
if (websockets_shared)
20+
target_link_libraries(${SAMP} websockets_shared ${LIBWEBSOCKETS_DEP_LIBS})
21+
add_dependencies(${SAMP} websockets_shared)
22+
else()
23+
target_link_libraries(${SAMP} websockets ${LIBWEBSOCKETS_DEP_LIBS})
24+
endif()
25+
endif()

0 commit comments

Comments
 (0)