Enable user-space for DP (part 4 of #10287) #10386

lyakh · 2025-11-21T16:39:57Z

The rest of DP-userspace commits, WiP.

scheduler_dp_recalculate() and dp_thread_fn() will have significant difference between the "proxy" and "native" versions. Extract them into a separate file for easy forking. Signed-off-by: Guennadi Liakhovetski <[email protected]>

DP tasks are recalculated on every LL tick (typically every 1ms). DP threads then run with varying periodicity and duration. They can take 10 or even 100ms to run. Also recauculating tasks in the end of those runs doesn't improve anything and only adds load and a need to protect data. This is even worse when DP threads run in user space. Skip recalculation for the native version. Signed-off-by: Guennadi Liakhovetski <[email protected]>

module_is_ready_to_process() can call the .is_ready_to_process() module method, therefore it can only be called from the DP thread context itself, not from the scheduler. Signed-off-by: Guennadi Liakhovetski <[email protected]>

With the native DP version IPCs should be processed on the thread itself. Prepare the scheduler for the transition. Signed-off-by: Guennadi Liakhovetski <[email protected]>

Instead of calling module hooks inline signal the DP thread to call them in the thread context in DP-scheduled module case. Signed-off-by: Guennadi Liakhovetski <[email protected]>

Add a heap parameter to buffer allocation functions. This makes buffer structures accessible to the user-space. Signed-off-by: Guennadi Liakhovetski <[email protected]>

The only functions that have to be converted to syscalls are mod_alloc_ext() and mod_free(), the rest of the API is implemented using inline functions. Signed-off-by: Guennadi Liakhovetski <[email protected]>

When a LLEXT module is freed, its partitioins should be removed from any memory domains. Add a function for that. Signed-off-by: Guennadi Liakhovetski <[email protected]>

Fix wrong sparse type-casting. Signed-off-by: Guennadi Liakhovetski <[email protected]>

Run DP threads in user-space. Move all the respective memory and kobjects to a dedicated memory domain. Work around Zephyr inability to remove memory domains on Xtensa. Signed-off-by: Guennadi Liakhovetski <[email protected]>

Copilot

Pull request overview

This pull request enables user-space support for the DP (Data Processing) scheduler in SOF (Sound Open Firmware), implementing part 4 of #10287. The changes introduce two implementations: a "native" version with full userspace capabilities and a "proxy" version with limited support, conditional on the CONFIG_SOF_USERSPACE_PROXY flag.

Key changes include:

Splitting DP scheduler implementation into native and proxy versions to support different userspace configurations
Adding syscall wrappers for memory allocation functions (mod_alloc_ext, mod_free, mod_fast_get) to enable userspace modules
Modifying buffer allocation API to accept heap parameters for DP module-specific heaps
Implementing memory domain management for userspace DP modules
Updating sparse annotations removal for stack allocation functions

Reviewed changes

Copilot reviewed 34 out of 34 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
src/schedule/zephyr_dp_schedule_native.c	New native implementation with full IPC handling and memory domain support for userspace DP modules
src/schedule/zephyr_dp_schedule_proxy.c	New proxy implementation with simplified userspace support
src/schedule/zephyr_dp_schedule.h	Shared header defining common structures for both DP scheduler implementations
src/schedule/zephyr_dp_schedule.c	Refactored to move implementation-specific code to native/proxy files
src/schedule/CMakeLists.txt	Updated build configuration to conditionally compile native or proxy implementation
src/audio/module_adapter/module/generic.c	Added syscall implementations and verification handlers for userspace module APIs
src/audio/module_adapter/module_adapter.c	Updated to use per-module heaps for DP components with reference counting
src/audio/buffers/comp_buffer.c	Modified buffer allocation to support DP-specific heaps and reference counting
src/include/sof/audio/module_adapter/module/generic.h	Added syscall declarations for module memory APIs
src/include/sof/audio/buffer.h	Updated buffer allocation API signatures to include heap parameter
src/ipc/ipc4/helper.c	Modified component connection to use DP heap for DP domain components
src/library_manager/llext_manager.c	Added `llext_manager_rm_domain` function to remove library partitions from memory domains
zephyr/lib/userspace_helper.c	Removed sparse annotations from stack allocation functions
test/cmocka/src/*/.c	Updated all test code to pass NULL heap parameter to buffer_new calls
app/prj.conf	Added dynamic thread allocation configuration
app/boards/intel_adsp_ace30_ptl.conf	Enabled userspace support

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-11-26T16:00:16Z

src/schedule/zephyr_dp_schedule_proxy.c

+
+#include "zephyr_dp_schedule.h"
+
+/* Go through all DP tasks and recalculate their readness and dedlines


Spelling error: "readness" should be "readiness".

Suggested change

/* Go through all DP tasks and recalculate their readness and dedlines

/* Go through all DP tasks and recalculate their readiness and deadlines

Copilot · 2025-11-26T16:00:16Z

src/schedule/zephyr_dp_schedule_proxy.c

+
+#include "zephyr_dp_schedule.h"
+
+/* Go through all DP tasks and recalculate their readness and dedlines


Spelling error: "dedlines" should be "deadlines".

Suggested change

/* Go through all DP tasks and recalculate their readness and dedlines

/* Go through all DP tasks and recalculate their readness and deadlines

Copilot · 2025-11-26T16:00:17Z

src/schedule/zephyr_dp_schedule_native.c

+		.size = size,
+		.attr = K_MEM_PARTITION_P_RW_U_RW,
+	};
+	/* IPC flattening buffer partitioin */


Spelling error: "partitioin" should be "partition".

Suggested change

/* IPC flattening buffer partitioin */

/* IPC flattening buffer partition */

Copilot · 2025-11-26T16:00:17Z

src/audio/module_adapter/module_adapter.c

+	list_item_del(&mod->dev->bsink_list);
 	sof_heap_free(mod_heap, mod->dev);
 	sof_heap_free(mod_heap, mod);
+	if (!mod_heap || !--mod_heap_user->client_count)


Logic error: The condition !mod_heap will always be false at this point because mod_heap_user is derived from mod_heap using container_of. If mod_heap were NULL, the container_of would have already caused undefined behavior. The check should be if (!mod_heap_user || !--mod_heap_user->client_count) or restructured to check mod_heap_user before calling container_of.

Copilot · 2025-11-26T16:00:17Z

src/audio/module_adapter/module_adapter.c

 static void module_adapter_mem_free(struct processing_module *mod)
 {
 	struct k_heap *mod_heap = mod->priv.resources.heap;
+	struct dp_heap_user *mod_heap_user = container_of(mod_heap, struct dp_heap_user, heap);


Incorrect use of container_of: When not in DP domain, mod_heap points to drv->user_heap (not part of a dp_heap_user structure), making container_of() invalid. This will compute an incorrect pointer. Should check if the module used a DP heap before using container_of, similar to how line 112 sets mod_heap_user = NULL for non-DP cases.

Copilot · 2025-11-26T16:00:18Z

src/audio/buffers/comp_buffer.c

-	rfree(buffer);
+	sof_heap_free(heap, buffer);
+	if (heap) {
+		struct dp_heap_user *mod_heap_user = container_of(heap, struct dp_heap_user, heap);


Incorrect use of container_of: When heap is NULL (non-DP buffers), container_of(heap, struct dp_heap_user, heap) will compute an invalid pointer. The container_of call should only be made inside the if (heap) block, or the check should be restructured to verify the heap is actually a DP heap before using container_of.

Copilot · 2025-11-26T16:00:18Z

src/audio/module_adapter/module_adapter.c

 emod:
-	if (mod_heap != drv->user_heap)
-		rfree(mod_heap);
+	rfree(mod_heap_user);


Potential NULL pointer dereference: mod_heap_user can be NULL when not in DP domain (line 112), but it's being passed to rfree() unconditionally. Should check if mod_heap_user is not NULL before calling rfree(mod_heap_user).

Suggested change

rfree(mod_heap_user);

if (mod_heap_user)

rfree(mod_heap_user);

Copilot · 2025-11-26T16:00:18Z

src/schedule/zephyr_dp_schedule_native.c

+			memcpy(flat->pipeline_state.source_sink + flat->pipeline_state.n_sources,
+			       param->pipeline_state.sinks,
+			       flat->pipeline_state.n_sinks *
+			       sizeof(flat->pipeline_state.source_sink[0]));


Missing break statement: The COMP_TRIGGER_PREPARE case (line 87) is missing a break statement, which will cause fall-through to the next case. This appears to be unintentional. Add break; after line 103.

Suggested change

sizeof(flat->pipeline_state.source_sink[0]));

sizeof(flat->pipeline_state.source_sink[0]));

break;

Copilot · 2025-11-26T16:00:19Z

src/schedule/zephyr_dp_schedule_native.c

+		}
+	}


Missing break statement: The COMP_TRIGGER_PREPARE case (line 139) is missing a break statement at the end of both the inner and outer switch. This will cause fall-through. Add break; after line 145 and possibly after the inner switch closing brace.

Suggested change

}

}

break;

}

break;

or the fall through is needed if intended.

kv2019i

Nothing major. A few comments inline on syscall verification, resta re minor.

kv2019i · 2025-11-27T10:36:51Z

src/schedule/CMakeLists.txt

 		zephyr_ll.c
 	)
 endif()



typo in commit: recauculating

kv2019i · 2025-11-27T10:43:03Z

src/schedule/zephyr_dp_schedule_native.c

 	ARG_UNUSED(p3);

 	do {
+		if (first) {


unlikely perhaps?

can we comment this as it looks like an first run or initialisation only flow.

I'll fix this - it'll just go above the loop

kv2019i · 2025-11-27T10:47:03Z

src/schedule/zephyr_dp_schedule_native.c

+
+				default:
+					/* illegal state, serious defect, won't happen */
+					k_panic();


Maybe use K_OOPS() so this could just erase the user thread (If/when implemented for xtensa)?

kv2019i · 2025-11-27T10:48:55Z

src/audio/module_adapter/module/generic.c

 	/* Now we can proceed with module specific initialization */
-	ret = interface->init(mod);
+#if CONFIG_IPC_MAJOR_4
+	if (mod->dev->ipc_config.proc_domain == COMP_PROCESSING_DOMAIN_DP)


Is this safe for the proxy-dp implementation?

src/audio/module_adapter/module/generic.c

lgirdwood

The first 2 patches are just moving/partitioning existing code, so are good to go now. Also some fixes in later patches too can go now.

lgirdwood · 2025-11-27T12:42:44Z

src/schedule/zephyr_dp_schedule_native.c

+		}
+	}


or the fall through is needed if intended.

lgirdwood · 2025-11-27T12:44:16Z

src/schedule/zephyr_dp_schedule_native.c

 	ARG_UNUSED(p3);

 	do {
+		if (first) {


can we comment this as it looks like an first run or initialisation only flow.

lgirdwood · 2025-11-27T12:56:45Z

src/include/sof/llext_manager.h

 int llext_manager_add_library(uint32_t module_id);

 int llext_manager_add_domain(const uint32_t component_id, struct k_mem_domain *domain);
+int llext_manager_rm_domain(const uint32_t component_id, struct k_mem_domain *domain);


spelling partitions in commit message.

lyakh · 2025-11-27T16:32:31Z

let's split this #10398

No issues remain, clearing my -1

lyakh force-pushed the user-p3 branch from c7b0fe9 to a985b5a Compare November 26, 2025 12:04

lyakh added 5 commits November 26, 2025 14:27

dp: extract 2 functions into a separate file

a877743

scheduler_dp_recalculate() and dp_thread_fn() will have significant difference between the "proxy" and "native" versions. Extract them into a separate file for easy forking. Signed-off-by: Guennadi Liakhovetski <[email protected]>

dp: native: move IPC execution to the thread

8f0ffee

With the native DP version IPCs should be processed on the thread itself. Prepare the scheduler for the transition. Signed-off-by: Guennadi Liakhovetski <[email protected]>

module-adapter: switch to DP signaling

80d2119

Instead of calling module hooks inline signal the DP thread to call them in the thread context in DP-scheduled module case. Signed-off-by: Guennadi Liakhovetski <[email protected]>

lyakh force-pushed the user-p3 branch 3 times, most recently from 790e01f to 917349f Compare November 26, 2025 14:14

lyakh added 4 commits November 26, 2025 15:38

buffer: allocate on specific heap

cfa2239

Add a heap parameter to buffer allocation functions. This makes buffer structures accessible to the user-space. Signed-off-by: Guennadi Liakhovetski <[email protected]>

mod_alloc: convert to syscalls

3bdf045

The only functions that have to be converted to syscalls are mod_alloc_ext() and mod_free(), the rest of the API is implemented using inline functions. Signed-off-by: Guennadi Liakhovetski <[email protected]>

llext: add a function to remove a memory domain

19e0971

When a LLEXT module is freed, its partitioins should be removed from any memory domains. Add a function for that. Signed-off-by: Guennadi Liakhovetski <[email protected]>

userspace: (cosmetic) clean up sparse use

9820996

Fix wrong sparse type-casting. Signed-off-by: Guennadi Liakhovetski <[email protected]>

lyakh force-pushed the user-p3 branch 2 times, most recently from a72e1f5 to f123bd6 Compare November 26, 2025 15:15

lyakh changed the title ~~[WiP][SKIP SOF-TEST] Finally enable user-space for DP (part 4 of #10287)~~ [WiP] Finally enable user-space for DP (part 4 of #10287) Nov 26, 2025

lyakh force-pushed the user-p3 branch from f123bd6 to 1d75c53 Compare November 26, 2025 15:41

dp: convert to user-space

364e1e1

Run DP threads in user-space. Move all the respective memory and kobjects to a dedicated memory domain. Work around Zephyr inability to remove memory domains on Xtensa. Signed-off-by: Guennadi Liakhovetski <[email protected]>

lyakh force-pushed the user-p3 branch from 1d75c53 to 364e1e1 Compare November 26, 2025 15:52

lyakh marked this pull request as ready for review November 26, 2025 15:53

Copilot AI review requested due to automatic review settings November 26, 2025 15:53

lyakh requested review from LaurentiuM1234, abonislawski, bardliao, dbaluta, iuliana-prodan, kv2019i, marcinszkudlinski, pblaszko and ranj063 as code owners November 26, 2025 15:53

lyakh requested review from RanderWang, jxstelter, lbetlej, lgirdwood, mmaka1 and plbossart as code owners November 26, 2025 15:53

Copilot started reviewing on behalf of lyakh November 26, 2025 15:53 View session

Copilot finished reviewing on behalf of lyakh November 26, 2025 15:56

Copilot AI reviewed Nov 26, 2025

View reviewed changes

lgirdwood changed the title ~~[WiP] Finally enable user-space for DP (part 4 of #10287)~~ Enable user-space for DP (part 4 of #10287) Nov 27, 2025

kv2019i previously requested changes Nov 27, 2025

View reviewed changes

lgirdwood reviewed Nov 27, 2025

View reviewed changes

lyakh mentioned this pull request Dec 1, 2025

DP: Fork "native" and "proxy" versions (part 4.1 of #10287) #10398

Open


		#include "zephyr_dp_schedule.h"

		/* Go through all DP tasks and recalculate their readness and dedlines

	/* IPC flattening buffer partitioin */
	/* IPC flattening buffer partition */

	rfree(mod_heap_user);
	if (mod_heap_user)
	rfree(mod_heap_user);

	sizeof(flat->pipeline_state.source_sink[0]));
	sizeof(flat->pipeline_state.source_sink[0]));
	break;

-		}
-	}
+			break;
+		}
+		break;

Enable user-space for DP (part 4 of #10287) #10386

Are you sure you want to change the base?

Enable user-space for DP (part 4 of #10287) #10386

Uh oh!

Conversation

lyakh commented Nov 21, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Nov 26, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 26, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 26, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 26, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 26, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 26, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 26, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 26, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 26, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kv2019i left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

lgirdwood left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lyakh commented Nov 27, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants