userspace: proxy: use a separate worker per core#10946
Draft
serhiy-katsyuba-intel wants to merge 2 commits into
Draft
userspace: proxy: use a separate worker per core#10946serhiy-katsyuba-intel wants to merge 2 commits into
serhiy-katsyuba-intel wants to merge 2 commits into
Conversation
Previously a single userspace worker (work queue) served requests for all userspace modules, regardless of the core they ran on, and the worker thread was re-pinned to the requesting module's core on every invocation. This actually only worked in single-core tests, but on multi-core pinning failed, because a thread can only be pinned while it is not running. Replace the single worker with a per-core worker array. Each worker thread is pinned to its core once, at creation time. Signed-off-by: Serhiy Katsyuba <serhiy.katsyuba@intel.com>
The proxy now uses a separate worker per core. Each work item is allocated, submitted and processed on the same core, so cross-core coherency is no longer required. Signed-off-by: Serhiy Katsyuba <serhiy.katsyuba@intel.com>
serhiy-katsyuba-intel
requested review from
dbaluta,
kv2019i,
lbetlej,
lgirdwood,
mmaka1 and
plbossart
as code owners
Contributor
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
This PR fixes multi-core userspace proxy handling by replacing a single shared userspace work queue (that was re-pinned per invocation) with a per-core worker array where each worker is pinned once at creation time.
Changes:
- Introduce a per-core
worker[CONFIG_CORE_COUNT]and route submissions/event usage through the current core’s worker. - Pin each worker thread to its core during worker creation (instead of re-pinning on each invoke).
- Remove the need for coherent allocation for work items by keeping work processing core-local.
|
|
||
| static int user_work_item_init(struct userspace_context *user_ctx, struct k_heap *user_heap) | ||
| { | ||
| int cpu = cpu_get_id(); |
| int ret; | ||
|
|
||
| ret = user_worker_get(); | ||
| ret = user_worker_get(cpu); |
Comment on lines
+173
to
+175
| work_item = sof_heap_alloc(user_heap, 0, sizeof(*work_item), 0); | ||
| if (!work_item) { | ||
| user_worker_put(); | ||
| user_worker_put(cpu); |
|
|
||
| #if !IS_ENABLED(CONFIG_SOF_USERSPACE_MOD_IPC_BY_DP_THREAD) | ||
| work_item->event = &worker.event; | ||
| work_item->event = &worker[cpu].event; |
| { | ||
| sof_heap_free(user_heap, user_ctx->work_item); | ||
| user_worker_put(); | ||
| user_worker_put(cpu_get_id()); |
Comment on lines
+214
to
+215
| int cpu = cpu_get_id(); | ||
| struct k_event * const event = &worker[cpu].event; |
| #if !IS_ENABLED(CONFIG_SOF_USERSPACE_MOD_IPC_BY_DP_THREAD) | ||
| /* Switch worker thread to module memory domain */ | ||
| ret = k_mem_domain_add_thread(user_ctx->comp_dom, worker.thread_id); | ||
| ret = k_mem_domain_add_thread(user_ctx->comp_dom, worker[cpu].thread_id); |
| #endif | ||
|
|
||
| ret = k_work_user_submit_to_queue(&worker.work_queue, &user_ctx->work_item->work_item); | ||
| ret = k_work_user_submit_to_queue(&worker[cpu].work_queue, &user_ctx->work_item->work_item); |
Comment on lines
+134
to
+137
| k_thread_suspend(worker[cpu].thread_id); | ||
|
|
||
| /* Pin worker thread to the same core as the module */ | ||
| k_thread_cpu_pin(worker[cpu].thread_id, cpu); |
|
|
||
| k_thread_access_grant(worker[cpu].thread_id, &worker[cpu].event); | ||
|
|
||
| k_thread_resume(worker[cpu].thread_id); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Previously a single userspace worker (work queue) served requests for all userspace modules, regardless of the core they ran on, and the worker thread was re-pinned to the requesting module's core on every invocation. This actually only worked in single-core tests, but on multi-core pinning failed, because a thread can only be pinned while it is not running.
Replace the single worker with a per-core worker array. Each worker thread is pinned to its core once, at creation time.