model_zoo.get_model silently drops sess_options → thread oversubscription in CPU-quota containers (50s vs 0.27s inference)

[sigma-plus]

Environment

• insightface 1.0.1 (PyPI), onnxruntime 1.29.0, CPU-only (CPUExecutionProvider)
• Linux arm64, Docker container with cgroup CPU quota = 4 cores on a 128-core host (cpuset-limited to cores 0-3)
• Model pack: buffalo_l (5 ONNX sessions)

Symptom

FaceAnalysis inference was ~50 seconds per call (recognize) / ~35s (register) in the container, vs ~0.27s after the workaround below — a ~185x degradation caused purely by thread oversubscription.

ls /proc/<pid>/task | wc -l showed 646 threads: 5 sessions x 128 threads + main thread. All 640 inference threads were pinned by the container's cpuset onto 4 physical cores, burning the quota on context switches and spin-waits.

Root cause

model_zoo.get_model() accepts arbitrary **kwargs but only forwards providers / provider_options to the session constructor — sess_options is silently dropped:

# insightface/model_zoo/model_zoo.py
def get_model(name, **kwargs):
    ...
    router = ModelRouter(model_file)
    providers = kwargs.get('providers', get_default_providers())
    provider_options = kwargs.get('provider_options', get_default_provider_options())
    model = router.get_model(providers=providers, provider_options=provider_options)
    return model

So this intuitive call does nothing, without any warning:

analyzer = FaceAnalysis(name="buffalo_l", root=..., providers=["CPUExecutionProvider"],
                        sess_options=so)   # <-- silently ignored

This matters because onnxruntime sizes its intra-op thread pool from the total system core count (sysconf(_SC_NPROCESSORS_ONLN)), which ignores both cgroup CFS quota and cpuset affinity. In any CPU-limited container this produces heavy oversubscription (see also #2558 and microsoft/onnxruntime#8313). OMP_NUM_THREADS does not help either, since the pypi build does not use OpenMP for its thread pool.

Secondary issue: monkeypatching onnxruntime.InferenceSession cannot work

The session is created via the subclass PickableInferenceSession, which binds to onnxruntime.InferenceSession at import time and is not exported (not in __all__, and not reachable from the insightface.model_zoo package namespace). Replacing ort.InferenceSession after importing insightface has no effect; the only workaround is patching insightface.model_zoo.model_zoo.PickableInferenceSession itself (submodule import required).

Suggested fix

Forward sess_options in get_model() (and/or accept it explicitly on FaceAnalysis), ~3 lines:

def get_model(name, **kwargs):
    ...
    sess_options = kwargs.get('sess_options', None)
    model = router.get_model(providers=providers, provider_options=provider_options,
                             sess_options=sess_options)

(or simply pass **kwargs through to ModelRouter.get_model / PickableInferenceSession.)

Workaround we use in production

import onnxruntime as ort
import insightface.model_zoo.model_zoo as model_zoo

class _PinnedSession(model_zoo.PickableInferenceSession):
    def __init__(self, model_path, **kw):
        so = ort.SessionOptions()
        so.intra_op_num_threads = 4  # container CPU limit
        so.inter_op_num_threads = 1
        kw["sess_options"] = so
        super().__init__(model_path, **kw)

model_zoo.PickableInferenceSession = _PinnedSession

Result: 646 → 26 threads, inference 50s → 0.27s on the same container.

Related: #2558 (feature request to expose session options — this report adds the container/CPU-quota failure mode, measurements and root-cause analysis).