Cache Minja Chat Template

[apaniukov]

Description

Avoid huge fix costs from chat template re-reading:

-    minja::chat_template minja_template(chat_tpl, m_bos_token, m_eos_token);
+    auto minja_template = get_cached_minja_chat_template(chat_tpl);

Before:

scenario                              msg tools    chars   mean(ms)    med(ms)    p95(ms)    min(ms)    max(ms)
---------------------------------------------------------------------------------------------------------------
single_user_no_system_no_tools          1     0      117     64.568     64.337     65.885     63.678     69.481
system_plus_single_user_no_tools        2     0      185     64.824     64.320     66.019     63.619     91.246
multi_turn_no_tools                     6     0      532     64.637     64.489     66.150     63.808     67.523
single_user_long_prompt_no_tools        2     0     2182     64.893     64.276     65.709     63.774     88.130
single_user_with_small_tools            2     1     1337     64.597     64.312     66.140     63.747     68.450
multi_turn_with_large_tools             6     3     2582     64.693     64.513     65.794     63.923     67.813

After:

scenario                              msg tools    chars   mean(ms)    med(ms)    p95(ms)    min(ms)    max(ms)
---------------------------------------------------------------------------------------------------------------
single_user_no_system_no_tools          1     0      117      0.102      0.100      0.123      0.081      0.160
system_plus_single_user_no_tools        2     0      185      0.107      0.106      0.115      0.102      0.122
multi_turn_no_tools                     6     0      532      0.175      0.174      0.187      0.166      0.202
single_user_long_prompt_no_tools        2     0     2182      0.129      0.128      0.138      0.123      0.154
single_user_with_small_tools            2     1     1337      0.146      0.144      0.155      0.138      0.164
multi_turn_with_large_tools             6     3     2582      0.302      0.300      0.321      0.290      0.329

CVS-189192

Fixes #(issue)

Checklist:

• This PR follows GenAI Contributing guidelines.
• Tests have been updated or added to cover the new code.
• This PR fully addresses the ticket.
• I have made corresponding changes to the documentation.

[Copilot]

Pull request overview

This PR adds an in-memory cache for compiled Minja chat templates inside TokenizerImpl to avoid repeatedly constructing/parsing the same template during apply_chat_template(), significantly reducing per-call overhead in chat-template application.

Changes:

• Introduced a small (size-limited) cache of minja::chat_template objects keyed by template string, guarded by a mutex.
• Updated apply_chat_template() to reuse cached templates and added locking around chat-template getters/setters to avoid data races.
• Cleared the cache on set_chat_template() to prevent reuse of stale compiled templates.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
src/cpp/src/tokenizer/tokenizer_impl.hpp	Adds mutex + unordered_map members for a Minja chat-template cache and declares a private cache accessor.
src/cpp/src/tokenizer/tokenizer_impl.cpp	Implements cached template retrieval and updates chat-template application and setters/getters to use locking and the cache.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.