Bug: No support for Template Jinja2 i18n (maybe I am wrong - Templates without i18n works)

[ChameleonRed]

Description

I read documentation and I found that I can use Templates - it works, but I try to use it with i18n.

Problem is that for all request language settings are same because is only one Jinja2 environment.

I can modify this environment per request with request.app.template_engine but what if I modify this and next parallel request modify it to - at the end Polish user can get Chinese translation since it is async.

Example of setup:

jinja_environment = Environment(
    loader=FileSystemLoader(
        searchpath=[
            'template',
            'authentication/template'
        ]
    ),
    extensions=[
        'jinja2.ext.i18n'
    ],
    autoescape=True,
)
# translations = get_translation(settings.locale_settings.possible_locales)
# jinja_environment.install_gettext_translations(translations)

listastar use request.app.engine to render do so if I do:

translations = get_translation(settings.locale_settings.possible_locales)
jinja_environment.install_gettext_translations('pl-PL')

No idea if request like this fail or not.

@get('/')
async def index(request: Request) -> Template:
    translations = get_translation(settings.locale_settings.possible_locales)
    # assume cookie read here
    jinja_environment.install_gettext_translations('pl-PL')
    return Template(
        template_name='index.jinja',
        media_type='text/html',
        context={
            'redirect_url': request.url.path,
        }
    )

since meanwhile can be 'en-US'

@get('/')
async def index(request: Request) -> Template:
    translations = get_translation(settings.locale_settings.possible_locales)
    # assume cookie read here
    jinja_environment.install_gettext_translations('en-US')
    return Template(
        template_name='index.jinja',
        media_type='text/html',
        context={
            'redirect_url': request.url.path,
        }
    )

Examples are simplified and handlers can be much longer.

URL to code causing the issue

No response

MCVE

Steps to reproduce

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

Screenshots

No response

Logs

Litestar Version

latest 2.19

Platform

• Linux
• Mac
• Windows
• Other (Please specify in the description above)

[ChameleonRed]

Some config to make easier analysis:

app = Litestar(
    route_handlers=[
        index,
    ],
    template_config=TemplateConfig(
        engine=JinjaTemplateEngine.from_environment(
            jinja_environment=jinja_environment
        ),
    ),
 )

[ChameleonRed]

I assume that parallel run race is possible.

I can use state to workaround this with make env[lang]for all translations - maybe it is solution. It will separate different languages.

[euri10]

is there a question or an issue here sorry I dont understand.

[ChameleonRed]

It is not question see this code - is there support for i18n or is it i18n mention in Documentation?

class Template(Response[bytes]):
    """Template-based response, rendering a given template into a bytes string."""

    __slots__ = (
        "context",
        "template_name",
        "template_str",
    )

    def __init__(
        self,
        template_name: str | None = None,
        *,
        template_str: str | None = None,
        background: BackgroundTask | BackgroundTasks | None = None,
        context: dict[str, Any] | None = None,
        cookies: ResponseCookies | None = None,
        encoding: str = "utf-8",
        headers: dict[str, Any] | None = None,
        media_type: MediaType | str | None = None,
        status_code: int = HTTP_200_OK,
    ) -> None:
        """Handle the rendering of a given template into a bytes string.

        Args:
            template_name: Path-like name for the template to be rendered, e.g. ``index.html``.
            template_str: A string representing the template, e.g. ``tmpl = "Hello <strong>World</strong>"``.
            background: A :class:`BackgroundTask <.background_tasks.BackgroundTask>` instance or
                :class:`BackgroundTasks <.background_tasks.BackgroundTasks>` to execute after the response is finished.
                Defaults to ``None``.
            context: A dictionary of key/value pairs to be passed to the temple engine's render method.
            cookies: A list of :class:`Cookie <.datastructures.Cookie>` instances to be set under the response
                ``Set-Cookie`` header.
            encoding: Content encoding
            headers: A string keyed dictionary of response headers. Header keys are insensitive.
            media_type: A string or member of the :class:`MediaType <.enums.MediaType>` enum. If not set, try to infer
                the media type based on the template name. If this fails, fall back to ``text/plain``.
            status_code: A value for the response HTTP status code.
        """
        if not (template_name or template_str):
            raise ValueError("Either template_name or template_str must be provided.")

        if template_name and template_str:
            raise ValueError("Either template_name or template_str must be provided, not both.")

        super().__init__(
            background=background,
            content=b"",
            cookies=cookies,
            encoding=encoding,
            headers=headers,
            media_type=media_type,
            status_code=status_code,
        )
        self.context = context or {}
        self.template_name = template_name
        self.template_str = template_str

    def create_template_context(self, request: Request) -> dict[str, Any]:
        """Create a context object for the template.

        Args:
            request: A :class:`Request <.connection.Request>` instance.

        Returns:
            A dictionary holding the template context
        """
        csrf_token = value_or_default(ScopeState.from_scope(request.scope).csrf_token, "")
        return {
            **self.context,
            "request": request,
            "csrf_input": f'<input type="hidden" name="_csrf_token" value="{csrf_token}" />',
        }

    def to_asgi_response(
        self,
        app: Litestar | None,
        request: Request,
        *,
        background: BackgroundTask | BackgroundTasks | None = None,
        cookies: Iterable[Cookie] | None = None,
        encoded_headers: Iterable[tuple[bytes, bytes]] | None = None,
        headers: dict[str, str] | None = None,
        is_head_response: bool = False,
        media_type: MediaType | str | None = None,
        status_code: int | None = None,
        type_encoders: TypeEncodersMap | None = None,
    ) -> ASGIResponse:
        if app is not None:
            warn_deprecation(
                version="2.1",
                deprecated_name="app",
                kind="parameter",
                removal_in="3.0.0",
                alternative="request.app",
            )

        if not (template_engine := request.app.template_engine):
            raise ImproperlyConfiguredException("Template engine is not configured")

        headers = {**headers, **self.headers} if headers is not None else self.headers
        cookies = self.cookies if cookies is None else itertools.chain(self.cookies, cookies)

        media_type = self.media_type or media_type
        if not media_type:
            if self.template_name:
                suffixes = PurePath(self.template_name).suffixes
                for suffix in suffixes:
                    if _type := guess_type(f"name{suffix}")[0]:
                        media_type = _type
                        break
                else:
                    media_type = MediaType.TEXT
            else:
                media_type = MediaType.HTML

        context = self.create_template_context(request)

        if self.template_str is not None:
            body = template_engine.render_string(self.template_str, context)
        else:
            # cast to str b/c we know that either template_name cannot be None if template_str is None
            template = template_engine.get_template(cast("str", self.template_name))
            body = template.render(**context).encode(self.encoding)

        return ASGIResponse(
            background=self.background or background,
            body=body,
            content_length=None,
            cookies=cookies,
            encoded_headers=encoded_headers,
            encoding=self.encoding,
            headers=headers,
            is_head_response=is_head_response,
            media_type=media_type,
            status_code=self.status_code or status_code,
        )

[ChameleonRed]

I think something like this is need:

class I18nTemplate(Response[bytes]):
    """Template-based response, rendering a given template into a bytes string."""

    __slots__ = (
        "context",
        "template_name",
        "template_str",
    )

    def __init__(
        self,
        template_name: str | None = None,
        language_code: str | None,
        *,
        template_str: str | None = None,
        background: BackgroundTask | BackgroundTasks | None = None,
        context: dict[str, Any] | None = None,
        cookies: ResponseCookies | None = None,
        encoding: str = "utf-8",
        headers: dict[str, Any] | None = None,
        media_type: MediaType | str | None = None,
        status_code: int = HTTP_200_OK,
    ) -> None:

Also need extra jinja2 engine for each engine with different translations.
Some very rough example ... - it is not match litestar (you can add translation to settings).

Currently litestar generate only one jinja2 engine for all language so race conflict looks possible.

def get_jinja_environment(language: str, state: State) -> jinja2.Environment:
    """
    Create a jinja environment for a given language.

    :param language: Language code like "en-US" or "pl-PL".
    :param state: Web server state.
    :return: Jinja environment.
    """

    if jinja_environment := state.jinja_environments.get(language):
        return jinja_environment

    jinja_environment = jinja2.Environment(
        loader=jinja2.FileSystemLoader(
            searchpath=[
                'template',
                'authentication/template'
            ]
        ),
        extensions=[
            'jinja2.ext.i18n'
        ],
        autoescape=True,
    )

    translation = get_translation((language,))
    jinja_environment.install_gettext_translations(translation)
    state.jinja_environments[language] = jinja_environment
    return jinja_environment

[ChameleonRed]

As you see current engine not allow multi languages - am I wrong.
Same with application configuration and Template.
I can use Template but can be problems with i18n.

I create some workaround but I think it should be fixed to allow i18n - what do you think is it possible to fix it.

[euri10]

I just crafted a quick and dirty test that seems to work, I'm not sure where you have race conditions happening, if that's the case please provide a MCVE to reprodice, as far as I can tell you can use the extension, this said I've never ever played with it so....

import gettext
from pathlib import Path
import subprocess
import pytest
from jinja2 import Environment, FileSystemLoader
from litestar import Litestar, Request, get
from litestar.response import Template
from litestar.contrib.jinja import JinjaTemplateEngine
from litestar.template.config import TemplateConfig
from litestar.testing import AsyncTestClient


async def test_litestar_jinja2_i18n_accept_language(tmp_path: Path):

    template_dir = tmp_path / "templates"
    template_dir.mkdir()
    template_path = template_dir / "hello.html"
    with open(template_path, "w") as f:
        f.write('{{ _("Hello") }}')

    locale_dir = tmp_path / "locale"
    langs = {"en": "Hello", "pl": "Cześć", "zh": "你好"}
    domain = "messages"

    for lang, translation in langs.items():
        lang_dir = locale_dir / lang / "LC_MESSAGES"
        lang_dir.mkdir(parents=True)
        po_path = lang_dir / f"{domain}.po"
        mo_path = lang_dir / f"{domain}.mo"
        with open(po_path, "w") as f:
            f.write(
                f'''msgid ""
msgstr ""
"Content-Type: text/plain; charset=utf-8\\n"
"Language: {lang}\\n"

#: {template_dir}/hello.html:1
msgid "Hello"
msgstr "{translation}"
'''
            )
        try:
            subprocess.run(["msgfmt", str(po_path), "-o", str(mo_path)], check=True)
        except Exception:
            pytest.skip("msgfmt not available to compile .po files")

    trans_cache = {}
    for lang in langs:
        try:
            trans_cache[lang] = gettext.translation(domain, localedir=locale_dir, languages=[lang])
        except FileNotFoundError:
            pass

    env = Environment(
        loader=FileSystemLoader(str(template_dir)),
        extensions=["jinja2.ext.i18n"],
        autoescape=True,
    )
    template_engine = JinjaTemplateEngine.from_environment(env)

    @get("/")
    async def handler(request: Request) -> Template:
        accept_language = request.headers.get("accept-language")
        lang = accept_language if accept_language in langs else "en"
        translator = trans_cache.get(lang, trans_cache["en"])
        return Template(template_name="hello.html", context={"_": translator.gettext})

    app = Litestar(route_handlers=[handler], template_config=TemplateConfig(engine=template_engine))

    async with AsyncTestClient(app) as client_en, AsyncTestClient(app) as client_pl, AsyncTestClient(app) as client_zh:
        resp_en = await client_en.get("/", headers={"accept-language": "en"})
        print(resp_en.text)
        assert resp_en.status_code == 200
        assert "Hello" in resp_en.text

        resp_pl = await client_pl.get("/", headers={"accept-language": "pl"})
        print(resp_pl.text)
        assert resp_pl.status_code == 200
        assert "Cześć" in resp_pl.text

        resp_zh = await client_zh.get("/", headers={"accept-language": "zh"})
        print(resp_zh.text)
        assert resp_zh.status_code == 200
        assert "你好" in resp_zh.text