Error when using VictoriaMetrics datasource with Grafana 12.4.0 which contains breaking change for plugins

[jiekun]

Describe the bug

Grafana released 12.4.0 with breaking change described in grafana/grafana#113412.

Starting in Grafana v12.4, plugin processes no longer receive all host environment variables by default.

In container environment, this may resulted in invalid path of the plugins (/tmp? /var/lib/grafana?) and caused the error. See the error logs below.

To Reproduce

Run Grafana v12.4.0 and VictoriaMetrics datasource in container.

Version

Grafana v12.4.0

Logs

logger=context userId=1 orgId=1 uname=admin t=2026-03-10T01:58:29.344928123Z level=error msg="Request Completed" method=GET path=/api/plugins/victoriametrics-metrics-datasource/settings status=500 remote_addr=127.0.0.1 time_ms=2 duration=2.757898ms size=36 referer=http://localhost:3000/connections/datasources/edit/victoria-metrics handler=/api/plugins/:pluginId/settings status_source=server error="Unrecognized remote plugin message: \nFailed to read any lines from plugin's stdout\nThis usually means\n  the plugin was not compiled for this architecture,\n  the plugin is missing dynamic-link libraries necessary to run,\n  the plugin is not executable by this process due to file permissions, or\n  the plugin failed to negotiate the initial go-plugin protocol handshake\n\nAdditional notes about plugin:\n  Path: /var/lib/grafana/plugins/victoriametrics-metrics-datasource/victoriametrics_metrics_backend_plugin_linux_arm64\n  Mode: -rwxr-xr-x\n  Owner: 10001 [?] (current: 10001 [?])\n  Group: 10001 [?] (current: 10001 [?])\n  ELF architecture: EM_AARCH64 (current architecture: arm64)\n"

Screenshots

No response

Used command-line flags

No response

Additional information

No response

[jiekun]

As an update:

The problem is in the hashicorp go-plugin library (used by ALL Grafana backend plugins) that needs a writable temp directory for Unix socket creation. We cannot fix it from the plugin side. There is an open issue - grafana/grafana#119424 in the Grafana repo.

[arturminchukov]

as workaround, you can use Migration step from the description.