google_cloud_run.md

title	Google Cloud Run
aliases	/serverless/gcp/gcr
further_reading	link tag text https://www.datadoghq.com/blog/instrument-cloud-run-with-datadog-sidecar/ Blog Instrument Google Cloud Run applications with the new Datadog Agent sidecar

Overview

Google Cloud Run is a fully managed serverless platform for deploying and scaling container-based applications in Google Cloud. Datadog provides metrics and logs collection for these services through our Google Cloud Integration. This page describes the process of instrumenting your application code running in Google Cloud Run.

Setup

To instrument your Google Cloud Run applications with in-process instrumentation, see Google Cloud Run In-Process. For details on the tradeoffs between the Sidecar instrumentation described here and In-Process instrumentation, see Sidecar vs. In-Process Instrumentation for Google Cloud Run.

The overall process for instrumenting Google Cloud Run applications is to install a tracer and use a Sidecar to collect the custom metrics and traces from your application. The application is configured to write its logs to a volume shared with the sidecar which then forwards them to Datadog.

Applications

Set up a Datadog tracing library, configure the application to send dogstatsd metrics to port 8125, and send correctly-formatted logs to the shared volume.

For custom metrics, use the Distribution Metrics to correctly aggregate data from multiple Google Cloud Run instances.

{{< tabs >}} {{% tab "Node.js" %}} Add the dd-trace-js library to your application.

app.js

// The tracer includes a dogstatsd client.
// It will send profiling information if configured with DD_PROFILING_ENABLED.
const tracer = require('dd-trace').init({
	logInjection: true,
});

const express = require("express");
const app = express();

const { createLogger, format, transports } = require('winston');

// We can use the DD_SERVERLESS_LOG_PATH environment variable if it is available.
// While this is not necessary, it the log forwarding configuration centralized
// in the cloud run configuration.
const log_filename = process.env.DD_SERVERLESS_LOG_PATH?.replace("*.log", "app.log") || "/shared-logs/logs/app.log";
console.log(`writing logs to ${log_filename}`);

const logger = createLogger({
	level: 'info',
	exitOnError: false,
	format: format.json(),
	transports: [new transports.File({ filename: log_filename })],
});

app.get("/", (_, res) => {
	logger.info("Hello!");
	tracer.dogstatsd.distribution("our-sample-app.sample-metric", 1);

	res.status(200).json({ msg: "A traced endpoint with custom metrics" });
});

const port = process.env.PORT || 8080;
app.listen(port);

You can use npm install dd-trace to add the tracer to your package.

Dockerfile

Your Dockerfile can look something like this. This will create a minmimal application container with metrics, traces, logs, (ADD PROFILING!). Note that the dockerfile needs to be built for the the x86_64 architecture (use the --platform linux/arm64 parameter for docker build).

FROM node:22-slim

COPY app.js package.json package-lock.json .
RUN npm ci --only=production

# Initialize the tracer
ENV NODE_OPTIONS="--require dd-trace/init"

EXPOSE 8080

CMD ["node", "app.js"]

Details

The dd-trace-js library provides support for Tracing, Metrics, and Profiling.

Set the NODE_OPTIONS="--require dd-trace/init" environment variable in your docker container to include the dd-trace/init module when the Node.js process starts.

Application Logs need to be sent to a file that the sidecar container can access. The container setup is detailed below. Log and Trace Correlation possible when logging is combined with the dd-trace-js library. The sidecar finds log files based on the DD_SERVERLESS_LOG_PATH environment variable, usually /shared-volume/logs/*.log which will forward all of files ending in .log in the /shared-volume/logs directory.

Set DD_PROFILING_ENABLED to enable Profiling.

{{% /tab %}} {{% tab "Python" %}}

Example Code

# add the example code here, with traces, custom metrics, profiling, and logs

Details

Tracing

Profiling

Metrics

Logs

{{% /tab %}} {{% tab "Java" %}}

Example Code

// add the example code here, with traces, custom metrics, profiling, and logs

Details

Tracing

Profiling

Metrics

Logs

{{% /tab %}} {{% tab "Go" %}}

Example Code

// add the example code here, with traces, custom metrics, profiling, and logs

Details

Tracing

Profiling

Metrics

Logs

{{% /tab %}} {{% tab ".NET" %}}

Example Code

// add the example code here, with traces, custom metrics, profiling, and logs

Details

Tracing

Profiling

Metrics

Logs

{{% /tab %}} {{% tab "PHP" %}}

Example Code

// add the example code here, with traces, custom metrics, profiling, and logs

Details

Tracing

Profiling

Metrics

Logs

{{% /tab %}} {{< /tabs >}}

Containers

A sidecar gcr.io/datadoghq/serverless-init:latest container is used to collect telemetry from your application container and send it to datadog. The sidecar container is configured with a healthcheck for correct starup, and a shared volume for log forwarding, and the environment variables documented below.

Environment Variables

Variable	Container	Description
DD_SERVERLESS_LOG_PATH	Sidecar (and Application, see notes)	The path where the agent will look for logs. For example /shared-volume/logs/*.log. - Required
DD_API_KEY	Sidecar	Datadog API key - Required
DD_SITE	Sidecar	Datadog site - Required
DD_LOGS_INJECTION	Sidecar	When true, enrich all logs with trace data for supported loggers in Java, Node, .NET, and PHP. See additional docs for Python, Go, and Ruby.
DD_SERVICE	Sidecar and Application	See Unified Service Tagging.
DD_VERSION	Sidecar	See Unified Service Tagging.
DD_ENV	Sidecar	See Unified Service Tagging.
DD_TAGS	Sidecar	See Unified Service Tagging.
DD_HEALTH_PORT	Sidecar	The port for sidecar health checks. For example 9999

The DD_SERVERLESS_LOG_PATH environment variable is not required on the application. But it can be set there and then used to configure the application's log filename. This avoids manually synchronizing the Cloud Run service's log path with the application code that writes to it.

The DD_LOGS_ENABLED environment variable is not required.

TODO: write something about DD_SOURCE.

{{< tabs >}} {{% tab "GCR UI" %}}

1. On the Cloud Run service page, select Edit & Deploy New Revision.
2. Open the Volumes main tab and create a new volume for log forwarding.
   1. Make an In-Memory volume called shared-logs.
   2. You may set a size limit if necessary.
3. Open the Containers main tab and click Add Container to add a new gcr.io/datadoghq/serverless-init:latest sidecar container.
4. Click Add health check to add a Startup check for the container.
   1. Select the TCP probe type.
   2. Choose any free port (9999, for example). We will need this port number shortly for the DD_HEALTH_PORT variable.
5. Click the Variables & Secrets tab and add the required environment variables.
   • The DD_HEALTH_PORT variable should be the port for the TCP health check you configured.
   • The DD_SERVERLESS_LOG_PATH variable should be set to /shared-logs/logs/*.log where /shared-logs is the volume mount point we will use in the next step.
   • See the table above for the other required and suggested Environment Variables.
6. Click the Volume Mounts tab and add the logs volume mount.
   • Mount it at the location that matches the prefix of DD_SERVERLESS_LOG_PATH, for example /shared-logs for a /shared-logs/logs/*.log log path.
7. Edit the application container.
8. Click the Volume Mounts tab and add the logs volume mount.
   • Mount it to the same location that you did for the sidecar container, for example /shared-logs.
9. Click the Variables & Secrets tab and set the DD_SERVICE environment variable as you did for the sidecar.
10. Click the Settings tab and set the Container start up order to Depends on the sidecar container.
11. Deploy the application. {{% /tab %}} {{% tab "YAML deploy" %}}
12. Step
13. by
14. step
15. instructions
    • with some details. {{% /tab %}} {{% tab "Terraform" %}}
16. Step
17. by
18. step
19. instructions
    • with some details. {{% /tab %}} {{< /tabs >}}

Add a service label

Add a service label which matches the DD_SERVICE value on the containers to the Google Cloud service. Access this through the service list, through the Labels button after selecting the service.

Futher Reading

{{< partial name="whats-next/whats-next.html" >}}