Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Protobuf scala span creation #2069

Merged
merged 17 commits into from
Apr 8, 2024
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/config/validate-links.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
"ignorePatterns": [
{ "pattern": "^https://mvnrepository\\.com" },
{ "pattern": "^http://127.0.0.1:8080" },
{ "pattern": "^https://site\\.mockito\\.org/" }
{ "pattern": "^http://jaeger:4317"},
{ "pattern": "^http://localhost:16686"}
]
}
60 changes: 51 additions & 9 deletions docs/src/modules/java-protobuf/pages/actions.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -449,29 +449,59 @@ include::example$java-protobuf-valueentity-counter/src/main/java/com/example/act
Please note that, the result of a side effect is ignored by the current command meaning that even if the call to
the `Counter` entity fails, the `Action` reply will succeed.


== Adding tracing spans

To add spans in your actions you can add the tracer available in the link:{attachmentsdir}/api/kalix/javasdk/action/ActionContext.html[`ActionContext`{tab-icon}, window="new"] and with it create the span.

To get the tracer from the link:{attachmentsdir}/api/kalix/javasdk/action/ActionContext.html[`ActionContext`{tab-icon}, window="new"]:
To create link:https://opentelemetry.io/docs/specs/otel/trace/api/#span[`spans`{tab-icon}, window="new"] in your actions you need the link:https://opentelemetry.io/docs/specs/otel/trace/api/#tracer[`tracer`{tab-icon}, window="new"] available through the `actionContext()` method.

[.tabset]
Java::
+
To add spans in your actions you can add the tracer available in the link:{attachmentsdir}/api/kalix/javasdk/action/ActionContext.html[`ActionContext`{tab-icon}, window="new"] and with it create the span.
+
To get the tracer from the link:{attachmentsdir}/api/kalix/javasdk/action/ActionContext.html[`ActionContext`{tab-icon}, window="new"]:
`actionContext()` gives you the link:{attachmentsdir}/api/kalix/javasdk/action/ActionContext.html[`ActionContext`{tab-icon}, window="new"]. That is, some methods that apply to the context of the action's request.
+
[source,java,indent=0]
.src/main/java/com/example/ControllerAction.java
----
include::example$java-protobuf-tracing/src/main/java/com/example/ControllerAction.java[tag=get-tracer]
----

Note that if link:https://docs.kalix.io/operations/observability-exports.html#_activating_tracing_beta[tracing] is not enabled in your service this `Optional` will be empty. Otherwise you can map over `tracerOpt` to retreive the tracer.
Scala::
+
`actionContext()` gives you the link:{attachmentsdir}/scala-api/kalix/scalasdk/action/ActionContext.html[`ActionContext`{tab-icon}, window="new"]. That is, some methods that apply to the context of the action's request.
+
[source,java,indent=0]
.src/main/scala/com/example/ControllerAction.java
----
include::example$scala-protobuf-tracing/src/main/scala/com/example/ControllerAction.scala[tag=get-tracer]
----

IMPORTANT: If your tracing is enabled, you will get a tracer that actually creates span and exports data as expected.
franciscolopezsancho marked this conversation as resolved.
Show resolved Hide resolved
But if tracing is not enabled, you will get a no operational tracer instead, which will not create traces.

Trace generation is disabled by default. To enable it in a service deployed to Kalix, see link:https://docs.kalix.io/operations/observability-exports.html#_activating_tracing_beta[`here`].
To enable it in a service running on your local machine you need to add the following `JAVA_TOOL_OPTIONS` to your `docker-compose.yml` in the base of your project.
[source,yaml]
.docker-compose.yml
---
services:
kalix-runtime:
image: ...
...
environment:
JAVA_TOOL_OPTIONS: >
-Dkalix.proxy.telemetry.tracing.enabled=true
-Dkalix.proxy.telemetry.tracing.collector-endpoint=http://jaeger:4317
...
jaeger:
image: jaegertracing/all-in-one:1.54
ports:
- 4317:4317
- 16686:16686
---

Here the traces are pushed to a `jaeger` docker image, to the port `4317`. And you can check them out at `http://localhost:16686`.
franciscolopezsancho marked this conversation as resolved.
Show resolved Hide resolved

To create a link:https://opentelemetry.io/docs/languages/java/instrumentation/#create-spans[span] and end it over an asynchronous call, you can do the following:

To link:https://opentelemetry.io/docs/languages/java/instrumentation/#create-spans[create a span] and end it over an asynchronous call, you can do the following:

[.tabset]
Java::
Expand All @@ -487,6 +517,18 @@ include::example$java-protobuf-tracing/src/main/java/com/example/ControllerActio
<4> Sets the status of the span as error.
<5> Closes the span.

Scala::
+
[source,java,indent=0]
.src/main/scala/com/example/ControllerAction.java
----
include::example$scala-protobuf-tracing/src/main/scala/com/example/ControllerAction.scala[tag=create-close-span]
----
<1> Sets the action's TraceContext the parent of this span. Linking the action's trace to this span.
<2> Creates and starts the span.
<3> Adds some attribute.
<4> Sets the status of the span as error.
<5> Closes the span.
NOTE: You can find how tracing is enabled and more info link:https://docs.kalix.io/operations/observability-exports.html#_activating_tracing_beta[here].

== Unit testing the side effects
Expand Down
1 change: 0 additions & 1 deletion samples/java-protobuf-tracing/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@ mvn kalix:runAll
This command will start your Kalix service and a companion Kalix Runtime as configured in [docker-compose.yml](./docker-compose.yml) file.
This will also start a Jaeger service to which the services above will push the traces. You can find Jaeger at `http://localhost:16686`


With both the Kalix Runtime and your service running, any defined endpoints should be available at `http://localhost:9000`. In addition to the defined gRPC interface, each method has a corresponding HTTP endpoint. Unless configured otherwise (see [Transcoding HTTP](https://docs.kalix.io/java-protobuf/writing-grpc-descriptors-protobuf.html#_transcoding_http)), this endpoint accepts POST requests at the path `/[package].[entity name]/[method]`.
For example, using [`grpcurl`](https://github.com/fullstorydev/grpcurl):

Expand Down
2 changes: 0 additions & 2 deletions samples/java-protobuf-tracing/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@
<dockerTag>${project.version}-${build.timestamp}</dockerTag>
<maven.build.timestamp.format>yyyyMMddHHmmss</maven.build.timestamp.format>
<mainClass>com.example.Main</mainClass>
<jvmArgs>-Dkalix.telemetry.tracing.collector-endpoint=http://localhost:4317</jvmArgs>
<jdk.target>17</jdk.target>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>

Expand Down Expand Up @@ -219,7 +218,6 @@
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.2</version>
<configuration>
<systemPropertyVariables>-Dkalix.telemetry.tracing.collector-endpoint="http://localhost:4317"</systemPropertyVariables>
<excludes>
<!-- ignore integration test classes -->
<exclude>**/*IntegrationTest</exclude>
Expand Down
Original file line number Diff line number Diff line change
@@ -1,24 +1,16 @@
package com.example;

import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.google.common.base.Charsets;
import com.google.protobuf.Empty;
import io.opentelemetry.api.trace.Span;
import io.opentelemetry.api.trace.StatusCode;
import io.opentelemetry.api.trace.Tracer;
import io.opentelemetry.context.Scope;
import kalix.javasdk.action.ActionCreationContext;

import java.io.IOException;
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.nio.charset.Charset;
import java.util.Optional;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.CompletionStage;

public class ControllerAction extends AbstractControllerAction {

Expand All @@ -30,29 +22,25 @@ public ControllerAction(ActionCreationContext creationContext) {}
@Override
public Effect<ControllerActionApi.MessageResponse> callAsyncEndpoint(Empty empty) {
// tag::get-tracer[]
Optional<Tracer> tracerOpt = actionContext().getOpenTelemetryTracer();
Tracer tracer = actionContext().getTracer();
// end::get-tracer[]

Optional<Span> span = tracerOpt.map(tracer -> {
return tracer
Span span = tracer
.spanBuilder(url+"/{}")
.setParent(actionContext().metadata().traceContext().asOpenTelemetryContext())// <1>
.startSpan() // <2>
.setAttribute("post", "1");// <3>
});

CompletableFuture<ControllerActionApi.MessageResponse> asyncComputation = callAsyncService()
.whenComplete((response, ex) -> {
if (ex != null) {
span.ifPresent(presentSpan ->
presentSpan
span
franciscolopezsancho marked this conversation as resolved.
Show resolved Hide resolved
.setStatus(StatusCode.ERROR, ex.getMessage())// <4>
.end());// <5>
.end();// <5>
} else {
span.ifPresent(presentSpan ->
presentSpan
span
.setAttribute("result", response.body().title)// <3>
.end());// <5>
.end();// <5>
}
})
.thenApply(response ->
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,5 +10,5 @@ import "kalix/annotations.proto";
// per component or method using annotations.
// Documentation at https://docs.kalix.io/java-protobuf/access-control.html
option (kalix.file).acl = {
allow: { service: "*" }
allow: { principal: INTERNET }
};
11 changes: 11 additions & 0 deletions samples/scala-protobuf-tracing/.env
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# this is the port where the kalix runtime container will be exposed
# when running multiple services on your local machine, make sure that this port is unique
ADVERTISED_HTTP_PORT=9000

# this is the port where the user services (your application) will open
# when running multiple services on your local machine, make sure that this port is unique
USER_SERVICE_PORT=8080

# this variable defines the host where the kalix runtime (running in docker)
# will reach the user service in local development
USER_SERVICE_HOST=host.docker.internal
33 changes: 33 additions & 0 deletions samples/scala-protobuf-tracing/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
boot/
lib_managed/
src_managed/
.bsp

# intellij
/src/intellij*/*.iml
/src/intellij*/*.ipr
/src/intellij*/*.iws
**/.cache
/.idea
/.settings

# vscode
/.vscode

# sbt's target directories
/target/
/project/target
/project/plugins/project/
/project/project
/project/**/target/
/test/macro-annot/target/
/test/files/target/
/test/target/
/build-sbt/
local.sbt
jitwatch.out

# metals
.metals
.bloop
project/**/metals.sbt
47 changes: 47 additions & 0 deletions samples/scala-protobuf-tracing/.scalafmt.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
version = 3.0.3

style = defaultWithAlign

docstrings.style = Asterisk
indentOperator.preset = spray
maxColumn = 120
rewrite.rules = [RedundantParens, SortImports, AvoidInfix]
unindentTopLevelOperators = true
align.tokens = [{code = "=>", owner = "Case"}]
align.openParenDefnSite = false
align.openParenCallSite = false
optIn.configStyleArguments = false
danglingParentheses.preset = false
spaces.inImportCurlyBraces = true
newlines.afterCurlyLambda = preserve
rewrite.neverInfix.excludeFilters = [
and
min
max
until
to
by
eq
ne
"should.*"
"contain.*"
"must.*"
in
ignore
be
taggedAs
thrownBy
synchronized
have
when
size
only
noneOf
oneElementOf
noElementsOf
atLeastOneElementOf
atMostOneElementOf
allElementsOf
inOrderElementsOf
theSameElementsAs
]
42 changes: 42 additions & 0 deletions samples/scala-protobuf-tracing/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
## This example show how to create Spans by the users

## Running Locally

When running a Kalix service locally, we need to have its companion Kalix Runtime running alongside it.

To start your service locally, run:

```shell
sbt runAll
```

This command will start your Kalix service and a companion Kalix Runtime as configured in [docker-compose.yml](./docker-compose.yml) file.
This will also start a Jaeger service to which the Kalix service will push the traces. You can find Jaeger at `http://localhost:16686`

With both the Kalix Runtime and your service running, any defined endpoints should be available at `http://localhost:9000`. In addition to the defined gRPC interface, each method has a corresponding HTTP endpoint. Unless configured otherwise (see [Transcoding HTTP](https://docs.kalix.io/java-protobuf/writing-grpc-descriptors-protobuf.html#_transcoding_http)), this endpoint accepts POST requests at the path `/[package].[entity name]/[method]`.
For example, using [`grpcurl`](https://github.com/fullstorydev/grpcurl):

```shell
grpcurl -plaintext localhost:9000 com.example.Controller/CallAsyncEndpoint
```
produces
```
{
"message": "sunt aut facere repellat provident occaecati excepturi optio reprehenderit"
}
```

## Deploying

To deploy your service, install the `kalix` CLI as documented in
[Install Kalix](https://docs.kalix.io/kalix/install-kalix.html)
and configure a Docker Registry to upload your docker image to.

You will need to update the `dockerImage` property in the `pom.xml` and refer to
[Configuring registries](https://docs.kalix.io/projects/container-registries.html)
for more information on how to make your docker image available to Kalix.

Finally, you use the `kalix` CLI to create a project as described in [Create a new Project](https://docs.kalix.io/projects/create-project.html). Once you have a project you can deploy your service into the project either
by using `mvn deploy kalix:deploy` which will package, publish your docker image, and deploy your service to Kalix,
or by first packaging and publishing the docker image through `mvn deploy` and
then [deploying the image through the `kalix` CLI](https://docs.kalix.io/services/deploy-service.html#_deploy).
38 changes: 38 additions & 0 deletions samples/scala-protobuf-tracing/build.sbt
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
organization := "com.example"

scalaVersion := "2.13.10"


enablePlugins(KalixPlugin, JavaAppPackaging, DockerPlugin)
dockerBaseImage := "docker.io/library/adoptopenjdk:11-jre-hotspot"
dockerUsername := sys.props.get("docker.username")
dockerRepository := sys.props.get("docker.registry")
dockerUpdateLatest := true
dockerBuildCommand := {
val arch = sys.props("os.arch")
if (arch != "amd64" && !arch.contains("x86")) {
// use buildx with platform to build supported amd64 images on other CPU architectures
// this may require that you have first run 'docker buildx create' to set docker buildx up
dockerExecCommand.value ++ Seq("buildx", "build", "--platform=linux/amd64", "--load") ++ dockerBuildOptions.value :+ "."
} else dockerBuildCommand.value
}
ThisBuild / dynverSeparator := "-"
run / fork := true
run / envVars += ("HOST", "0.0.0.0")
run / javaOptions ++= Seq("-Dlogback.configurationFile=logback-dev-mode.xml")

Compile / scalacOptions ++= Seq(
"-release:11",
"-deprecation",
"-feature",
"-unchecked",
"-Xlog-reflective-calls",
"-Xlint")
Compile / javacOptions ++= Seq("-Xlint:unchecked", "-Xlint:deprecation", "-parameters" // for Jackson
)

libraryDependencies ++= Seq(
"com.softwaremill.sttp.client4" %% "core" % "4.0.0-M11" % Compile,
"org.json4s" %% "json4s-native" % "4.1.0-M5"% Compile,
"org.scalatest" %% "scalatest" % "3.2.7" % Test
)
24 changes: 24 additions & 0 deletions samples/scala-protobuf-tracing/docker-compose.yml
franciscolopezsancho marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# If you're looking to use eventing with Google PubSub, to get an emulator running:
# - add property "-Dkalix.proxy.eventing.support=google-pubsub-emulator" to the JAVA_TOOL_OPTIONS environment map under the kalix-runtime service
# - uncomment the env var PUBSUB_EMULATOR_HOST and the section below for gcloud-pubsub-emulator service
version: "3"
services:
kalix-runtime:
image: gcr.io/kalix-public/kalix-runtime:1.1.34
container_name: tracing
ports:
- "${ADVERTISED_HTTP_PORT}:9000"
extra_hosts:
- "host.docker.internal:host-gateway"
environment:
JAVA_TOOL_OPTIONS: >
-Dkalix.proxy.telemetry.tracing.enabled=true
-Dkalix.proxy.telemetry.tracing.collector-endpoint=http://jaeger:4317
ADVERTISED_HTTP_PORT: ${ADVERTISED_HTTP_PORT}
USER_SERVICE_HOST: ${USER_SERVICE_HOST}
USER_SERVICE_PORT: ${USER_SERVICE_PORT}
jaeger:
image: jaegertracing/all-in-one:1.54
ports:
- 4317:4317
- 16686:16686
1 change: 1 addition & 0 deletions samples/scala-protobuf-tracing/project/build.properties
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
sbt.version=1.9.9
franciscolopezsancho marked this conversation as resolved.
Show resolved Hide resolved
Loading
Loading