Protobuf scala span creation (#2069)

* Added tracing context in Scala SDK and using sttp in the Scala sample * simpler sync example * added async calling endpoint * using core sttp and some cleaning * refactoring * added readme * removing sync * implementing interface for ScalaActionContextAdapter * Cleaned and created docs * Clean up scala-protobuf-traicing sample and adding NoOps tracer for a better DX * docs and new implementation for the java-protobuf-tracing example * avoid validating local urls and validate back * Update docs/src/modules/java-protobuf/pages/actions.adoc Co-authored-by: Renato Cavalcanti <[email protected]> * Update samples/scala-protobuf-tracing/project/plugins.sbt Co-authored-by: Renato Cavalcanti <[email protected]> * Update sdk/java-sdk-protobuf/src/main/java/kalix/javasdk/action/ActionCreationContext.java Co-authored-by: Renato Cavalcanti <[email protected]> * addressing comments * simplifying request/response types --------- Co-authored-by: Renato Cavalcanti <[email protected]>
lightbend · Apr 8, 2024 · 05f78a5 · 05f78a5
1 parent 8aceaa8
commit 05f78a5
Show file tree

Hide file tree

Showing 36 changed files with 637 additions and 97 deletions.
diff --git a/docs/config/validate-links.json b/docs/config/validate-links.json
@@ -2,6 +2,7 @@
   "ignorePatterns": [
     { "pattern": "^https://mvnrepository\\.com" },
     { "pattern": "^http://127.0.0.1:8080" },
-    { "pattern": "^https://site\\.mockito\\.org/" }
+    { "pattern": "^http://jaeger:4317"},
+    { "pattern": "^http://localhost:16686"}
   ]
 }
diff --git a/docs/src/modules/java-protobuf/pages/actions.adoc b/docs/src/modules/java-protobuf/pages/actions.adoc
@@ -449,29 +449,60 @@ include::example$java-protobuf-valueentity-counter/src/main/java/com/example/act
 Please note that, the result of a side effect is ignored by the current command meaning that even if the call to
 the `Counter` entity fails, the `Action` reply will succeed.
 
-
 == Adding tracing spans
 
-To add spans in your actions you can add the tracer available in the link:{attachmentsdir}/api/kalix/javasdk/action/ActionContext.html[`ActionContext`{tab-icon}, window="new"] and with it create the span. 
-
-To get the tracer from the link:{attachmentsdir}/api/kalix/javasdk/action/ActionContext.html[`ActionContext`{tab-icon}, window="new"]: 
+To create link:https://opentelemetry.io/docs/specs/otel/trace/api/#span[`spans`{tab-icon}, window="new"] in your actions you need the link:https://opentelemetry.io/docs/specs/otel/trace/api/#tracer[`tracer`{tab-icon}, window="new"] available through the `actionContext()` method.
 
 [.tabset]
 Java::
 +
-To add spans in your actions you can add the tracer available in the link:{attachmentsdir}/api/kalix/javasdk/action/ActionContext.html[`ActionContext`{tab-icon}, window="new"] and with it create the span.
-+
-To get the tracer from the link:{attachmentsdir}/api/kalix/javasdk/action/ActionContext.html[`ActionContext`{tab-icon}, window="new"]:
+`actionContext()` gives you the link:{attachmentsdir}/api/kalix/javasdk/action/ActionContext.html[`ActionContext`{tab-icon}, window="new"]. That is, some methods that apply to the context of the action's request.
 +
 [source,java,indent=0]
 .src/main/java/com/example/ControllerAction.java
 ----
 include::example$java-protobuf-tracing/src/main/java/com/example/ControllerAction.java[tag=get-tracer]
 ----
 
-Note that if link:https://docs.kalix.io/operations/observability-exports.html#_activating_tracing_beta[tracing] is not enabled in your service this `Optional` will be empty. Otherwise you can map over `tracerOpt` to retreive the tracer.
+Scala::
++
+`actionContext()` gives you the link:{attachmentsdir}/scala-api/kalix/scalasdk/action/ActionContext.html[`ActionContext`{tab-icon}, window="new"]. That is, some methods that apply to the context of the action's request.
++
+[source,java,indent=0]
+.src/main/scala/com/example/ControllerAction.java
+----
+include::example$scala-protobuf-tracing/src/main/scala/com/example/ControllerAction.scala[tag=get-tracer]
+----
+
+IMPORTANT: If tracing is enabled, you will get a tracer that actually creates span and exports data as expected.
+But if tracing is not enabled, you will get a no operational tracer instead, which will not create traces.
+
+Trace generation is disabled by default. To enable it in a service deployed to Kalix, see link:https://docs.kalix.io/operations/observability-exports.html#_activating_tracing_beta[`here`].
+To enable it in a service running on your local machine you need to add the following `JAVA_TOOL_OPTIONS` to your `docker-compose.yml` in the base of your project.
+It's also necessary to have where to collect the traces. For example, an extra container, like the `jaeger` below.
+[source,yaml]
+.docker-compose.yml
+---
+services:
+  kalix-runtime:
+    image: ...
+    ...
+    environment:
+      JAVA_TOOL_OPTIONS: >
+        -Dkalix.proxy.telemetry.tracing.enabled=true
+        -Dkalix.proxy.telemetry.tracing.collector-endpoint=http://jaeger:4317
+    ...
+  jaeger:
+    image: jaegertracing/all-in-one:1.54
+    ports:
+      - 4317:4317
+      - 16686:16686
+---
+
+Here the traces are pushed to a `jaeger` docker image, to the port `4317`. And you can check them out at `http://localhost:16686`.
 
-To create a link:https://opentelemetry.io/docs/languages/java/instrumentation/#create-spans[span] and end it over an asynchronous call, you can do the following:
+
+To link:https://opentelemetry.io/docs/languages/java/instrumentation/#create-spans[create a span] and end it over an asynchronous call, you can do the following:
 
 [.tabset]
 Java::
@@ -487,6 +518,18 @@ include::example$java-protobuf-tracing/src/main/java/com/example/ControllerActio
 <4> Sets the status of the span as error.
 <5> Closes the span.
 
+Scala::
++
+[source,java,indent=0]
+.src/main/scala/com/example/ControllerAction.java
+----
+include::example$scala-protobuf-tracing/src/main/scala/com/example/ControllerAction.scala[tag=create-close-span]
+----
+<1> Sets the action's TraceContext the parent of this span. Linking the action's trace to this span.
+<2> Creates and starts the span.
+<3> Adds some attribute.
+<4> Sets the status of the span as error.
+<5> Closes the span.
 NOTE: You can find how tracing is enabled and more info  link:https://docs.kalix.io/operations/observability-exports.html#_activating_tracing_beta[here].
 
 == Unit testing the side effects

diff --git a/samples/java-protobuf-tracing/README.md b/samples/java-protobuf-tracing/README.md
@@ -16,7 +16,6 @@ mvn kalix:runAll
 This command will start your Kalix service and a companion Kalix Runtime as configured in [docker-compose.yml](./docker-compose.yml) file.
 This will also start a Jaeger service to which the services above will push the traces. You can find Jaeger at `http://localhost:16686`
 
-
 With both the Kalix Runtime and your service running, any defined endpoints should be available at `http://localhost:9000`. In addition to the defined gRPC interface, each method has a corresponding HTTP endpoint. Unless configured otherwise (see [Transcoding HTTP](https://docs.kalix.io/java-protobuf/writing-grpc-descriptors-protobuf.html#_transcoding_http)), this endpoint accepts POST requests at the path `/[package].[entity name]/[method]`.
 For example, using [`grpcurl`](https://github.com/fullstorydev/grpcurl):
 

diff --git a/samples/java-protobuf-tracing/pom.xml b/samples/java-protobuf-tracing/pom.xml
@@ -18,7 +18,6 @@
       <dockerTag>${project.version}-${build.timestamp}</dockerTag>
     <maven.build.timestamp.format>yyyyMMddHHmmss</maven.build.timestamp.format>
     <mainClass>com.example.Main</mainClass>
-    <jvmArgs>-Dkalix.telemetry.tracing.collector-endpoint=http://localhost:4317</jvmArgs>
     <jdk.target>17</jdk.target>
     <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
 
@@ -219,7 +218,6 @@
         <artifactId>maven-surefire-plugin</artifactId>
         <version>2.22.2</version>
         <configuration>
-          <systemPropertyVariables>-Dkalix.telemetry.tracing.collector-endpoint="http://localhost:4317"</systemPropertyVariables>
           <excludes>
             <!-- ignore integration test classes -->
             <exclude>**/*IntegrationTest</exclude>

diff --git a/samples/java-protobuf-tracing/src/main/java/com/example/ControllerAction.java b/samples/java-protobuf-tracing/src/main/java/com/example/ControllerAction.java
@@ -1,24 +1,16 @@
 package com.example;
 
-import com.fasterxml.jackson.core.JsonProcessingException;
-import com.fasterxml.jackson.databind.ObjectMapper;
-import com.google.common.base.Charsets;
 import com.google.protobuf.Empty;
 import io.opentelemetry.api.trace.Span;
 import io.opentelemetry.api.trace.StatusCode;
 import io.opentelemetry.api.trace.Tracer;
-import io.opentelemetry.context.Scope;
 import kalix.javasdk.action.ActionCreationContext;
 
-import java.io.IOException;
 import java.net.URI;
 import java.net.http.HttpClient;
 import java.net.http.HttpRequest;
 import java.net.http.HttpResponse;
-import java.nio.charset.Charset;
-import java.util.Optional;
 import java.util.concurrent.CompletableFuture;
-import java.util.concurrent.CompletionStage;
 
 public class ControllerAction extends AbstractControllerAction {
 
@@ -30,29 +22,25 @@ public ControllerAction(ActionCreationContext creationContext) {}
   @Override
   public Effect<ControllerActionApi.MessageResponse> callAsyncEndpoint(Empty empty) {
     // tag::get-tracer[]
-    Optional<Tracer> tracerOpt = actionContext().getOpenTelemetryTracer();
+    Tracer tracer = actionContext().getTracer();
     // end::get-tracer[]
 
-    Optional<Span> span = tracerOpt.map(tracer -> {
-      return tracer
+    Span span = tracer
               .spanBuilder(url+"/{}")
               .setParent(actionContext().metadata().traceContext().asOpenTelemetryContext())// <1>
               .startSpan() // <2>
               .setAttribute("post", "1");// <3>
-    });
 
     CompletableFuture<ControllerActionApi.MessageResponse> asyncComputation = callAsyncService()
       .whenComplete((response, ex) -> {
         if (ex != null) {
-          span.ifPresent(presentSpan ->
-            presentSpan
+          span
               .setStatus(StatusCode.ERROR, ex.getMessage())// <4>
-              .end());// <5>
+              .end();// <5>
         } else {
-          span.ifPresent(presentSpan ->
-            presentSpan
+          span
               .setAttribute("result", response.body().title)// <3>
-              .end());// <5>
+              .end();// <5>
         }
       })
       .thenApply(response ->

diff --git a/samples/java-protobuf-tracing/src/main/proto/com/example/kalix_policy.proto b/samples/java-protobuf-tracing/src/main/proto/com/example/kalix_policy.proto
@@ -10,5 +10,5 @@ import "kalix/annotations.proto";
 // per component or method using annotations.
 // Documentation at https://docs.kalix.io/java-protobuf/access-control.html
 option (kalix.file).acl = {
-  allow: { service: "*" }
+  allow: { principal: INTERNET }
 };
diff --git a/samples/scala-protobuf-tracing/.env b/samples/scala-protobuf-tracing/.env
@@ -0,0 +1,11 @@
+# this is the port where the kalix runtime container will be exposed
+# when running multiple services on your local machine, make sure that this port is unique
+ADVERTISED_HTTP_PORT=9000
+
+# this is the port where the user services (your application) will open
+# when running multiple services on your local machine, make sure that this port is unique
+USER_SERVICE_PORT=8080
+
+# this variable defines the host where the kalix runtime (running in docker)
+# will reach the user service in local development
+USER_SERVICE_HOST=host.docker.internal
diff --git a/samples/scala-protobuf-tracing/.gitignore b/samples/scala-protobuf-tracing/.gitignore
@@ -0,0 +1,33 @@
+boot/
+lib_managed/
+src_managed/
+.bsp
+
+# intellij
+/src/intellij*/*.iml
+/src/intellij*/*.ipr
+/src/intellij*/*.iws
+**/.cache
+/.idea
+/.settings
+
+# vscode
+/.vscode
+
+# sbt's target directories
+/target/
+/project/target
+/project/plugins/project/
+/project/project
+/project/**/target/
+/test/macro-annot/target/
+/test/files/target/
+/test/target/
+/build-sbt/
+local.sbt
+jitwatch.out
+
+# metals
+.metals
+.bloop
+project/**/metals.sbt
diff --git a/samples/scala-protobuf-tracing/.scalafmt.conf b/samples/scala-protobuf-tracing/.scalafmt.conf
@@ -0,0 +1,47 @@
+version = 3.0.3
+
+style = defaultWithAlign
+
+docstrings.style           = Asterisk
+indentOperator.preset      = spray
+maxColumn                  = 120
+rewrite.rules              = [RedundantParens, SortImports, AvoidInfix]
+unindentTopLevelOperators  = true
+align.tokens               = [{code = "=>", owner = "Case"}]
+align.openParenDefnSite    = false
+align.openParenCallSite    = false
+optIn.configStyleArguments = false
+danglingParentheses.preset = false
+spaces.inImportCurlyBraces = true
+newlines.afterCurlyLambda = preserve
+rewrite.neverInfix.excludeFilters = [
+  and
+  min
+  max
+  until
+  to
+  by
+  eq
+  ne
+  "should.*"
+  "contain.*"
+  "must.*"
+  in
+  ignore
+  be
+  taggedAs
+  thrownBy
+  synchronized
+  have
+  when
+  size
+  only
+  noneOf
+  oneElementOf
+  noElementsOf
+  atLeastOneElementOf
+  atMostOneElementOf
+  allElementsOf
+  inOrderElementsOf
+  theSameElementsAs
+]
diff --git a/samples/scala-protobuf-tracing/README.md b/samples/scala-protobuf-tracing/README.md
@@ -0,0 +1,42 @@
+## This example show how to create Spans by the users
+
+## Running Locally
+
+When running a Kalix service locally, we need to have its companion Kalix Runtime running alongside it.
+
+To start your service locally, run:
+
+```shell
+sbt runAll
+```
+
+This command will start your Kalix service and a companion Kalix Runtime as configured in [docker-compose.yml](./docker-compose.yml) file.
+This will also start a Jaeger service to which the Kalix service will push the traces. You can find Jaeger at `http://localhost:16686`
+
+With both the Kalix Runtime and your service running, any defined endpoints should be available at `http://localhost:9000`. In addition to the defined gRPC interface, each method has a corresponding HTTP endpoint. Unless configured otherwise (see [Transcoding HTTP](https://docs.kalix.io/java-protobuf/writing-grpc-descriptors-protobuf.html#_transcoding_http)), this endpoint accepts POST requests at the path `/[package].[entity name]/[method]`.
+For example, using [`grpcurl`](https://github.com/fullstorydev/grpcurl):
+
+```shell
+grpcurl -plaintext localhost:9000  com.example.Controller/CallAsyncEndpoint                                 
+```
+produces
+```
+{
+  "message": "sunt aut facere repellat provident occaecati excepturi optio reprehenderit"
+}
+```
+
+## Deploying
+
+To deploy your service, install the `kalix` CLI as documented in
+[Install Kalix](https://docs.kalix.io/kalix/install-kalix.html)
+and configure a Docker Registry to upload your docker image to.
+
+You will need to update the `dockerImage` property in the `pom.xml` and refer to
+[Configuring registries](https://docs.kalix.io/projects/container-registries.html)
+for more information on how to make your docker image available to Kalix.
+
+Finally, you use the `kalix` CLI to create a project as described in [Create a new Project](https://docs.kalix.io/projects/create-project.html). Once you have a project you can deploy your service into the project either
+by using `mvn deploy kalix:deploy` which will package, publish your docker image, and deploy your service to Kalix,
+or by first packaging and publishing the docker image through `mvn deploy` and
+then [deploying the image through the `kalix` CLI](https://docs.kalix.io/services/deploy-service.html#_deploy).
diff --git a/samples/scala-protobuf-tracing/build.sbt b/samples/scala-protobuf-tracing/build.sbt
@@ -0,0 +1,38 @@
+organization := "com.example"
+
+scalaVersion := "2.13.10"
+
+
+enablePlugins(KalixPlugin, JavaAppPackaging, DockerPlugin)
+dockerBaseImage := "docker.io/library/adoptopenjdk:11-jre-hotspot"
+dockerUsername := sys.props.get("docker.username")
+dockerRepository := sys.props.get("docker.registry")
+dockerUpdateLatest := true
+dockerBuildCommand := {
+  val arch = sys.props("os.arch")
+  if (arch != "amd64" && !arch.contains("x86")) {
+    // use buildx with platform to build supported amd64 images on other CPU architectures
+    // this may require that you have first run 'docker buildx create' to set docker buildx up
+    dockerExecCommand.value ++ Seq("buildx", "build", "--platform=linux/amd64", "--load") ++ dockerBuildOptions.value :+ "."
+  } else dockerBuildCommand.value
+}
+ThisBuild / dynverSeparator := "-"
+run / fork := true
+run / envVars += ("HOST", "0.0.0.0")
+run / javaOptions ++= Seq("-Dlogback.configurationFile=logback-dev-mode.xml")
+
+Compile / scalacOptions ++= Seq(
+  "-release:11",
+  "-deprecation",
+  "-feature",
+  "-unchecked",
+  "-Xlog-reflective-calls",
+  "-Xlint")
+Compile / javacOptions ++= Seq("-Xlint:unchecked", "-Xlint:deprecation", "-parameters" // for Jackson
+)
+
+libraryDependencies ++= Seq(
+  "com.softwaremill.sttp.client4" %% "core" % "4.0.0-M11" % Compile,
+  "org.json4s" %% "json4s-native" % "4.1.0-M5"% Compile,
+  "org.scalatest" %% "scalatest" % "3.2.7" % Test
+)
diff --git a/samples/scala-protobuf-tracing/docker-compose.yml b/samples/scala-protobuf-tracing/docker-compose.yml
@@ -0,0 +1,24 @@
+# If you're looking to use eventing with Google PubSub, to get an emulator running:
+# - add property "-Dkalix.proxy.eventing.support=google-pubsub-emulator" to the JAVA_TOOL_OPTIONS environment map under the kalix-runtime service
+# - uncomment the env var PUBSUB_EMULATOR_HOST and the section below for gcloud-pubsub-emulator service
+version: "3"
+services:
+  kalix-runtime:
+    image: gcr.io/kalix-public/kalix-runtime:1.1.34
+    container_name: tracing
+    ports:
+      - "${ADVERTISED_HTTP_PORT}:9000"
+    extra_hosts:
+      - "host.docker.internal:host-gateway"
+    environment:
+      JAVA_TOOL_OPTIONS: >
+        -Dkalix.proxy.telemetry.tracing.enabled=true
+        -Dkalix.proxy.telemetry.tracing.collector-endpoint=http://jaeger:4317
+      ADVERTISED_HTTP_PORT: ${ADVERTISED_HTTP_PORT}
+      USER_SERVICE_HOST: ${USER_SERVICE_HOST}
+      USER_SERVICE_PORT: ${USER_SERVICE_PORT}
+  jaeger:
+    image: jaegertracing/all-in-one:1.54
+    ports:
+      - 4317:4317
+      - 16686:16686
diff --git a/samples/scala-protobuf-tracing/project/build.properties b/samples/scala-protobuf-tracing/project/build.properties
@@ -0,0 +1 @@
+sbt.version=1.9.9