Fix AWS API Gateway endpoints correlation HTTP span tags - Inferred Proxy Spans#10561
Open
Fix AWS API Gateway endpoints correlation HTTP span tags - Inferred Proxy Spans#10561
Conversation
jandro996
added
tag: do not merge
jandro996
force-pushed
the
alejandro.gonzalez/RFC-1081
branch
from
7163ef9 to
85a368c
Compare
jandro996
force-pushed
the
alejandro.gonzalez/RFC-1081
branch
from
c681b6c to
c8a926f
Compare
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 63 metrics, 8 unstable metrics. Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.60.0-SNAPSHOT~762de576c1, baseline=1.60.0-SNAPSHOT~f3e5e5b89b
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.079 s) : 0, 1079228
Total [baseline] (8.749 s) : 0, 8749208
Agent [candidate] (1.072 s) : 0, 1071632
Total [candidate] (8.734 s) : 0, 8733964
section iast
Agent [baseline] (1.231 s) : 0, 1230781
Total [baseline] (9.348 s) : 0, 9347539
Agent [candidate] (1.231 s) : 0, 1231459
Total [candidate] (9.358 s) : 0, 9357679
gantt
title insecure-bank - break down per module: candidate=1.60.0-SNAPSHOT~762de576c1, baseline=1.60.0-SNAPSHOT~f3e5e5b89b
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.2 ms) : 0, 1200
crashtracking [candidate] (1.216 ms) : 0, 1216
BytebuddyAgent [baseline] (638.532 ms) : 0, 638532
BytebuddyAgent [candidate] (633.491 ms) : 0, 633491
AgentMeter [baseline] (29.359 ms) : 0, 29359
AgentMeter [candidate] (29.207 ms) : 0, 29207
GlobalTracer [baseline] (260.852 ms) : 0, 260852
GlobalTracer [candidate] (259.217 ms) : 0, 259217
AppSec [baseline] (33.299 ms) : 0, 33299
AppSec [candidate] (33.04 ms) : 0, 33040
Debugger [baseline] (62.819 ms) : 0, 62819
Debugger [candidate] (61.473 ms) : 0, 61473
Remote Config [baseline] (637.346 µs) : 0, 637
Remote Config [candidate] (622.644 µs) : 0, 623
Telemetry [baseline] (10.864 ms) : 0, 10864
Telemetry [candidate] (12.35 ms) : 0, 12350
Flare Poller [baseline] (6.093 ms) : 0, 6093
Flare Poller [candidate] (5.347 ms) : 0, 5347
section iast
crashtracking [baseline] (1.193 ms) : 0, 1193
crashtracking [candidate] (1.174 ms) : 0, 1174
BytebuddyAgent [baseline] (795.628 ms) : 0, 795628
BytebuddyAgent [candidate] (796.74 ms) : 0, 796740
AgentMeter [baseline] (11.238 ms) : 0, 11238
AgentMeter [candidate] (11.251 ms) : 0, 11251
GlobalTracer [baseline] (248.09 ms) : 0, 248090
GlobalTracer [candidate] (248.177 ms) : 0, 248177
AppSec [baseline] (34.902 ms) : 0, 34902
AppSec [candidate] (33.906 ms) : 0, 33906
Debugger [baseline] (64.798 ms) : 0, 64798
Debugger [candidate] (65.446 ms) : 0, 65446
Remote Config [baseline] (537.738 µs) : 0, 538
Remote Config [candidate] (536.024 µs) : 0, 536
Telemetry [baseline] (8.578 ms) : 0, 8578
Telemetry [candidate] (8.572 ms) : 0, 8572
Flare Poller [baseline] (3.476 ms) : 0, 3476
Flare Poller [candidate] (3.477 ms) : 0, 3477
IAST [baseline] (27.023 ms) : 0, 27023
IAST [candidate] (26.959 ms) : 0, 26959
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.60.0-SNAPSHOT~762de576c1, baseline=1.60.0-SNAPSHOT~f3e5e5b89b
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.064 s) : 0, 1063506
Total [baseline] (10.881 s) : 0, 10881419
Agent [candidate] (1.063 s) : 0, 1062877
Total [candidate] (10.95 s) : 0, 10950263
section appsec
Agent [baseline] (1.247 s) : 0, 1246534
Total [baseline] (11.078 s) : 0, 11078219
Agent [candidate] (1.239 s) : 0, 1239360
Total [candidate] (11.024 s) : 0, 11023505
section iast
Agent [baseline] (1.242 s) : 0, 1242349
Total [baseline] (11.177 s) : 0, 11176996
Agent [candidate] (1.234 s) : 0, 1234003
Total [candidate] (11.177 s) : 0, 11177180
section profiling
Agent [baseline] (1.189 s) : 0, 1189075
Total [baseline] (10.885 s) : 0, 10884806
Agent [candidate] (1.198 s) : 0, 1197883
Total [candidate] (10.975 s) : 0, 10975129
gantt
title petclinic - break down per module: candidate=1.60.0-SNAPSHOT~762de576c1, baseline=1.60.0-SNAPSHOT~f3e5e5b89b
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.17 ms) : 0, 1170
crashtracking [candidate] (1.177 ms) : 0, 1177
BytebuddyAgent [baseline] (627.616 ms) : 0, 627616
BytebuddyAgent [candidate] (627.767 ms) : 0, 627767
AgentMeter [baseline] (28.89 ms) : 0, 28890
AgentMeter [candidate] (28.781 ms) : 0, 28781
GlobalTracer [baseline] (256.956 ms) : 0, 256956
GlobalTracer [candidate] (257.049 ms) : 0, 257049
AppSec [baseline] (32.853 ms) : 0, 32853
AppSec [candidate] (32.841 ms) : 0, 32841
Debugger [baseline] (61.901 ms) : 0, 61901
Debugger [candidate] (61.846 ms) : 0, 61846
Remote Config [baseline] (615.416 µs) : 0, 615
Remote Config [candidate] (694.854 µs) : 0, 695
Telemetry [baseline] (12.147 ms) : 0, 12147
Telemetry [candidate] (13.695 ms) : 0, 13695
Flare Poller [baseline] (6.129 ms) : 0, 6129
Flare Poller [candidate] (3.777 ms) : 0, 3777
section appsec
crashtracking [baseline] (1.191 ms) : 0, 1191
crashtracking [candidate] (1.173 ms) : 0, 1173
BytebuddyAgent [baseline] (663.198 ms) : 0, 663198
BytebuddyAgent [candidate] (658.568 ms) : 0, 658568
AgentMeter [baseline] (12.069 ms) : 0, 12069
AgentMeter [candidate] (12.01 ms) : 0, 12010
GlobalTracer [baseline] (259.497 ms) : 0, 259497
GlobalTracer [candidate] (258.169 ms) : 0, 258169
AppSec [baseline] (168.577 ms) : 0, 168577
AppSec [candidate] (167.633 ms) : 0, 167633
Debugger [baseline] (67.1 ms) : 0, 67100
Debugger [candidate] (67.239 ms) : 0, 67239
Remote Config [baseline] (653.991 µs) : 0, 654
Remote Config [candidate] (659.791 µs) : 0, 660
Telemetry [baseline] (9.64 ms) : 0, 9640
Telemetry [candidate] (9.66 ms) : 0, 9660
Flare Poller [baseline] (3.722 ms) : 0, 3722
Flare Poller [candidate] (3.819 ms) : 0, 3819
IAST [baseline] (25.491 ms) : 0, 25491
IAST [candidate] (25.086 ms) : 0, 25086
section iast
crashtracking [baseline] (1.191 ms) : 0, 1191
crashtracking [candidate] (1.182 ms) : 0, 1182
BytebuddyAgent [baseline] (803.778 ms) : 0, 803778
BytebuddyAgent [candidate] (798.239 ms) : 0, 798239
AgentMeter [baseline] (11.544 ms) : 0, 11544
AgentMeter [candidate] (11.297 ms) : 0, 11297
GlobalTracer [baseline] (249.707 ms) : 0, 249707
GlobalTracer [candidate] (248.425 ms) : 0, 248425
AppSec [baseline] (32.433 ms) : 0, 32433
AppSec [candidate] (32.19 ms) : 0, 32190
Debugger [baseline] (68.486 ms) : 0, 68486
Debugger [candidate] (68.14 ms) : 0, 68140
Remote Config [baseline] (532.801 µs) : 0, 533
Remote Config [candidate] (543.116 µs) : 0, 543
Telemetry [baseline] (8.621 ms) : 0, 8621
Telemetry [candidate] (8.545 ms) : 0, 8545
Flare Poller [baseline] (3.456 ms) : 0, 3456
Flare Poller [candidate] (3.425 ms) : 0, 3425
IAST [baseline] (27.156 ms) : 0, 27156
IAST [candidate] (26.78 ms) : 0, 26780
section profiling
crashtracking [baseline] (1.217 ms) : 0, 1217
crashtracking [candidate] (1.221 ms) : 0, 1221
BytebuddyAgent [baseline] (680.656 ms) : 0, 680656
BytebuddyAgent [candidate] (687.296 ms) : 0, 687296
AgentMeter [baseline] (8.593 ms) : 0, 8593
AgentMeter [candidate] (8.676 ms) : 0, 8676
GlobalTracer [baseline] (215.657 ms) : 0, 215657
GlobalTracer [candidate] (217.257 ms) : 0, 217257
AppSec [baseline] (32.272 ms) : 0, 32272
AppSec [candidate] (32.748 ms) : 0, 32748
Debugger [baseline] (67.265 ms) : 0, 67265
Debugger [candidate] (67.39 ms) : 0, 67390
Remote Config [baseline] (616.865 µs) : 0, 617
Remote Config [candidate] (623.233 µs) : 0, 623
Telemetry [baseline] (8.896 ms) : 0, 8896
Telemetry [candidate] (8.995 ms) : 0, 8995
Flare Poller [baseline] (3.742 ms) : 0, 3742
Flare Poller [candidate] (3.741 ms) : 0, 3741
ProfilingAgent [baseline] (100.422 ms) : 0, 100422
ProfilingAgent [candidate] (99.98 ms) : 0, 99980
Profiling [baseline] (101.001 ms) : 0, 101001
Profiling [candidate] (100.567 ms) : 0, 100567
LoadParameters
See matching parameters
SummaryFound 3 performance improvements and 8 performance regressions! Performance is the same for 9 metrics, 16 unstable metrics.
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.60.0-SNAPSHOT~762de576c1, baseline=1.60.0-SNAPSHOT~f3e5e5b89b
dateFormat X
axisFormat %s
section baseline
no_agent (1.199 ms) : 1187, 1211
. : milestone, 1199,
iast (3.111 ms) : 3070, 3152
. : milestone, 3111,
iast_FULL (5.653 ms) : 5597, 5710
. : milestone, 5653,
iast_GLOBAL (3.629 ms) : 3569, 3689
. : milestone, 3629,
profiling (2.047 ms) : 2029, 2066
. : milestone, 2047,
tracing (1.74 ms) : 1725, 1754
. : milestone, 1740,
section candidate
no_agent (1.177 ms) : 1165, 1188
. : milestone, 1177,
iast (3.259 ms) : 3214, 3303
. : milestone, 3259,
iast_FULL (5.939 ms) : 5879, 5999
. : milestone, 5939,
iast_GLOBAL (3.546 ms) : 3484, 3609
. : milestone, 3546,
profiling (2.26 ms) : 2239, 2281
. : milestone, 2260,
tracing (1.814 ms) : 1798, 1831
. : milestone, 1814,
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.60.0-SNAPSHOT~762de576c1, baseline=1.60.0-SNAPSHOT~f3e5e5b89b
dateFormat X
axisFormat %s
section baseline
no_agent (17.206 ms) : 17034, 17377
. : milestone, 17206,
appsec (18.564 ms) : 18377, 18752
. : milestone, 18564,
code_origins (17.656 ms) : 17481, 17831
. : milestone, 17656,
iast (17.406 ms) : 17234, 17578
. : milestone, 17406,
profiling (19.845 ms) : 19644, 20045
. : milestone, 19845,
tracing (18.095 ms) : 17914, 18275
. : milestone, 18095,
section candidate
no_agent (17.234 ms) : 17062, 17406
. : milestone, 17234,
appsec (19.045 ms) : 18850, 19241
. : milestone, 19045,
code_origins (18.52 ms) : 18331, 18709
. : milestone, 18520,
iast (18.385 ms) : 18198, 18571
. : milestone, 18385,
profiling (18.551 ms) : 18370, 18731
. : milestone, 18551,
tracing (17.526 ms) : 17352, 17700
. : milestone, 17526,
DacapoParameters
See matching parameters
SummaryFound 1 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 1 unstable metrics.
Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.60.0-SNAPSHOT~762de576c1, baseline=1.60.0-SNAPSHOT~f3e5e5b89b
dateFormat X
axisFormat %s
section baseline
no_agent (14.918 s) : 14918000, 14918000
. : milestone, 14918000,
appsec (14.977 s) : 14977000, 14977000
. : milestone, 14977000,
iast (18.622 s) : 18622000, 18622000
. : milestone, 18622000,
iast_GLOBAL (17.624 s) : 17624000, 17624000
. : milestone, 17624000,
profiling (14.794 s) : 14794000, 14794000
. : milestone, 14794000,
tracing (15.002 s) : 15002000, 15002000
. : milestone, 15002000,
section candidate
no_agent (15.506 s) : 15506000, 15506000
. : milestone, 15506000,
appsec (15.011 s) : 15011000, 15011000
. : milestone, 15011000,
iast (18.282 s) : 18282000, 18282000
. : milestone, 18282000,
iast_GLOBAL (17.758 s) : 17758000, 17758000
. : milestone, 17758000,
profiling (14.692 s) : 14692000, 14692000
. : milestone, 14692000,
tracing (15.142 s) : 15142000, 15142000
. : milestone, 15142000,
Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.60.0-SNAPSHOT~762de576c1, baseline=1.60.0-SNAPSHOT~f3e5e5b89b
dateFormat X
axisFormat %s
section baseline
no_agent (1.484 ms) : 1472, 1495
. : milestone, 1484,
appsec (3.798 ms) : 3577, 4020
. : milestone, 3798,
iast (2.251 ms) : 2182, 2320
. : milestone, 2251,
iast_GLOBAL (2.282 ms) : 2213, 2351
. : milestone, 2282,
profiling (2.073 ms) : 2019, 2128
. : milestone, 2073,
tracing (2.072 ms) : 2019, 2126
. : milestone, 2072,
section candidate
no_agent (1.474 ms) : 1462, 1485
. : milestone, 1474,
appsec (2.519 ms) : 2465, 2574
. : milestone, 2519,
iast (2.251 ms) : 2182, 2320
. : milestone, 2251,
iast_GLOBAL (2.299 ms) : 2230, 2369
. : milestone, 2299,
profiling (2.498 ms) : 2333, 2662
. : milestone, 2498,
tracing (2.07 ms) : 2017, 2124
. : milestone, 2070,
|
jandro996
force-pushed
the
alejandro.gonzalez/rfc-1076
branch
from
35abdf1 to
9e4cfe6
Compare
jandro996
requested review from
daniel-romano-DD,
manuel-alvarez-alvarez and
smola
Contributor
|
Hi! 👋 Thanks for your pull request! 🎉 To help us review it, please make sure to:
If you need help, please check our contributing guidelines. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 06ba1cf584
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // Http.url - value of x-dd-proxy-domain-name + x-dd-proxy-path | ||
| span.setTag(HTTP_URL, domainName != null ? domainName + path : path); | ||
| // Http.url - https:// + x-dd-proxy-domain-name + x-dd-proxy-path | ||
| span.setTag(HTTP_URL, domainName != null ? "https://" + domainName + path : path); |
There was a problem hiding this comment.
This now treats any non-null x-dd-proxy-domain-name as usable, so an empty header value produces http.url like https:///path instead of a valid path-only URL. The code already treats empty domain as missing for service-name fallback, and before this change an empty domain naturally yielded just path; this regression can generate malformed URL tags and hurt endpoint correlation/parsing when gateways forward blank domain headers.
Useful? React with 👍 / 👎.
Comment on lines
+212
to
+213
| if (rootSpan != null && rootSpan != this.span) { | ||
| // Copy _dd.appsec.enabled metric (always 1 if present) |
There was a problem hiding this comment.
The new AppSec copy path is gated on rootSpan != this.span, but inferred proxy spans are created before the framework server span in HttpServerDecorator.startSpan, so for incoming distributed requests this inferred span is typically the local root and the condition is false. In the exact distributed-tracing scenario this change targets, _dd.appsec.enabled/_dd.appsec.json therefore never get copied, making the new propagation behavior ineffective.
Useful? React with 👍 / 👎.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What Does This Do
This PR implements standardized tags for inferred proxy spans produced by the Java tracer when instrumenting AWS API Gateway (v1 REST and v2 HTTP APIs). The changes align proxy spans with the cross-platform contract defined in RFC-1081 for endpoint discovery and correlation.
Mandatory tags implemented:
aws.httpapi(v2 HTTP API) in addition toaws.apigateway(v1 REST API)serverfor all proxy spansweb, maintained for consistencyhttps://scheme (prevents backend parsing issues)x-dd-proxy-resource-pathheader (resource template path)<Method> <Route>when route available, fallback to<Method> <Path>_dd.appsec.enabledmetric and_dd.appsec.jsontag from root span to proxy span in distributed tracing scenariosOptional tags implemented:
x-dd-proxy-account-idheaderx-dd-proxy-api-idheaderx-dd-proxy-regionheaderarn:aws:apigateway:{region}::/restapis|apis/{api-id}Motivation
This implementation is required by RFC-1081: Endpoint Discovery & Correlation from Inferred Spans
This PR covers the Inferred Proxy Spans portion of the RFC. The Inferred Lambda Spans portion will be addressed in a separate PR #10576.
Additional Notes
aws_user exclusion: The optional
aws_usertag was intentionally excluded per RFC guidance due to PII concerns (assumed-role session names may contain user identifiers). Implementation requires explicit approval.Contributor Checklist
type:and (comp:orinst:) labels in addition to any other useful labelsclose,fix, or any linking keywords when referencing an issueUse
solvesinstead, and assign the PR milestone to the issueJira ticket: APPSEC-61198
Note: Once your PR is ready to merge, add it to the merge queue by commenting
/merge./merge -ccancels the queue request./merge -f --reason "reason"skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see this doc.