Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade storage integration test to v2 Trace Reader #6388

Merged
merged 15 commits into from
Dec 26, 2024

Conversation

ekefan
Copy link
Contributor

@ekefan ekefan commented Dec 19, 2024

Which problem is this PR solving?

Description of the changes

  • Incrementally swaps the fields of StorageIntegration to align with v2 storage api while supporting v1 api
  • Updates test functions accordingly to work with the updated fields

How was this change tested?

  • make test

Checklist

@ekefan ekefan requested a review from a team as a code owner December 19, 2024 13:41
@ekefan ekefan requested a review from albertteoh December 19, 2024 13:41
@ekefan ekefan force-pushed the upgrage-storage-integration branch 3 times, most recently from 7b5ae83 to b3a2eb1 Compare December 19, 2024 14:36
@ekefan
Copy link
Contributor Author

ekefan commented Dec 19, 2024

Hello @yurishkuro,

please review this implementation.

@yurishkuro
Copy link
Member

I need to go through it more carefully but on a brief scan looks very much in the right direction.

@yurishkuro yurishkuro added the changelog:ci Change related to continuous integration / testing label Dec 19, 2024
@ekefan ekefan force-pushed the upgrage-storage-integration branch from b3a2eb1 to 3908c5a Compare December 20, 2024 10:19
@ekefan ekefan requested a review from yurishkuro December 21, 2024 22:09
@ekefan ekefan force-pushed the upgrage-storage-integration branch 3 times, most recently from 7ccf5f7 to 1760562 Compare December 23, 2024 17:06
@yurishkuro
Copy link
Member

sorry, I don't follow what recent changes are you making, the code looks the same as before.

@ekefan
Copy link
Contributor Author

ekefan commented Dec 24, 2024

sorry, I don't follow what recent changes are you making, the code looks the same as before.

Apologies, I haven't been able to make changes yet. I made a mistake trying to update this branch with upstream/main - I added commits, I didn't author. The recent push helped me fix that mistake so this branch can sync properly.

@yurishkuro
Copy link
Member

make sure to hit Update Branch button before continuing.

@ekefan ekefan force-pushed the upgrage-storage-integration branch from 1760562 to ad46425 Compare December 25, 2024 23:21
`StorageIntegrationV2` supports v2 storage api interfaces as StorageIntegration still supports v1 api
Provide a function for extracting flat array of spans from otel traces
Signed-off-by: Emmanuel Emonueje Ebenezer <[email protected]>
Accept combination of change in the file

Signed-off-by: Emmanuel Emonueje Ebenezer <[email protected]>
…ckend

Signed-off-by: Ebenezer <[email protected]>
Signed-off-by: Emmanuel Emonueje Ebenezer <[email protected]>
…2ToModel`

Signed-off-by: Emmanuel Emonueje Ebenezer <[email protected]>
Upgraded to TraceReader for v2 storage api

Signed-off-by: Emmanuel Emonueje Ebenezer <[email protected]>
Signed-off-by: Emmanuel Emonueje Ebenezer <[email protected]>
@ekefan ekefan force-pushed the upgrage-storage-integration branch from ad46425 to aa18bc4 Compare December 25, 2024 23:24
Comment on lines 67 to 69
span.Process = &model.Process{}
span.Process.ServiceName = batch.GetProcess().GetServiceName()
span.Process.Tags = batch.GetProcess().GetTags()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what we usually do in this case is

if span.Process == nil {
  span.Process = batch.Process
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright...

I tried updating the span process everytime, regardless, but the comparator would pick up a difference I couldn't notice by looking at json data. But I'll try with this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reusing the Process from Batch avoids unnecessary memory allocations.

Copy link
Contributor Author

@ekefan ekefan Dec 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay,

I tried again though, I don't know why yet, but for badger storage (the test I tried) it does this only for those cases.
Screenshot (83)
Screenshot (84)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I spent an hour debugging but couldn't find the root cause. I even wrote a simple test that does a roundtrip from model -> ptrace -> model where we can see that if Process.Tags==[] at the start it becomes nil after roundtrip. However, pretty.Diff did not complain about that, so I am not sure why it was complaining in the badger test (it complained exactly about that, that actual Process.Tags == nil while expected process had [].

Signed-off-by: Emmanuel Emonueje Ebenezer <[email protected]>
@yurishkuro yurishkuro changed the title Upgrage storage integration test to storage v2 API Upgrade storage integration test to v2 Trace Reader Dec 26, 2024
Signed-off-by: Yuri Shkuro <[email protected]>
Signed-off-by: Yuri Shkuro <[email protected]>
Copy link

codecov bot commented Dec 26, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 96.25%. Comparing base (b02c559) to head (e22f53f).
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #6388      +/-   ##
==========================================
- Coverage   96.26%   96.25%   -0.01%     
==========================================
  Files         368      368              
  Lines       20979    21008      +29     
==========================================
+ Hits        20196    20222      +26     
- Misses        599      601       +2     
- Partials      184      185       +1     
Flag Coverage Δ
badger_v1 10.55% <70.00%> (+1.60%) ⬆️
badger_v2 3.03% <70.00%> (+1.18%) ⬆️
cassandra-4.x-v1-manual 16.43% <70.00%> (+1.55%) ⬆️
cassandra-4.x-v2-auto 2.96% <70.00%> (+1.17%) ⬆️
cassandra-4.x-v2-manual 2.96% <70.00%> (+1.17%) ⬆️
cassandra-5.x-v1-manual 16.43% <70.00%> (+1.55%) ⬆️
cassandra-5.x-v2-auto 2.96% <70.00%> (+1.17%) ⬆️
cassandra-5.x-v2-manual 2.96% <70.00%> (+1.17%) ⬆️
elasticsearch-6.x-v1 20.16% <70.00%> (+1.60%) ⬆️
elasticsearch-7.x-v1 20.23% <70.00%> (+1.61%) ⬆️
elasticsearch-8.x-v1 20.39% <70.00%> (+1.60%) ⬆️
elasticsearch-8.x-v2 3.05% <70.00%> (+1.20%) ⬆️
grpc_v1 12.20% <70.00%> (+1.59%) ⬆️
grpc_v2 9.31% <70.00%> (+1.19%) ⬆️
kafka-2.x-v1 10.39% <70.00%> (+1.08%) ⬆️
kafka-2.x-v2 3.06% <70.00%> (+1.21%) ⬆️
kafka-3.x-v1 10.39% <70.00%> (+1.08%) ⬆️
kafka-3.x-v2 3.06% <70.00%> (+1.21%) ⬆️
memory_v2 3.06% <70.00%> (+1.21%) ⬆️
opensearch-1.x-v1 20.29% <70.00%> (+1.61%) ⬆️
opensearch-2.x-v1 20.29% <70.00%> (+1.61%) ⬆️
opensearch-2.x-v2 3.06% <70.00%> (+1.22%) ⬆️
tailsampling-processor 0.56% <0.00%> (+0.09%) ⬆️
unittests 95.12% <100.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@yurishkuro yurishkuro merged commit b02900c into jaegertracing:main Dec 26, 2024
55 of 56 checks passed
@yurishkuro
Copy link
Member

great start @ekefan !

I changed the description of PR a bit because it does not fully resolve the original issue.

@ekefan ekefan deleted the upgrage-storage-integration branch December 28, 2024 10:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/storage changelog:ci Change related to continuous integration / testing v2
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants