Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: ingestion data flow #1100

Merged
merged 20 commits into from
Jan 23, 2025
Merged

refactor: ingestion data flow #1100

merged 20 commits into from
Jan 23, 2025

Conversation

de-sh
Copy link
Contributor

@de-sh de-sh commented Jan 13, 2025

Fixes #XXXX.

Description

Rids code of unnecessary conversions and copying


This PR has:

  • been tested to ensure log ingestion and log query works.
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added documentation for new or modified features or behaviors.

@coveralls
Copy link

coveralls commented Jan 13, 2025

Pull Request Test Coverage Report for Build 12906543338

Details

  • 83 of 442 (18.78%) changed or added relevant lines in 28 files are covered.
  • 12 unchanged lines in 8 files lost coverage.
  • Overall coverage increased (+0.1%) to 12.856%

Changes Missing Coverage Covered Lines Changed/Added Lines %
src/catalog/mod.rs 0 1 0.0%
src/kafka.rs 0 1 0.0%
src/metadata.rs 0 1 0.0%
src/handlers/http/modal/query/querier_ingest.rs 0 2 0.0%
src/event/writer/mem_writer.rs 0 3 0.0%
src/handlers/http/kinesis.rs 0 4 0.0%
src/handlers/http/modal/query/querier_role.rs 0 4 0.0%
src/handlers/http/role.rs 0 4 0.0%
src/handlers/http/modal/ingest/ingestor_role.rs 0 5 0.0%
src/validator.rs 0 5 0.0%
Files with Coverage Reduction New Missed Lines %
src/handlers/http/modal/utils/ingest_utils.rs 1 28.89%
src/otel/traces.rs 1 0.0%
src/handlers/http/modal/query/querier_ingest.rs 1 0.0%
src/event/writer/mod.rs 1 0.0%
src/event/mod.rs 1 0.0%
src/handlers/http/ingest.rs 1 67.23%
src/handlers/http/logstream.rs 2 1.68%
src/storage/mod.rs 4 0.0%
Totals Coverage Status
Change from base Build 12873819985: 0.1%
Covered Lines: 2477
Relevant Lines: 19268

💛 - Coveralls

Copy link
Contributor

@nikhilsinhaparseable nikhilsinhaparseable left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good to merge

@nitisht nitisht merged commit 4d2897e into parseablehq:main Jan 23, 2025
11 checks passed
@de-sh de-sh deleted the ingest branch January 23, 2025 05:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants