Code

refactor: ingestion data flow#1100

Code

Merged

nitisht merged 20 commits intoparseablehq:mainparseablehq/parseable:mainfrom de-sh:ingestde-sh/parseable:ingestCopy head branch name to clipboardJan 23, 2025

+597-618

Commits

Commits on Jan 20, 2025

• refactor: kinesis message construction may panic

  de-sh

  committed
  df70d0e

  Copy full SHA for df70d0e

• refactor: replace BTreeMap with serde_json::Map

  de-sh

  committed
  c57d931

  Copy full SHA for c57d931

• refactor: get rid of clone

  de-sh

  committed
  c7f401f

  Copy full SHA for c7f401f

• refactor: use Value for JSON data

  de-sh

  committed
  9ff72e7

  Copy full SHA for 9ff72e7

• refactor: HeaderMap::get and let Some else

  de-sh

  committed
  c0bee1b

  Copy full SHA for c0bee1b

• refacror: ingest utils don't need http context anymore

  de-sh

  committed
  0270aa4

  Copy full SHA for 0270aa4

• refactor: more descriptive error variants

  de-sh

  committed
  569d132

  Copy full SHA for 569d132

• refactor: PUT stream header extraction

  de-sh

  committed
  d0682cf

  Copy full SHA for d0682cf

• refactor: use Path and Json extractor

  de-sh

  committed
  7af329d

  Copy full SHA for 7af329d

• don't extract where not required

  de-sh

  committed
  1eaf769

  Copy full SHA for 1eaf769

• refactor: serde date_list

  de-sh

  committed
  0b26394

  Copy full SHA for 0b26394

• refactor: serde DefaultPrivilege

  de-sh

  committed
  d27a763

  Copy full SHA for d27a763

• refactor: serde Dashboard

  de-sh

  committed
  7583832

  Copy full SHA for 7583832

• refactor: serde Filter

  de-sh

  committed
  9248ef7

  Copy full SHA for 9248ef7

Commits on Jan 21, 2025

• Merge branch 'main' into ingest

  de-sh

  authored
  9a30847

  Copy full SHA for 9a30847

• refactor: move up p_timestamp addition to recordbatch

  de-sh

  committed
  2a28285

  Copy full SHA for 2a28285

• refactor: refer over clone

  de-sh

  committed
  ff50822

  Copy full SHA for ff50822

• fix: don't hog write privileges

  de-sh

  committed
  6a1175a

  Copy full SHA for 6a1175a

• refactor: DRY stream writer creation

  de-sh

  committed
  e9c53de

  Copy full SHA for e9c53de

Commits on Jan 22, 2025

• refactor: serde StreamType

  de-sh

  committed
  d61d72d

  Copy full SHA for d61d72d