Skip to content

uttom-akash/data-migration-dynamo-aurora

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

4 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Data Migration from Dynamo Database to Aurora PostgreSQL

Dynamo Database Segmented Scan

Divide the DyanamoDb tables into N Segments logically, and distribute those segments among worker process/threads. Each worker process will read at max 1MB of data in each read request then request for next 1MB using the LastEvaluatedKey.

Medium Blog: Reading Millions of DynamoDB Records in Distributed Parallel Batches

Spring Batch

Processing same table in parallel and different table sequentially.

PostgreSQL Merge update

Write or update data enabling retry mechanism.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors