BREAKING CHANGE - updates to Store setup

Breaking changes:

* Now only one DynamoDB wrapper / client per store instance
* A whole bunch of changes to store setup and config

Also documentation for setup

The driver for this was that configuration with multiple
DynamoDB clients was messy, and I don't think is that useful. For different clients the user can just instantiate different instances of the entity store.

Author: mikebroberts

README.md

@@ -75,16 +75,18 @@ We are using a ["Single Table Design"](https://www.alexdebrie.com/posts/dynamodb
 
 Now we add / install Entity Store in the usual way [from NPM](https://www.npmjs.com/package/@symphoniacloud/dynamodb-entity-store) , e.g.
 
-```% npm install @symphoniacloud/dynamodb-entity-store```
+```
+% npm install @symphoniacloud/dynamodb-entity-store
+```
 
 Let's assume that our DynamoDB Table is named `AnimalsTable`.
 
 We can create an entity store by using
 the [`createStore`](https://symphoniacloud.github.io/dynamodb-entity-store/functions/createStore.html)
-and  [`createStandardSingleTableStoreConfig`](https://symphoniacloud.github.io/dynamodb-entity-store/functions/createStandardSingleTableStoreConfig.html) functions:
+and  [`createStandardSingleTableConfig`](https://symphoniacloud.github.io/dynamodb-entity-store/functions/createStandardSingleTableConfig.html) functions:
 
 ```typescript
-const entityStore = createStore(createStandardSingleTableStoreConfig('AnimalsTable'))
+const entityStore = createStore(createStandardSingleTableConfig('AnimalsTable'))
 ```
 
 `entityStore` is an object that implements
@@ -116,7 +118,7 @@ We only need to create this object **once per type** of entity in our applicatio
 * **Optional:** express how to convert an object to a DynamoDB record ("formatting")
 * **Optional:** Create Global Secondary Index (GSI) key values
 
-> A complete discussion of _Entities_ is available in [the manual, here](./documentation/1-Entities.md).
+> A complete discussion of _Entities_ is available in [the manual, here](./documentation/Entities.md).
 
 We can now call `.for(...)` on our entity store. This returns an object that implements [`SingleEntityOperations`](https://symphoniacloud.github.io/dynamodb-entity-store/interfaces/SingleEntityOperations.html) - **this is the object that you'll likely work with most when using this library**.
 
@@ -223,9 +225,8 @@ When you're working on setting up your entities and queries you'll often want to
 doing. You can do this by turning on logging:
 
 ```typescript
-const config = createStandardSingleTableStoreConfig('AnimalsTable')
-config.store.logger = consoleLogger
-const entityStore = createStore(config)
+const config = createStandardSingleTableConfig('AnimalsTable')
+const entityStore = createStore(createStandardSingleTableConfig('AnimalsTable'), { logger: consoleLogger })
 ```
 
 With this turned on we can see the output from our last query:
@@ -313,7 +314,7 @@ Write operations are no different than before - Entity Store handles generating
 generator functions. So if we have the following...
 
 ```typescript
-const entityStore = createStore(createStandardSingleTableStoreConfig('AnimalsTable'))
+const entityStore = createStore(createStandardSingleTableConfig('AnimalsTable'))
 const chickenStore = entityStore.for(CHICKEN_ENTITY)
 
 await chickenStore.put({ breed: 'sussex', name: 'ginger', dateOfBirth: '2021-07-01', coop: 'bristol' })
@@ -403,16 +404,11 @@ We create the entity store using a custom configuration:
 
 ```typescript
 const entityStore = createStore(
-  createSingleTableConfiguration({
-    tableName: 'FarmTable',
-    metaAttributeNames: { pk: 'Name' }
-  })
+  createMinimumSingleTableConfig('FarmTable', { pk: 'Name' })
 )
 ```
 
-We're only using one table (here "single table" just means one table, rather than the "standard" configuration), and we
-override
-the `metaAttributeNames` settings to only store the partition key, with the correct attribute name.
+> See [_Setup_ in the manual](documentation/Setup.md) for more on custom table configuration.
 
 The `Entity` this time is a bit more complicated:
 
@@ -496,16 +492,13 @@ results in:
 Error: Scan operations are disabled for this store
 ```
 
-However, we can change our store configuration to be the following:
+However, we can change our table configuration to be the following:
 
 ```typescript
-const entityStore = createStore(
-  createSingleTableConfiguration({
-    tableName: 'FarmTable',
-    metaAttributeNames: { pk: 'Name' },
-    allowScans: true
-  })
-)
+const entityStore = createStore({
+  ...createMinimumSingleTableConfig('FarmTable', { pk: 'Name' }),
+  allowScans: true
+})
 ```
 
 and now we can run our scan:

documentation/2-Setup.md

@@ -136,7 +136,9 @@ function pk({ name }: Pick<Farm, 'name'>) {
 
 ### `.convertToDynamoFormat()` (optional)
 
-`convertToDynamoFormat()` is an optional function you may choose to implement in order to change how DynamoDB Entity Store writes an object to DynamoDB during `put` operations. Since DynamoDB Entity Store uses the [AWS Document Client](https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/Package/-aws-sdk-lib-dynamodb/) library under the covers, this is more about choosing which fields to save, and any field-level modification, rather than lower-level "marshalling". If you need to change marshalling options at the AWS library level please refer to the [Setup chapter](2-Setup.md).
+`convertToDynamoFormat()` is an optional function you may choose to implement in order to change how DynamoDB Entity Store writes an object to DynamoDB during `put` operations.
+Since DynamoDB Entity Store uses the [AWS Document Client](https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/Package/-aws-sdk-lib-dynamodb/) library under the covers, this is more about choosing which fields to save, and any field-level modification, rather than lower-level "marshalling".
+If you need to change marshalling options at the AWS library level please refer to the [Setup chapter](Setup.md).
 
 By default DynamoDB Entity Store will store all the fields of an object, unmanipulated, using the field names of the object. E.g. going back to our `Sheep` example, let's say we're writing the following object:
 
@@ -214,7 +216,7 @@ The `gsis` field defines _generator_ functions for all of the Global Secondary I
 
 The type of `gsis` is `Record<string, GsiGenerators>`, a map from a GSI identifier to a GSI PK generator, and optionally a GSI SK generator.
 
-The **GSI identifier** will typically be the same as, or similar to, the name of your actual DynamoDB GSI. The mapping from _Entity_ GSI ID to DynamoDB GSI Name is configured in [Table Setup](2-Setup.md), but as an example the "standard" configuration uses `gsi` as the _Entity_ GSI ID, and `GSI` for the corresponding index name.
+The **GSI identifier** will typically be the same as, or similar to, the name of your actual DynamoDB GSI. The mapping from _Entity_ GSI ID to DynamoDB GSI Name is configured in [Table Setup](Setup.md), but as an example the "standard" configuration uses `gsi` as the _Entity_ GSI ID, and `GSI` for the corresponding index name.
 
 If you understand the table `pk()` and `sk()` generators then you'll understand the GSI Generators too. See the [example in the project README](https://github.com/symphoniacloud/dynamodb-entity-store/blob/main/README.md#example-2-adding-a-global-secondary-index) for an example. 
 

documentation/1-Entities.md renamed to documentation/Entities.md

@@ -0,0 +1,217 @@
+# Chapter 2 - Setup
+
+## Library and Module
+
+[_DynamoDB Entity Store_](https://github.com/symphoniacloud/dynamodb-entity-store) is available [from NPM](https://www.npmjs.com/package/@symphoniacloud/dynamodb-entity-store), and can be installed in the usual way, e.g.:
+
+```
+% npm install @symphoniacloud/dynamodb-entity-store
+```
+
+The library is provided in both CommonJS and ESModule form. All entrypoints are available from the root _index.js_ file.
+
+> _I tried using package.json [exports](https://nodejs.org/api/packages.html#exports) but IDE support seems flakey, so I've reverted for now to just supporting a root "barrel" file_
+
+## Instantiating Entity Store
+
+The main entry point for Entity Store is the function [`createStore(config)`](https://symphoniacloud.github.io/dynamodb-entity-store/functions/createStore.html). This function returns an instance of the [`AllEntitiesStore`](https://symphoniacloud.github.io/dynamodb-entity-store/interfaces/AllEntitiesStore.html) type which you can use to perform operations on your DynamoDB table(s).
+
+`createStore(config)` takes one required argument and one optional argument:
+* `tablesConfig` defines the names and configuration of all the tables you want to access through an instance of the Store.
+* `context` provides implementations of various behaviors. If you don't use it then defaults are used.
+
+For some scenarios using all the default values of DynamoDB Entity Store will be sufficient. In such a case you can instantiate your store as follows:
+
+```typescript
+const entityStore = createStore(createStandardSingleTableConfig('AnimalsTable')) // "AnimalsTable" is an example
+```
+
+Typically though you'll need to change behavior in some form. I'll start with describing how to update `context`.
+
+### Overriding `context`
+
+`context` is an object of type [`TableBackedStoreContext`](https://symphoniacloud.github.io/dynamodb-entity-store/interfaces/TableBackedStoreContext.html), defined as follows:
+
+```typescript
+{
+  logger: EntityStoreLogger
+  dynamoDB: DynamoDBInterface
+  clock: Clock
+}
+```
+
+If you don't specify a context when calling `createStore()` then the default values are used, as follows:
+
+* `logger` : No-op logger (Don't log)
+* `dynamoDB` : Wrapper using default DynamoDB document client. (See below for details)
+* `clock` : Real clock based on system time (it can be useful to override this in tests)
+
+Use the [`createStoreContext()`](https://symphoniacloud.github.io/dynamodb-entity-store/functions/createStoreContext.html) function to create a context with different values.
+With no arguments it provides precisely the same default values, but you can provide overrides as necessary. Here are a few such scenarios.
+
+#### Overriding the DynamoDB Document Client or DynamoDB wrapper
+
+By default DynamoDB Entity Store uses the default [DynamoDB Document Client](https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/Package/-aws-sdk-lib-dynamodb/) object, which uses the AWS account and region in the current context (e.g. from environment variables) and default marshalling / unmarshalling (see the official [AWS Documentation](https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/Package/-aws-sdk-lib-dynamodb/) for more details).
+
+If you want to override any of this behavior you can provide your own Document Client object as the second argument to `createStoreContext()`.
+
+For example to override the region you might call the following:
+
+```typescript
+const storeContext = createStoreContext({}, DynamoDBDocumentClient.from(new DynamoDBClient({ region: 'us-east-1' })))
+```
+
+DynamoDB Entity Store uses a "wrapper" around the Document Client, which is the `dynamoDB` property on the Store Context. You can also override this, but typically you'd only do so for unit / in-process tests.
+
+#### Specifying a logger
+
+DynamoDB Entity Store will log various behavior at **debug level**. You can override the library's logger when calling `createStoreContext()`, e.g. `createStoreContext({ logger: consoleLogger })`.
+
+The default implementation is a "no-op" logger, i.e. don't actually log anywhere.
+However you can instead use the [`consoleLogger`](https://symphoniacloud.github.io/dynamodb-entity-store/variables/consoleLogger.html), or you can provide your own implementation of [`EntityStoreLogger`](https://symphoniacloud.github.io/dynamodb-entity-store/interfaces/EntityStoreLogger.html). 
+E.g. here's an implementation that uses the [AWS Powertools Logger](https://docs.powertools.aws.dev/lambda/typescript/latest/core/logger/) :
+
+```typescript
+// Create an implementation of DynamoDB Entity Store Logger using an underlying AWS Powertools Logger
+function createPowertoolsEntityStoreLogger(logger: Logger): EntityStoreLogger {
+  return {
+    getLevelName() {
+      return logger.getLevelName()
+    },
+    debug(input: LogItemMessage, ...extraInput) {
+      logger.debug(input, ...extraInput)
+    }
+  }
+}
+```
+
+#### Overriding the Clock
+
+DynamoDB Entity Store uses a clock when generating the Last Updated field on items. By default this is the system clock, but you
+can override this - typically you'd only want to do so in tests. For an example see [`FakeClock`](https://github.com/symphoniacloud/dynamodb-entity-store/blob/main/test/unit/testSupportCode/fakes/fakeClock.ts) in the project's own test code.
+
+### Configuring Tables
+
+DynamoDB entity store can use one or more tables when performing operations.
+You specify your entire table configuration as the first argument of `createStore(config)`.
+I'll first explain how to configure Entity Store when using one table, and then will expand this to why and how you might want to configure multiple tables.
+
+#### Single-table Configuration
+
+A single table is configured using the `TableConfig` interface:
+
+```typescript
+export interface TableConfig {
+  tableName: string
+  metaAttributeNames: {
+    pk: string
+    sk?: string
+    gsisById?: Record<string, { pk: string; sk?: string }>
+    ttl?: string
+    entityType?: string
+    lastUpdated?: string
+  }
+  allowScans?: boolean
+  gsiNames?: Record<string, string>
+}
+```
+
+If you want you can "hand-roll" this object, however there are support functions in [_setupSupport.ts_](../src/lib/support/setupSupport.ts) to help out. 
+
+For example, say you want to use a "standard single table" configuration. To create one of these you can call `createStandardSingleTableConfig()`, just passing your underlying table name. The resulting configuration will be as follows:
+
+```typescript
+{
+  tableName: 'testTable',
+  allowScans: false,
+  metaAttributeNames: {
+    pk: 'PK',
+    sk: 'SK',
+    ttl: 'ttl',
+    entityType: '_et',
+    lastUpdated: '_lastUpdated',
+    gsisById: {
+      gsi: {
+        pk: 'GSIPK',
+        sk: 'GSISK'
+      }
+    }
+  },
+  gsiNames: {
+    gsi: 'GSI'
+  }
+}
+```
+
+This configuration is valid when:
+
+* Your table partition key attribute is named `PK`
+* You have a table sort key and the attribute is named `SK`
+* You have one GSI (Global Secondary Index) which is named `GSI`. It has a string partition key named `GSIPK` and a string sort key named `GSISK`. You reference this in entities using the "logical" ID `gsi`.
+* You want to automatically create `_et` and `_lastUpdated` attributes for each item.
+* If you specify a TTL (Time-To-Live) value when writing an object then it will be stored in an attributed named `ttl`
+* You don't want to allow scans
+
+If any of your configuration is different from this you can do the following:
+
+* Use the `createMinimumSingleTableConfig()` function, providing the table name and meta attribute names, and then add any other necessary properties
+* Use the `createStandardSingleTableConfig()` function above, and replace properties
+* Build your own implementation of `TableConfig`
+
+The particular behaviors of this configuration will be explained in later parts of this manual.
+
+#### When to use multi-table configuration
+
+Some projects will use multiple DynamoDB tables. While I'm a fan of DynamoDB "single table design", I think there's often a place to use different tables for different operational reasons. And sometimes you'll be working in a project that doesn't use single table design.
+
+When your project has multiple tables you can choose one of the following:
+
+* Create an `AllEntitiesStore` per table, each store using single-table configuration
+* Create one or several `AllEntitiesStore`(s) that have a multi-table configuration
+
+The way that multi-table configuration works with DynamoDB Entity Store is that each entity can only be stored in one table, and in the setup configuration each table includes the list of entities contained within it.
+To be clear - **one table can store multiple entities, but each entity can only be stored in one table**, for each Entity Store instance.
+
+So if you want to store the same entity in multiple tables that immediately drives to using multiple `AllEntitiesStore` instances.
+
+DynamoDB Entity Store uses one underlying Document Client per instance.
+Another reason to use multiple instances therefore is if the different tables have different DynamoDB document client configuration. For example:
+
+* Different tables are in different accounts / regions / have different credentials
+* Different tables use different marshalling / unmarshalling options
+
+However it's often the case that the constraint of one-table-per-entity, and common-document-client-per-table, is absolutely fine, and in such a case using a multi-table configuration of DynamoDB Entity Store can be used. This has the following advantages:
+
+* Less code / state in your application
+* Ability to perform transactions across multiple entities in different tables 
+
+#### How to use multi-table configuration
+
+To use a multi-table configuration call `createStore(config)` just as you would do for single-table, but
+the config object needs to be of type `MultiTableConfig`, as follows:
+
+```typescript
+export interface MultiTableConfig {
+  entityTables: MultiEntityTableConfig[]
+  defaultTableName?: string
+}
+
+export interface MultiEntityTableConfig extends TableConfig {
+  entityTypes?: string[]
+}
+```
+
+In other words a multi-table config consists of:
+
+* An array of regular `TableConfig` objects, each having the addition of array of entity type names stored in the table
+* An optional default table name
+
+The entity type names must be precisely the same as those specified in the `type` field of the _Entities_ you'll be using when performing operations.
+When you make calls to the operations functions in Entity Store the library will first find the table configuration used for that _Entity_.
+
+The `defaultTableName` property is useful if you have a situation where _most_ entities are in one table, but you have a few "special cases" of other entities being in different tables.
+
+You have a few options of how to create a `MultiTableConfig` object:
+
+* Use the `createStandardMultiTableConfig()` function if all of your tables use the same "standard" configuration described earlier
+* Build your own configuration, optionally using the other support functions in [_setupSupport.ts_](../src/lib/support/setupSupport.ts).

documentation/Setup.md

@@ -2,6 +2,6 @@
 
 For an overview of using DynamoDB Entity Store, please see the [README](../README.md).
 
-1. [Entities](1-Entities.md)
-2. [Setup](2-Setup.md)
-3. [Simple Usage](3-SimpleUsage.md)
+1. [Entities](Entities.md)
+2. [Setup](Setup.md)
+3. [Simple Usage](SimpleUsage.md)

documentation/3-SimpleUsage.md renamed to documentation/SimpleUsage.md

@@ -1,6 +1,6 @@
 import {
   createEntity,
-  createStandardSingleTableStoreConfig,
+  createStandardSingleTableConfig,
   createStore,
   DynamoDBValues,
   rangeWhereSkBetween
@@ -29,7 +29,7 @@ export const SHEEP_ENTITY = createEntity(
 
 async function run() {
   // Create entity store using default configuration
-  const config = createStandardSingleTableStoreConfig('AnimalsTable')
+  const config = createStandardSingleTableConfig('AnimalsTable')
   // config.store.logger = consoleLogger
   const entityStore = createStore(config)
 

documentation/manual.md

@@ -1,5 +1,5 @@
 import {
-  createStandardSingleTableStoreConfig,
+  createStandardSingleTableConfig,
   createStore,
   Entity,
   rangeWhereSkBeginsWith,
@@ -68,7 +68,7 @@ export function gsiBreed(breed: string) {
 
 async function run() {
   // Create entity store using default configuration
-  const entityStore = createStore(createStandardSingleTableStoreConfig('AnimalsTable'))
+  const entityStore = createStore(createStandardSingleTableConfig('AnimalsTable'))
   const chickenStore = entityStore.for(CHICKEN_ENTITY)
 
   await chickenStore.put({ breed: 'sussex', name: 'ginger', dateOfBirth: '2021-07-01', coop: 'bristol' })