Skip to content

Commit 50b3189

Browse files
tmpaultmpaul
committed
Fixed errors in find/destroy
1 parent 29003de commit 50b3189

File tree

2 files changed

+168
-49
lines changed

2 files changed

+168
-49
lines changed

‎README.md‎

Lines changed: 127 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -81,7 +81,7 @@ If this file is missing, the adapter will try to read host, port , IDs and key f
8181
});
8282
```
8383

84-
####Explanation
84+
###Explanation
8585
- DynamoDB stores information in the form of tables. Each table has a collection of items. Think of items as a row in a table. Each
8686
item in turn is a collection of attributes. Each attribute is stored in the database as a key:value pair.
8787

@@ -91,50 +91,57 @@ one hash key or one hash key and a range key.
9191
- To specify a given attribute as hash key, use `keyType: "hash"` along with its definition as shown in the above example.
9292
- DynamoDB expects a hash key for every item that is created. If this value is not available at the time of creation, there is an option to use a UUID generator to generate the hash key. To use this option, specify `uuid: true` in the model definition.
9393
- Similarly, attribute becomes a range key if keyType is set to "range".
94-
- Make sure that the hash key and range keys are specified during CRUD operations. For most of these operations, hash key is a must and range key is optional, or conditional. Delete operation requires both hash and range keys. JugglingDB `destroy` method by default only
95-
sends in the id of the model. To support destroy operation for items with range keys, add the following method in `lib/model.js` of JugglingDB
94+
95+
96+
###USAGE
97+
98+
#####HASH KEY ONLY
99+
- If a model only has a hash key, any attribute can be specified as the hash key. However, if `uuid` is set to `true`, then the attribute name
100+
must be `id`. This restriction comes from JugglingDB.
101+
- If no unique ID generation is present, the value of the hash key must be provided at the time of creating an item in the table. In this case, the attribute name can be anything; not just `id`.
96102
```javascript
97-
AbstractClass.prototype.remove = function (cb) {
98-
if (stillConnecting(this.constructor.schema, this, arguments)) return;
99-
100-
var hashKey = this.schema.adapter._models[this.constructor.modelName].hashKey;
101-
var rangeKey = this.schema.adapter._models[this.constructor.modelName].rangeKey;
102-
103-
this.trigger('destroy', function (destroyed) {
104-
this._adapter().remove(this.constructor.modelName, this[hashKey], this[rangeKey], function (err) {
105-
if (err) {
106-
return cb(err);
107-
}
108-
109-
destroyed(function () {
110-
if(cb) cb();
111-
});
112-
}.bind(this));
113-
}, this.toObject(), cb);
114-
};
103+
var User = schemaDynamo.define('User', {
104+
someId : { type: String, keyType: "hash", uuid: true} ....
105+
//Not allowed
106+
107+
var User = schemaDynamo.define('User', {
108+
someId : { type: String, keyType: "hash"}, .....
109+
// Allowed
115110
```
116-
- To destroy an object with both hash and range key, use the following:
111+
#####HASH & RANGE KEYS
112+
- If a model has both hash & range keys, a primary key attribute called `id` must be present in the table. The attribute name cannot be anything else other than `id`, or the adapter will throw an error.
113+
114+
- The primary key must be defined as follows:
117115
```javascript
118-
user.remove(function(err){
119-
if (err) {
116+
var User = schemaDynamo.define('User', {
117+
id : { type: String, keyType: "pk", separator : "--oo--"},
118+
companyId : { type: Number, keyType: "hash"},
119+
name: { type: String },
120+
age: { type: Number , keyType: "range"},....
120121
....
121-
}
122-
});
123122
```
124-
#####Data Types
123+
- The separator is used to define the primary key based on the hash and range keys. If the hash key is `1` and the range key is `xyz`, then according to the above example, the primary key `id` will be `1--oo--xyz`. Any random separator can be used to store the primary key, but make sure that you do not include separators like `###` or `?`. These separators might cause problems in the view pages of the model, wherein they will be interpreted as part of the url. The default separator is `--x--`.
124+
125+
- The important thing to note is that the primary key is purely a virtual attribute to identify a particular item. It does not get persisted in the database.
126+
127+
#####DATATYPES
125128
- DynamoDB supports only String, Binary, and Number datatypes along with their corresponding Sets.
126129
- The adapter currently supports String, Number, Date and Boolean datatypes.
127130
- Null, undefined and empty strings are handled by the adapter. However, invalid date or missing numbers might cause
128131
DynamoDB to throw an error.
129132
- Date is stored internally as a number, and boolean is stored as a string => `true` or `false`.
130133
131-
#####Read Write / Capacity Units
134+
#####READ/ WRITE CAPCACITY UNITS
132135
- Provisioned Throughput for each table can be specified by using the `set` property function of Jugglingdb. In the above example, the
133136
read and write capacity units are set for `User`.
134137
- The defaults for read and write capacity units are 5 and 10 respectively.
135138
136-
#####Database Limitations
137-
- DynamoDB has an item size limit of 64 kb. Typically data is stored in the form of big strings in NoSQL tables (e.g objects with complex data structures). Eventually these strings might exceed 64 kb in size. To overcome this limitation, the adapter uses a sharding technique.
139+
#####DATABASE LIMITATIONS
140+
- DynamoDB has an item size limit of 64 kb. Typically data is stored in the form of big strings in NoSQL tables (e.g objects with complex data structures). Eventually these strings might exceed 64 kb in size. To overcome this limitation, the adapter uses a sharding technique. Sharding is done if `sharding` is set to `true` in the model property.
141+
142+
```javascript
143+
tasks: { type: String, sharding : true, splitter : "63kb"}
144+
```
138145
139146
- When an attribute is being sharded, the attribute is separated from the parent table, and is stored in a different (child) table. According to the example above, tasks attribute will be stored in a new table called `User_tasks`. The primary key of User table `id`, is stored in the new table as `user#id`, and a new range key is assigned for every chunk that exceeds a given size. So if `splitter` is set to 60 kb, and the tasks string exceeds 60 kb, the structure of child table will be as shown below:
140147
@@ -151,15 +158,104 @@ read and write capacity units are set for `User`.
151158
tasks: { "S" : "down into two different pieces" }
152159
}
153160
```
161+
- The value of `splitter` can be anywhere from 1 to 63 kb. This is to make sure that there is enough room to store the primary key of parent table and the range key in the child table.
154162
155163
- The attribute value (String) is broken down based on the size specified by the `splitter` attribute. For example, in the above given model definition, the tasks string is broken down into 60 kb chunks. Each chunk is stored as a new item with a range key and the primary key of parent item.
156164
157165
- Due to the large size of items being retrieved/written to the child tables, these tables require more read and write capacity compared to the original table. Read and write capacities for the child table can be specified as follows:
158166
```javascript
159-
tasks: { type: String, sharding : true, splitter : "60kb" , read : 15, write: 20}
167+
tasks: { type: String, sharding : true, splitter : "63kb" , read : 15, write: 20}
160168
```
169+
- If read and write are not specified, the read / write capacities of the parent table is used.
170+
161171
- When the main item is being retrieved from the database, the adapter queries each child table, and builds back the string. As a result, the data structure is still intact after retrieval.
162172
173+
###CRUD OPERATIONS
174+
175+
####Create
176+
```javascript
177+
178+
// Only hash key
179+
180+
User = db.define('User', {
181+
email: { type: String, keyType: "hash"},
182+
name: { type: String }
183+
});
184+
185+
// Both hash and range keys
186+
187+
Book = db.define('Book', {
188+
id : { type: String, keyType: "pk"},
189+
title : { type: String, keyType: "hash"},
190+
subject : { type: String, keyType: "range"},
191+
});
192+
193+
var user = new User();
194+
var book = new Book();
195+
196+
user.email = "[email protected]";
197+
user.name = "John Doe";
198+
199+
/* Note that book's `id` is not specified. `id` being
200+
a primary key is automatically created from the hash and range keys:
201+
title and subject
202+
*/
203+
204+
book.title = "A Lost Cause";
205+
book.subject = "Fiction";
206+
207+
User.create(user, function(err, _user) {
208+
.....
209+
console.log(_user);
210+
/*
211+
{
212+
id : "[email protected]",
213+
// Note that id is set to the same value as hashKey. This ensures that views don't break
214+
email: "[email protected]",
215+
name: "John Doe",
216+
}
217+
*/
218+
});
219+
220+
Book.create(book, function(err, _book) {
221+
...
222+
console.log(_book);
223+
/*
224+
{
225+
id : "A Lost Cause--x--Fiction", // Value is simply returned by create. It does not exist in database.
226+
title: "A Lost Cause",
227+
subject: "Fiction"
228+
}
229+
*/
230+
});
231+
```
232+
####Find
233+
```javascript
234+
var id = "0e203f96-8edc-437a-b1f0-625a584a49bd";
235+
User.find(id, function (err, user) {
236+
.....
237+
...
238+
});
239+
```
240+
241+
####All
242+
```javascript
243+
User.all({
244+
where : {
245+
age : { gt : 20 }
246+
/*
247+
or age : { between : [10,20] } - Between
248+
or age : { eq : 20 } - Equal to
249+
or age : [10,20] - In range 10 to 20
250+
or age : { le : 35} - Less than or equal to
251+
*/
252+
}
253+
}, function(err, users){
254+
.....
255+
});
256+
```
257+
258+
163259
####Upcoming features
164260
- Support for `limit`, `order` keywords in query filters.
165261
- Event emitters to notify that table has been created & is active.

‎lib/dynamodb.js‎

Lines changed: 41 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -715,13 +715,24 @@ function query(model, filter, hashKey, rangeKey, queryString) {
715715
tableParams.KeyConditions[key] = {};
716716
tableParams.KeyConditions[key].ComparisonOperator = query[key].operator;
717717
tableParams.KeyConditions[key].AttributeValueList = [];
718-
tableParams.KeyConditions[key].AttributeValueList.push(DynamoFromJSON(query[key].attrs));
718+
var attrResult = DynamoFromJSON(query[key].attrs);
719+
if (attrRestult instanceof Array) {
720+
tableParams.KeyConditions[key].AttributeValueList = DynamoFromJSON(query[key].attrs);
721+
} else {
722+
tableParams.KeyConditions[key].AttributeValueList.push(DynamoFromJSON(query[key].attrs));
723+
}
724+
719725
queryString = queryString + "& RANGEKEY: `" + String(key) + "` " + String(query[key].operator) + " `" + String(query[key].attrs) + "`";
720726
} else {
721727
tableParams.QueryFilter[key] = {};
722728
tableParams.QueryFilter[key].ComparisonOperator = query[key].operator;
723729
tableParams.QueryFilter[key].AttributeValueList = [];
724-
tableParams.QueryFilter[key].AttributeValueList.push(DynamoFromJSON(query[key].attrs));
730+
var attrResult = DynamoFromJSON(query[key].attrs);
731+
if (attrRestult instanceof Array) {
732+
tableParams.QueryFilter[key].AttributeValueList = DynamoFromJSON(query[key].attrs);
733+
} else {
734+
tableParams.QueryFilter[key].AttributeValueList.push(DynamoFromJSON(query[key].attrs));
735+
}
725736
queryString = queryString + "& `" + String(key) + "` " + String(query[key].operator) + " `" + String(query[key].attrs) + "`";
726737
}
727738
}
@@ -781,7 +792,15 @@ function scan(model, filter, queryString) {
781792
tableParams.ScanFilter[key] = {};
782793
tableParams.ScanFilter[key].ComparisonOperator = query[key].operator;
783794
tableParams.ScanFilter[key].AttributeValueList = [];
784-
tableParams.ScanFilter[key].AttributeValueList.push(DynamoFromJSON(query[key].attrs));
795+
var attrResult = DynamoFromJSON(query[key].attrs);
796+
797+
if (attrResult instanceof Array) {
798+
tableParams.ScanFilter[key].AttributeValueList = DynamoFromJSON(query[key].attrs);
799+
} else {
800+
tableParams.ScanFilter[key].AttributeValueList.push(DynamoFromJSON(query[key].attrs));
801+
}
802+
803+
785804
queryString = queryString + "& `" + String(key) + "` " + String(query[key].operator) + " `" + String(query[key].attrs) + "`";
786805
}
787806
}
@@ -932,29 +951,31 @@ DynamoDB.prototype.find = function find(model, pKey, callback) {
932951
hk = temp[0];
933952
rk = temp[1];
934953
queryString = queryString + " WHERE " + hashKey + " `EQ` " + hk + " " + rangeKey + " `EQ` " + rk;
954+
if (this._attributeSpecs[model][rangeKey] === "number") {
955+
rk = parseInt(rk);
956+
} else if (this._attributeSpecs[model][rangeKey] === "date") {
957+
rk = Number(rk);
958+
}
935959
} else {
936960
hk = pKey;
937961
queryString = queryString + " WHERE " + hashKey + " `EQ` " + hk;
938962
}
939963
logger.log("info", queryString.blue);
940-
// If hashKey is of type Number use parseInt
964+
965+
// If hashKey is of type Number use parseInt
941966
if (this._attributeSpecs[model][hashKey] === "number") {
942967
hk = parseInt(hk);
943968
} else if (this._attributeSpecs[model][hashKey] === "date") {
944969
hk = Number(hk);
945970
}
946-
if (this._attributeSpecs[model][rangeKey] === "number") {
947-
rk = parseInt(rk);
948-
} else if (this._attributeSpecs[model][rangeKey] === "date") {
949-
rk = Number(rk);
950-
}
951-
952971

953972
var tableParams = {};
954973
tableParams.Key = {};
955974
tableParams.Key[hashKey] = DynamoFromJSON(hk);
956-
tableParams.Key[rangeKey] = DynamoFromJSON(rk);
957-
975+
if (pk !== undefined) {
976+
tableParams.Key[rangeKey] = DynamoFromJSON(rk);
977+
}
978+
958979
tableParams.TableName = this.tables(model);
959980

960981
tableParams.ReturnConsumedCapacity = "TOTAL";
@@ -1298,20 +1319,22 @@ DynamoDB.prototype.destroy = function (model, pKey, callback) {
12981319
var temp = pKey.split(pkSeparator);
12991320
hk = temp[0];
13001321
rk = temp[1];
1322+
if (this._attributeSpecs[model][rangeKey] === "number") {
1323+
rk = parseInt(rk);
1324+
} else if (this._attributeSpecs[model][rangeKey] === "date") {
1325+
rk = Number(rk);
1326+
}
13011327
} else {
13021328
hk = pKey;
13031329
}
1304-
// If hashKey is of type Number use parseInt
1330+
1331+
// If hashKey is of type Number use parseInt
13051332
if (this._attributeSpecs[model][hashKey] === "number") {
13061333
hk = parseInt(hk);
13071334
} else if (this._attributeSpecs[model][hashKey] === "date") {
13081335
hk = Number(hk);
13091336
}
1310-
if (this._attributeSpecs[model][rangeKey] === "number") {
1311-
rk = parseInt(rk);
1312-
} else if (this._attributeSpecs[model][rangeKey] === "date") {
1313-
rk = Number(rk);
1314-
}
1337+
13151338
// Use updateItem function of DynamoDB
13161339
var tableParams = {};
13171340
// Set table name as usual

0 commit comments

Comments
 (0)