Skip to content

Added .loop' API #18

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Added .loop' API #18

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
e4c1f88
Enable Strings as choice keys
autarc Jul 11, 2015
5dd2752
Add $parent property for passing scoped references
autarc Jul 11, 2015
165c61a
improve skip parameter description
autarc Jul 15, 2015
f8a4ac2
Added 'loop' call to API
autarc Jul 22, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 35 additions & 6 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -151,6 +151,8 @@ Parse bytes as an array. `options` is an object; following options are available
Use number for statically sized arrays.
- `readUntil` - (either `length` or `readUntil` is required) If `'eof'`, then this parser
reads until the end of `Buffer` object. If function it reads until the function returns true.
- `$parent` - (Optional) An array with selected properties from the parental scope. References can be
accessed inside functions using `this.$parent[...]`.

```javascript
var parser = new Parser()
Expand Down Expand Up @@ -193,9 +195,11 @@ Combining `choice` with `array` is useful for parsing a typical

- `tag` - (Required) The value used to determine which parser to use from the `choices`
Can be a string pointing to another field or a function.
- `choices` - (Required) An object which key is an integer and value is the parser which is executed
- `choices` - (Required) An object which key is an integer/string and value is the parser which is executed
when `tag` equals the key value.
- `defaultChoice` - (Optional) In case of the tag value doesn't match any of `choices` use this parser.
- `$parent` - (Optional) An array with selected properties from the parental scope. References can be
accessed inside functions using `this.$parent[...]`.

```javascript
var parser1 = ...;
Expand All @@ -207,21 +211,46 @@ var parser = new Parser()
.choice('data', {
tag: 'tagValue',
choices: {
1: parser1, // When tagValue == 1, execute parser1
4: parser2, // When tagValue == 4, execute parser2
5: parser3 // When tagValue == 5, execute parser3
1: parser1, // if tagValue == 1, execute parser1
4: parser2, // if tagValue == 4, execute parser2
5: parser3 // if tagValue == 5, execute parser3
}
});
```

### loop(lookahead, [,options])
Choose and execute parsers which are matching the referenced tag value.
The `lookahead` parser provides information about upcoming fields to determine the selection.
As long as one of the `choices` options fits, it continues parsing the buffer.

- `tag` - (Required) The value used to determine which parser to use from the `choices`
Can be a string pointing to another field or a function.
- `choices` - (Required) An object which key is an integer/string and value is the parser which is executed
when `tag` equals the key value. The key of a choice is used as the property name and will hold an array
of results if multiple instances are matched.

```javascript
var parser1 = ...;
var parser2 = ...;

var parser = new Parser()
.loop(new Parser().skip(4).string('type', { length: 4 })), {
tag: 'type',
choices: {
ftyp: parser1, // if type == 'ftyp', execute parser1
moov: parser2 // if type == 'ftyp', execute parser2
}
});
```

### nest(name [,options])
Nest a parser in this position. Parse result of the nested parser is stored in the variable
`name`.

- `type` - (Required) A `Parser` object.

### skip(length)
Skip parsing for `length` bytes.
Skip parsing of bytes. `length` can be either a number, a string or a function.

### endianess(endianess)
Define what endianess to use in this parser. `endianess` can be either `'little'` or `'big'`.
Expand Down Expand Up @@ -254,7 +283,7 @@ These are common options that can be specified in all parsers.
```javascript
var parser = new Parser()
.array('ipv4', {
type: uint8,
type: 'uint8',
length: '4',
formatter: function(arr) { return arr.join('.'); }
});
Expand Down
95 changes: 82 additions & 13 deletions lib/binary_parser.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,8 @@ var SPECIAL_TYPES = {
'Skip' : null,
'Choice' : null,
'Nest' : null,
'Bit' : null
'Bit' : null,
'Loop' : null
};

var BIT_RANGE = [];
Expand Down Expand Up @@ -148,9 +149,6 @@ Parser.prototype.choice = function(varName, options) {
throw new Error('Choices option of array is not defined.');
}
Object.keys(options.choices).forEach(function(key) {
if (isNaN(parseInt(key, 10))) {
throw new Error('Key of choices must be a number.');
}
if (!options.choices[key]) {
throw new Error('Choice Case ' + key + ' of ' + varName + ' is not valid.');
}
Expand All @@ -174,6 +172,17 @@ Parser.prototype.nest = function(varName, options) {
return this.setNextParser('nest', varName, options);
};

Parser.prototype.loop = function (lookahead, options) {
if (!(lookahead instanceof Parser)) {
throw new Error('The lookahead function must be a Parser object.');
}
if (!options.tag) {
throw new Error('Tag option of loop is not defined.');
}

return this.setNextParser('loop', lookahead, options);
};

Parser.prototype.endianess = function(endianess) {
switch (endianess.toLowerCase()) {
case 'little':
Expand Down Expand Up @@ -291,24 +300,23 @@ Parser.prototype.setNextParser = function(type, varName, options) {

// Call code generator for this parser
Parser.prototype.generate = function(ctx) {

if (this.type) {
this['generate' + this.type](ctx);
this.generateAssert(ctx);
if (this.options.assert) {
this.generateAssert(ctx);
}
}

var varName = ctx.generateVariable(this.varName);
if (this.options.formatter) {
var varName = ctx.generateVariable(this.varName);
this.generateFormatter(ctx, varName, this.options.formatter);
}

return this.generateNext(ctx);
};

Parser.prototype.generateAssert = function(ctx) {
if (!this.options.assert) {
return;
}

var varName = ctx.generateVariable(this.varName);

switch (typeof this.options.assert) {
Expand Down Expand Up @@ -485,7 +493,17 @@ Parser.prototype.generateArray = function(ctx) {
ctx.pushCode('var {0} = buffer.read{1}(offset);', item, NAME_MAP[type]);
ctx.pushCode('offset += {0};', PRIMITIVE_TYPES[NAME_MAP[type]]);
} else if (type instanceof Parser) {
ctx.pushCode('var {0} = {};', item);
if (!this.options.$parent) {
ctx.pushCode('var {0} = {};', item);
} else {
ctx.pushCode('var {0} = { "$parent": {1} };', item,
'[' + this.options.$parent.map(function(prop){ return '"' + prop + '"'; }).toString() +
'].reduce(function($parent, key){\
$parent[key] = ' + ctx.generateVariable() + '[key];\
return $parent;\
}, {})'
);
}

ctx.pushScope(item);
type.generate(ctx);
Expand Down Expand Up @@ -519,12 +537,22 @@ Parser.prototype.generateChoiceCase = function(ctx, varName, type) {
Parser.prototype.generateChoice = function(ctx) {
var tag = ctx.generateOption(this.options.tag);

ctx.pushCode('{0} = {};', ctx.generateVariable(this.varName));
if (!this.options.$parent) {
ctx.pushCode('{0} = {};', ctx.generateVariable(this.varName));
} else {
ctx.pushCode('{0} = { "$parent": {1} };', ctx.generateVariable(this.varName),
'[' + this.options.$parent.map(function(prop){ return '"' + prop + '"'; }).toString() +
'].reduce(function($parent, key){\
$parent[key] = ' + ctx.generateVariable() + '[key];\
return $parent;\
}, {})'
);
}
ctx.pushCode('switch({0}) {', tag);
Object.keys(this.options.choices).forEach(function(tag) {
var type = this.options.choices[tag];

ctx.pushCode('case {0}:', tag);
ctx.pushCode('case ' + (isNaN(tag) ? '"{0}"' : '{0}') + ':', tag);
this.generateChoiceCase(ctx, this.varName, type);
ctx.pushCode('break;');
}, this);
Expand All @@ -551,6 +579,47 @@ Parser.prototype.generateFormatter = function(ctx, varName, formatter) {
}
};

Parser.prototype.generateLoop = function(ctx) {
var originalScopes = ctx.scopes;
var scopeReference = originalScopes[0].join('.');
var loopVar = ctx.generateTmpVariable();
var tagVar = ctx.generateTmpVariable();
var caseVar = ctx.generateTmpVariable();
ctx.pushCode('var {0} = true;', loopVar);
ctx.pushCode('var {0} = {};', tagVar);
ctx.pushCode('var {0} = {};', caseVar);

ctx.pushCode('while ({0}) {', loopVar);
ctx.scopes = [[tagVar]];
var before = ctx.code;
this.varName.generate(ctx);
ctx.scopes = [[]];
ctx.pushCode('offset -= {0};',
ctx.code.replace(before, '').match(/offset \+= (\d+);/g).reduce(function(length, modifier) {
return length + parseInt(modifier.replace(/\D+/, ''), 10);
}, 0)
);
ctx.pushCode('switch({0}.{1}) {', tagVar, this.options.tag);
Object.keys(this.options.choices).forEach(function(tag) {
var type = this.options.choices[tag];

ctx.pushCode('case ' + (isNaN(tag) ? '"{0}"' : '{0}') + ':', tag);
ctx.pushCode('{0} = {};', caseVar);
this.generateChoiceCase(ctx, caseVar, type);
ctx.pushCode('if (!{0}.{1}) {\
{0}.{1} = {2}; } else {\
if (!Array.isArray({0}.{1})) {0}.{1} = [{0}.{1}];\
{0}.{1}.push({2}); }', scopeReference, tag, caseVar);
ctx.pushCode('break;');
}, this);
ctx.pushCode('default:');
ctx.pushCode('{0} = false;', loopVar);
ctx.pushCode('}');

ctx.pushCode('}');
ctx.scopes = originalScopes;
};

Parser.prototype.isInteger = function() {
return !!this.type.match(/U?Int[8|16|32][BE|LE]?|Bit\d+/);
};
Expand Down
18 changes: 18 additions & 0 deletions test/composite_parser.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -262,6 +262,24 @@ describe('Composite parser', function(){
});
});

describe('Loop parser', function(){
it('should parse looped parsers', function(){
var parser = Parser.start()
.loop(Parser.start().uint8('tag'), {
tag: 'tag',
choices: {
0: 'int32le',
1: 'int16le'
}
});
var buffer = new Buffer([0x0, 0x4e, 0x61, 0xbc, 0x00, 0x01, 0xd2, 0x04]);
assert.deepEqual(parser.parse(buffer), {
0: 12345678,
1: 1234
});
});
});

describe('Nest parser', function() {
it('should parse nested parsers', function() {
var nameParser = new Parser()
Expand Down