|
| 1 | +# Stepper Documentation |
| 2 | + |
| 3 | +<!-- @import "[TOC]" {cmd="toc" depthFrom=1 depthTo=6 orderedList=false} --> |
| 4 | + |
| 5 | +<!-- code_chunk_output --> |
| 6 | + |
| 7 | +- [Stepper Documentation](#stepper-documentation) |
| 8 | + - [Quickstart](#quickstart) |
| 9 | + - [High-level implementation details](#high-level-implementation-details) |
| 10 | + - [Expression stepper](#expression-stepper) |
| 11 | + - [Statements](#statements) |
| 12 | + - [Constant declaration](#constant-declaration) |
| 13 | + - [Functions](#functions) |
| 14 | + - [Alpha renaming](#alpha-renaming) |
| 15 | + - [Augmenting functionalities](#augmenting-functionalities) |
| 16 | + - [Entry point](#entry-point) |
| 17 | + - [Generating explanations](#generating-explanations) |
| 18 | + - [Some important decisions](#some-important-decisions) |
| 19 | + - [Builtin math functions](#builtin-math-functions) |
| 20 | + |
| 21 | +<!-- /code_chunk_output --> |
| 22 | + |
| 23 | + |
| 24 | +## Quickstart |
| 25 | +First of all, make sure that you have already installed `js-slang` using `yarn`. There are many possible ways that you can work and test the code. One of my personal solution is using `yarn test`. During the development, you can edit the file from `../__test__/tracer_debug.ts` and run it with the following command: |
| 26 | +```bash |
| 27 | +yarn test -- tracer_debug.ts > testOutput.log |
| 28 | +``` |
| 29 | +Note that the flag `--silence=false` is set in order to see the output from `console.log`. In order to fully test the stepper, you can execute the following command. |
| 30 | +```bash |
| 31 | +yarn test -- tracer_full.ts |
| 32 | +``` |
| 33 | +## High-level implementation details |
| 34 | +Our stepper is a program that reduces a piece of code to its simplest form until it can no longer be reduced. While the idea sounds abstract, the step-by-step reduction of code allows us to add explanations, further aiding student learning when they want to scrutinize how the code is evaluated. The method is formally defined as beta reduction. |
| 35 | + |
| 36 | +### Expression stepper |
| 37 | +In order to implement the program with such functionalities, we have to first parse the program string into certain structure that we can approach to, so-called Abstract Syntax Tree (AST). For instance, string `2 * 3` can be converted to AST as `BinExp[* Lit[2] Lit[3]]`, where `BinExp` is a binary expression node consisting of two literals `2` and `3` combined with operator `*`. Note that our programming language consists of various node types, it's difficult to implement a parser from scratch to cover all cases. Luckily, the library `acorn` helps us generate the AST from a piece of string which is very convenient. |
| 38 | + |
| 39 | +Here comes the fun part. How are we supposed to evaluate `BinExp[* Lit[2] Lit[3]]`? Since we cannot do anything further, we just simply **contract** them to `Lit[6]` and we are done. However, consider the AST `BinExp[* Lit[2] BinExp[+ Lit[3] Lit[4]]]` generated from `2 * (3 + 4)`, if we contract this expression directly, we will get `2 * {OBJECT}` which is not computable since one of the operands is not a number. Hence, we have to contract `BinExp[+ Lit[3] Lit[4]]]` first before contracting the outer `BinExp`. Note that our stepper only contracts one node for each step. This is our main constraint that we use during the implementation. |
| 40 | + |
| 41 | + |
| 42 | + |
| 43 | +To implement this, we should have two methods `oneStep` and `contract` to perform this reduction. In addition, we also have another two methods `oneStepPossible` and `isContractible` to safeguard our AST before actually proceeding with our reduction procedure. |
| 44 | + |
| 45 | +- `isContractible: StepperBaseNode => boolean` returns true if all of its children cannot be proceed further (`oneStepPossible` is false). |
| 46 | +- `oneStepPossible: StepperBaseNode => boolean` returns true if the AST can be proceed further. |
| 47 | +- `oneStep: StepperBaseNode => StepperBaseNode` either perform `contract` if the AST is contractible or performing `oneStep` on child nodes. |
| 48 | +- `contract: StepperBaseNode => StepperBaseNode` contracts the AST. |
| 49 | + |
| 50 | +These methods can be applied for all types of AST nodes, such as expressions, statements, and program. The details about `StepperBaseNode` are discussed in [this](#augmenting-functionalities) section. |
| 51 | + |
| 52 | +### Statements |
| 53 | +Each program consists of one or more statements. Consider the simplest case where every statement is an expression statement. |
| 54 | +```ts |
| 55 | +1 + 1; |
| 56 | +2 + 3; |
| 57 | +4 + 5; |
| 58 | +``` |
| 59 | +Initially, our stepper must reduce the first two statements while keeping the second statement. For each expression statement, the simplest form where the statement contains literal value is called _value statement_ (e.g. `2;`). |
| 60 | +```ts |
| 61 | +1 + 1; 2 + 3; 4 + 5; |
| 62 | +2; 2 + 3; 4 + 5; |
| 63 | +2; 5; 4 + 5; // First two statements are value statements |
| 64 | +5; 4 + 5; // first value statement is ignored |
| 65 | +5; 9; // Now, the first two statements are value statements |
| 66 | +9; // second statement is kept |
| 67 | +``` |
| 68 | +Not all statements, however, are value-inducing. For instance, constant declaration statement `const x = 1;` and function declarations do not produce value. Here, we assume that non-value inducing statements will be reduced to `undefined;` (an expression statement containing literal value `undefinedNode`). The fix is really simple; we just ignore it during the search for the first two statements. |
| 69 | +```ts |
| 70 | +const x = 1; 1 + 1; const y = 3; 1 + 5; |
| 71 | +1 + 1; const y = 3; 1 + 5; |
| 72 | +2; const y = 3; 1 + 5; // we got our first value statement |
| 73 | +2; 1 + 5; // ignore the non-value inducing |
| 74 | +2; 6; |
| 75 | +6; // second value statement is kept. |
| 76 | +``` |
| 77 | +If there is no value inducing statement, we simply output `undefined';` |
| 78 | +```ts |
| 79 | +const x = 1; |
| 80 | +undefined; // after substituting x |
| 81 | +``` |
| 82 | +Note that the `oneStep` method determines which child nodes will be contracted. You can check the implementation of `oneStep` in `nodes/Program.ts` for more details. |
| 83 | + |
| 84 | + |
| 85 | +### Constant declaration |
| 86 | +To be updated |
| 87 | + |
| 88 | +To make sure that the substituted variable has not been declared in the scope, we can simply check whether the variable name is in the list generated by `scanAllDeclarationNames()`. |
| 89 | +```ts |
| 90 | +scanAllDeclarationNames(): string[] { |
| 91 | + return this.body |
| 92 | + .filter(ast => ast.type === 'VariableDeclaration' || ast.type === 'FunctionDeclaration') |
| 93 | + .flatMap((ast: StepperVariableDeclaration | StepperFunctionDeclaration) => { |
| 94 | + if (ast.type === 'VariableDeclaration') { |
| 95 | + return ast.declarations.map(ast => ast.id.name) |
| 96 | + } else { |
| 97 | + // Function Declaration |
| 98 | + return [(ast as StepperFunctionDeclaration).id.name] |
| 99 | + } |
| 100 | + }) |
| 101 | + } |
| 102 | +``` |
| 103 | +### Functions |
| 104 | +To be updated |
| 105 | +### Alpha renaming |
| 106 | +To be updated |
| 107 | +## Augmenting functionalities |
| 108 | +Since our stepper takes AST (of any types) as an input and recursively navigate along the tree to find the next node to contract. There are many functionalities that we have to perform on each AST (such as contracting the AST as discussed in the previous section). Intuitively, we should add these functionalities as methods for each of the AST classes. Since the AST obtaining from the library is an `ESTree` interface, we have to implement our own concrete classes inherited from the ESTree AST Nodes. We also have to create our own convertor to convert the former ESTree into our own Stepper AST: |
| 109 | + |
| 110 | +```typescript |
| 111 | +// interface.ts |
| 112 | +export interface StepperBaseNode { |
| 113 | + type: string |
| 114 | + isContractible(): boolean |
| 115 | + isOneStepPossible(): boolean |
| 116 | + contract(): StepperBaseNode |
| 117 | + oneStep(): StepperBaseNode |
| 118 | + substitute(id: StepperPattern, value: StepperExpression): StepperBaseNode |
| 119 | + freeNames(): string[] |
| 120 | + allNames(): string[] |
| 121 | + rename(before: string, after: string): StepperBaseNode |
| 122 | +} |
| 123 | +``` |
| 124 | +Conversion from `es.BaseNode` (AST interface from `ESTree`) to `StepperBaseNode` is handled by function `convert` in `generator.ts`. |
| 125 | + |
| 126 | +```typescript |
| 127 | +// generator.ts |
| 128 | +const nodeConverters: { [Key: string]: (node: any) => StepperBaseNode } = { |
| 129 | + Literal: (node: es.SimpleLiteral) => StepperLiteral.create(node), |
| 130 | + UnaryExpression: (node: es.UnaryExpression) => StepperUnaryExpression.create(node), |
| 131 | + BinaryExpression: (node: es.BinaryExpression) => StepperBinaryExpression.create(node), |
| 132 | + // ... (omitted) |
| 133 | +} |
| 134 | + |
| 135 | +export function convert(node: es.BaseNode): StepperBaseNode { |
| 136 | + const converter = nodeConverters[node.type as keyof typeof nodeConverters] |
| 137 | + return converter ? converter(node as any) : undefinedNode |
| 138 | + // undefinedNode is a global variable with type StepperLiteral |
| 139 | +} |
| 140 | +``` |
| 141 | + |
| 142 | +### Entry point |
| 143 | +The starting point of our stepper is at `steppers.ts` with the function `getSteps`. This function is responsible for triggering reduction until it cannot be proceed. The result from our evaluation is then stored in array `steps`. Here is the shorten version of `getSteps`. |
| 144 | + |
| 145 | +```typescript |
| 146 | +export function getSteps(node: StepperBaseNode) { |
| 147 | + const steps = [] |
| 148 | + function evaluate(node) { |
| 149 | + const isOneStepPossible = node.isOneStepPossible() |
| 150 | + if (isOneStepPossible) { |
| 151 | + const oldNode = node |
| 152 | + const newNode = node.oneStep() // entry point |
| 153 | + // read global state redex and add them to steps array |
| 154 | + return evaluate(newNode) |
| 155 | + } else { |
| 156 | + return node |
| 157 | + } |
| 158 | + } |
| 159 | + evaluate(node) |
| 160 | + return steps |
| 161 | +} |
| 162 | + |
| 163 | +``` |
| 164 | + |
| 165 | +In order to keep track of the node that is reduced (i.e., the node highlighted in orange and green in the current SICP stepper), we use the global state redex to track all nodes that should be highlighted. This state is then updated dynamically during the contraction process. |
| 166 | + |
| 167 | +```typescript |
| 168 | +export let redex: { preRedex: StepperBaseNode[]; postRedex: StepperBaseNode[] } = { |
| 169 | + preRedex: [], |
| 170 | + postRedex: [] |
| 171 | +} |
| 172 | +// How to use global state |
| 173 | +redex.preRedex = [node] |
| 174 | +const ret = someSortOfReduction(node) |
| 175 | +redex.postRedex = [ret] |
| 176 | +``` |
| 177 | +### Generating explanations |
| 178 | +Explanations are generated based on `preRedex`. We use the same approach with the convert function. We have several dispatchers corresponding to each of the AST types. The implementation of `explainer` is in `generator.ts`. |
| 179 | +### Some important decisions |
| 180 | +There are some design decisions that diverge from the original Source 1 and 2. Here are some changes we have made. |
| 181 | +#### Builtin math functions |
| 182 | +Calling math function with non number arguments is prohibited in stepper. |
| 183 | +```ts |
| 184 | +// Test Incorrect type of argument for math function |
| 185 | +math_sin(true); // error! |
| 186 | +``` |
0 commit comments