From 6aa80f2a41f798042c33fc375e9f3634d434b9e6 Mon Sep 17 00:00:00 2001 From: marty1885 Date: Mon, 25 Nov 2019 19:43:52 +0800 Subject: [PATCH 1/3] reorder toctree entries --- docs/source/index.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/index.rst b/docs/source/index.rst index 235d698..132b311 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -42,13 +42,13 @@ Be aware that Numenta holds the rights to HTM related patents. And only allows f Tensor PythonBindings UsingWithClingROOT - Contribution GUI .. toctree:: :caption: DEVELOPER ZONE :maxdepth: 2 + Contribution Backends DeveloperNotes OpenCLBackend From c9118a146cc4cab9d3208e5bba5272502f5580db Mon Sep 17 00:00:00 2001 From: marty1885 Date: Mon, 25 Nov 2019 19:46:42 +0800 Subject: [PATCH 2/3] update Python binding document --- docs/source/PythonBindings.md | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/docs/source/PythonBindings.md b/docs/source/PythonBindings.md index 7eae4cb..bc33597 100644 --- a/docs/source/PythonBindings.md +++ b/docs/source/PythonBindings.md @@ -28,7 +28,7 @@ et = ROOT.et Then you can call Etaler functions from Python. ```Python -s = et.Shape(); +s = et.Shape() [s.push_back(v) for v in [2, 2]] #HACK: ROOT doesn't support initalizer lists. t = et.ones(s) print(t) @@ -40,4 +40,11 @@ print(t) ``` ## PyEtaler -The offical Python binding - [PyEtaler](https://guthub.com/etaler/pyetaler) in currently work in progress. But we recomment using ROOT to bind from Python before PyEtaler leaves WIP. +The offical Python binding - [PyEtaler](https://guthub.com/etaler/pyetaler) in currently work in progress. We recomment using ROOT to bind from Python before PyEtaler leaves WIP. + +``` +>>> from etaler import et +>>> et.ones([2, 2]) +{{ 1, 1}, + { 1, 1}} +``` \ No newline at end of file From 73875b97099b732d2dcf8b002b37692a8dd21240 Mon Sep 17 00:00:00 2001 From: marty1885 Date: Mon, 25 Nov 2019 21:36:25 +0800 Subject: [PATCH 3/3] update Intdoruction --- docs/source/Introduction.md | 197 +++++++++++++++++++++++++++++++++--- docs/source/Tensor.md | 2 +- 2 files changed, 185 insertions(+), 14 deletions(-) diff --git a/docs/source/Introduction.md b/docs/source/Introduction.md index 5658982..a6e1863 100644 --- a/docs/source/Introduction.md +++ b/docs/source/Introduction.md @@ -1,44 +1,215 @@ +With a lack of writing ablilty. The introduction will be a clone of PyTorch's introduction document. This is also a good chance to see how well Etaler is doing at it's tensor operators. + # A quick introduction to Etaler -Include Etaler's headers. +At Etaler's core are Tensors. They are generalized matrix in more then 2 dimentions; or called N-Dimentional Arrays in some places. We'll see how they are used in-depth later. Now, let's look what we could do with tensors ```C++ +// Etaler.hpp packs most of the core headers together #include +#include +#include using namespace et; + +#include +using namespace std; ``` -Declare a HTM layer. +## Creating Tensors + +Tensors can be created from pointers holding the data and an appropriate shape. ```C++ -#include -SpatialPooler sp(/*input_shape=*/{256}, /*output_shape=*/{64}); +int d[] = {1, 2, 3, 4, 5, 6, 7, 8}; +Tensor v = Tensor({8}, d); +cout << v << endl; + +// Create a matrix +Tensor m = Tensor({2, 4}, d); +cout << m << endl; + +// Create a 2x2x2 tensor +Tensor t = Tensor({2, 2, 2}, d); +cout << t << endl; ``` -Encode values +Out + +``` +{ 1, 2, 3, 4, 5, 6, 7, 8} +{{ 1, 2, 3, 4}, + { 5, 6, 7, 8}} +{{{ 1, 2}, + { 3, 4}}, + + {{ 5, 6}, + { 7, 8}}} +``` + +Just like a vector is a list of scalars, a matrix is a list of vectors. A 3D Tensor is a list of matrices. Think it like so: indexing into a 3D Tensor gives you a matrix, indexing into a matrix gives you a vector and indexing. into a vector gives you a scalar, + +Just for clearifcation. When I say "Tensor" I ment a `et::Tensor` object. ```C++ -#include -Tensor x = encoders::scalar(/*value=*/0.3, +// Index into v wii give you a scalar +cout << v[{0}] << endl; + +// Scalars are special as you can convert them into native C++ types +cout << v[{0}].item() << endl; + +// Indexing a matrix gives you a vector +cout << m[{0}] << endl; + +// And indexing into a 3d Tensor to get a matrix +cout << t[{0}] << endl; +``` + +Out + +``` +{ 1} +1 +{ 1, 2, 3, 4} +{{ 1, 2}, + { 3, 4}} +``` + +You can also create tensors of other types. The default, as you can see, is whatever the pointer is point to. To create floating-point Tensors, just point to an floating point array then call `Tensor()`. + +You can also create a tensor of zeros with an supplied shape using `zeros()` + +```C++ +Tensor x = zeros({3, 4, 5}, DType::Float); +cout << x << endl; +``` + +Out + +``` +{{{ 0, 0, 0, 0, 0}, + { 0, 0, 0, 0, 0}, + { 0, 0, 0, 0, 0}, + { 0, 0, 0, 0, 0}}, + + {{ 0, 0, 0, 0, 0}, + { 0, 0, 0, 0, 0}, + { 0, 0, 0, 0, 0}, + { 0, 0, 0, 0, 0}}, + + {{ 0, 0, 0, 0, 0}, + { 0, 0, 0, 0, 0}, + { 0, 0, 0, 0, 0}, + { 0, 0, 0, 0, 0}}} +``` + +## Operation with Tensors + +You can operate on tensors in the ways you would expect. + +```C++ +Tensor x = ones({3}); +Tensor y = constant({3}, 7); +Tensor z = x + y; +cout << z << endl; +``` + +Out + +``` +{ 8, 8, 8} +``` + +One helpful operation that we will make use of later is concatenation. + +```C++ +// By default, the cat/concat/concatenate function works on the 1st axis +Tensor x_1 = zeros({2, 5}); +Tensor y_1 = zeros({3, 5}); +Tensor z_1 = cat({x_1, y_1}); +cout << z_1 << endl; + +// Concatenate columns: +x_2 = zeros({2, 3}); +y_2 = zeros({2, 5}); +z_2 = cat({x_2, y_2}, 1); +cout << z_2 << endl; +``` + +Out + +``` +{{ 0, 0, 0, 0, 0}, + { 0, 0, 0, 0, 0}, + { 0, 0, 0, 0, 0}, + { 0, 0, 0, 0, 0}, + { 0, 0, 0, 0, 0}} +{{ 0, 0, 0, 0, 0, 0, 0, 0}, + { 0, 0, 0, 0, 0, 0, 0, 0}} +``` + +## Reshaping Tensors +Use the `reshape` method to reshape a tensor. Unlike PyTorch using `view()` to reshapce tensots. view in Etaler works like the one in [xtensor](https://github.com/xtensor-stack/xtensor); it performs indexing. + + +```C++ +Tensor x = zeros({2, 3, 4}); +cout << x << endl; +cout << x.reshape({2, 12}) << endl; // Reshape to 2 rows, 12 columns +``` + +Out + +``` +{{{ 0, 0, 0, 0}, + { 0, 0, 0, 0}, + { 0, 0, 0, 0}}, + + {{ 0, 0, 0, 0}, + { 0, 0, 0, 0}, + { 0, 0, 0, 0}}} +{{ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, + { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}} +``` + +## HTM Algorithms + +HTM algorithms, like what the name indicates. Implements HTM related algorithms. They are the sole perpose why Etaler exists. + +Without getting too deep. Typically the first thing we do in HTM is encode values into SDRs. SDRs are sparse binary tensors. i.e. Elements in the tensor are either 1 or 0 and most of them are 0s. + +```C++ +Tensor x = encoder::scalar(/*value=*/0.3, /*min_val=*/0.f, /*max_val=*/5.f); +cout << x << endl; ``` -Run (i.e. inference) the SpatialPooler. +Out -```c++ -Tensor y = sp.compute(x); ``` +{ 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0} +``` + +[Spatial Pooler](https://numenta.com/neuroscience-research/research-publications/papers/htm-spatial-pooler-neocortical-algorithm-for-online-sparse-distributed-coding/) is a very commonly used layer in HTM. Trough an unsurpervised learning process, it can extract patterns from SDRs fed to it. -If you want the SpatialPooler to learn from the data. Call the `learn()` function. ```C++ +SpatialPooler sp(/*input_shape=*/{32}, /*output_shape=*/{64}); +// Run (i.e. inference) the SpatialPooler. +Tensor y = sp.compute(x); +cout << y.cast(DType::Bool) << endl; +// If you want the SpatialPooler to learn from the data. Call the `learn()` function. sp.learn(x, y); ``` -Save the parameters of the SpatialPooler: +Out +``` +{ 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0} +``` + +After traning, you might want like to save the Spatial Pooler's weights for use in the future. ```C++ -#include auto states = sp.states(); save(states, "sp.cereal"); diff --git a/docs/source/Tensor.md b/docs/source/Tensor.md index 2848250..3f6c04a 100644 --- a/docs/source/Tensor.md +++ b/docs/source/Tensor.md @@ -180,7 +180,7 @@ std::cout << (a+b).shape() << std::endl; Unlike PyTorch and NumPy, Etaler does not support the lagecy brodcasting rule. It doesn't allow certain tensors with different shapes but have the same amount of elements to brodcast together. -``` +```C++ //This would get you a warning in PyTorch and works in NumPy. //But not in Etaler a = ones({4})