From 51866998a6b7dda66e4be458d09b102b543d384d Mon Sep 17 00:00:00 2001 From: Avital Oliver Date: Fri, 26 Feb 2021 06:20:20 +0100 Subject: [PATCH 1/2] Release v0.3.1 --- CHANGELOG.md | 44 +++++++++++++++++++++++++++++++++----------- flax/version.py | 2 +- 2 files changed, 34 insertions(+), 12 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 80afd3357..f090fc815 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,34 +8,56 @@ vNext - - - - - Bug Fix `flax.core.apply` and `Module.apply`. Now it returns a tuple containing the output and a frozen empty collection when `mutable` is specified as an empty list. - - - - Add `sow` method to `Module` and `capture_intermediates` argument to `Module.apply`. - See [howto](https://flax.readthedocs.io/en/latest/howtos/extracting_intermediates.html) for usage patterns. - - - - - Some Module arguments can now be passed either as dataclass attribute or - as argument to `__call__`. See [design note](https://flax.readthedocs.io/en/latest/design_notes/arguments.html) - - `use_running_average` and `deterministic` no longer have a default. They should be passed explicitly - - `broadcast_dims` is now a attribute to `Dropout` instead of a `__call__` argument. - - - - - - - - Added OptimizedLSTM: ~33% faster than the original LSTM when using <=1024 units - - Bug Fix `Scope.variable` mutability check, before a variable could only be initialized - if the 'params' collection was mutable. - - Linen `Module` instances are now Frozen after `setup` has been called. + - + - + - + - + - + - + - + - + - + +0.3.1 +------ + +Many improvements to Linen! Notably, there's a clean API for extracting intermediates from modules +defined using `@nn.compact`, a more ergonomic API for using Batch Norm and Dropout in modules +defined using `setup`, support for `MultiOptimizer` with Linen, and multiple safety, performance +and error message improvements. + +Breaking Changes: + - Linen `Module` instances are now frozen after `setup` has been called. Previously mutations after setup could be dropped silently. Now the stateless requirement is enforced by raising a TypeError in `__setattr__` after `setup`. - Pytrees of dicts and lists are transformed into FrozenDict and tuples during attribute assignment. This avoids undetected submodules and inner state. + - Bug Fix `flax.core.apply` and `Module.apply`. Now it returns a tuple containing the output and a frozen empty + collection when `mutable` is specified as an empty list. + - `broadcast_dims` is now a attribute to `Dropout` instead of a `__call__` argument. + - `use_running_average` and `deterministic` no longer have a default. They should be passed explicitly + - Bug Fix `Scope.variable` mutability check, before a variable could only be initialized + if the 'params' collection was mutable. + +Other Improvements: + - Some Module arguments can now be passed either as dataclass attribute or + as argument to `__call__`. See [design note](https://flax.readthedocs.io/en/latest/design_notes/arguments.html) + - Add `sow` method to `Module` and `capture_intermediates` argument to `Module.apply`. + See [howto](https://flax.readthedocs.io/en/latest/howtos/extracting_intermediates.html) for usage patterns. - Add support for dict/FrozenDict when using `ModelParamTraversal` As a result `MultiOptimizer` can be used properly with linen modules. + - Added OptimizedLSTM: ~33% faster than the original LSTM when using <=1024 units - Added `is_mutable()` method to `Variable` and `is_mutable_collection()` to `flax.linen.Module`. v0.3 diff --git a/flax/version.py b/flax/version.py index fced632a6..7073d710d 100644 --- a/flax/version.py +++ b/flax/version.py @@ -12,5 +12,5 @@ # See the License for the specific language governing permissions and # limitations under the License. -__version__ = "0.3.0" +__version__ = "0.3.1" From 75334e082ca1080d9cead55cc165da36c7530d0c Mon Sep 17 00:00:00 2001 From: Avital Oliver Date: Fri, 26 Feb 2021 07:39:35 +0100 Subject: [PATCH 2/2] Add missing CHANGELOG entries from Git history --- CHANGELOG.md | 36 ++++++++++++++++++++++++++++-------- 1 file changed, 28 insertions(+), 8 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index f090fc815..a851c0ee2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -32,33 +32,53 @@ vNext 0.3.1 ------ -Many improvements to Linen! Notably, there's a clean API for extracting intermediates from modules +Many improvements to Linen, and the old `flax.nn` is officially reprecated! + +Notably, there's a clean API for extracting intermediates from modules defined using `@nn.compact`, a more ergonomic API for using Batch Norm and Dropout in modules defined using `setup`, support for `MultiOptimizer` with Linen, and multiple safety, performance and error message improvements. -Breaking Changes: +Possible breaking changes: + - Call setup lazily. See #938 for motivation and more details. - Linen `Module` instances are now frozen after `setup` has been called. Previously mutations after setup could be dropped silently. Now the stateless requirement is enforced by raising a TypeError in `__setattr__` after `setup`. - - Pytrees of dicts and lists are transformed into FrozenDict and tuples during attribute assignment. + - Pytrees of dicts and lists are transformed into FrozenDict and tuples during + attribute assignment. This avoids undetected submodules and inner state. - - Bug Fix `flax.core.apply` and `Module.apply`. Now it returns a tuple containing the output and a frozen empty + - Bug Fix `flax.core.apply` and `Module.apply`. Now it returns a tuple + containing the output and a frozen empty collection when `mutable` is specified as an empty list. - - `broadcast_dims` is now a attribute to `Dropout` instead of a `__call__` argument. - - `use_running_average` and `deterministic` no longer have a default. They should be passed explicitly - - Bug Fix `Scope.variable` mutability check, before a variable could only be initialized - if the 'params' collection was mutable. + - `broadcast_dims` is now a attribute to `Dropout` instead of a `__call__` + argument. + - `use_running_average` and `deterministic` no longer have a default. They + should be passed explicitly + - Bug Fix `Scope.variable` mutability check, before a variable could only be + initialized if the 'params' collection was mutable. Other Improvements: + - Re-introduced the `lm1b` language modeling example + - Recognizes batch free inputs in pooling layers. (for use with vmap) + - Add Adadelta optimizer + - Fully deprecate all "pre-Linen" `flax.nn` classes and methods. - Some Module arguments can now be passed either as dataclass attribute or as argument to `__call__`. See [design note](https://flax.readthedocs.io/en/latest/design_notes/arguments.html) - Add `sow` method to `Module` and `capture_intermediates` argument to `Module.apply`. See [howto](https://flax.readthedocs.io/en/latest/howtos/extracting_intermediates.html) for usage patterns. + - Support passing in modules directly as attributes to other modules, and + deal with them correctly both in top-level modules and in submodules. + - Don't require the `variable` argument to `Module.apply` to be a FrozenDict - Add support for dict/FrozenDict when using `ModelParamTraversal` As a result `MultiOptimizer` can be used properly with linen modules. - Added OptimizedLSTM: ~33% faster than the original LSTM when using <=1024 units + - Fix dtype handling for Adam and LAMB optimizers in 64bit mode. - Added `is_mutable()` method to `Variable` and `is_mutable_collection()` to `flax.linen.Module`. + - Add `axis_name` arg to `flax.linen.vmap` + - Enable broadcast in `flax.linen.scan` + - Fix behavior when inner module classes were defined in another module + - Add automatic giant array chunking in msgpack checkpoints. + - Log info message when a checkpoint is not found in the directory. v0.3 -----