Skip to content

Commit f0fa224

Browse files
committed
Use TensorFlow 2.3 for remote tests
This causes a regression in two benchmark tests: - net4: Random network with RectifiedLinear neurons - net6: LMU with eager mode off Two other benchmarks are adjusted slightly to re-center them on the times currently achieved under TensorFlow 2.2. Here are the full results before this commit: test_benchmarks.py::test_performance[net0-False-64-True-1.0-1.15] Exec times (15): 1.092 (min), 1.146 (max), 1.122 (mean), 0.016 (std) test_benchmarks.py::test_performance[net1-False-64-True-2.25-2.55] Exec times (15): 2.468 (min), 2.507 (max), 2.488 (mean), 0.014 (std) test_benchmarks.py::test_performance[net2-True-64-True-0.6-0.9] Exec times (15): 0.793 (min), 0.842 (max), 0.812 (mean), 0.015 (std) test_benchmarks.py::test_performance[net3-True-64-True-0.95-1.15] Exec times (15): 1.034 (min), 1.082 (max), 1.057 (mean), 0.013 (std) test_benchmarks.py::test_performance[net4-False-None-True-0.5-0.7] Exec times (15): 0.647 (min), 0.656 (max), 0.649 (mean), 0.002 (std) test_benchmarks.py::test_performance[net5-True-100-True-1.25-1.45] Exec times (15): 1.345 (min), 1.399 (max), 1.367 (mean), 0.015 (std) test_benchmarks.py::test_performance[net6-True-100-False-1.05-1.25] Exec times (15): 1.180 (min), 1.197 (max), 1.186 (mean), 0.005 (std) Here are the full results after this commit: test_benchmarks.py::test_performance[net0-False-64-True-1.0-1.15] Exec times (15): 1.109 (min), 1.186 (max), 1.134 (mean), 0.020 (std) test_benchmarks.py::test_performance[net1-False-64-True-2.25-2.55] Exec times (15): 2.488 (min), 2.539 (max), 2.509 (mean), 0.015 (std) test_benchmarks.py::test_performance[net2-True-64-True-0.6-0.9] Exec times (15): 0.684 (min), 0.728 (max), 0.700 (mean), 0.013 (std) test_benchmarks.py::test_performance[net3-True-64-True-0.95-1.15] Exec times (15): 1.023 (min), 1.064 (max), 1.040 (mean), 0.011 (std) test_benchmarks.py::test_performance[net4-False-None-True-0.5-0.7] Exec times (15): 0.700 (min), 0.717 (max), 0.705 (mean), 0.004 (std) test_benchmarks.py::test_performance[net5-True-100-True-1.25-1.45] Exec times (15): 1.326 (min), 1.357 (max), 1.339 (mean), 0.009 (std) test_benchmarks.py::test_performance[net6-True-100-False-1.05-1.25] Exec times (15): 1.237 (min), 1.262 (max), 1.245 (mean), 0.006 (std)
1 parent 76afa44 commit f0fa224

3 files changed

Lines changed: 7 additions & 7 deletions

File tree

‎.nengobones.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -244,7 +244,7 @@ travis_yml:
244244
- script: remote-test
245245
env:
246246
NENGO_VERSION: git+https://github.com/nengo/nengo.git#egg=nengo[tests]
247-
TF_VERSION: tensorflow<2.3.0
247+
TF_VERSION: tensorflow==2.3.0
248248
- script: test-coverage
249249
env:
250250
NENGO_VERSION: git+https://github.com/nengo/nengo.git#egg=nengo[tests]

‎.travis.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ jobs:
4040
-
4141
env:
4242
NENGO_VERSION="git+https://github.com/nengo/nengo.git#egg=nengo[tests]"
43-
TF_VERSION="tensorflow<2.3.0"
43+
TF_VERSION="tensorflow==2.3.0"
4444
SCRIPT="remote-test"
4545
-
4646
env:

‎nengo_dl/tests/test_benchmarks.py‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -207,9 +207,9 @@ def test_lmu(Simulator, native_nengo, pytestconfig):
207207
64,
208208
True,
209209
1.0,
210-
1.15,
210+
1.2,
211211
),
212-
(benchmarks.cconv(128, 64, nengo.LIF()), False, 64, True, 2.25, 2.55),
212+
(benchmarks.cconv(128, 64, nengo.LIF()), False, 64, True, 2.3, 2.6),
213213
(
214214
benchmarks.integrator(128, 32, nengo.RectifiedLinear()),
215215
True,
@@ -231,11 +231,11 @@ def test_lmu(Simulator, native_nengo, pytestconfig):
231231
False,
232232
None,
233233
True,
234-
0.5,
235-
0.7,
234+
0.6,
235+
0.8,
236236
),
237237
(benchmarks.lmu(1000, 1, native_nengo=True), True, 100, True, 1.25, 1.45),
238-
(benchmarks.lmu(1000, 1, native_nengo=True), True, 100, False, 1.05, 1.25),
238+
(benchmarks.lmu(1000, 1, native_nengo=True), True, 100, False, 1.15, 1.35),
239239
],
240240
)
241241
def test_performance(net, train, minibatch_size, eager, min, max, capsys):

0 commit comments

Comments
 (0)