Skip to content

test_softmax.py does not test argmax and latency implementations; latency fails #1443

@vasdommes

Description

@vasdommes

Quick summary

  1. test_softmax.py does not actually test non-default implementations (argmax and latency).
  2. If we fix that, tests for latency fail, reporting low accuracy.

Details

Steps to Reproduce

  1. Clone the hls4ml repository
  2. Checkout the main branch, commit hash: 6900257
  3. Run pytest test/pytest/test_softmax.py
  4. Go e.g. to test/pytest/test_softmax_test_softmax_9,6-input_shape6-18,8-io_parallel-False-latency-Vivado/parameters.h, line 29, see:
    static const nnet::softmax_implementation implementation = nnet::softmax_implementation::stable;

instead of expected nnet::softmax_implementation::latency.

  1. Now, if we replace the line
    cfg['LayerName']['softmax']['Strategy'] = strategy

    with
    cfg['LayerName']['softmax']['implementation'] = strategy

then all latency tests fail with messages like:

Accuracy hls4ml relative to keras: 0.1908
  1. Now, if we replace
    return np.clip(d, -32, 31)

    with
    return np.clip(d, -32, 0)

the the tests pass.

(Catapult backend fails on my laptop because it doesn't create .so file, but that's unrelated).

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions