Question on reduction with empty axes and scalar input

[philloooo]

In MLReduceOptions

• If empty, no dimensions are reduced, and the shape of the output tensor is the same as the shape of the input tensor.

Does that mean it should be an no-op (a.k.a identity)?

Is that true when you specifies empty axes for scalar input too?

If that's the case, our WPTs are wrong - it's clearly still reducing?

[inexorabletash]

Not identity - some of the reductions apply a function to each input value before reduction, or the reduced value after reduction. So there's no dimension reduction for a scalar, but the pre/post function still applies.

• L1, L2, Max, Min, Mean, Product, Sum - these should be identity for a scalar input.
• LogSum - same as log(n)
• LogSumExp - same as log(exp(n)) (which is identity except for precision loss?)
• SumSquare - same as pow(n, 2)

[philloooo]

So if you specify empty axes for none scalar input, do these pre/post function apply too?
builder.reduceSumSquare([2,2,2,2], {axes: []}) -> [4,4,4,4]?

Or you only apply these functions for empty axes and scalar input case?

[inexorabletash]

Pre/post functions are always applied.

So for reduceSumSquare, pre function is "square", reduction is "sum", post function is "identity".

reduceSumSquare([2,2,2,2]) → 16 - each input squared, all dimensions reduced
reduceSumSquare([2,2,2,2], {axes: []}) → [4,4,4,4] - each input squared, no dimensions reduced
reduceSumSquare([2]) → 4 - each input squared, all dimensions reduced
reduceSumSquare([2], {axes: []}) → [4] - each input squared, no dimensions reduced
reduceSumSquare(2) → 4 - each input squared, all dimensions reduced (but scalar so that's a no-op)
reduceSumSquare(2, {axes: []}) → 4 - each input squared, no dimensions reduced

For reduceLogSumExp, pre function is "exp", reduction is "sum", post function is "log"

reduceLogSumExp([2,2,2,2]) → 3.386... - each input exp'd, all dimensions reduced, sum log'd
reduceLogSumExp([2,2,2,2], {axes: []}) → [2,2,2,2] - each input exp'd, no dimensions reduced, each value log'd
reduceLogSumExp([2]) → 2 - each input exp'd, all dimensions reduced, sum log'd
reduceLogSumExp([2], {axes: []}) → [2] - each input exp'd, no dimensions reduced, each value log'd
reduceLogSumExp(2) → 2 - each input exp'd, all dimensions reduced (no-op), sum log'd
reduceLogSumExp(2, {axes: []}) → 2 - each input exp'd, no dimensions reduced, each value log'd

[inexorabletash]

... but all of this implies that the spec text could be improved. Reducing a dimension is different from applying the reduction operation to an input, so writing "no dimensions are reduced" is unclear. And behavior for scalars should be called out, since those don't have a dimension to reduce.

[fdwr]

So there's no dimension reduction for a scalar, but the pre/post function still applies.

👍

Pre/post functions are always applied.

So for reduceSumSquare, pre function is "square", reduction is "sum", post function is "identity".

reduceSumSquare([2,2,2,2]) → 16 - each input squared, all dimensions reduced reduceSumSquare([2,2,2,2], {axes: []}) → [4,4,4,4] - each input squared, no dimensions reduced reduceSumSquare([2]) → 4 - each input squared, all dimensions reduced reduceSumSquare([2], {axes: []}) → [4] - each input squared, no dimensions reduced reduceSumSquare(2) → 4 - each input squared, all dimensions reduced (but scalar so that's a no-op) reduceSumSquare(2, {axes: []}) → 4 - each input squared, no dimensions reduced

For reduceLogSumExp, pre function is "exp", reduction is "sum", post function is "log"

reduceLogSumExp([2,2,2,2]) → 3.386... - each input exp'd, all dimensions reduced, sum log'd reduceLogSumExp([2,2,2,2], {axes: []}) → [2,2,2,2] - each input exp'd, no dimensions reduced, each value log'd reduceLogSumExp([2]) → 2 - each input exp'd, all dimensions reduced, sum log'd reduceLogSumExp([2], {axes: []}) → [2] - each input exp'd, no dimensions reduced, each value log'd reduceLogSumExp(2) → 2 - each input exp'd, all dimensions reduced (no-op), sum log'd reduceLogSumExp(2, {axes: []}) → 2 - each input exp'd, no dimensions reduced, each value log'd

Great examples.