Skip to content

Commit

Permalink
wgsl: Convert quantizeToF16 to used hfround (#3118)
Browse files Browse the repository at this point in the history
Instead of passing the input through a F16Array, use the library
provided function hfround. hfround is a fast look up table based
rounding function for f16.

Benchmarking locally this provides a ~20% improvement to fma interval
calculations, which are particularly sensitive to quantization
cost. Overall I was seeing more on the order of ~10% improvement.
  • Loading branch information
zoddicus authored Oct 31, 2023
1 parent ccee5a9 commit fc58db8
Showing 1 changed file with 2 additions and 5 deletions.
7 changes: 2 additions & 5 deletions src/webgpu/util/math.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ import { assert } from '../../common/util/util.js';
import {
Float16Array,
getFloat16,
hfround,
setFloat16,
} from '../../external/petamoriken/float16/float16.js';

Expand Down Expand Up @@ -2021,13 +2022,9 @@ export function quantizeToF32(num: number): number {
return quantizeToF32Data[0];
}

/** Statically allocate working data, so it doesn't need per-call creation */
const quantizeToF16Data = new Float16Array(new ArrayBuffer(2));

/** @returns the closest 16-bit floating point value to the input */
export function quantizeToF16(num: number): number {
quantizeToF16Data[0] = num;
return quantizeToF16Data[0];
return hfround(num);
}

/** Statically allocate working data, so it doesn't need per-call creation */
Expand Down

0 comments on commit fc58db8

Please sign in to comment.