Skip to content

Commit

Permalink
add layerNorm GB national standard operator's explanation
Browse files Browse the repository at this point in the history
  • Loading branch information
DoorKickers committed Oct 17, 2024
1 parent 25901cd commit d8a1c54
Show file tree
Hide file tree
Showing 3 changed files with 7 additions and 1 deletion.
1 change: 1 addition & 0 deletions diopi_test/python/configs/diopi_configs.py
Original file line number Diff line number Diff line change
Expand Up @@ -7986,6 +7986,7 @@
),
),

#This test config is for the national standard GB operator version of diopiLayerNorm, which is different from the original interface definition.
'layer_norm': dict(
name=["layer_norm"],
dtype=[np.float32, np.float64, np.float16],
Expand Down
2 changes: 2 additions & 0 deletions diopi_test/python/conformance/diopi_functions.py
Original file line number Diff line number Diff line change
Expand Up @@ -5270,6 +5270,8 @@ def layer_normGB(input, normalized_shape, weight=None, bias=None, eps=1e-05):
bias = None if bias is None else bias

out = raw_like(input)

# Note that, this is the national standard GB operator version, which is different from the diopiLayerNorm interface definition, normalized_shape has changed to begin_norm_axis.
func = check_function("diopiLayerNormGB")
ret = func(
input.context(),
Expand Down
5 changes: 4 additions & 1 deletion proto/include/diopi/functions.h
Original file line number Diff line number Diff line change
Expand Up @@ -3488,7 +3488,8 @@ DIOPI_API diopiError_t diopiNormBackward(diopiContextHandle_t ctx, diopiTensorHa

/**
* @brief Applies Layer Normalization over a mini-batch of inputs.
* type=[float32, float64, float16].
* Note that, this is the national standard GB operator version, which is different from the diopiLayerNorm interface definition, normalized_shape has changed
* to begin_norm_axis. type=[float32, float64, float16].
* @param[in] ctx Context environment.
* @param[in] save_mean Mean tensor,the mean value for each feature channel of the input tensor. type=[float32, float64, float16].
* @param[in] save_invstd Backup of inverse standard deviation computed during training. type=[float32, float64, float16].
Expand Down Expand Up @@ -3933,6 +3934,8 @@ DIOPI_API diopiError_t diopiLayerNormBackward(diopiContextHandle_t ctx, diopiTen

/**
* @brief Compute the backward pass for diopiLayerNormGB(). Computes gradients for input, weight, and bias.
* Note that, this is the national standard GB operator version, which is different from the diopiLayerNormBackward interface definition, normalized_shape has
* changed to begin_norm_axis
* @param[in] ctx Context environment.
* @param[in] grad_output the grad tensor of output. type=[float32, float64, float16].
* @param[in] grad_bias the grad of bias. type=[float32, float64, float16].
Expand Down

0 comments on commit d8a1c54

Please sign in to comment.