Skip to content

Commit

Permalink
highbd_quantize_intrin_sse2: quiet int sanitizer warnings
Browse files Browse the repository at this point in the history
add a missing cast in ^ operations; quiets warnings of the form:
implicit conversion from type 'int' of value -1 (32-bit, signed) to type
'unsigned int' changed the value to 4294967295 (32-bit, unsigned)

Bug: b/229626362
Change-Id: I56f74981050b2c9d00bad20e68f1b73ce7454729
  • Loading branch information
jzern committed Aug 17, 2022
1 parent d939886 commit b77b6b6
Showing 1 changed file with 3 additions and 2 deletions.
5 changes: 3 additions & 2 deletions vpx_dsp/x86/highbd_quantize_intrin_sse2.c
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,8 @@ void vpx_highbd_quantize_b_sse2(const tran_low_t *coeff_ptr, intptr_t count,
const int64_t tmp4 = ((tmp3 * quant_ptr[k != 0]) >> 16) + tmp3;
const uint32_t abs_qcoeff =
(uint32_t)((tmp4 * quant_shift_ptr[k != 0]) >> 16);
qcoeff_ptr[k] = (int)(abs_qcoeff ^ coeff_sign[j]) - coeff_sign[j];
qcoeff_ptr[k] =
(int)(abs_qcoeff ^ (uint32_t)coeff_sign[j]) - coeff_sign[j];
dqcoeff_ptr[k] = qcoeff_ptr[k] * dequant_ptr[k != 0];
if (abs_qcoeff) eob_i = iscan[k] > eob_i ? iscan[k] : eob_i;
}
Expand Down Expand Up @@ -143,7 +144,7 @@ void vpx_highbd_quantize_b_32x32_sse2(
const int64_t tmp2 = ((tmp1 * quant_ptr[rc != 0]) >> 16) + tmp1;
const uint32_t abs_qcoeff =
(uint32_t)((tmp2 * quant_shift_ptr[rc != 0]) >> 15);
qcoeff_ptr[rc] = (int)(abs_qcoeff ^ coeff_sign) - coeff_sign;
qcoeff_ptr[rc] = (int)(abs_qcoeff ^ (uint32_t)coeff_sign) - coeff_sign;
dqcoeff_ptr[rc] = qcoeff_ptr[rc] * dequant_ptr[rc != 0] / 2;
if (abs_qcoeff) eob = iscan[idx_arr[i]] > eob ? iscan[idx_arr[i]] : eob;
}
Expand Down

0 comments on commit b77b6b6

Please sign in to comment.