Issue in numbers with period after update

[seloumi]

Brief outline of the issue

After some update (bidi,latex,xetex,...) I'm facing problem in numbers with . or - which is not happening previously, numbering direction is correct with maghrib form 0,1,2,...,9 but with mashriq form ٠,١,٢,٣,٤,٥,٦,٧,٨,٩ direction of numbering do not change with command (\setlatin - \setnonlatin)

I guess the problem come from this part (bidi) code do not work for mashriq form

bidi/bidi.dtx

Lines 2169 to 2178 in 3b57786

	\newcount\bidi@digits
	\XeTeXinterchartokenstate=\@ne
	\newXeTeXintercharclass\bidi@digits@charclass
	\newXeTeXintercharclass\bidi@sepmark@charclass
	\XeTeXcharclass `\. \bidi@sepmark@charclass
	\XeTeXcharclass `\- \bidi@sepmark@charclass
	\bidi@digits=`\0 \loop \XeTeXcharclass \bidi@digits \bidi@digits@charclass \ifnum\bidi@digits<`\9 \advance\bidi@digits \@ne \repeat
	\bidi@digits=`\۰ \loop \XeTeXcharclass \bidi@digits \bidi@digits@charclass \ifnum\bidi@digits<`\۹ \advance\bidi@digits \@ne \repeat
	\XeTeXinterchartoks \bidi@sepmark@charclass \bidi@digits@charclass = {\DigitsDotDashInterCharToks}
	\newcommand*{\DigitsDotDashInterCharToks}{\if@nonlatin\char"200F \fi}

Minimal example showing the issue

% !TEX TS-program = XeLaTeX
% !TEX encoding = UTF-8 Unicode
\documentclass[12pt]{article}
\usepackage{amsmath}
\usepackage{fontspec}
\usepackage{bidi}
\setmainfont[Script=Arabic]{Amiri}
\parindent=0pt

\begin{document}

\setRTL
\setnonlatin

\section{text}
\subsection{text text}
\subsection{text text}

1.23 % change direction with \setnonlatin - \setlatin 

١.٢٣ % do not change direction with \setnonlatin - \setlatin

\end{document}

If we remove the loop for mashriq form the result is fine

\documentclass[12pt]{article}
\usepackage{amsmath}
\usepackage{fontspec}
\usepackage{bidi}
\setmainfont[Script=Arabic]{Amiri}
\parindent=0pt

\makeatletter
\newcount\bidi@digits 
 \XeTeXinterchartokenstate=\@ne 
 \newXeTeXintercharclass\bidi@digits@charclass 
 \newXeTeXintercharclass\bidi@sepmark@charclass 
 \XeTeXcharclass `\. \bidi@sepmark@charclass 
 \XeTeXcharclass `\- \bidi@sepmark@charclass 
 \bidi@digits=`\0 \loop \XeTeXcharclass \bidi@digits \bidi@digits@charclass \ifnum\bidi@digits<`\9 \advance\bidi@digits \@ne \repeat
 \XeTeXcharclass  `\۰ \bidi@digits@charclass
 \XeTeXcharclass  `\١ \bidi@digits@charclass
 \XeTeXcharclass  `\٢ \bidi@digits@charclass
 \XeTeXcharclass  `\٣ \bidi@digits@charclass 
 \XeTeXcharclass  `\٤ \bidi@digits@charclass
 \XeTeXcharclass  `\٥ \bidi@digits@charclass
 \XeTeXcharclass  `\٦ \bidi@digits@charclass
 \XeTeXcharclass  `\٧ \bidi@digits@charclass
 \XeTeXcharclass  `\٨ \bidi@digits@charclass
 \XeTeXcharclass  `\٩ \bidi@digits@charclass     
 \XeTeXinterchartoks \bidi@sepmark@charclass  \bidi@digits@charclass = {\DigitsDotDashInterCharToks} 
\makeatother

\begin{document}

\setRTL
\setnonlatin

\section{text}
\subsection{text text}
\subsection{text text}

1.23 % change direction with \setnonlatin - \setlatin 

١.٢٣ % change direction with \setnonlatin - \setlatin

\end{document}

Log and PDF files

test.log
test.pdf

[davidcarlisle]

@seloumi probably this was after I re synced from Vafa's latest version so that this repository matches ctan again. I can make technical TeX fixes to keep things working like

a825300

which just made the patch match the updated version of \begin in the last latex release.

But I can't actually read these scripts so I would need more than just a @ ping and two ?? to pinpoint the issue.

Vafa is in this group but may not be available at present. You might be better asking on stackexchange, if a user familiar with the bidi coding can see what the issue is.

I may look later but this is hard for me as I would not necessarily notice if the characters were in the wrong order.

If a fix is suggested I can apply it and arrange ctan uploads etc.

[seloumi]

What puzzles me is the loop to add \XeTeXcharclass looks like it don't work

\bidi@digits=`\۰ \loop \XeTeXcharclass \bidi@digits \bidi@digits@charclass 
\ifnum\bidi@digits<`\۹ \advance\bidi@digits \@ne \repeat

Even though it was working before !
Without \loop it work fine

\XeTeXcharclass  `\۰ \bidi@digits@charclass
 \XeTeXcharclass  `\١ \bidi@digits@charclass
 \XeTeXcharclass  `\٢ \bidi@digits@charclass
 \XeTeXcharclass  `\٣ \bidi@digits@charclass 
 \XeTeXcharclass  `\٤ \bidi@digits@charclass
 \XeTeXcharclass  `\٥ \bidi@digits@charclass
 \XeTeXcharclass  `\٦ \bidi@digits@charclass
 \XeTeXcharclass  `\٧ \bidi@digits@charclass
 \XeTeXcharclass  `\٨ \bidi@digits@charclass
 \XeTeXcharclass  `\٩ \bidi@digits@charclass

[davidcarlisle]

Ah that is probably something I can trace (but I'm in the middle of preparing a latex patch level 2 release, so not this evening:-) Thanks for the pointer.

…

On Sun, 3 Nov 2019 at 19:41, Salim Bou ***@***.***> wrote: What puzzles me is the loop to add \XeTeXcharclass looks like they don't work ***@***.***=`\۰ \loop \XeTeXcharclass ***@***.*** ***@***.***@charclass ***@***.***<`\۹ ***@***.*** ***@***.*** \repeat Even though it was working before ! Without \loop it work fine \XeTeXcharclass `\۰ ***@***.***@charclass \XeTeXcharclass `\١ ***@***.***@charclass \XeTeXcharclass `\٢ ***@***.***@charclass \XeTeXcharclass `\٣ ***@***.***@charclass \XeTeXcharclass `\٤ ***@***.***@charclass \XeTeXcharclass `\٥ ***@***.***@charclass \XeTeXcharclass `\٦ ***@***.***@charclass \XeTeXcharclass `\٧ ***@***.***@charclass \XeTeXcharclass `\٨ ***@***.***@charclass \XeTeXcharclass `\٩ ***@***.***@charclass — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#11?email_source=notifications&email_token=AAJVYAQ7DB2OET7Y5BWIHJDQR4SM5A5CNFSM4JHIJQLKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEC527EY#issuecomment-549171091>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJVYAVSWVOY27XN4XCEVD3QR4SM5ANCNFSM4JHIJQLA> .

[davidcarlisle]

@seloumi OK I can say why the loop and your expanded version differ.

the loop sets up a contiguous series of unicode slots from

U+06f0 EXTENDED ARABIC-INDIC DIGIT ZERO

to

U+06f9 EXTENDED ARABIC-INDIC DIGIT NINE

Your expanded list uses the same 0:

U+06f0 EXTENDED ARABIC-INDIC DIGIT ZERO

but then 1 to 9 are ١ to ٩

which is

U+0661 ARABIC-INDIC DIGIT ONE

U+0669 ARABIC-INDIC DIGIT NINE

I have no view on what digits should get which character class. Both sets?

[seloumi]

Thanks, so I think another loop must be added for ARABIC-INDIC DIGIT

[davidcarlisle]

I assume so but I don't really know the implications of the different styles of arabic digits however I assume the directionality implications for bidi are the same so looping over 0-9 for both sets would seem to make sense. However I'd prefer if someone who can read Arabic confirmed this. Your posting ends with a question mark which doesn't inspire me with confidence to push this change....

…

On Mon, 4 Nov 2019 at 12:24, Salim Bou ***@***.***> wrote: Thanks, so I think another loop must be added for ARABIC-INDIC DIGIT ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#11?email_source=notifications&email_token=AAJVYAX7ZDU6SHGGCE4RJKTQSAIBJA5CNFSM4JHIJQLKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEC7B7JA#issuecomment-549330852>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJVYAWLBYYZ65D2XXBFWR3QSAIBJANCNFSM4JHIJQLA> .

[bykhov]

Since I have recently updated Latex distribution (probably since the change in polyglossia v1.46?), I have similar problems in Hebrew text with ordinary numbers when both numbers with period and section numbering reverted. Example follows.

\documentclass[12pt]{article}
\usepackage{polyglossia,bidi}
\setmainlanguage{hebrew}
\setotherlanguage{english}
\setmainfont{David}
\setmonofont{Courier New}
\begin{document}
 בדיקה 1.23
\begin{english}
Test 1.23
\end{english}
\section{א}
\subsection{א}
\subsection{א}
\end{document}

See also reutenauer/polyglossia#213