ctags for .tex files

[TryerGit]

As suggested here, I am breaking the issue into two issues. This issue is for ctags generation of .tex files.

The file on which I run ctags is latextest.tex:

latextest.tex:

\documentclass[11pt]{article}
\usepackage{amsmath}
\begin{document}
\begin{alignat}{2}
	\text{ }& asdfasdf              &&\quad\nonumber\\
	\text{ }&89999\times 45\leq 88	&&\quad \forall\label{eq:1}\\
		&abc	                &&\quad\exists\label{eq:okay}\\
		&\times x	        &&\quad\forall\nonumber\\
		&abcd                   &&\quad\nonumber
\end{alignat}
\bibliographystyle{apalike}
\bibliography{bibfile.bib}
\end{document}

On running the (universal) ctags as : ctags latextest.tex, I obtain the following tags file:

!_TAG_FILE_FORMAT	2	/extended format; --format=1 will not append ;" to lines/
!_TAG_FILE_SORTED	1	/0=unsorted, 1=sorted, 2=foldcase/
!_TAG_OUTPUT_MODE	u-ctags	/u-ctags or e-ctags/
!_TAG_PROGRAM_AUTHOR	Universal Ctags Team	//
!_TAG_PROGRAM_NAME	Universal Ctags	/Derived from Exuberant Ctags/
!_TAG_PROGRAM_URL	https://ctags.io/	/official site/
!_TAG_PROGRAM_VERSION	0.0.0	//
eq:1	latextest.tex	/^	\\text{ }&89999\\times 45\\leq 88	&&\\quad \\forall\\label{eq:1}\\\\$/;"	l
eq:okay	latextest.tex	/^		&abc	                &&\\quad\\exists\\label{eq:okay}\\\\$/;"	l

On running the (exuberant) ctags as : ctags latextest.tex, I obtain the following tags file:

!_TAG_FILE_FORMAT	2	/extended format; --format=1 will not append ;" to lines/
!_TAG_FILE_SORTED	1	/0=unsorted, 1=sorted, 2=foldcase/
!_TAG_PROGRAM_AUTHOR	Darren Hiebert	/dhiebert@users.sourceforge.net/
!_TAG_PROGRAM_NAME	Exuberant Ctags	//
!_TAG_PROGRAM_URL	http://ctags.sourceforge.net	/official site/
!_TAG_PROGRAM_VERSION	5.9~svn20110310	//
BIBLIOGRAPHY	latextest.tex	/^\\bibliography{bibfile.bib}$/;"	s
eq:1	latextest.tex	/^	\\text{ }&89999\\times 45\\leq 88	&&\\quad \\forall\\label{eq:1}\\\\$/;"	l
eq:okay	latextest.tex	/^		&abc	                &&\\quad\\exists\\label{eq:okay}\\\\$/;"	l

One can notice that in the exuberant ctags version, there is an extra tag generated called BIBLIOGRAPHY. Note that to run the exuberant ctags executable, I had the following in ~/.ctags file:

--langdef=latex
--langmap=latex:.tex
--regex-latex=/^\\tableofcontents/TABLE OF CONTENTS/s,toc/
--regex-latex=/^\\frontmatter/FRONTMATTER/s,frontmatter/
--regex-latex=/^\\mainmatter/MAINMATTER/s,mainmatter/
--regex-latex=/^\\backmatter/BACKMATTER/s,backmatter/
--regex-latex=/^\\bibliography\{/BIBLIOGRAPHY/s,bibliography/
--regex-latex=/^\\part[[:space:]]*(\[[^]]*\])?[[:space:]]*\{([^}]+)\}/PART \2/s,part/
--regex-latex=/^\\part[[:space:]]*\*[[:space:]]*\{([^}]+)\}/PART \1/s,part/
--regex-latex=/^\\chapter[[:space:]]*(\[[^]]*\])?[[:space:]]*\{([^}]+)\}/CHAP \2/s,chapter/
--regex-latex=/^\\chapter[[:space:]]*\*[[:space:]]*\{([^}]+)\}/CHAP \1/s,chapter/
--regex-latex=/^\\section[[:space:]]*(\[[^]]*\])?[[:space:]]*\{([^}]+)\}/\. \2/s,section/
--regex-latex=/^\\section[[:space:]]*\*[[:space:]]*\{([^}]+)\}/\. \1/s,section/
--regex-latex=/^\\subsection[[:space:]]*(\[[^]]*\])?[[:space:]]*\{([^}]+)\}/\.\. \2/s,subsection/
--regex-latex=/^\\subsection[[:space:]]*\*[[:space:]]*\{([^}]+)\}/\.\. \1/s,subsection/
--regex-latex=/^\\subsubsection[[:space:]]*(\[[^]]*\])?[[:space:]]*\{([^}]+)\}/\.\.\. \2/s,subsubsection/
--regex-latex=/^\\subsubsection[[:space:]]*\*[[:space:]]*\{([^}]+)\}/\.\.\. \1/s,subsubsection/
--regex-latex=/^\\includegraphics[[:space:]]*(\[[^]]*\])?[[:space:]]*(\[[^]]*\])?[[:space:]]*\{([^}]+)\}/\3/g,graphic+listing/
--regex-latex=/^\\lstinputlisting[[:space:]]*(\[[^]]*\])?[[:space:]]*(\[[^]]*\])?[[:space:]]*\{([^}]+)\}/\3/g,graphic+listing/
--regex-latex=/\\label[[:space:]]*\{([^}]+)\}/\1/l,label/
--regex-latex=/\\ref[[:space:]]*\{([^}]+)\}/\1/r,ref/
--regex-latex=/\\pageref[[:space:]]*\{([^}]+)\}/\1/p,pageref/

The above is obtained from tagbar github documentation. I notice that there is a line in the ~/.ctags file that instructs ctags to generate a BIBLIOGRAPHY label.

The tagbar page also states the following:

Note: As of 2012-05-05 the ctags development version contains an improved
LaTeX parser that works better than the example configuration presented here.
So if you are using a development build newer than that or a stable version
newer than 5.8 you should use the built-in support instead of this example.

Given the above, and my understanding that universal ctags is expected to be a superset of the features provided by exuberant ctags, I open this issue since exuberant ctags seems to do something that universal ctags does not (given my current understanding that I am running ctags with correct options and that its built-in support for parsing .tex files is exhaustive)

Could you please clarify on the above and also please guide me to documentation about how the .tex parser works in universal ctags from a user's point of view? In particular, I would like to know whether each line in the ~/.ctags file above for exuberant ctags is built-into universal ctags automatically under the default settings. Clearly, for example, the line for generating BIBLIOGRAPHY is not, for instance.

Thank you.

[masatake]

Please read this comment after updating your u-ctags as I wrote in #3518.

u-ctags mainly focus on definition.

\bibliography{bibfile.bib}

This line defines nothing. So u-ctags reports nothing.

\bibliography{bibfile.bib}

This line refers to bibliography, a macro.
This lines refers to bibfile.bib, an external file.

If you give more options, u-ctags can report the PARTS of referenced items.
Till the year 3000, u-ctags will be able to extract and report ALL referenced items.
However, in the year 2022, u-ctags can report the PARTS of referenced items.
The Tex parser can extract and report parts of external files.

--extras=+{reference} (or --extras=+r in short) let ctags report the reference items if the parser for the input language supports:

[yamato@dev64]~/var/ctags-github% cat /tmp/foo.tex 
\documentclass[11pt]{article}
\usepackage{amsmath}
\begin{document}
\begin{alignat}{2}
	\text{ }& asdfasdf              &&\quad\nonumber\\
	\text{ }&89999\times 45\leq 88	&&\quad \forall\label{eq:1}\\
		&abc	                &&\quad\exists\label{eq:okay}\\
		&\times x	        &&\quad\forall\nonumber\\
		&abcd                   &&\quad\nonumber
\end{alignat}
\bibliographystyle{apalike}
\bibliography{bibfile.bib}
\end{document}
[yamato@dev64]~/var/ctags-github% ./ctags --options=NONE --extras=+'{reference}' -o - /tmp/foo.tex 
ctags: Notice: No options will be read from files or environment
bibfile.bib	/tmp/foo.tex	/^\\bibliography{bibfile.bib}$/;"	i
eq:1	/tmp/foo.tex	/^	\\text{ }&89999\\times 45\\leq 88	&&\\quad \\forall\\label{eq:1}\\\\$/;"	l
eq:okay	/tmp/foo.tex	/^		&abc	                &&\\quad\\exists\\label{eq:okay}\\\\$/;"	l
[yamato@dev64]~/var/ctags-github% ./ctags --options=NONE --extras=+'{reference}' --fields='{roles}' -o - /tmp/foo.tex

--fields=+K{roles} makes the output more descriptive like:

./ctags --options=NONE --extras=+'{reference}' --fields=+K'{roles}' -o - /tmp/foo.tex
ctags: Notice: No options will be read from files or environment
bibfile.bib	/tmp/foo.tex	/^\\bibliography{bibfile.bib}$/;"	xinput	roles:bibliography
eq:1	/tmp/foo.tex	/^	\\text{ }&89999\\times 45\\leq 88	&&\\quad \\forall\\label{eq:1}\\\\$/;"	label	roles:def
eq:okay	/tmp/foo.tex	/^		&abc	                &&\\quad\\exists\\label{eq:okay}\\\\$/;"	label	roles:def

The newer ctags has man pages. man ctags explains the usage of --extras and --fields. The latest man pages are available on our website:
https://docs.ctags.io/en/latest/man/ctags.1.html#options .

You can remove many parts in your .ctags if you introduce newer u-ctags.

If you think ctags misses something, open an issue. I will investigate it (till the year 3000:-).

[masatake]

The biggest issue may be that vim, a client tool you use, doesn't understand roles:. In this aspect, I cannot help you. ctags is a tool just extracting tags.

[masatake]

• includegraphics
• lstinputlisting

These two should be tagged. They are external entities.
I updated #3330. It is related to #2428.