Global TeX Macro file

[FelixBenning]

Description

LaTeX writers typically learn to use macros. And this makes sense: you can replace formatting instructions with semantic macros. For example

\newcommand{\real}{\mathbb{R}}
\newcommand{\nat}{\mathbb{N}}
\newcommand{\int}{\mathbb{Z}}
\newcommand{\complex}{\mathbb{C}}

How LaTeX macros interact with quarto

Quarto consists of a bunch of technologies that result in a complicated interaction

1. Pandoc has a latex_macros extension. If it is enabled (it is by default in quarto) it substitutes \newcommand macros itself. This interaction only happens if \newcommand is placed outside the typical TeXfences $$ and pandoc is applied to the file. This is only the case if the file is converted by pandoc, e.g. when it is a .qmd file.
2. mathjax can also deal with \newcommands but the \newcomand have to be read before the macro obviously so one has to make sure that the execution order is correct. One can see the issues when this is used in a forum like math stackexchange: If you add a \newcommand in the text field and later remove it, it is "sticky" since mathjax already executed it.
3. KaTeX assumes every equation delimited by $$ as independent for speed and is therefore unable to deal with \newcommands defined in a different equation.
4. LaTeX does not like \newcommand used twice for the same macro and throws a compilererror in this case. A redefinition is usually achieved with \renewcommand. Another alternative is \providecommand which only defines the macro if it does not exist yet.

TL;DR: Ideally, pandoc processes the macros such that mathjax/KaTeX do not have to deal with them. If pandoc does not process the macros, then only mathjax can be used. For html the macros need to be included in every file, whearas for LaTeX they should only be included once.

Ways to include snippets in quarto

There are also multiple ways to include snippets in quarto. Let's assume we are working with a book type of project. Then there is a global _quarto.yml configuration file and multiple .qmd chapter files.

1. Includes of the form include-in-header, include-before-body in the _quarto.yml configuration file for the html format (similar for the pdf format). It is important to note that every such include has the note "Include contents of file, verbatim" so
   • it is not possible to combine this with another include mechanism
   • pandoc does not read the contents and therefore does not process \newcommand macros
2. Templates/Template partials use the template mechanism of pandoc to provide a scaffolding to place the pandoc converted files into. These templates are already in the target format (e.g. .tex or .html and are therefore not converted by pandoc. Pandoc will therefore not convert \newcommands in them. Furthermore it is only possible to replace existing templates or template partials and not possible to introduce new ones
3. Inside a .qmd file one can use Includes of the form

   {{< include _macros.qmd >}}

Neither of these options is really suited to what we want to do: The first two do not interact nicely with the latex_macros extension and in both cases we would have to mix the macros with other stuff. In the first case, we would have to mix the macros with the rest of the _quarto.yml configuration file (because we cannot include another file) and in the second we would have to mix it with the rest of the template because we cannot create a new blank template just for the macros.

This essentially only leaves the last option for the html target.

Best method (to my knowledge) at the moment

1. Create a _macros.qmd file. Only use \newcommand in it (no markdown), it should also be valid as a .tex file!
2. Create a before-title.tex template partial (this is a predefined partial that is empty by default) with the following content

   % before-title.tex
   \include{_macros.qmd} % the \include acts like copy & paste (filetype does not matter)

   this is why it is important that the _macros.qmd is also valid as a .tex file.
3. In every .qmd chapter file include at the beginning:

   ::: {.content-hidden}
   {{< include _macros.qmd >}}
   :::

   This means the macros will be included in every html page, while the .content-hidden property ensures it is removed from the TeX output. This ensures that \newcommand is not used multiple times for the same macro.

Pros & Cons

The strategy above

• ensures that pandoc processes the \newcommand macros because the macros are in a .qmd file
• allows us to use a clean, single purpose macros file for both TeX and html

However, it has the drawbacks:

• we need boilerplate for every .qmd chapter file
• we cannot use comments in the _macros.qmd file to explain macros because .tex comments would break .qmd syntax and therefore the html target whereas html comments would break .tex syntax and therefore the \include used to import the macros into the tex target.

Alternative

The user baptiste had another approach (#7518 (comment)) that may give you the ability to use comments:

1. Instead of a _macros.qmd file, use a _macros.tex file and change the \include in the before-title.tex

2. Create the following _macros.qmd file

   ```{r, echo=FALSE, results='asis'}
   if (knitr::is_html_output()) {
       system("latexpand macros.tex -o _macros.tex")
       macros <- readLines('_macros.tex')
       system("rm _macros.tex")
       writeLines(macros)
   }
   ```

   latexpand is bundled with TexLive (which is the most common tex distribution) so it is likely pre-installed.
   The following is a python version, but I cannot find a way to suppress the output when the target is not html, i.e. it appears there is no corresponding command to knitr::is_html_output() in python so you have to deal with WARNINGS about repeated \newcommands

    ```{python}
    #| echo: false
    #| output: asis
   
    from tex_comment_strip import strip_comments
    with open("macros.tex", "r") as f:
        result = strip_comments(f.read())
    print(result)
    ```

   the file tex_comment_strip.py should be filled with the contents of https://gist.github.com/dzhuang/dc34cdd7efa43e5ecc1dc981cc906c85 or similar. The script in question also requires https://github.com/dabeaz/ply. Alternatively one can of course also call latexpand from python.

Discussion

The boilerplate necessary to get global macros is seriously annoying. Every file requiring

    ::: {.content-hidden}
    {{< include _macros.qmd >}}
    :::

is not very elegant. Ideally, there would be a way to have a include-in-header that is not treated like raw text. Perhaps global macros should also be a first class citizen and get its own yaml argument.

References

• https://stackoverflow.com/questions/76398554/how-to-define-latex-macros-globally-for-a-quarto-book
• Defining custom macros is not possible with Katex #7518 (comment)
• Including LaTeX macros from a file into a quarto book #2845

[cderv]

Thanks a lot for this great summary with a lot to think about.

Ideally, there would be a way to have a include-in-header that is not treated like raw text. Perhaps global macros should also be a first class citizen and get its own yaml argument.

I do believe that this is Pandoc's current behavior. latex_macros extensions will only happen in the Markdown reader, and there is no global default option like include-in-header to do this.

So with Pandoc itself, I think the problem will be the same - the following content

\newcommand{\real}{\mathbb{R}}
\newcommand{\nat}{\mathbb{N}}
\newcommand{\int}{\mathbb{Z}}
\newcommand{\complex}{\mathbb{C}}

will need to be included in all source files rendered by Pandoc.

Quarto's include mechanism simplifies this.

Better support in Pandoc itself may be required to go further.

But that is just first thought.

Again, thank you.

[mcanouil]

For reference and information, for macros there is the .hidden class as documented in the markdown basics guide.

[FelixBenning]

Unfortunately the .hidden class actually makes its way into LaTeX and causes an error during compilation due to the already defined macros. Conversely the .content-hidden class seems to work (although with warnings)

[FelixBenning]

One may hope that the Code Insertion Extension may help reduce the boilerplate, but the inserted code appears to not be processed but instead inserted raw. So I could not get this to work to insert the \newcommand for html. Even though it is trivial to only activate this filter for html so if one could get this to be processed then this should work perfectly

[FelixBenning]

Another hope may be that you could use pre-render and post-render scripts to create the _macros.qmd file. If you could restrict these to html you could essentially make sure that _macros.qmd is empty whenever you render pdf to avoid the duplicate \usepackage. But you can only use these project wide and not target specific. Most likely this is not something you want to change because if the targets where rendered in parallel you would get race conditions if you wanted to rely on this

[mcanouil]

If you want for the include to be conditional, set it to be: https://quarto.org/docs/authoring/conditional.html

::: {.content-hidden when-format="pdf"}
{{< include _macros.qmd >}}
:::

or

::: {.content-visible when-format="html"}
{{< include _macros.qmd >}}
:::

[cderv]

@mcanouil AFAIK this does not help because this problem with Pandoc Macro feature logic is reading time.

Quarto include mechanism is output logic - the source will have all include, no matter conditional.

This has been discussed already in discussion about this Macro problem.

Doing pre-processing on the file allows to control what will be included in the .md source for Quarto to read before rendering.

@FelixBenning, this type of conditional document inclusion can be done using an engine (R, Python, or Julia) by producing Raw markdown conditionally to some information. This is the "Alternative" part you shared in OP.
Pre-render scripts are more global.

We are at the limit of Quarto and Pandoc here, because this is a Pandoc feature that does not play well with Quarto multi format logic. Ultimately, maybe this feature needs to be integrate as a Quarto feature directly which would allow to make it work differently, and only for some format

[mcanouil]

Thanks Christophe.

Does Quarto have environment variable allowing to know the formats? You can access the output files paths but I did not find any variable allowing to know if the project is rendering files to PDF, HTML, etc.

I think there was a discussion about this somewhere or I'm hallucinating this from my work on the following post-render script: https://github.com/mcanouil/quarto-workflows/blob/main/.github/workflows/assets/slides-to-pdf.sh

[cderv]

Does Quarto have environment variable allowing to know the formats?

Not yet, still an opened feature request.

• communicate format output to all engines  #2216

[FelixBenning]

I managed to create an R version of the script above that can use the conditional (I will update the code as soon as I can). This solves this part of the problem if R is installled (which I had to do). So to make this independent of the language and to avoid the boilerplate include in every file it would still be nice if there were some way to include the file in every .qmd and process it. Because the include-before-* yaml configuration is always raw at the moment. It would be nice if there was a non-raw option