Contributing a new Lua filter for Font styling and alignment

[nandac]

Dear Folks,

I would like to contribute a new Lua filter to the community that is available here: https://github.com/nandac/fonts-and-alignment

This filter currently only works for LaTeX documents, but I will be making it work for HTML and ePub in the future.

As I am a beginner in Lua and LaTeX and would like to

1. Get a review of the Lua code.
2. Get suggestions on refactoring and improvement
3. Get some comments on if I should be splitting this into two different filters etc.

I hope this would not be too much of an imposition on the community. The filter is not very large at present.

[tarleb]

Thanks for this, that's a nice little filter. Below are a few small suggestions.

• Whether an element is a Span or Div can be tested by checking elem.t (or elem.tag).
• The filter uses for loops combined with if statements to find the correct field in a table. A more idiomatic way would be to use table indexing. E.g.,

  local tag = elem.t
  local code_for_class = LATEX_CODES_FOR_TAGS[tag]

• It's not necessary to pass the tag to create_handler if the tag is taken from elem.t. The inner function could be named handler and then be used directly:

  local function handler (elem)
    ...
  end
  Span = handler
  Div = handler

Keeping everything in a single file seems like a good choice at this moment.

Hope this is helpful.

[bpj]

This is very similar to a filter which I posted to the Pandoc mailing list recently https://v.gd/GQL7mL. You might want to look at how I did things. In particular:

• It is probably a good idea to use \textsf and friends for spans whenever possible for spans, they being the LaTeX standard and set up to handle italic correction etc. automatically.

• You might want to include alternative classes corresponding the two-letter Plain TeX font switch commands bf, it, rm, sc, sf, sl, tt. They are less intrusive and faster to type, and should be clear enough for most people targeting LaTeX (and you can include a table in the documentation! :-) Something like this to keep it DRY:

local abbrev = {
  bold      = 'bf',
  italic    = 'it',
  serif     = 'rm',
  smallcaps = 'sc',
  sans      = 'sf',
  slanted   = 'sl',
  monospace = 'tt',
}

for _,style in pairs(LATEX_CODE_FOR_TAGS) do
  for long, short in pairs(abbrev) do
    style[short] = style[long]
  end
end

• You may want to include a shortcut bolditalic so that people won't have to type two classes for that combination, unless you include the short class names.

• You might want to include the commands from ulem which AFAIK Pandoc's LaTeX template loads anyway. I use the following abbreviations for them BTW:

  Command	Short
  \uline	u
  \uuline	uu
  \uwave	uw
  \sout	so
  \xout	xo
  \dashuline	da/dau
  \dotuline	do/dou

I would as you see create the RawInline and RawBlock objects only once during initialization and reuse them to save resources, unless @tarleb thinks that that may have adverse effects:

end_span = pandoc.RawInline('latex', '}')
local span = LATEX_CODES.Span
for class,code in pairs(span) do
  code = pandoc.RawInline(code)
  span[class] = { code, end_span }
end
local div = LATEX_CODES.Div
for class,code1 in pairs(div) do
  local code2 = code1:gsub('begin','end')
  div[class] = {
    pandoc.RawBlock('latex', code1),
    pandoc.RawBlock('latex', code2),
  }
end

local function handler (elem)
  local style = LATEX_CODES[elem.tag]
  -- Unpack the stuff we need
  local class   = elem.classes
  local content = elem.content
  -- Looping over the classes in the element is more efficient!
  -- And we loop backwards so the commands come in the same order as the classes!
  for i=#class,1,-1 do
    if style[class[i]] then
      -- The content is a pandoc.List object
      content:insert(1, style[class][1])
      content:insert(style[class][2])
    end
  end
  elem.content = content
  return elem
end

Span = handler
Div  = handler

I hope this helps!

[nandac]

@tarleb and @bpj thank you for your valuable comments.

@bpj I was using the code you posted on Pandoc discuss as a base as you were answering my question.

I value the opportunity to collaborate with both of you on this filter.

I will take up your suggestions and message you both once I am done.

I hope you won't mind giving me another review later.

[nandac]

@tarleb and @bpj.

I have made improvements to the filter according to your suggestions above.

1. I am now predominantly using table indexes to access values and reduced the number of for loops considerably. Thank you so much for pointing out the capabilities of the elem object.
2. Inline elements now use the \text* LaTeX commands instead of the long hand commands.
3. Abbreviated classes are now supported.
4. Fleshed out the README and created specimen files of the output using the filter.

What I have not done are the following:

1. The addition of underlining capabilities has not been added because the Pandoc LaTeX template only conditionally adds the ulem package.

If the two of you don't mind giving my code another review I would appreciate it.

My main concern now is the maintainability of the LATEX_CODE_FOR_TAGS table. If I was using a language I am familiar with I would have broken it up into three tables. In Lua, I find that I would have to loop through three tables to find the correct class which I want to avoid as merging tables is a little involved.

In addition, I am open to suggestions for a better name for the filter that is more descriptive.

Thanks in advance.

[tarleb]

Nice, looks good to me!

I agree that the LATEX_CODE_FOR_TAGS is a bit verbose, but it's not that bad. However, if you want to remove duplication, how about doing something like

local latex_font_sizes = {
  xxsmall = 'scriptsize',
  xsmall = 'footnotesize',
  -- ...
}

for k, v in pairs(latex_font_sizes) do
  LATEX_CODES_FOR_TAGS.Span[k] = {'\\'.. v .. ' ', '}'}
  LATEX_CODES_FOR_TAGS.Div[k] = {'\\begin{'.. v .. '}', '\\end{' .. v .. '}'}
end

[nandac]

Thanks, @tarleb for the code above, but considering the fact that the code is not the same for inlines and divs for LaTeX it might not be worth the work. I think I might end up with something even more unwieldy. So perhaps I should leave the code as it is right now.

I am now facing an issue where the expected output and the generated output do not match.

This only seems to occur when I specify more than one style for a piece of text for example:

::: {.sc .bf .it}
Some text
:::

The classes when accessed in the code do not seem to follow a set order as far as I can tell. Because the classes get applied in a different order each time the corresponding LaTeX code also gets applied in a different order, and as a result, the expected and actual do not match.

Is there a way to preserve the order of the classes and not have it change every time? I thought that because the .classes attribute is a list it would follow a set order.

I hope you will be able to enlighten me on why this is the case and what would be a good workaround.

[tarleb]

The reason for the non-determinism is that pairs does not guarantee a specific order when called on tables. Only ipairs ensures a certain order, but it only works with sequences.

The easiest would be to iterate over the elem.classes and then check if code_for_class[class] is non-nil.

[tarleb]

Aside: we do guarantee a certain order when using pairs on a pandoc.AttributeList element, thanks to implementing a custom __pairs metamethod. But that'd be overkill here.

[nandac]

Thanks, @tarleb the first method works but I need some explanation on how iterating over elem.classes with pairs preserves the order of the classes.

I think I am missing something here conceptually and would like to understand the mechanism being used here.

I have uploaded the new code to the repository.

[tarleb]

The elem.classes table is a sequence, so we can iterate over it with the deterministic ipairs instead of pairs. I'm actually surprised that using pairs appears to be working as well.

[nandac]

Ah @tarleb that makes much more sense now. I have changed pairs to ipairs and everything looks right.

What would be the next steps now?

[tarleb]

We still haven't decided how to deal with new filters in this repo.

You could announce the new filter on the pandoc-discuss mailing list, so others can start to use it.

[nandac]

@tarleb Before I announced the filter I wanted to make the change regarding separating the large style list into three and I am happy to say that I have succeeded in that endeavor.

I am now happy with the filter code and would like to request one more review of the code.

I have a question about the "magical" elem variable is this something Pandoc gives us access to whenever we attach a handler to an element in the AST. I could not find much documentation about it as far as I could tell.

[tarleb]

Sorry, I currently don't have time for another detailed review.