Adapt micromark-extension-directive for fenced divs

[dmca-glasgow]

Hello again 👋

Further to my question adding attributes to images, I'm now trying to replicate markdown syntax from Pandoc for a lot of stuff, and so I'm attempting to write my own, or hack existing, parsing plugins. I have tried this a few times before actually, but been scared off due to not getting anywhere!

I looked all over for existing plugins compatible with Pandoc fenced-divs, and there have been a few attempts, but none are recommended for production use.

While hacking around with micromark-extension-directive I did figure out that if I comment this line, all of a sudden I am tanalisingly close to all I need, which is defaulting to a div when no name is given, with attributes:

Markdown:

::: {#hello}
test
:::

MDAST:

{
  type: 'containerDirective',
  name: ' ',
  attributes: { id: 'hello' },
  _fenceCount: 2,
  content: '<p>test</p>\n'
}

HTML:

<div id="hello">
<p>test</p>
</ >

So as you can see in the HTML here, the opening tag for the container directive seems to fall back to div, but the closing doesn't.

I have also tried feeding 'div' as the container name in factoryName, in the event of a space character, but so far computer says no:

/** @type {State} */
function start(code) {
  if (unicodeWhitespace(code)) {
    effects.enter(type)
    effects.consume(100) // d
    effects.consume(105) // i
    effects.consume(118) // v
    effects.exit(type)
    return ok(code)
  }

  if (
    code === codes.eof ||
    markdownLineEnding(code) ||
    unicodePunctuation(code)
    // || unicodeWhitespace(code)
  ) {
    return nok(code)
  }

  effects.enter(type)
  effects.consume(code)
  return name
}

Assertion: expected given code to equal expected code
      at assert (file:///Users/staff/Work/micromark-extension-directive/node_modules/devlop/lib/development.js:230:9)
      at ok (file:///Users/staff/Work/micromark-extension-directive/node_modules/devlop/lib/development.js:182:3)
      at Object.consume (file:///Users/staff/Work/micromark-extension-directive/node_modules/micromark/dev/lib/create-tokenizer.js:247:5)
      at start (file:///Users/staff/Work/micromark-extension-directive/dev/lib/factory-name.js:28:15)
      at sequenceOpen (file:///Users/staff/Work/micromark-extension-directive/dev/lib/directive-container.js:68:6)
      at go (file:///Users/staff/Work/micromark-extension-directive/node_modules/micromark/dev/lib/create-tokenizer.js:242:13)
      at main (file:///Users/staff/Work/micromark-extension-directive/node_modules/micromark/dev/lib/create-tokenizer.js:220:11)
      at Object.write (file:///Users/staff/Work/micromark-extension-directive/node_modules/micromark/dev/lib/create-tokenizer.js:143:5)
      at writeToChild (file:///Users/staff/Work/micromark-extension-directive/node_modules/micromark/dev/lib/initialize/document.js:287:15)
      at flowContinue (file:///Users/staff/Work/micromark-extension-directive/node_modules/micromark/dev/lib/initialize/document.js:260:7) {
    code: 'ERR_ASSERTION',
    actual: false,
    expected: true,
    generated: false,
    operator: 'ok'
  }

Can you please give me some pointers as to where I should be looking in the code to accomplish this? Or how I should be thinking when working with this code, because so far I'm a little stumped!

Thanks.

[dmca-glasgow]

I've just found I can fix the closing div tag by swapping this line for:

const newToken = this.sliceSerialize(token).trim() || 'div'
stack[stack.length - 1].name = newToken

Updates the MDAST:

{
  type: 'containerDirective',
  name: 'div',
  attributes: { id: 'hello' },
  _fenceCount: 2,
  content: '<p>test</p>\n'
}

and the HTML:

<div id="hello">
<p>test</p>
</div>

[wooorm]

I would recommend spending some more time with the existing directives and the existing code.
There are a lot of comments for functions, e.g., if you click consume you should see docs there.
consume works on the passed code. You cannot “consume” random characters that are not in the input.
And that’s what the error says: you are consuming something that isn’t there.

which is defaulting to a div when no name is given, with attributes:

This is not what you want, here.
Here, is something dealing with input.
What you want thus, is optionality, here.
If the input does not contain foo you cannot read foo here.

In a different place, you can handle that omission, and default to div: that would be in the to-html or to-mdast part of things.

I can fix the closing div tag by swapping this line for:

I do not understand, where is a closing div?

[dmca-glasgow]

Thanks for the reply @wooorm, I'll read the comments on functions and see if I can get my head round it :)

I do not understand, where is a closing div?

There wasn't a closing div and now there is one.

After commenting this line, the following markdown:

::: {#hello}
test
:::

Is now captured as a container directive (instead of just a paragraph containing the string), and produces:

{
  type: 'containerDirective',
  name: ' ',
  attributes: { id: 'hello' },
  _fenceCount: 2,
  content: '<p>test</p>\n'
}

<div id="hello">
<p>test</p>
</ >

Then defaulting to 'div' here for a name consisting of only a space using:

this.sliceSerialize(token).trim() || 'div'

Correctly parses and serializes the syntax of Pandoc's fenced divs:

{
  type: 'containerDirective',
  name: 'div',
  attributes: { id: 'hello' },
  _fenceCount: 2,
  content: '<p>test</p>\n'
}

<div id="hello">
<p>test</p>
</div>

This aligns with your advice above— don't do it in the syntax part, do it in the to-html or to-mdast part, but if you can suggest a better way to go about it I'm keen to learn!

All original tests pass. I'll write some more today to check nesting etc. works for container directives with no name.