mdast-util-to-markdown: expected output with nested lists

[triaeiou]

Hi (I seem to be asking a lot of questions lately),

The following

import {fromHtml as hastFromHtml} from 'hast-util-from-html';
import {toMarkdown as mdastToMarkdown} from 'mdast-util-to-markdown';
import {toMdast as hastToMdast} from 'hast-util-to-mdast';

const html = '<ul><li>one</li><li>two</li></ul>';

const hast = hastFromHtml(html);
const mdast = hastToMdast(hast);
const md = mdastToMarkdown(mdast);
console.log(md);

results in (i.e. no blank rows between list items)

*   one
*   two

Is the following the expected output from const html = '<ul><li>one</li><li>two<ul><li>1</li><li>2</li></ul></li></ul>' ; (i.e. a blank row between root level list items)?

*   one

*   two

    *   nested one
    *   nested two

Thanks

[wooorm]

Hey. This is vague. If you want to improve the way lists are made loose or tight, which might be possible, a proposal on how it would work would be more useful?

[wooorm]

but for completeness I think it should newline - however it seems unrelated to tight/loose lists.

This seems like a separate discussion, which we can have, but we don‘t need to ;)
In short: mdast is an AST, so it’s going to loose info. It can’t roundtrip completely, and that’s fine.
That blank line is one such “meaningless“ thing, which it doesn’t roundtrip. And I think that’s fine.

Attached is a print out

P.S., I like using Gists for attachments like this!

---

301 looks weird? How is mdast-util-to-markdown turning an unordered list into an ordered one?
302: Edge case, that can be solved with https://github.com/syntax-tree/mdast-util-to-markdown#optionsbulletorderedother. I like always using one marker, and forcing a break with an empty comment tho.
303: More clear.
304: Is mdast-util-to-markdown removing the EOL? 🤔
305/306/307: I think the output from mdast-util-to-markdown is an improvement.
308: Again an unordered list turning into an ordered one?
...rest I often don’t understand, or IMO better, too?

Well, there is a difference in the DOM as well as in how it is rendered without any CSS but that is fine.

Right. Not to semantics, but a different tree, that you can select with CSS (or JS I guess).

but for completeness

Yeah, I’m looking for a good solution that works for everyone, outside of your specific use case! A real improvement, rather than just a change.

There is a blank line prior to the next item

If I understand what you are proposing correctly, that would not work (well) on minified HTML. I think it is very important to work on minified HTML too.

---

As an alternative, what if we detect that phrasing content only, is used directly inside a <li>? Similar to your recent patch, use hast-util-phrasing to check that (and some extra handling to ignore comments, the doctype)?

[triaeiou]

301 looks weird? How is mdast-util-to-markdown turning an unordered list into an ordered one?
308: Again an unordered list turning into an ordered one?

Sorry that's due to my unclear log print out (I regexed out the commonmark samples into a dictionary which i looped through and printed the commonmark markdown and the mdast-to-markdown output for each - however as mentioned I sometimes changed the HTML, without changing the commonmark markdown, to make the markdown print out more clearer in list separations) in 301 and 308 I changed <ul> to <ol> to see that the list separation was where it should be.

304: Is mdast-util-to-markdown removing the EOL? 🤔

That's a soft break I believe so it doesn't have any <br> in the HTML and so disappears in HTML ⇒ markdown.

305/306/307: I think the output from mdast-util-to-markdown is an improvement.

I think those commonmark samples are to demonstrate syntax latitude, not "standard" and mdast-util-to-markdown obviously outputs "standard"

There is a blank line prior to the next item

If I understand what you are proposing correctly, that would not work (well) on minified HTML. I think it is very important to work on minified HTML too.

I was discussing the commonmark spec notes (regarding markdown syntax). To translate that to HTML I was suggesting that <ul><li>Some text<br></li></ul>, <ul><li><p>Some text</p></li></ul> and <ul><li><ul><li>Nested text</li></ul><p>Non-nested text</p></li></ul> would result in loose lists (the outer list in case 3, inner is tight). That would work the same on any format HTML I believe?

As an alternative, what if we detect that phrasing content only, is used directly inside a <li>? Similar to your recent patch, use hast-util-phrasing to check that (and some extra handling to ignore comments, the doctype)?

I think that wouldn't meet these differences?

- # A header
  with some text
- and next item

outputs <ul><li><h1>A header</h1>with some text</li><li>and next item</li></ul> (newlines removed) in the commonmark dingus whereas

- # A header
  with some text

- and next item

outputs <ul><li><h1>A header</h1><p>with some text</p></li><li><p>and next item</p></li></ul>

Granted, I think these are minor details. It would be "neat" to be able to run the commonmark samples in both directions but the question is how useful that is for HTML ⇒ markdown as there are so many more variants of HTML that would produce similar looking output.

[wooorm]

Hmm, I don‘t really understand exactly what you are proposing here now. But perhaps a PR to hast-util-to-mdast will help clear that confusion out. And then we have a lot of tests to see if it breaks or improves things.

For the <li>Some text<br></li></ul> example, according to HTML, a <br> should not be used like that: it should always be preceded and followed by text. That’s why it can’t occur in markdown. And why they are filtered out by us. So I don’t think we should worry or care about them!

I think that wouldn't meet these differences?

Granted, I think these are minor details. It would be "neat" to be able to run the commonmark samples in both directions but the question is how useful that is for HTML ⇒ markdown as there are so many more variants of HTML that would produce similar looking output.

Correct. I don’t think that is very important. I don’t think we can roundtrip all CommonMark output examples to their input anyway? The goal here I believe is to create good and readable markdown that resembles the intent of the HTML.

[triaeiou]

Ok, I've thought about it some more and made a "demo" PR with minimal changes (for better code clarity one should probably rename one variable but in this branch I kept the edit to as few lines as possible, besides npm test fails as it compares to remark output which I guess has the same logic).

The current logic is: a list is loose if at least one list item contains more than one child. This interprets <ul><li>Outer 1<ul><li>Inner 1</li><li>Inner 2</li></ul></li><li>Outer 2</li></ul> outer list as loose even though it is the output from a tight list (had it been the output of a loose outer list Outer 1 would have been wrapped in a <p>.

My suggested logic, when disregarding any <br>, is *a list is loose if at least one list item contains a <p> (which is the output effect of a markdown loose list). Granted this logic misses lists that were loose in Markdown but contained no paragraphs, e.g.

- # Outer 1

  # Second block
- # Outer 2

The spec says A list is loose [snip] any of its constituent list items directly contain two block-level elements with a blank line between them but two block level elements with a blank line becomes the same HTML as without.

[wooorm]

Response in the PR: syntax-tree/hast-util-to-mdast#75 (comment).

[triaeiou]

They're just working as expected here AFAIK.

How do you mean? Just to avoid confusion, the dingus reference was just to illustrate what the reverse does, the discussion is about what markdown to output when coming from HTML/mdast.

[JounQin]

See the source codes and rendered result on GitHub at https://github.com/orgs/syntax-tree/discussions/77#discussioncomment-3368384

[triaeiou]

See the source codes and rendered result on GitHub at https://github.com/orgs/syntax-tree/discussions/77#discussioncomment-3368384

Yes, but the issue is going the other direction, i.e. what markdown to output from <ul><li>one</li><li>two<ul><li>subone</li><li>subtwo</li></ul></li></ul>

What "seems" to be the reverse behaviour of commonmark and micromark:

- one
- two

    - subone
    - subtwo

or the current implementation of toMarkdown():

- one

- two

    - subone
    - subtwo