Skip to content

Tokenizing empty fenced code block doesn't apply codeFenceLine to opening fence line #255

@DivineDominion

Description

@DivineDominion

Hey Fletcher, hope things are fine!

I found a weird token thing again today.

Given a document that has this string content:

```\n```

I.e. opening and closing code block triple backticks with 1 newline, only the closing backticks are tokenized as generic fenced code block lines. A printout of the tree with ranges and token types:

  {0, 7}.blockCodeFenced
    {0, 4}.lineFenceBacktick3
      {0, 3}.codeFence
      {3, 1}.textNl
    {4, 3}.codeFenceLine
      {4, 3}.codeFence

Once you add anything in the code block, even another line break, the lineFenceBacktick3 will be "downgraded" to the generic one:

```\n\n```

produces

  {0, 8}.blockCodeFenced
    {0, 4}.codeFenceLine
      {0, 3}.codeFence
      {3, 1}.textNl
    {4, 1}.textNl
    {5, 3}.codeFenceLine
      {5, 3}.codeFence

I'm using this information for syntax highlighting of course, and I wonder how you deal with these inconsistencies in behavior in your own apps. Given previous questions about this were not received with the same level of 'OCD' that I show, I guess you don't mind this in practice somehow :)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions