-
Notifications
You must be signed in to change notification settings - Fork 98
Open
Description
Hey Fletcher, hope things are fine!
I found a weird token thing again today.
Given a document that has this string content:
```\n```
I.e. opening and closing code block triple backticks with 1 newline, only the closing backticks are tokenized as generic fenced code block lines. A printout of the tree with ranges and token types:
{0, 7}.blockCodeFenced
{0, 4}.lineFenceBacktick3
{0, 3}.codeFence
{3, 1}.textNl
{4, 3}.codeFenceLine
{4, 3}.codeFence
Once you add anything in the code block, even another line break, the lineFenceBacktick3 will be "downgraded" to the generic one:
```\n\n```
produces
{0, 8}.blockCodeFenced
{0, 4}.codeFenceLine
{0, 3}.codeFence
{3, 1}.textNl
{4, 1}.textNl
{5, 3}.codeFenceLine
{5, 3}.codeFence
I'm using this information for syntax highlighting of course, and I wonder how you deal with these inconsistencies in behavior in your own apps. Given previous questions about this were not received with the same level of 'OCD' that I show, I guess you don't mind this in practice somehow :)
Metadata
Metadata
Assignees
Labels
No labels