💈 CSS tokenisation inconsistencies

[luxonauta]

Hey 👋🏻 @huozhi, thanks for the work with this library! 💜

I'm updating the Dracula Theme website and trying it.

I identified some inconsistencies in the CSS highlighting that affect tokenisation accuracy. Specifically, CSS properties and hexadecimal values are not being categorised correctly according to the available TokenTypes.

For the following CSS snippet:

body {
  background: #000;
}

The tokenisation I would expect:

• body → identifier;
• { → sign;
• background → property;
• : → sign;
• #000 → string or number;
• ; → sign;
• } → sign.

Current tokenisation:

• body → identifier;
• { → sign;
• background → identifier → (should be property);
• : → sign;
• # → sign → (should be part of string or number);
• 000 → number → (should be part of string or number);
• ; → sign;
• } → sign.

If this is expected and correct, would the next step be to change the colours based on the language? 🤔

This part is crucial to us, because on the new page dedicated to the Dracula Theme and Alucard specifications, we have sections like these to exemplify what is expected:

/* Pink - White */
body {
  /* Cyan - Pink - Purple */
  background: #000;
}

/* Green - White */
#dracula {
  display: none;
  visibility: hidden;
  height: calc(var(--deathDate) - var(--birthDate));
  /* Cyan - Pink - Yellow */
  font-family: "Transylvania";
  /* Cyan - Pink - Purple */
  opacity: 0;
}

This should also be accurate on the website to avoid generating unnecessary noise. 😅

[huozhi]

Thanks for the feedback, I'll look into the design of the custom token type for specific language!

[luxonauta]

Hey 👋🏻 @huozhi, small update: It's live here.

My temporary solution for this specific case was the following:

&.language-css {
  .sh__line {
    >.sh__token--identifier:first-child {
      --sh-identifier: var(--pink);
    }

    .sh__token--keyword+.sh__token--sign {
      --sh-keyword: var(--pink);
      --sh-sign: var(--pink);
    }

    .sh__token--keyword+.sh__token--sign+.sh__token--sign+.sh__token--sign+.sh__token--identifier+.sh__token--sign {
      --sh-sign: var(--pink);
    }

    &:has(> .sh__token--sign:first-child + .sh__token--identifier) {
      >.sh__token--sign:first-child {
        --sh-sign: var(--green);
      }

      >.sh__token--identifier {
        --sh-identifier: var(--green);
      }
    }

    &:has(> .sh__token--space:first-child) {

      >.sh__token--identifier:nth-child(2),
      >.sh__token--identifier:nth-child(2)+.sh__token--sign,
      >.sh__token--identifier:nth-child(2)+.sh__token--sign+.sh__token--identifier {
        --sh-identifier: var(--cyan);
        --sh-sign: var(--cyan);
      }

      >.sh__token--identifier:nth-child(2)+.sh__token--sign:has(+ .sh__token--space) {
        --sh-sign: var(--pink);
      }
    }

    &:has(.sh__token--space + .sh__token--sign + .sh__token--class) {
      .sh__token--sign {
        &:not(:nth-last-of-type(2)) {
          --sh-sign: var(--purple);
        }
      }
    }

    &:not(:has(> .sh__token--space:first-child)):not( :has(> .sh__token--sign:first-child)) {
      >.sh__token--identifier:first-child {
        --sh-identifier: var(--pink);
      }
    }
  }
}

However, this approach is not very scalable. Let me know if I can help with anything else or if this isn't something planned for the library in the short term! 🖖🏻

[huozhi]

Not an easy one, probably need to redesign a bit the ast structure a bit for css if we want to always highligh the rule name

[luxonauta]

Haha, I thought it would be, but I can help! 💜 I can also reach out on our Discord – there are plenty of syntax nerds better than me there! 😊