Lightning CSS doesn't escape content: "\..."

[ghost]

We have @charset "UTF-8"; in our CSS which Lightning CSS is skipping when bunding/minifying the styles.
Without it, it's possible to encounter issues with displaying unicode like in #164.
There are other ways to insert the character encoding like HTTP headers and the HTML but this is not possible for people in some use cases.

Is it possible to respect the @charset rule?

I've also prepared a Playground link demonstrating the issue.

[ghost]

At least from my understanding, this conversion of the Unicode is due to the missing @charset but I may be mistaken. It would be great if we could respect the Unicode.

[devongovett]

UTF-8 is the default character set for CSS, so I don't think this is necessary?

[ghost]

UTF-8 is the default character set for CSS, so I don't think this is necessary?

I might be misunderstanding the problem then but isn't this causing the change from content:"\f67f" to content: "";?

[devongovett]

Escaping isn't necessary since the output is utf8.

[ghost]

Something must be going on because the content doesn't always render correctly:


And I've only managed to trace it to this difference in the CSS. If I minify the CSS with SWC, it escapes the content and I can't reproduce the rendering issue. But that product is not mature enough in my opinion.

[neoGeneva]

This is something I've encountered too. I'm currently unsure what causes Chrome to suddenly think that the CSS file isn't UTF-8, but I have a hunch that it's something to do with devtools.

There's a discussion over in the Font Awesome repo about a similar issue: FortAwesome/Font-Awesome#17644 The consensus there seems that @charset "UTF-8"; doesn't always solve the issue, so it would be good if Lighning CSS left characters in their encoded form or had an option to toggle it.

[ghost]

This is something I've encountered too. I'm currently unsure what causes Chrome to suddenly think that the CSS file isn't UTF-8, but I have a hunch that it's something to do with devtools.

There's a discussion over in the Font Awesome repo about a similar issue: FortAwesome/Font-Awesome#17644 The consensus there seems that @charset "UTF-8"; doesn't always solve the issue, so it would be good if Lighning CSS left characters in their encoded form or had an option to toggle it.

Our QA also managed to narrow it down to having DevTools open and refreshing the page.
The surest way of avoiding this that I've found is avoiding escaping content:"\f67f" to content: ""; but Lightning is not giving us an option to do that.
Hopefully, with this issue, they will consider it.

[rkumorek]

We've had the same issue recently with migrated sites that were still using font icons. Had to revert back to nanocss for the time being as clients reported issues with icons.

Same thing:
content:"\f67f" becomes content: "" (square bracket) which seems not to work on some browsers... for some clients... sometimes.

[devongovett]

See https://developer.mozilla.org/en-US/docs/Web/CSS/@charset for the rules of how a charset is chosen by browsers when loading a CSS file. The problem could be that you are setting a non-UTF8 charset as part of the Content-Type HTTP header, as that would override other methods such as @charset. You could also prepend a @charset rule to the file, but this is usually unnecessary because if no other value is overriding it, the default is UTF-8.

[domenkozar]

I've hacked around parcel bug to include @charset "UTF-8" at the beginning of the main.css, but it still sometimes just renders it in correctly. Opening the debugging tools triggers the bug almost always.

[devongovett]

Honestly this sounds like a chrome dev tools bug? Has anyone filed an issue there?

[woody-li]

Met the same issue.
Font icon content converted as unicode "box" char, and con't be resolved by dynamic font.
Is it posibile to provide a option for convert unicode or not?

[nclemeur]

I have also seen this problem, and I would also greatly appreciate if this behavior could be made optional...

[dbt-padberg]

I can confirm this issue. For me it's the same using vite==4.4.9 (depends on lightningcss^1.21.5) in combination with @mdi/font==7.2.96

[musosoft]

In the Lightning CSS documentation, it states:

Lightning CSS does not do this – the output CSS should always behave identically to the input. Keep this in mind when comparing file sizes between tools.

However, this doesn't seem to be true for projects using font icons. Lightning CSS converts all Unicode escapes into direct Unicode characters. Browsers can't render these.

To address this, I suggest you to implement a feature that will compare characters against a list of most common used UTF-8 characters and encode only those that match.

As this issue is still open for over a year now, I'm moving back to PostCSS. Hopefully, it could be resolved soon as this is an incredible project that suits all CSS needs nowadays.

[Fedik]

Would be good to have an option to keep the content value unchanged.
Currently

let { code } = transform({
  code: Buffer.from(`
    .foo:before {
      content: "\\31";
    }
    .bar:before {
      content: "\\e4c2";
    }
  `),
});

Produce:

.foo:before {
  content: "1";
}
.bar:before {
  content: "";
}

But I expect that it keep the original value

[monochromer]

The problem could be that you are setting a non-UTF8 charset as part of the Content-Type HTTP header

Some cloud providers do not provide the ability to modify HTTP headers.

[amirsarfarW]

Would be good to have an option to keep the content value unchanged. Currently

let { code } = transform({
  code: Buffer.from(`
    .foo:before {
      content: "\\31";
    }
    .bar:before {
      content: "\\e4c2";
    }
  `),
});

Produce:

.foo:before {
  content: "1";
}
.bar:before {
  content: "";
}

But I expect that it keep the original value

I have the same issue

Is this gonna be fixed or not?

[liulinboyi]

To solve the issue I have a try to use lightningcss's visitor to transform value of content to Unicode

vite.config.ts

import { defineConfig } from "vite";
import vue from "@vitejs/plugin-vue";
import { browserslistToTargets } from "lightningcss";

function replaceWithUnicodeEscape(str: any) {
  return [...str]
    .map((char) => {
      // Check if the character is a non-ASCII character
      if (char.codePointAt(0) > 0x7f) {
        // Get the Unicode code point and convert it to hexadecimal
        const codePoint = char.codePointAt(0).toString(16);
        // Add a backslash
        return `\\${codePoint}`;
      } else {
        // If it is an ASCII character, return the original character
        return char;
      }
    })
    .join("");
}

export default defineConfig({
  plugins: [vue()],
  css: {
    transformer: "lightningcss",
    lightningcss: {
      targets: browserslistToTargets(["> 0.25%", "not dead"]),
      visitor: {
        Declaration(decl: any) {
          // Check if it is a "custom" property with the value name "content"
          if (decl.property === "custom" && decl.value.name === "content") {
            decl.value.value = decl.value.value.map((x: any) => {
              if (x.value.value) {
                let unicodeContent = replaceWithUnicodeEscape(x.value.value);
                return {
                  ...x,
                  value: {
                    ...x.value,
                    value: unicodeContent,
                  },
                };
              } else {
                return x;
              }
            });
            return decl;
          }
          // If it is not a "content" property or does not require special handling, return the original declaration
          return decl;
        },
      },
    },
  },
  build: {
    cssMinify: false,
    cssCodeSplit: true,
    rollupOptions: {},
  },
});

but i found after transform, the output code is error, like follow

the code before transform:

.ant-select-multiple .ant-select-selector:after {
  content: "\a0";
  width: 0;
  margin: 2px 0;
  line-height: 24px;
  display: inline-block;
}

the code after transform:

.ant-select-multiple .ant-select-selector:after {
  content: "\\a0";
  width: 0;
  margin: 2px 0;
  line-height: 24px;
  display: inline-block;
}

I debugged the source code for lightningcss and found out that the reason was rust-cssparser's way of handling backslashes.
If "\\" is found in the value of content, it is replaced with "\\\\", This approach needs to be more detailed judgment on whether the backslash and the following hexadecimal characters are valid unicode.

At the end, I submitted a PR to fix this. servo/rust-cssparser#398