Question on when we should consider :is() browser support to be at 98%

[mkrause]

As per the blog post on native CSS nesting, Sass would consider emitting nesting rules using :is() when browser support for this feature is at least 98%. It links to the caniuse page as a reference for this statement, which is currently listing 95.91% of global marketshare with full support for :is().

However, as per the discussion here, any percentage reported by caniuse is capped at 97.98% due to the way it considers untracked browsers to (pessimistically) always have 0% support. This means that :is() will never reach 98% as per the default metric. If we change to "all tracked" instead, then caniuse reports 98+% support for :is().

So then, should we consider :is() to be at the required level of support now?

[nex3]

This is a tricky question. We really do want to be very conservative about when we unavoidably emit code that's known to be incompatible with older browsers: in almost all cases, Sass will only emit new CSS features if the author explicitly writes them, so our browser compatibility window is very long by default, and we don't want to break users who are relying on that (e.g. to support internal tools, to support markets with high retention of older devices, etc). On the other hand, there's no point in having a bar that's literally impossible to reach.

I think the most robust solution here would be to say that in order to demonstrate that it's sufficiently safe to use a given feature, someone needs to actually do the legwork and test that feature against enough of the remaining browsers not covered by caniuse to show that it does in practice have 98%+ penetration. Unfortunately, the caniuse data doesn't make it very clear what those browsers are or what the relative usage percentages of each version is, so this is substantially difficult in its own right. But I do think it's worth investigating.

It's also worth noting that although the "all tracked" support for :is() is 98.09%, the "tracked desktop" support is only 96.47%, which possibly should be enough to disqualify it on its own.

From a broader perspective: :is() isn't even 5 years old yet in its current form. We only started emitting hsl() in 2022, twenty years after it first appeared in a working draft for CSS Color 3 and 14 years after full support first landed in Firefox. This situation is definitely a little different, in that moving to :is() aligns us with CSS compatibility in other ways, but I'm still in no hurry to start supporting it.

[mkrause]

Hi @nex3, thanks for the response. I'm not really trying to rush it, but would be good to have some kind of objective target so it's possible to track "progress" towards sufficient support.

I think the most robust solution here would be to say that in order to demonstrate that it's sufficiently safe to use a given feature, someone needs to actually do the legwork and test that feature against enough of the remaining browsers not covered by caniuse to show that it does in practice have 98%+ penetration. Unfortunately, the caniuse data doesn't make it very clear what those browsers are or what the relative usage percentages of each version is, so this is substantially difficult in its own right. But I do think it's worth investigating.

Yeah, as far as I can tell the StatCounter data has no way of telling us what that "other" category is. Could even just be mainline browsers with some kind of user agent spoofing I'm not sure. Based on what I can tell from the caniuse discussion there also doesn't seem to be an alternative data source? No one suggested any alternative in any case.

How would you feel about using an estimate for the "other" category? Like the formula suggested in the original comment. And we could even split that in the desktop vs mobile categories and consider both to have to fit the minimum.

[nex3]

I see, so it's not just that no one is collecting data on what the 2.02% of untracked browsers support, no one's even collecting data on what they are. That's kind of shocking for a selection that represents a pretty dramatic portion of non-Chromium traffic.

I appreciate the logic of the suggested formula of "assume the unknown browsers have a similar compatibility profile to known browsers", but again without more specific evidence to the contrary I want to generally lean conservative—after all, they could be things like feature-phone browsers that are much less likely to support modern CSS. So here's a proposal for an updated browser support metric:

Browser support for a given CSS feature is considered "universal" when desktop and mobile browsers each (separately) have an estimated 98% global support for a feature, per caniuse. For this estimate, we consider half the caniuse/StatCounter untracked browsers to support no features, and the other half of them to support an equivalent distribution of features to the tracked browsers.

Given the current 2.22% untracked browser rate at time of writing, that would mean we'd multiply the tracked percentage by (100% - 2.22%/2) = 98.89% to find the estimated real percentage. For :is(), that gives us 96.47% * 98.89% = 95.4% on desktop and 98.88% * 98.89% = 97.78% on mobile.

---

Out of curiosity, I also looked into StatCounter's own data and found it to be oddly different from what caniuse lists. It has far more explicitly listed browsers—only 0.01% "Unknown" and 0.03% "Other", everything else is listed. It also has substantially different numbers for some browsers. For example, it lists "Android" as 0.55% market share, while caniuse lists "Android Browser" as 0.31%. I'm curious if you (or anyone) have any insight into where that discrepancy comes from.

We could potentially do a separate analysis using these numbers and manual testing for the browsers that aren't tracked by caniuse, but first I'd like to get a better understanding of how caniuse consumes these statistics in the first place.

[mkrause]

Out of curiosity, I also looked into StatCounter's own data and found it to be oddly different from what caniuse lists. It has far more explicitly listed browsers—only 0.01% "Unknown" and 0.03% "Other", everything else is listed. It also has substantially different numbers for some browsers. For example, it lists "Android" as 0.55% market share, while caniuse lists "Android Browser" as 0.31%. I'm curious if you (or anyone) have any insight into where that discrepancy comes from.

I'm not really sure. The usage numbers are part of the data.json file here which is a periodically updated export of the site's database. I have noticed that the numbers haven't been updated in a while now, so maybe it's just out of date.

[nex3]

If that's the case, then we do actually have a pretty good idea of what the untracked browsers are, even if what they support is still a mystery.