Clarify the methodology for verifying APCA’s efficacy

[ericwbailey]

This is a sibling issue to w3c/wcag3#307 and Myndex/SAPC-APCA#35.

I want to frame this issue as someone who isn’t a member of the Silver working group, but who has a vested interest in its success.

I have been paying close attention to the SAPC-APCA (APCA for short), in that it addresses an important gap in the current WCAG criterion. As a graphic designer by trade, I think it is incredibly important to have this area of the WCAG updated to match the current technological landscape.

I also want to acknowledge that the stakes for APCA’s success are incredibly high, in that it affects both:

• All visible content on the internet, as well as
• Public perception of the WCAG’s legitimacy.

The ask

Claims of scientific legitimacy must be answered by the rigor of scientific process. I request the working group outlines a process by which specialists can be identified who can vet the APCA’s effectiveness.

I understand that visual contrast is incredibly complicated. It lives in the intersection of the physics of waveforms, the biology of the eye, and cultural considerations. I also understand that vision disability may further this complication.

I do not have the specialized knowledge required to comment on these three disciplines and how they relate to how the formulates of an algorithm. However, I believe that this should not be an impediment to request clarification of process.

In my mind, these factors further reinforce the need for constructive, critical analysis of the APCA. I am very worried about confirmation bias—input from automotive or military researchers would be a good starting point, as they both possess the expertise needed to review these areas.

EDIT: Removed links to published work.

An invited expert may be convinced of the validity of their work, but that does not necessarily mean their work is partially or completely valid.

I am questioning if the methodology behind the APCA is accurate, effective, and reproducible.

I am also not saying current color contrast calculation is effective, nor am I asking if other sections of the WCAG have, or will be peer reviewed—I hope that discussions about these two subjects can be left out of this issue so as to not derail it.

The gravity of this ask is also not lost on me. Candidly, improving color contrast calculation could be one of the most impactful changes for digital accessibility on the web. This is why I am so concerned about ensuring its success.

[caitlynmayers]

I'd like to second Eric's statements and ask — rigorous analysis of this new method is an important step that I think the W3C should follow as they form WCAG 3. Thank you!

[frankelavsky]

Yes, thank you Eric (and thanks for linking to my issue on APCA)! This is probably my single biggest ask of the silver WG right now. I want WCAG 3 to be successful and I want APCA to be successful too.

And part of that success is verification: claims of an empirical foundation need to be backed up with clear references, not just a working model. We need to see under the hood and how all of the constants and variables are informed by science, since that is the primary argument for APCA as a better choice. It will set a dangerous precedence if we claim to be scientific and cannot demonstrate why clearly.

To me this means some time spent on documentation. If we think in terms of an academic paper, we already have a "related works" and "results" section, but we are missing "methodology."

[bruce-usab]

I am questioning if the methodology behind the APCA is accurate, effective, and reproducible.

@ericwbailey (and everyone) have you dug into the algorithms a little bit? See the APCA read.me.

I am not a mathematician, but I can follow the logic well enough to appreciate that is essentially of the same form/structure as the current luminosity contrast algorithms. The RGB weights are the same to a couple decimal points.

• APCA is using updated values, which WCAG 2.0/2.1 never incorporated. (Fortunately, the obsolete numbers do not impact the calculations anywhere near the 4.5/3.0 thresholds).
• APCA is returning a number which has more real-world face validity than the rather arbitrary ratios which have been working well enough. But the 2.0 values are really kind of odd thing! 1 to 21? Who rates/grades that way? It has been okay, but there is no reason not to want something better.
• APCA is using more refined algorithms, so light-on-dark is a different result than dark-on-light. It is a bit of quirk that the current algorithm is agnostic to this, when a perceptual difference is quite apparent from casual inspection. People have been okay with shrugging that off.
• APCA is catching (failing) color combinations that are obviously terrible (but permissible with the current SC). It is a bit of a quirk with the current algorithm that these are missed, but the combinations are terrible enough for everyone, so designers generally avoid them. It would certainly be an improvement to have a WCAG metric to flag those!
• APCA might permit use of some color combinations (for example, white foreground text on a red button), which the current algorithm excludes. It would be a good thing to be able to provide developers some some additional flexibility, if accessibility is not impacted.

The numbers APCA generates are accurate and reproducible. The effective quality has to be true as well, since we know 4.5/3.0 thresholds have proven, over time, to at least be good enough.

The only open question in my mind is picking defensible numbers for the thresholds for what is good enough as a minimum baseline. APCA is not strict with this, and returns a sampling. Which is exactly what I think folks should be looking for a this time.

As a bit of background, there was very little empirical data for the thresholds used for WCAG 2.0. The WG did the best we could at the time, but there just was not much research available. It has worked out. I don't think there was ever a peer review per se. Yes, we need more testing with end-users. But if anything, the real-world validation is so that thresholds picked for using APAC are not too relaxed.

I am not understanding the demand for the scientific rigor for APCA when (1) we did not have that with the current approach (and still do not), and (2) by inspection, APCA is better (by just about any objective metric [except simplicity and familiarity]) and (3) mathematically more defensible.

[frankelavsky]

I am not understanding the demand for the scientific rigor for APAC when (1) we did not have that with the current approach (and still do not), and (2) by inspection, APCA is better (by any objective metric) and (3) mathematically more defensible.

As a researcher myself, when there are claims made about something being backed by empirical science,[1,2,3,4] I need to know what those studies are. If it was just a mathematically more defensible model that produces better results, I'd be fine with it. But stating that it is built with a certain level of academic rigor is something that the entire research community is going to be interested in.

In addition, there are several references to ongoing "empirical work" as well[5,6]. While this is interesting and promising, this also seems vague and potentially suspicious. While there could be "ongoing" empirical work or studies (plural), I haven't seen an open call for participation and I am not sure how someone could possibly recruit all the necessary participants across the broad spectrum of affected disabilities without multiple open calls. In addition, the budget and logistics for this work is non-trivial. We're talking some serious funding and time involved. While it is possible to do this work entirely from a protected IP perspective, I am also considering the possibility that these are attempts to make APCA appear more authoritative.

That second possibility raises questions about why we are choosing APCA and not perhaps another model which is easy to use and understand, still backed by science (and can demonstrate it), and with an open/permissive license. The closed nature of APCA and the IP concerns involved just make me nervous.

Individually, each of these pieces are not cause for suspicion. But adding up claims that this is backed by empirical work, that empirical work is ongoing (and is likely a significant undertaking), and that there are IP reasons things are not brought to light... this makes me wary from a research perspective. Can we trust it? Where are the honest signals? As I said, it perhaps would have been better if it was just a mathematically more defensible model that produces better results.

The accessibility research community has long had issues with WCAG, so to claim that part of WCAG 3 has a level of rigor that hasn't been seen before is pretty important. Researchers have wanted this for a while! So if this claim turns out to be false after WCAG 3 is out in the world (the worst-case scenario), then there could be significant damage between the standards and research communities.

But all this being said: I do want more rigor in WCAG. Just because that wasn't what was done with previous contrast models doesn't mean that should be how things are always done. The research community does have decades of work on color perception and even existing models, all of which I am sure the researchers would be happy to see have contributed to the standards community (even if towards APCA). In addition many researchers (myself included) would love to contribute to projects that have such broad impact, especially in situations where there is still work that needs to be done.

APCA really does have the chance to do this, so to me the stakes are quite high. Despite my suspicions about the worst-case scenario, I am still hopeful for the best-case scenario too. And if there is an opportunity to bring some empirical research into silver's efforts on contrast, please reach out. Perhaps claims of "ongoing research" mean that there is still work to be done. And perhaps APCA could use some assistance with that, in whatever stage it is in. (If not me, then I certainly know folks in research spaces with a willingness to contribute.)

[Myndex]

Well, hasn't this all become a little firestorm.

Hello @ericwbailey @bruce-usab @frankelavsky @caitlynmayers

EDITS IN PLACE:

GENERALLY

The APCA and the related work is a product of a research and development project following the issue thread I started in April of 2019 in the WCAG GitHub issues tracker. While much of the work has been conducted at our lab, it has also been a function of the Visual Contrast Subgroup (VCS) of Silver, and the work has been under the review and oversight of the members of the VCS, one of whom is also a member of the US Access Board, as well as members of W3 and members of the public in general.

The issues and discussions tab has been open at the APCA repo since I moved the project to GitHub as open source, and I am proactive and responsive to comments and discussions there.

The VCS merged into the newly reformed LVTF, something I suggested last year as there are many more visually related issues besides contrast that need examination in terms of guidance and standards within the W3.

As it stands, there have been several independent reviews that demonstrate efficacy. There are threads in the LVTF discussions section with evaluations and review by other peers. But moreover, the evidence regarding APCA is prima facia. I have presented it openly for years, and have written about it extensively, and always with clear visual examples, and evidence that is open and easy to recreate and verify.

---

Funding for rigorous analysis

@caitlynmayers

I'd like to second Eric's statements and ask — rigorous analysis of this new method is an important step that I think the W3C should follow as they form WCAG 3. Thank you!

I whole heartedly agree. Such analysis has been an ongoing part of the development. There are limited resources, as the W3 is a non profit and this work is being donated to the W3 for free. But the work and the studies are continuing, and we are about "in the middle" of that process.

Something you might find shocking about academic work is that authors of scientific research do not get paid for their articles: in fact they have to pay for them to be published, and they receive nothing from the publish. And it is worse for open-access. For instance Nature charges over $11,000 to the author for publishing their paper in the open access section of Nature's website. Eleven thousand, that was not a typo.

More to the point though, peer review for publication is not a "stamp certifying efficacy", and claims that it is misunderstand the purpose and process.

---

Straight out of the hood

@frankelavsky

We need to see under the hood and how all of the constants and variables are informed by science

Absolutely!!! When I publish that information will be laid bare.

I am considering issuing NDAs for a small team for review earlier than later. To be honest, the evidence is prima facia, and so I am a little surprised by the last couple days of "firestorm", but perhaps we should have a private conversation to discuss away from the distractions.

In the meantime, the basic math for your enjoyment:

when there are claims made about something being backed by empirical science,... But stating that it is built with a certain level of academic rigor is something that the entire research community is going to be interested in.

Yes, well understood. There is so much more I am eager to disclose, including modules that more directly account for certain CVD types in a minimally invasive manner, which while slightly less critical for sRGB is going to be very critical for UHD Rec2020 and Rec2100 color spaces.

We are in the MIDDLE of a multi-year process.

I haven't seen an open call for participation

That is why I am posting on Twitter, among other places. And there have been many calls for comments, usage, evaluation — The software for the larger scale studies is not complete, the new target date is spring summer 2022.

If you are good at grant writing, that might help. The project is underfunded to say the least.

That second possibility raises questions about why we are choosing APCA and not perhaps another model which is easy to use and understand, still backed by science (and can demonstrate it), and with an open/permissive license. The closed nature of APCA and the IP concerns involved just make me nervous.

It is not closed what makes you say that? Have you read anything about it? I have listed all the links. It will take you some time to get through it. As far as I can tell, this just leapt up at you yesterday. Have you digested all the material yet? By all I do mean the links and their sub-links listed above.

Had you, you would know that I am bound by my membership to donate the essential IP for the basic APCA for use in the WCAG_3 guidelines.

Evaluating existing models was done very early in the project in 2019 — the only reason I began development of the SAPC project is the lack of suitable models for self-illuminated displays, and a pilot study in 2019 indicated a more suitable method could be developed.

to claim that part of WCAG 3 has a level of rigor that hasn't been seen before is pretty important

I never made any such claim. As you don't know me or the project, you are not aware of the work. I have been performing all due diligence. the fact things are just now becoming more publicly noticeable is in preparation of the greater awareness needed for the larger part of the Perceptex project.

THE HIGH STAKES OF NOT READING

so to me the stakes are quite high.

The stakes are incredibly high — I'm the first one to say that, and here's why:

• The Internet destroyed the print industry.
• By some estimates reading is down 40% over the last two decades.
• Reading on devices and computers screens & tablets is generally difficult.
  • Personally I really miss reading print on paper, and that's not a unique view.
  • There are characteristics of reading on screens that require unique treatment and consideration, that have often not been considered.
• Part of the reason that reading and readability is so poor right now, is due to the well-known problems of WCAG_2 contrast, particularly 1.4.3.
  • There's been a vast increase in misunderstanding of what and how contrast should be used.
  • Classical design seems to have been thrown out the window.
  • There has been a lot of good research regarding critical contrast and critical font size for best readability, and none of it is incorporated in the existing standards.

Consider, projecting forward, the implications of any society, of a world society, that no longer reads.

Consider how a public that does not read is a public that is poised on the edge of a dangerous slippery slope — the same kind of slippery slope we've seen in the past that results in devastating sociopolitical consequences.

Yes, the stakes are really that high. It is why I have devoted the last three years of my life to this project.

---

Hi Bruce! @bruce-usab

For the last week I've been thinking "what would Bruce do"...

Thank you for providing the helpful added context. I'm adding a bit as well below:

I am not a mathematician, but I can follow the logic well enough to appreciate that is essentially of the same form/structure as the current luminosity contrast algorithms. The RGB weights are the same to a couple decimal points.

Indeed the reduction to luminance Y is similar. The changes may seem subtle, but are part of the modeling the output of real world self-illuminated displays.

APCA is using more refined algorithms, so light-on-dark is a different result than dark-on-light. It is a bit of quirk that the current algorithm is agnostic to this, when a perceptual difference is quite apparent from casual inspection. People have been okay with shrugging that off.

Yes the "APCA engine" is uniquely different, but is related to modern color appearance models. It is not creating a ratio, but instead a perceptual difference value using a series of power curves.

APCA is catching (failing) color combinations that are obviously terrible (but permissible with the current SC). It is a bit of a quirk with the current algorithm that these are missed, but the combinations are terrible enough for everyone, so designers generally avoid them. It would certainly be an improvement to have a WCAG metric to flag those!

Not just an improvement, but functionally important. There is movement in the CSS group for automated contrast properties — such automated properties are going to be a VERY bad thing if they are based on WCAG_2!!!

As you know, my studies in 2019 showed that WCAG_2 incorrectly passed 49% and incorrectly failed 23% of random colors. Sam recently did his own independent study and found it to be 55% and 30% respectively. YIKES!!!! Just think of the problems if CSS uses that for an automated contrast property?!?!

APCA might permit use of some color combinations (for example, white foreground text on a red button), which the current algorithm excludes. It would be a good thing to be able to provide developers some some additional flexibility, if accessibility is not impacted.

I've actually shown that for CVD types, accessibility is markedly improved here. I suppose it's my bad hitting the hornet's nest when I created the APCA logo using what I call "APCA Orange" #f60 which fails all WCAG_2 contrast, but passes APCA (of course) and as you can see in this slide, all CVD types do well, despite WCAG_2 wrongtrast failing it.

The only open question in my mind is picking defensible numbers for the thresholds for what is good enough as a minimum baseline. APCA is not strict with this, and returns a sampling. Which is exactly what I think folks should be looking for a this time.

That is in part the purpose of the public beta test, and also of the spring 2022 studies.

As a bit of background, there was very little empirical data for the thresholds used for WCAG 2.0. The WG did the best we could at the time, but there just was not much research available. It has worked out. I don't think there was ever a peer review per se. Yes, we need more testing with end-users. But if anything, the real-world validation is so that thresholds picked for using APAC are not too relaxed.

I am not understanding the demand for the scientific rigor for APCA when (1) we did not have that with the current approach (and still do not), and (2) by inspection, APCA is better (by just about any objective metric [except simplicity and familiarity]) and (3) mathematically more defensible.

Thank you very much Bruce for that recap. I don't seem capable of getting things across as effortlessly as you!!

Thank you ALL for the comments and I hope we can move forward postively and proactively.

Andy

THE WORLD IS READING^™

[vavroom]

I am a "member of the public", no longer directly involved with WAI. I was active on the EOWG for 18 months and was going to work on AG but life got in the way of that. I have been involved with accessibility since the mid 1990's. I currently am an independent accessibility consultant and I deal everyday with having to explain accessibility and WCAG to clients who find the guidelines hard to understand. I need to justify success criterias all the time. I need to explain why elements of accessibility, such as color contrast work, or don't work.

In that context, the prospect of having to explain a tool that hasn't been peer reviewed by experts concerns me greatly.

APCA may, or may not, be the right approach. I don't know. I do know that the aspect of having to make a subjective call around font style fills me with dread. The last thing we need is more subjectivity in decisions. Maybe I'm misunderstanding the documents I've seen about it but from my read, after taking into account background and foreground colors, and font weight and size, which are all calculatable, the last step is relying on judgment about the font. If I'm correct, this in and of itself will be the cause of endless problems.

That said. It seems to me to be verging on conflict of interest for a tool/methodology creator to push it so strongly for adoption into a standard. It would be another matter if the tool/methodology had been peer reviewed and was approaching industry standard already.

What bothers me even more is the apparent high level of defensiveness anytime someone expresses a sentiment that could be perceived as negative towards either the tool or the tool's author. Having been part of discussions with @Myndex here and on Twitter, and having reviewed other interactions, I am seeing a lot of passive aggressiveness that has no place in standards creation. I have heard and seen complaints from multiple individuals who felt bullied by Andrew's approach. When comments are made to Andrew about these, he dismisses them out of hand, blaming a small group of people. Having known several of these people for many years, not just online, I know their integrity and if it's a question of taking sides, I'll side with them any day against someone who consistently exhibit passive aggressive behaviors verging on bullying.

To whit: His refusal to address @ericwbailey's concerns directly. Offering to go privately - this is pretty typical behavior from abusers. Note, I'm NOT accusing Andrew of being an abuser. But his behavior is on the edge. Interacting with him feels very much like being gaslighted at every turn.

I hope I'm wrong, but I fear I'm not.

Circling back to APCA, however, and dissociating the person from the tool/methodology, we still need to ensure that whatever makes it in the final version of AG has been peer reviewed and is accepted by the group through the consensus process of the W3C. And consensus cannot happen when individuals in the group behave in such a way that other group members just "give up". I suggest that because of the risk of conflict of interest, Andrew Somers should recuse himself from discussions around this tool/methodology.

[ericwbailey]

Hi @bruce-usab, thank you for your reply. I am also not a mathematician, and I also was able to follow the general gist of the algorithm.

When you mention accurate and reproducible, that's exactly what I'm wondering about. Internally accurate and reproducible is not the same as external, especially with confirmation bias in play.

I would also like to re-emphasize this point:

I am also not saying current color contrast calculation is effective,

I am not questioning if the current color contrast algorithm is effective enough. I am asking if the APCA is a suitable replacement, or if another algorithm should be sourced. The way to determine this is to see if the APCA can pass muster.

by inspection, APCA is better

I am not sure about this, given how supporting content falls apart at closer inspection. @frankelavsky expounds on this far better than I could in #574 (comment).

[ericwbailey]

@Myndex You previously mentioned you would prefer GitHub communication. I am doing that here.

The standards process is open and transparent, and I am observing that by posting this issue. I would like to say that your request for private communication does add to my perception of a pattern of behavior of deflection and obfuscation when pressed for direct answers. This leads to an erosion of trust.

I also would like to re-stress this point:

Andrew’s communication patterns both on and off GitHub create an obfuscating effect, where longform messaging dodge or deflect direct asks. This could be intentional, accidental, or both.

We all understand the value, and impact of an updated contrast algorithm and how it affects reading on digital screens. In addition, I am not questioning your passion for your project. There is no need to expound on it here.

[modulist]

One of my major responsibilities at work is to create content that explains accessibility to designers, developers, and managers — who are not accessibility professionals. My standard is to create content that's not only simple enough for a designer to understand, but also simple enough for them to explain to a co-worker.

Before this flared up into a personal issue, I was hoping to be able to research APCA and be able to write a series of pieces explaining it to my audience. Here are some sample questions I was hoping to cover:

• what's being measured? If this is a two-dimensional formula, what are the x and y axes?
• how were the formulas behind APCA created?
• how were the constants in the formulas arrived at? can we check them?
• if testing was done, what was the test experience like? how could someone else reproduce the tests?
• can we improve or better calibrate APCA in the future?
• what impact do factors like font-weight, reversed text, and ambient light have on APCA?

The intent was to be able to explain this to someone with a high-school level math background and a high-school level of English, which may not be their first language.

APCA and its documentation are not self-explanatory. The image of the Contrast Prediction Equation in this thread isn't particularly helpful to me or my audience. It needs a legend or a step-by-step explanation, which is missing.

Those of us who aren't familiar with APCA and its development will need a learning path through this, and it will require patience and civility from all parties.

I sincerely hope we can find a way to gain more transparency so we can put our own support behind APCA and promote it.

[bruce-usab]

@modulist — do you think you will be able to make your APCA explanation public facing? I think it would be helpful to a have a skeptical third-party write that up. You might make it three parts:

1. Problems with the current 2.x approach/algorithm. @Myndex has provided a great deal of evidence here! Personally, I would also like something (1) a little bit more oriented to the layperson (which might describe your audience), and (2) less pejorative when describing the status quo.
2. Your analysis, good or bad, of how APCA addresses those defects.
3. Your prognosis for APCA.

My own experience with @Myndex is that he is listening and does make changes with his model implementations in response to feedback and criticism. I am taking the liberty to suggest this 3-step approach, since (1) is valuable regardless (even if only for your internal audience). Also both (1) and (2) can be agnostic to an appreciation (or not) of APCA. (3) is probably the most important, but needs to be informed by (2). And even if you conclude that you cannot endorse APCA, that still gets the world closer to something better than what we have with the 2.0/2.1/2.2 metric. I do not know of anyone asserting that the current 2.x approach is sufficient for WCAG3.

@ericwbailey and @vavroom et al. -- What I have been doing for the past year or so is trying to remember to check APCA scores any time I have had call to use the WebAIM contrast checker. This habit has greatly increased my comfort with APCA. With the benefit of hindsight, (1) I wish had been more disciplined with this practice, and (2) taken notes. Maybe that should be my New Year's resolution?

For the last week I've been thinking "what would Bruce do"...

Thank you for that kindness, but that way madness lies...

[vavroom]

@bruce-usab Who is being pejorative here?

As for "getting comfortable with APCA", that's not an issue for me. Telling @ericwbailey and i to familiarize ourselves with the tool doesn't address the very real concerns several people have raised.

Whether or not I like the tool/methodology doesn't stop it from not having been peer reviewed. Doesn't stop it from having claims of scientific rigor behind it that have no transparent support around it.

I stand by what I said earlier.

[Myndex]

Hi @vavroom Thank you for your comments.

In that context, the prospect of having to explain a tool that hasn't been peer reviewed by experts concerns me greatly.

Yet you are doing that now by using the WCAG_2 SCs. They are a product of internal consensus, not independent peer review. Moreover, there is a process in place, and a process that is being followed toward consensus, and peer review publication, and this has been part of the plan from the very beginning, along with the complete openness I have provided as the studies have progressed, from the beginning in april 2019.

The rumors that "there is no peer review" are misleading at best, but otherwise somewhat faux concerns. This project has had ample oversight and review.

The last thing we need is more subjectivity in decisions. Maybe I'm misunderstanding the documents I've seen about it but from my read, after taking into account background and foreground colors, and font weight and size, which are all calculatable, the last step is relying on judgment about the font. If I'm correct, this in and of itself will be the cause of endless problems.

You can not calculate font weight and size today — and that is an actual problem we are working on.

Here is an example. All of these fonts samples are set to font-size: 14px (except of course non-black headers) do you see the problem?

The propertyfont-size: is not a valid way to determine the actual readability of a font. This is equally a problem with font weight.

That said. It seems to me to be verging on conflict of interest for a tool/methodology creator to push it so strongly for adoption into a standard.

I am not a tool creator. I am an independent researcher, with no interest in creating tools. The tools that I have presented were only created to demonstrate the technology, a common practice in research. There are dozens of tool creators that have incorporated APCA into their tools, along side the existing WCAG_2, for the purposes of beta testing and evaluation.

I am not "pushing strongly" for adoption — I was brought in as an invited expert after starting thread 695, with the express goal of finding a workable solution for Silver.

For the backstory, you can start with issue thread 695 in WCAG GitHub where I identified the significant issues with the WCAG_2 contrast method. I am not someone to complain and leave, I am proactive in finding the solution. In this case I have devoted nearly three years of full time research focus to solving the WCAG_2 contrast problem.

It would be another matter if the tool/methodology had been peer reviewed and was approaching industry standard already.

APCA is a project in development. That it is visible is because it is being developed OPENLY and to encourage public comment while doing so. The issues and discussions tab are open for any and all questions comments and open discussions regarding APCA. This is what is done with an open project before formal peer review.

Nevertheless, peers have already been involved. Studies and live interactive experiments have been public facing for years. If you have only heard about this recently in social media is because we are working to increase awareness and generate interest for gathering comments and for future planned studies.

What bothers me even more is the apparent high level of defensiveness anytime someone expresses a sentiment that could be perceived as negative towards either the tool or the tool's author.

This is not true

The project is open to all criticism and discussion as I mentioned before. The claim that it is not is part of some rumors of unknown origin. Other than certain individuals have been invited numerous times to formally discuss any issues they desire as part of the consensus and peer process — yet they have refused to do so.

This is a public beta. If you have an actual problem with the math or the methods, please file an issue so we can track and address it.

Links to links of publicly accessible information

Linktree of the most key chunks of data Start at the top and work your way down.

The W3 Visual Contrast Wiki This page is divided into several pages that fairly completely cover the subject matter with additional referencing links.

List of articles/gists here on GitHub A number of writings on contrast and color include several key comparative articles that demonstrate the WCAG_2 and APCA differences.

In depth issue threads These threads contain a mass of the early research, all out in the open.

SAPC Contrast Research Tools The WCAG3/Silver Contrast Algorithm beta site. Includes interactive experiments that demonstrate the concepts. Click on "research mode" and then use the experiments to evaluate aspects of contrast.

Color Vision Deficiency Simulator This clinically accurate simulator demonstrates the way someone with a Color Vision Deficiency (incorrectly labeled "color blind") sees colors.

And the GitHub repo here, see the Discussion tab.

When Does Peer Review Happen

Peer review occurs when the "work" on the project is complete and a subsequent paper is submitted for publication. In some cases there may be concurrent review, or concurrent oversight, and there is some of that going on. But again, the evidence is prima facia—clear to see.

[Myndex]

@Myndex You previously mentioned you would prefer GitHub communication. I am doing that here.

Regarding APCA directly, and at the APCA repository which is over there. All questions or issues with the APCA project should be made in the appropriate repo.

[Myndex]

Hello Claudio @modulist

One of my major responsibilities at work is to create content that explains accessibility to designers, developers, and managers — who are not accessibility professionals. My standard is to create content that's not only simple enough for a designer to understand, but also simple enough for them to explain to a co-worker.

This is an area I am not succeeding with, LOL.

Before this flared up into a personal issue, I was hoping to be able to research APCA and be able to write a series of pieces explaining it to my audience. Here are some sample questions I was hoping to cover:

Excellent, this is the very kind of discussion/criticism that is needed. Please do not let the trolls or flame wars impede this progress.

I probably have answered all of your questions in this white-paper in progress:
https://www.w3.org/WAI/GL/task-forces/silver/wiki/Visual_Contrast_of_Text_Subgroup/Whitepaper

But I will answer individually below as well. I am going to answer in "not" plain language initially, and then make my own ham fisted plain language attempt thereafter when appropriate. I tend to avoid this as I always worry about "talking down" to anyone, but I have been recently told that what I write is "obtuse and hard to grok" or words to that effect. So apparently, finding the right tone is challenging for me.

• what's being measured? If this is a two-dimensional formula, what are the x and y axes?

NOT plain language ↓

APCA is measuring the supra-threshold perceptual contrast of certain kinds of stimuli on a self illuminated monitor, device, or display. The larger SAPC model is multi-dimensional. The simple W3 licensed APCA is derived from this larger model, and an attempt has been made to fit to the 3D to 7D empirical data sets in as close to 2D or 3D as possible, but you might call it more like a "2.5D" as things like polarity and spatial frequency are important (and sometimes more important) than the color difference. Some of these are presently separated in a LUT, but some aspects are on their way to being incorporated as parametric curves. At that point, it is at least a 3D construct. There you might say that Y is the total luminance context, X as the Lc value (difference), Z as the spatial component. That is probably the irreducible de minimus.

The stages of analysis are:

• Reduce a given sRGB CSS color value to an estimated display relative luminance.
• Soft clamp the black levels to acknowledge flare, black clamping from user adjustment, and other factors.
• Apply separate exponents to the text and background luminances
  • Said exponents are dependent on the polarity, as in black text on light BG or light text on dark BG.
  • Said exponents are specific for the BG and the text stimuli in accordance with the power curves fit to empirical studies conducted here, as well as the well understood contrast models as discussed by Barten (91), Fairchild (R-Lab), Hunt, and CIE (76, 02, 16)
• Subtract the resultant processed text value from the processed background value.
• Scale and clamp to create the output value as Lc (lightness contrast).

The Lc value can be used directly as a guide, as it is in Bridge-PCA, the WCAG_2 100% backwards compatible version. Or design flexibility can be further improved by matching it to a lookup table of font values to find a minimum font size and weight.

My Plain Language attempt: ↓

The amount of predicted lightness for the text color is subtracted from the predicted lightness for the background color to arrived at the predicted lightness contrast, or Lc value.

---

• how were the formulas behind APCA created?

Some of this is part of proprietary information, and will be presented at the appropriate time. It is not open for public discussion YET but I am committed to it being open to the public.

In the meantime, I will say that it is a combination of research into all of the existing appearance and color models (like RLab, CIELAB and CIELUV, CIECAM02 and CAM16, Hunt, etc) and a lengthy series of experiments with the existing contrast methods (publicly available on or through the Myndex site) and fitting to empirically created data sets here in the lab. And other fun things that I am bursting at the seams to talk about but that my patent attorney tells me I have to shut up about for the time being.

I am only going to hint at one little thing: there is a lot more that goes beyond contrast. (In Hollywood we call that a "teaser").

• how were the constants in the formulas arrived at? can we check them?

Same as above. And yes you can check them, they are public. (!!!) I may be misunderstanding your question?

All the math for the W3 version (contrasting color pairs) is public.

• if testing was done, what was the test experience like? how could someone else reproduce the tests?

The testing environment is that of the SAPC standard observer, which is detailed at the APCA github repo.

• can we improve or better calibrate APCA in the future?

There is a planned wide-area study for 2022. Also, the entire function is designed to be extensible and adjustable. I have hinted at what some of the extended functions are in the source code, which is publicly available.

However, right now, there seems to be a cry for "keeping it simple" so I am not going to get into the future possibilities.

---

• what impact do factors like font-weight, reversed text, and ambient light have on APCA?

These things are directly taken into account!!!!!

Not Plain Language ↓

Font weight and font size are critical to understanding the APCA guidelines, as these relate to spatial frequency. Spatial frequency is a primary determiner of perceptual contrast. Here is a crude infographic that outlines this (not for publication, I'm making a nicer one, LOL)

What we see here is that as spatial frequency increases the perception of contrast (and importantly readability contrast) decreases dramatically. This is reflected in the font lookup tables for APCA.

Regarding "reversed" text, I assume you mean reversed polarity, as in light text on a dark background. I already did mention polarity as part of the fundamental, basic APCA function (above). But we can stop for a moment and look at some factors as to why. One of them is local adaptation; the HVS (human vision system) is an amazingly complex system with an incredible range of sensitivity — from a flickering candle miles away on a moonless night to the blazing noon sun. But we can't see all that at one time, we have to adapt to the conditions, known as lightness (or darkness) adaptation. (think for instance the effect of being in a dark theater for a Saturday matinee and then walking out into the sunshine.)

There is also local adaptation — in this case, I mean the background immediately around the text. Our adaptation level is a combination of our environment's ambient light, the overall light from the screen, and the very local light from the screen and text. It should be obvious that with light text on dark, the overall area lightness is darker, resulting in a darker adaptation and thus causing light text to appear "brighter" that dark text for the same (inverted) pair of colors. But this is also dependent on the overall location of that pair relative to the ambient lightness adaptation.

In the case of the basic APCA, there is some pre-estimation. There are future versions that are pending release upon IP filings that directly account for some of these factors. Those are for more critical applications.

My Plain Language attempt: ↓

Yes, font weight and size are very important, defined in a lookup table to provide more accurate guidance to designers. Small thin fonts need more lightness/darkness contrast to compensate for their inherently decreased contrast.

Reverse text, light on dark, is indicated with a negative number. but is scaled for similar readability. I.e. Lc 60 and Lc-60 are essentially the same perceived contrast.

---

---

The intent was to be able to explain this to someone with a high-school level math background and a high-school level of English, which may not be their first language.

January Edit: Bad math joke deleted, and I've reviewed and re-written the "Why APCA" document, and tested the "readability score" to fit that of a high school senior (ot at least my best attempt at that).

---

APCA and its documentation are not self-explanatory. The image of the Contrast Prediction Equation in this thread isn't particularly helpful to me or my audience. It needs a legend or a step-by-step explanation, which is missing.

Okay, on the page in the APCA repo is a plain language step-by-step. Please let me know if I can help break it down further.

---

Those of us who aren't familiar with APCA and its development will need a learning path through this, and it will require patience and civility from all parties.

Yes, Alastair @alastc made me aware that I need to write a one page explainer, and I am working on that among the many other year-end tasks I am buried under. At the moment, I've generated a massive amount of material, and I'm not that certain what would be best for you to start.

I did create a linktree with what "I think" is the best path to climb the learning curve. Start near the top and work your way down:

https://linktr.ee/Myndex

---

---

I sincerely hope we can find a way to gain more transparency so we can put our own support behind APCA and promote it.

To be honest I do find statements like this a bit frustrating, as I have been completely open internet rumors notwithstanding. Comments made to the APCA repo discussion area are promptly responded to.

Here is a further link list, feel free to read it all and please let me know if you have any questions:

Links to links of publicly accessible information

The W3 Visual Contrast Wiki This page is divided into several pages that fairly completely cover the subject matter with additional referencing links.

List of articles/gists here on GitHub A number of writings on contrast and color include several key comparative articles that demonstrate the WCAG_2 and APCA differences.

In depth issue threads These threads contain a mass of the early research.

SAPC Contrast Research Tools The WCAG3/Silver Contrast Algorithm beta site. Includes interactive experiments that demonstrate the concepts. Click on "research mode" and then use the experiments to evaluate aspects of contrast.

Color Vision Deficiency Simulator This clinically accurate simulator demonstrates the way someone with a Color Vision Deficiency (incorrectly labeled "color blind") sees colors.

And the GitHub repo here, see the Discussion tab.

---

Thank you for your comments

[chaals]

Dear all,

This is a request from a very long-term but now very occasional WAI contributor to keep to technical issues in W3C's issue trackers, including in this issue.

W3C is a very diverse community of people who have many skills, talents, perspectives, histories, cultures and personalities. To help ensure a respectful and productive work environment, there are several different formal mechanisms to deal with communication problems and inappropriate behaviour. They don't include "discuss the merits of individuals on Github", because that is not appropriate behaviour.

It is possible to edit recorded comments in Github. Sometimes, that is a way to improve a situation. I suggest it would be helpful in this case.

(This isn't the only issue where this problem has arisen, it is the one that most recently stood out to me).