Skip to content

Commit 1fbf670

Browse files
authored
Merge pull request #208 from PublicisSapient/automated-lighthouse
WIP: Lighthouse test step, CI step and output formatter
2 parents 18ad928 + dba551c commit 1fbf670

File tree

7 files changed

+1909
-2393
lines changed

7 files changed

+1909
-2393
lines changed

.gitignore

+1
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@ node_modules
33
api-cache
44
tmp
55
.vscode
6+
report/*
67
.idea
78

89
# auto-gen PHP files/logs

README.md

+3
Original file line numberDiff line numberDiff line change
@@ -101,6 +101,7 @@ A place to learn and share with developers what makes web work accessible. This
101101
- Axe CLI: Uses a browser webdriver to open pages and run accessibility tests on it.
102102
- pa11y CLI: Uses Puppeteer to run its own headless Chrome browser to run accessibility tests.
103103
- Jest + Puppeteer: Used to run and validate code in unit tests.
104+
- Lighthouse CLI: Uses a browser webdriver to open and run accessibility audits on each page and then provides reports and a summary.
104105

105106
Read [the Enable Code Quality article](https://www.useragentman.com/enable/code-quality.php) for the full details behind the testing tools being used and how.
106107

@@ -112,6 +113,8 @@ Read [the Enable Code Quality article](https://www.useragentman.com/enable/code-
112113
- Run only the v.Nu tests: `npm run test-vnu`
113114
- Run only the Axe tests: `npm run test-axe`
114115
- Run only the Pa11y tests: `npm run test-pa11y`
116+
- Run only the Lighthouse tests on all URL's: `npm run test-lighthouse`
117+
- Run only the Lighthouse tests on a single URL: `npm run test-lighthouse-url {Valid URL}`
115118
116119
If you are noticing that the jest tests are taking a long time, you might want to run `npm run jest-debug-memory-leak`. If the heap size for each test group increases a lot, there is a memory leak in the tests. More information about that can be found at in the article [Your Jest Tests are Leaking Memory](https://chanind.github.io/javascript/2019/10/12/jest-tests-memory-leak.html) by [David Chanin](https://chanind.github.io/about/)
117120

bin/checkHTML.sh

+20
Original file line numberDiff line numberDiff line change
@@ -459,6 +459,22 @@ function runPa11yTests() {
459459
fi
460460
}
461461

462+
463+
464+
function runLighthouseTests() {
465+
#. Download the HTML files if they have not already been downloaded
466+
if ! [ -f tmp/downloaded-urls.txt ]
467+
then
468+
bin/generateSiteMap.sh
469+
downloadHTML
470+
else
471+
: "${DOWNLOADED_URLS:=`cat tmp/downloaded-urls.txt`}"
472+
fi
473+
474+
node bin/lighthouse-accessibility-scan.js
475+
}
476+
477+
462478
#.. let's wipe the tmp directory if it exists
463479
if [ -z "$(ls -A tmp)" ]
464480
then
@@ -475,6 +491,9 @@ then
475491
elif [ "$1" = "pa11y" ]
476492
then
477493
runPa11yTests
494+
elif [ "$1" = "lighthouse" ]
495+
then
496+
runLighthouseTests
478497
else
479498
#.. Run checks and preparation for tests
480499
bin/generateSiteMap.sh
@@ -484,6 +503,7 @@ else
484503
runVNUTests
485504
runAXETests
486505
runPa11yTests
506+
runLighthouseTests
487507

488508
#.. Remove temporary files on success
489509
rm tmp/* 2> /dev/null

bin/lighthouse-accessibility-scan.js

+207
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,207 @@
1+
/**************************************************
2+
* Usage:
3+
*
4+
* 1. npm run test
5+
*
6+
* Will run the entire set of tests including this lighthouse test against all URL's
7+
* found in the tmp/downloaded-urls.txt file.
8+
*
9+
*
10+
* 2. npm run test-lighthouse
11+
*
12+
* Will run only the lighthouse accessibility tests against all URL's
13+
* found in the tmp/downloaded-urls.txt file.
14+
*
15+
*
16+
* 3. npm run test-lighthouse-url {VALID URL}
17+
*
18+
* Will run the lighthouse accessibilty tests against a single URL.
19+
* For example: npm run test-lighthouse-url https://www.useragentman.com/enable/index.php
20+
*
21+
*
22+
* Note:
23+
*
24+
* If the tmp/downloaded-urls.txt file does not exist, you can manually create one
25+
* or you can generate one by first running npm run test-lighthouse.
26+
*********************************************** */
27+
28+
const { spawn } = require('child_process');
29+
const fs = require('fs');
30+
31+
const SUMMARY_PATH = 'report/lighthouse/summary.json';
32+
const REPORT_PATH = 'report/lighthouse/';
33+
const RED_TXT = '\x1b[31m%s\x1b[0m';
34+
const GREEN_TXT = '\x1b[32m%s\x1b[0m';
35+
const YELLOW_TXT = '\x1b[33m%s\x1b[0m';
36+
const SCORE_THRESHOLD = 1;
37+
38+
const getOptions = () => {
39+
const singleUrl = process.argv[2];
40+
const downloadedUrls = 'tmp/downloaded-urls.txt';
41+
const isValidUrl = URL.canParse(singleUrl);
42+
43+
if (singleUrl !== undefined && !isValidUrl) {
44+
console.error(
45+
`Error: ${singleUrl} is not a valid URL. Please include http:// or https://`,
46+
);
47+
process.exit(1);
48+
}
49+
50+
const args = isValidUrl
51+
? [`-s ${singleUrl} --params "--only-categories=accessibility"`]
52+
: [`-f ${downloadedUrls} --params "--only-categories=accessibility"`];
53+
54+
const numPages = isValidUrl ? 1 : getNumPages(downloadedUrls);
55+
56+
return {
57+
command: './node_modules/.bin/lighthouse-batch',
58+
args,
59+
numPages,
60+
};
61+
};
62+
63+
const readFileSync = (path, encoding = 'utf-8') => {
64+
try {
65+
return fs.readFileSync(path, encoding);
66+
} catch (err) {
67+
throw new Error(`Error reading file at ${path}: ${err.message}`);
68+
}
69+
};
70+
71+
const fileExists = (path) => {
72+
if (!fs.existsSync(path)) {
73+
throw new Error(`File not found: ${path}`);
74+
}
75+
};
76+
77+
const getNumPages = (downloadedUrls) => {
78+
fileExists(downloadedUrls);
79+
return readFileSync(downloadedUrls).split('\n').filter(Boolean).length;
80+
};
81+
82+
const getReport = (fileName) => {
83+
fileExists(fileName);
84+
return JSON.parse(readFileSync(fileName));
85+
};
86+
87+
const logPageStatus = ({ fileName }) => {
88+
const { runtimeError, requestedUrl, categories } = getReport(fileName);
89+
90+
if (runtimeError) {
91+
console.log(
92+
YELLOW_TXT,
93+
`🚫 Error scanning: ${requestedUrl} - ${runtimeError.message}\n`,
94+
);
95+
return;
96+
}
97+
98+
const statusColor =
99+
categories.accessibility.score >= SCORE_THRESHOLD ? GREEN_TXT : RED_TXT;
100+
const statusMessage =
101+
categories.accessibility.score >= SCORE_THRESHOLD
102+
? '✅ Pass'
103+
: '❌ Fail';
104+
105+
console.log(statusColor, `${statusMessage}: ${requestedUrl}\n`);
106+
};
107+
108+
const printIssuesSummary = (audits, url, fileName, score) => {
109+
console.log(RED_TXT, `\n${url} failed scan with score: ${score}%:\n`);
110+
console.log(
111+
RED_TXT,
112+
`Visit https://googlechrome.github.io/lighthouse/viewer/ and upload ${fileName} to see the full accessibility report.\n`,
113+
);
114+
115+
Object.values(audits).forEach(
116+
({ score, id, title, description, details }, index) => {
117+
if (score < 1 && score !== null) {
118+
console.log(` Issue ${index + 1}: ${id}\n`);
119+
console.log(` Title: ${title}\n`);
120+
console.log(
121+
` Selector: ${details?.items[0]?.node?.selector}\n`,
122+
);
123+
console.log(` Description: ${description}\n\n`);
124+
}
125+
},
126+
);
127+
};
128+
129+
const formatSummary = () => {
130+
fileExists(SUMMARY_PATH);
131+
const summary = JSON.parse(readFileSync(SUMMARY_PATH));
132+
133+
let passCount = 0,
134+
failCount = 0,
135+
errorCount = 0;
136+
137+
console.log('\nLighthouse Scan Summary ...\n');
138+
139+
summary.forEach((item) => {
140+
const fileName = `${REPORT_PATH}${item.file}`;
141+
const report = getReport(fileName);
142+
const score = Number(item.score) * 100;
143+
144+
if (item?.error) return errorCount++;
145+
if (report.categories.accessibility.score === SCORE_THRESHOLD)
146+
return passCount++;
147+
148+
printIssuesSummary(report.audits, item.url, fileName, score);
149+
failCount++;
150+
});
151+
152+
const totalCount = passCount + failCount + errorCount;
153+
const statusColor = failCount === 0 ? GREEN_TXT : RED_TXT;
154+
console.log(
155+
statusColor,
156+
`Scan complete: ${passCount}/${totalCount} URLs passed\n`,
157+
);
158+
};
159+
160+
const runLighthouseBatch = () => {
161+
const { numPages } = getOptions();
162+
163+
console.log(
164+
`\nLighthouse Scan Started on ${numPages} pages, this may take awhile ...\n`,
165+
);
166+
167+
return new Promise((resolve, reject) => {
168+
const { command, args } = getOptions();
169+
const child = spawn(command, args, { shell: true });
170+
171+
child.stderr.on('data', (data) => {
172+
data.toString()
173+
.split('\n')
174+
.filter((line) =>
175+
line.includes('Printer json output written to'),
176+
)
177+
.forEach((line) => {
178+
const match = line.match(
179+
/(?<=Printer json output written to\s).*$/,
180+
);
181+
if (match) logPageStatus({ fileName: match[0] });
182+
});
183+
});
184+
185+
child.on('error', (err) =>
186+
reject(
187+
new Error(`Failed to run the lighthouse scan: ${err.message}`),
188+
),
189+
);
190+
191+
child.on('close', (code) => {
192+
if (code === 0) resolve();
193+
else
194+
reject(
195+
new Error(
196+
`Lighthouse batch scan failed with exit code ${code}.`,
197+
),
198+
);
199+
});
200+
});
201+
};
202+
203+
runLighthouseBatch()
204+
.then(formatSummary)
205+
.catch((err) =>
206+
console.error(`Error running runLighthouseBatch: ${err.message}`),
207+
);

content/body/code-quality.php

+22
Original file line numberDiff line numberDiff line change
@@ -79,6 +79,28 @@
7979
20% to 50% of accessibility issues. Is there any way to improve upon that?
8080
</p>
8181

82+
<h2>Using Lighthouse CLI for Accessibilty Audits</h2>
83+
84+
<p>Enable runs Lighthouse accessibility scans on all of our pages. Lighthouse generates a report per page and an overall summary with scoring associated to the results. The reports generated by the
85+
scans provide another layer of accessibility testing which ensures that pages are marked up properly for screen readers. It also checks for sufficient
86+
text element contrast. Once reports are generated you can easily view them in the same format that Lighthouse through the chrome browser's inspect window provides by
87+
navigating to <a href="https://googlechrome.github.io/lighthouse/viewer/">https://googlechrome.github.io/lighthouse/viewer/</a> and uploading a generated
88+
report file.
89+
</p>
90+
91+
<p>Enable makes use of <a href="https://github.com/GoogleChrome/lighthouse-ci">https://github.com/GoogleChrome/lighthouse-ci</a>
92+
and <a href="https://github.com/noetibbl/lighthouse-batch">https://github.com/noetibbl/lighthouse-batch</a> to run the Lighthouse accessibility scans. It does this by:</p>
93+
94+
<ol>
95+
<li>Generating all the HTML for all the PHP pages on the site.</li>
96+
<li>Calling the lighthouse node scripts and passing the page or pages as a URL parameter or a file paremeter.</li>
97+
<li>Scans each page one by one and generates reports.</li>
98+
<li>Creates a log for each pages success, failure and error status.</li>
99+
<li>Ingests all the reports and runs some calculations on scoring based on the metrics Lighthouse scans for.</li>
100+
<li>Creates a summary report of all the pages scanned.</li>
101+
<li>Finally, if issues are found, it will log out to a terminal window all the problem areas and the suggested fixes.</li>
102+
</ol>
103+
82104

83105
<h2>Unit Testing</h2>
84106

0 commit comments

Comments
 (0)