Add Stryker dynamic analysis evaluation

[emw868]

StrykerJS Dynamic Analysis PR

Summary

This PR adds StrykerJS as the team's dynamic analysis tool on branch tool/stryker-dynamic-analysis.

Concrete Installation Evidence

• install/package.json
  Adds the new dev dependency @stryker-mutator/core and the scripts dynamic:stryker and dynamic:stryker:test.
• stryker.config.json
  Adds the Stryker configuration used for mutation testing.
• .github/workflows/stryker-dynamic-analysis.yml
  Adds a dedicated GitHub Actions workflow that installs dependencies, runs Stryker, and uploads the generated artifacts.

Concrete Run Artifacts

• artifacts/dynamic-analysis/stryker/npm-ls-stryker.txt
  Confirms that @stryker-mutator/core is installed in the project.
• artifacts/dynamic-analysis/stryker/terminal-output.txt
  Captures the local terminal output from a successful Stryker run.
• GitHub Actions artifact: stryker-dynamic-analysis
  The workflow uploads the terminal output and generated Stryker report files for the PR.

Quantitative Results

Local run results:

• Mutated file count: 1 (src/pagination.js)
• Total mutants: 137
• Killed mutants: 103
• Survived mutants: 33
• Timeout mutants: 1
• Mutation score: 75.91%
• Runtime: about 8 seconds on the dev container

Qualitative Evaluation

Pros

• Very little project-specific wiring was required - Stryker worked with a simple command runner and node:test, so I did not need to reuse NodeBB's existing Mocha setup.
• The reports are immediately useful - The terminal output shows which mutants survived, and the generated HTML/JSON reports are straightforward evidence for the PR.
• It scales from "quick smoke check" to stricter enforcement - The mutate target can stay narrow for cheap experimentation or expand later to broader source directories.

Cons

• Vanilla startup was not fully plug-and-play in this repository - The first Stryker run failed while copying generated symlinked assets under build/public/plugins/core/inter.
• It is easy to get misleadingly low or high scores depending on scope - Restricting the mutation target to a single file makes the tool easy to adopt, but it is not representative of whole-project mutation strength.
• Runtime will grow quickly if the mutation scope expands - The current run is fast because it mutates one small file with a tiny dedicated test suite.

[alexbock2]

Looks good

[emw868]

Project Part 3B:

1. I used StrykerJS, a mutation testing tool for JavaScript/Node.js. Instead of only checking whether tests pass, it makes small changes, or mutations(mutants) to the code and reruns the tests to see whether the tests detect those changes. If a mutant survives, that suggests the test suite is weak or missing an assertion. This can be seen in the documentation of Stryker(https://stryker-mutator.io/docs/stryker-js/introduction/).

2. This is dynamic analysis. Stryker changes code and executes tests against the mutated program, so it depends on runtime behavior rather than just inspecting source text.

3. Stryker is good at catching weak tests, missing assertions, and untested branches/logic. For example, it can reveal cases where conditionals, boolean checks, arithmetic, or edge-case behavior can be changed without causing tests to fail. In our repository, it mutated src/pagination.js and showed which logic changes were still not being caught by the smoke tests.

4. In our setup, the important customizations were:

   • using the command runner instead of NodeBB’s built-in Mocha flow
   • pointing it at a dedicated node:test smoke suite
   • limiting mutate to src/pagination.js so runtime stayed short
   • adding ignorePatterns for generated folders like build, coverage, and logs because the near-vanilla setup
     initially failed on generated assets
   • enabling clear-text, HTML, and JSON reports for easier review
     In general, Stryker also supports customization of other things as listed in the documentation files.
     https://stryker-mutator.io/docs/stryker-js/configuration/

5. The best use is as a PR-stage quality signal for logic-heavy code, not as a replacement for unit tests or linting. In our project, we integrated it into GitHub Actions with a dedicated workflow and kept the mutation scope narrow. Based on our run, this made the tool practical: it processed 137 mutants, killed 103, left 33 surviving, had 1 timeout, and finished in about 8 seconds. That is fast enough for review-time feedback, but broad whole-project mutation testing would likely be too expensive/noisy.

6. Stryker does not really produce “false positives” in the same way as a linter. A survived mutant usually does mean the tests did not detect some behavioral change. However, it can produce true positives that are low-priority or not very useful, especially when the changed behavior is minor or not as important. It can technically also produce false negatives at the project level if the mutation scope is intentionally narrow. The tool is valuable, but the results need human judgment and careful scoping.