chore: setup Loki for visual regression testing

[Anxhul10]

Mentor(s) :

• Douglas Fabris
• Guilherme Gazzo

Mentee:

• Anshul Ivan Ekka

Project Overview

Visual regression testing is used to detect visual bugs caused by styling issues, misalignments, font rendering, or dependency updates. Common problems include overlapping modules, missing or hidden elements, or components rendering outside the visible area.

This project reintroduces visual testing for Fuselage with two execution modes:

• Offline mode – run tests locally against stored baseline snapshots.

• Online mode – run tests on a remote environment (e.g., CI/CD).

Additionally, it implements a smart testing mode that selectively runs visual tests only for affected components, rather than executing the entire suite each time. This significantly improves efficiency while maintaining reliability in catching visual regressions early in the development cycle.

Real-Life Example: Why This Project is Needed

When Fuselage was running on Storybook 8, the visuals were stable. However, after upgrading to Storybook 9, several visual changes appeared.

The pull request (PR) that introduced the Storybook upgrade passed the CI/CD pipeline even though it caused visual regressions — because no visual tests were in place. As a result, the PR was merged, and visual issues made it into the main branch.

Examples of issues after the Storybook upgrade:

• #1697 – Regression in component visuals
• #1695 – Unexpected style changes after upgrade

This highlights the importance of visual regression testing — it ensures that even if functional tests pass, unintended visual changes are caught before merging.

Usage Guide

1. Offline Mode

Run the following command at the root of the repository to generate and compare visual snapshots locally:

yarn fuselageSnap

In the Storybook setup containing packages such as fuselage, fuselage-toastbar, onboarding-ui, and layout, the following scripts are available:

• update-storybook
  Builds Storybook in the current package where the command is executed.
  After building, it removes old refs for that package and creates new refs in .loki/refs.

• loki:test-ci
  Used in the Docker action to compare refs and diffs visually during CI.

• loki:test
  Runs visual tests for the current package locally inside Docker.
  This script uses the flag:

  --chromeDockerImage=zenika/alpine-chrome
  

2. Online Mode

Push your changes to GitHub.
This will automatically trigger visual regression tests for the affected packages and components in the CI/CD pipeline.

Sequence Diagram: Loki Visual Regression Testing Workflow

Related Pull Requests

• #42 – Added logic for fuselageSnap.
• #43 – Function to trim the stats file for easier readability.
• #49 – Trigger full tests when running fuselageSnap.
• #66 – Initialized Docker with zenika/alpine-chrome:with-node.
• #75 – Used fuselageSnap in Docker action to ensure consistency.
• #80 – Added local fuselageSnap initialization at repo root.
• #84 – Added transitive dependency support and visual tests triggered on SCSS file changes.

What Can Be Improved Further?

• Refactoring certain functions to improve maintainability and readability.
• Stabilizing some components to ensure consistent and reliable visual tests.

Expansion Possibilities

• Extending visual regression testing to the Rocket.Chat repository.
• Integrating the smart testing feature so that only affected components are tested, reducing CI time and resources.

Acknowledgements

I would like to thank my mentors and the Rocket.Chat community for giving me the opportunity to work on this awesome project.
This experience pushed me out of my comfort zone, encouraged me to read and understand documentation deeply, and taught me how to approach real-world open source challenges.

It truly made my summer an amazing journey of learning and growth.

[changeset-bot]

⚠️ No Changeset found

Latest commit: d409701

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[Anxhul10]

Loki Test Errors

I've encountered several issues while running yarn loki test, primarily due to dynamic or non-visible components in the Storybook canvas. Here's a breakdown of the problems and actions taken:

---

• ### Skipped: Layout/Box (Invisible Component)
• Location: packages/fuselage → LAYOUT → Box
• Issue: This component is not visible on the Storybook canvas.
• Cause: Loki retries multiple times to capture it and eventually fails.
• Action: Skipped this story in the Loki test configuration.

---

• ### skipped Needs Fix: Navigation/Dropdown
• Location: packages/fuselage → Navigation → Dropdown
• Issue: Dropdown has ~20 examples, but on canvas, only items 3 to 20 or 1.5 to 20 are visible, depending on render state.
• Impact: Loki captures inconsistent visuals on every run.
• Suggested Fix:
  • Limit the number of examples.

---

• ### Needs Fix: Navigation/SidebarV2
• Location: packages/fuselage → Navigation → SidebarV2
• Issue: The notification count changes on each refresh.
• Impact: Leads to inconsistent snapshots and test failures.
• Suggested Fix: Stabilize the data (mock/freeze values) for test consistency.

---

• ### Needs Fix: pages/LoaderPage
• Location: packages/onboarding-ui → pages → LoaderPage
• Issue: Loader animation is still running when Loki takes the snapshot.
• Impact: Causes different visual results on each run.
• Suggested Fix:
  • Mock or disable animation during test or
  • Use a delay before capture.

---

• ### Needs Fix: AudioPlayer
• Location: packages/fuselage → pages
• Issue: getting 1 request fails to load it works fine when the test is on the one screen size . If tested on multiple viewports the error arises. Another problem faced is the preloaded timestamp sometimes it is 0:0 and sometimes it is 11:15.
• Suggested Fix:
  • To solve this delay can be used l before each snap shot.
  • ensure on each refresh the timestamp is at start

---

• ### Needs Fix: ToastBar not captured in Loki snapshot
• Location: packages/fuselage-toastbar
• Issue: Only the triggering button is captured in the Loki snapshot — the actual toast notification is missing. This happens even with persistent toasts. The ToastBar requires interaction (e.g., a button click) to trigger the toast, which Loki does not simulate by default.
• Suggested Fix:
  • Update .loki/config.js with chromeSelector: 'body > *' to ensure Loki captures portal-rendered toasts.

---

• ### Needs Fix: : Tooltip not captured in Loki snapshot

Location: packages/layout

Issue:
The tooltip in Components/TooltipWrapper is not captured in the Loki snapshot. This happens because the tooltip only appears on hover, which Loki does not simulate during snapshot tests. As a result, only the base element (e.g., button or icon) is visible, while the tooltip message is missing from the snapshot.

** Suggested Fix:**

• use isLokiRunning conditinaly to show the tooltip without hover

** Notes: **

• Loki does not trigger hover states by default.
• Without this fix, visual regression tests will fail to capture tooltip UI regressions.

[gabriellsh]

We had Loki for visual regression before and we removed it. I don't remember clearly why, but I don't think we're planning on re-introducing it.

[Anxhul10]

We had Loki for visual regression before and we removed it. I don't remember clearly why, but I don't think we're planning on re-introducing it.

rocket chat fuselage use to have loki in the past be but started to get some issues because of apple silicon discrepancies when taking screenshots from macOS

But I noticed that Rocket.Chat main repo doesn't used loki in the past .

but isn't this is gsoc 2025 project list that has been selected , @dougfabris and @ggazzo is my mentor.

currently this is wip.

[dougfabris]

@Anxhul10 don't worry, he wasn't aware about the project 😆

[Anxhul10]

@Anxhul10 don't worry, he wasn't aware about the project 😆

I got little heart attack right now ...
Thanks for the clarification mentor