Golden Tests Examples with Flutter

Languages: English · Português

Sample project demonstrating how to write and automate golden tests (visual snapshot tests) in Flutter, using the Alchemist package and continuous integration (GitHub Actions, Codemagic, and similar).

What is a golden test?

A golden is a reference image (a PNG snapshot) of how a widget or screen looks at a given moment. Think of it as the “approved screenshot” of your UI layout.

When you run a golden test:

Flutter renders the widget in a test environment (no real device required).
The test captures that render as an image.
The image is compared pixel by pixel with the golden file already stored in the repo (test/goldens/ci/).
If the UI changed and the diff exceeds the allowed threshold, the test fails — so accidental layout, color, or structure regressions are caught in CI.
If you intentionally changed the UI, you regenerate the goldens with flutter test --update-goldens, review the new PNGs, and commit them.

Golden tests are visual regression tests: they do not replace logic or interaction tests, but they complement them by guarding what users actually see.

Platform goldens vs CI goldens (Alchemist)

Type	Where	Purpose
CI goldens	`test/goldens/ci/`	Stable across macOS, Linux, and Windows; text is obscured so fonts do not break CI. These are committed to Git.
Platform goldens	`test/goldens/<platform>/`	Readable previews on your machine for debugging. Not committed (output varies by OS).

Locally, both can be generated. In CI (CI=true, e.g. Codemagic), only CI goldens run.

What this repository includes

A minimal Flutter app with a “Hello, world!” screen
Alchemist golden tests in test/golden_test.dart
Reference images under test/goldens/ci/
A GitHub Actions workflow for Alchemist CI golden tests on Ubuntu

Project structure

lib/
  main.dart                    # Sample app (MyHomePage)
test/
  flutter_test_config.dart     # Alchemist config (CI vs local)
  golden_test.dart             # Golden tests
  goldens/
    ci/                        # CI reference images (versioned)
.github/workflows/
  golden_tests.yml             # Pipeline on GitHub Actions

Prerequisites

Flutter SDK
Git (to version CI golden images)

Running tests

Install dependencies:

flutter pub get

Run all tests:

flutter test

Run only golden tests (Alchemist tags them with golden):

flutter test --tags golden

Run only the golden test file:

flutter test test/golden_test.dart

Simulate CI locally (only CI goldens, as on Codemagic):

flutter test --dart-define=CI=true

More readable output in the terminal (lists each test and variant):

flutter test test --reporter expanded

Running tests in Android Studio

Open the project root folder (the one that contains pubspec.yaml), not only android/ or lib/.

Quick run

Open test/golden_test.dart.
Click Run (▶) next to void main() or the group('Golden Tests') line,
or right-click the test folder → Run 'tests in test.dart'.

Running the whole test folder is valid. With Alchemist you should see 4 tests passed (2 golden tests × 2 variants: Windows + CI).

Run Configuration (recommended)

The default run button does not pass --update-goldens or a verbose reporter. For clearer runs, create dedicated configurations:

Run → Edit Configurations… → + → Flutter Test

Configuration	Test file	Additional run args
Golden tests	`test/golden_test.dart`	`--reporter expanded`
Golden tests (update)	`test/golden_test.dart`	`--update-goldens --reporter expanded`
Golden tests (CI only)	`test/golden_test.dart`	`--dart-define=CI=true --tags golden --reporter expanded`

Use Golden tests day to day (compare only). Use Golden tests (update) after intentional UI changes. Use Golden tests (CI only) to mimic Codemagic locally.

To update goldens from the IDE without a custom configuration, use the Terminal tab:

flutter test test/golden_test.dart --update-goldens --reporter expanded

Updating golden images

When you intentionally change the UI:

flutter test --update-goldens

Review the new files under test/goldens/ci/ before committing. Do not pass --update-goldens in CI pipelines — that would always refresh references and hide real regressions.

Quick checklist

Goal	What to do
Validate UI against goldens	Run ▶ on `test/` or `test/golden_test.dart` — expect 4 tests passed (2 tests × Windows + CI).
See each test name in the log	`flutter test test --reporter expanded` or a Flutter Test config with that flag.
Update images after a UI change	`flutter test test/golden_test.dart --update-goldens` (or Golden tests (update) run configuration).
Inspect PNG files	Open `test/goldens/ci/` (committed) and `test/goldens/<platform>/` (e.g. `windows/` — local only).
Verify failures are detected	Follow How to force a real failure (quick test), then update goldens if the change was intentional.
Match Codemagic / CI locally	`flutter test --dart-define=CI=true --tags golden` (only CI variants, 2 tests).

How to force a real failure (quick test)

Use this to confirm that golden tests actually detect UI changes. Run without --update-goldens.

Changing only MyApp (e.g. primarySwatch: Colors.red) usually does not fail the test, because test/golden_test.dart builds its own MaterialApp and does not pump MyApp. See I changed main.dart but tests still pass below.

Option 1 — change text in `MyHomePage` (`lib/main.dart`)

Affects the MyHomePage golden only.

const Text('Hello, mundo!'),  // was 'Hello, world!'

Option 2 — change text in the test (`test/golden_test.dart`)

Affects the Hello World golden only.

child: Text('Hello, mundo!'),  // was 'Hello, world!'

Option 3 — change theme in the test (not in `MyApp`)

Affects the MyHomePage golden (e.g. red AppBar).

child: MaterialApp(
  theme: ThemeData(primarySwatch: Colors.red),
  home: const MyHomePage(title: 'Golden Test Example'),
),

Run and what to expect

flutter test test/golden_test.dart --reporter expanded

Or use your Golden tests run configuration in Android Studio (without --update-goldens).

Tests should fail with a golden mismatch.
Review diffs under test/failures/ (ignored by Git).
Restore the code or run flutter test test/golden_test.dart --update-goldens after an intentional UI change, then commit updated files in test/goldens/ci/.

Troubleshooting

“Nothing happened” when I ran the tests

If the Run tool window only shows a green Test Results line (or “4 tests passed”) and no obvious file changes, the tests may still have run successfully. Golden tests compare existing PNGs by default; they do not always create new files or open a visual diff when everything passes.

What you see	What it usually means
4 tests passed in ~1–2 s	Normal for this repo: Hello World + MyHomePage, each with Windows and CI variants.
Collapsed tree, no test names	Android Studio summary view. Use `--reporter expanded` or expand the run log / use a Flutter Test configuration on `golden_test.dart`.
`goldens/` looks empty in the Project view	Expand subfolders: `test/goldens/ci/` (versioned) and `test/goldens/<platform>/` (e.g. `windows/` — local only, often not in Git).
No new PNGs after Run ▶	Expected without `--update-goldens`. Run only validates against committed CI goldens.
No `failures/` folder	Expected when tests pass. Diffs are written under `test/failures/` mainly when a golden fails.

Confirm in the terminal:

flutter test test --reporter expanded

Expected lines:

Golden Tests renders Hello World layout (variant: Windows)
Golden Tests renders Hello World layout (variant: CI)
Golden Tests renders MyHomePage layout (variant: Windows)
Golden Tests renders MyHomePage layout (variant: CI)
All tests passed!

To prove the pipeline end-to-end, follow How to force a real failure (quick test).

I changed `main.dart` (e.g. `Colors.red` in `MyApp`) but tests still pass

The golden file is compared against the widget tree built inside test/golden_test.dart, not against runApp(MyApp()).

Change in `main.dart`	Affects golden tests?
`MyApp` → `theme: ThemeData(primarySwatch: Colors.red)`	No — tests use their own `MaterialApp` without that theme.
`MyHomePage` → text, `AppBar` color via `Theme.of(context)`, layout	Partially — only the `MyHomePage` scenario; not the Hello World scenario.
`main()` / `runApp`	No — never executed in widget tests.

See How to force a real failure (quick test) for step-by-step examples.

To test the real app shell (MyApp + theme), wrap the scenario with const MyApp() instead of a bare MaterialApp in golden_test.dart.

First time / missing golden files

If CI golden PNGs are missing from test/goldens/ci/, generate them once:

flutter pub get
flutter test test/golden_test.dart --update-goldens

Commit the files under test/goldens/ci/.

Tests fail after pulling the repo

Someone may have updated UI and CI goldens. Either align your code with the new images or, if your UI change is correct, regenerate locally and commit updated test/goldens/ci/ after review.

IDE shows errors on `package:flutter/material.dart`

That is an analyzer/SDK setup issue, not Alchemist. Run flutter pub get, point the IDE Flutter SDK to your install (e.g. C:\Android\flutter-sdk), and restart the Dart Analysis Server.

What the tests do

In test/golden_test.dart:

Hello World — MaterialApp with centered “Hello, world!” text.
MyHomePage — the sample screen from lib/main.dart (AppBar + centered text).

Each scenario is declared with GoldenTestScenario inside a GoldenTestGroup and executed via goldenTest from Alchemist.

CI/CD

GitHub Actions

The workflow .github/workflows/golden_tests.yml runs on Ubuntu with Flutter stable (Dart ≥ 3.8, required by Alchemist) on pushes and pull requests to master:

flutter pub get
flutter test --tags golden --dart-define=CI=true

flutter_test_config.dart also reads the CI environment variable when present; the workflow sets both for compatibility.

Commit the reference PNGs under test/goldens/ci/ so CI can compare them. Update goldens locally with --update-goldens; do not regenerate them in CI on every push.

Codemagic

Codemagic sets CI=true automatically. Use a workflow that runs:

flutter pub get
flutter test --tags golden

CI goldens are comparable on Linux or macOS builders. You do not need a Mac-only machine for Alchemist CI goldens (unlike raw matchesGoldenFile with real text).

Example codemagic.yaml script section:

scripts:
  - flutter pub get
  - flutter test --tags golden

Firebase Test Lab

Alchemist golden tests are not meant to run on Firebase Test Lab.

	Alchemist / `flutter test` goldens	Firebase Test Lab
Where it runs	Host machine or CI (VM)	Real devices / emulators in Google’s cloud
Test type	Widget tests (`flutter test`)	Instrumentation, Robo, or integration tests on device
Comparison	PNG golden files in the repo	Screenshots, performance, crashes — different tooling

Use Alchemist on Codemagic, GitHub Actions, etc. for visual regression of widgets. Use Firebase Test Lab for running the app on many devices (integration tests, smoke tests, Robo). You can use both in the same project for different purposes, but not the same Alchemist golden pipeline on FTL.

Relevant dependencies

Package	Purpose
`flutter_test`	Flutter test framework
alchemist	Golden tests with CI-stable and platform-readable modes

Sample app

The app in lib/main.dart shows an AppBar titled “Golden Test Example” and “Hello, world!” centered on screen.

flutter run

References

License

Sample project for educational purposes. Adapt and reuse as needed.

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
android		android
ios		ios
lib		lib
test		test
web		web
.gitignore		.gitignore
.metadata		.metadata
README.md		README.md
README.pt.md		README.pt.md
analysis_options.yaml		analysis_options.yaml
dart_test.yaml		dart_test.yaml
pubspec.lock		pubspec.lock
pubspec.yaml		pubspec.yaml

Latest commit

History

Folders and files

Repository files navigation

Golden Tests Examples with Flutter

What is a golden test?

Platform goldens vs CI goldens (Alchemist)

What this repository includes

Project structure

Prerequisites

Running tests

Running tests in Android Studio

Quick run

Run Configuration (recommended)

Updating golden images

Quick checklist

How to force a real failure (quick test)

Option 1 — change text in MyHomePage (lib/main.dart)

Option 2 — change text in the test (test/golden_test.dart)

Option 3 — change theme in the test (not in MyApp)

Run and what to expect

Troubleshooting

“Nothing happened” when I ran the tests

I changed main.dart (e.g. Colors.red in MyApp) but tests still pass

First time / missing golden files

Tests fail after pulling the repo

IDE shows errors on package:flutter/material.dart

What the tests do

CI/CD

GitHub Actions

Codemagic

Firebase Test Lab

Relevant dependencies

Sample app

References

License

About

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages

Option 1 — change text in `MyHomePage` (`lib/main.dart`)

Option 2 — change text in the test (`test/golden_test.dart`)

Option 3 — change theme in the test (not in `MyApp`)

I changed `main.dart` (e.g. `Colors.red` in `MyApp`) but tests still pass

IDE shows errors on `package:flutter/material.dart`