Document steps for updating prod screenshots in tests #469

mon4our · 2025-08-31T15:10:31Z

Updated the acceptance tests wiki with steps on how to update screenshots

mon4our · 2025-09-02T09:50:49Z

@seanlip PTAL, thanks!

seanlip

Thanks @mon4our for doing this! Left a few notes, PTAL.

seanlip · 2025-09-02T12:36:41Z

Acceptance-Tests.md

 On the other hand, if the screenshot fails locally (in desktop environment), the screenshot `teachPage-diff.png` will be generated and stored inside a new folder `diff-snapshots` under `logged-out-user/dev-desktop-screenshots`. 

+### Adding Prod screenshots for Acceptance Tests
+On making changes that affect a user journey tested through the acceptance tests or introduce a new feature, a contributor needs to update/add screenshots supporting their changes.


On --> When

seanlip · 2025-09-02T12:37:48Z

Acceptance-Tests.md


+### Adding Prod screenshots for Acceptance Tests
+On making changes that affect a user journey tested through the acceptance tests or introduce a new feature, a contributor needs to update/add screenshots supporting their changes.
+It has been noted that screenshots obtained by running the Acceptance tests locally using the prod flag do not match those captured by the CI. To overcome this, you can commit any image in the prod screenshot folder and follow these steps:


Drop the first sentence, replace with: "These screenshots should be obtained from the CI (not local) run, so that the environment matches future runs." Update the second sentence to follow on from that.

In the second sentence, what do you mean by "commit any image in the prod screenshot folder"?

seanlip · 2025-09-02T12:38:39Z

Acceptance-Tests.md

+### Adding Prod screenshots for Acceptance Tests
+On making changes that affect a user journey tested through the acceptance tests or introduce a new feature, a contributor needs to update/add screenshots supporting their changes.
+It has been noted that screenshots obtained by running the Acceptance tests locally using the prod flag do not match those captured by the CI. To overcome this, you can commit any image in the prod screenshot folder and follow these steps:
+1. Go to the failing run and click on the 'uploading diff screenshots as an artifact' section:


Capitalize "uploading"

"click on the" --> "open the"

seanlip · 2025-09-02T12:39:02Z

Acceptance-Tests.md

+1. Go to the failing run and click on the 'uploading diff screenshots as an artifact' section:
+  <img width="349" height="35" alt="image" src="https://github.com/user-attachments/assets/827d8557-3346-443a-aaf8-1fd0168a3f24" />
+
+2. Then download the artifact from the artifact download url:


Highlight that URL in the image.

seanlip · 2025-09-02T12:39:16Z

Acceptance-Tests.md

+2. Then download the artifact from the artifact download url:
+  <img width="1221" height="301" alt="image" src="https://github.com/user-attachments/assets/c2caa55f-b4f9-40cc-a47a-7ddc2de9a479" />
+
+3. Extract the contents of the artifact. There will be a folder with an image. The image will be like this:


be like this --> look like this

seanlip · 2025-09-02T12:42:17Z

Acceptance-Tests.md

+
+(Its curriculum-admin in this case)
+
+14. Navigate to the spec


This is unclear, please specify what you mean by "spec". How do they figure out which spec to navigate to?

seanlip · 2025-09-02T12:42:42Z

Acceptance-Tests.md

+15. Check if the failure was for desktop or mobile based on the screenshot or the subfolder in the artifact:
+<img width="102" height="127" alt="image" src="https://github.com/user-attachments/assets/c1b902ee-5ccb-4218-a247-d4e19da9d3ed" />
+
+16. Go to the prod-desktop-screenshots/prod-mobile-screenshots folder depending on your failure


Use "or" instead of "/" to avoid ambiguity, since a slash is a folder name separator too

seanlip · 2025-09-02T12:42:50Z

Acceptance-Tests.md

+
+16. Go to the prod-desktop-screenshots/prod-mobile-screenshots folder depending on your failure
+
+17. Paste the image we split.


Refer to the step number where the splitting action was taken

seanlip · 2025-09-02T12:43:57Z

Acceptance-Tests.md

+
+17. Paste the image we split.
+
+18. Check the name of the image from the artifact. It will be {screenshot_name}-diff.png


I think you can combine steps 18-21 into something along the lines of "replace the old screenshot with the new one"

seanlip · 2025-09-02T12:44:23Z

Acceptance-Tests.md

+
+20. Rename the pasted image to {screenshot_name}-snap.png
+
+21. Check that the correct image got replaced, commit and push your changes!


"Commit and push your changes" should be a separate step

Maybe at the end you can also ask them to do a self-review of the PR to verify that the correct images are used.

seanlip · 2025-09-26T06:51:38Z

@mon4our Just to check, any reason this isn't wrapped up yet? It seems like it'd be quite useful to have on the wiki.

Updated instructions for adding production screenshots in acceptance tests to clarify the process and improve readability.

mon4our · 2025-09-29T01:31:06Z

@seanlip Made the requested changes. PTAL, thanks!

seanlip

Looks great. Thanks @mon4our!

Document steps for updating prod screenshots in tests

e001b48

mon4our requested a review from imchristie August 31, 2025 15:10

mon4our assigned mon4our, imchristie and seanlip Aug 31, 2025

seanlip requested changes Sep 2, 2025

View reviewed changes

seanlip removed their assignment Sep 2, 2025

NITISH084 mentioned this pull request Sep 4, 2025

FIX #22110: Skill Editor UI enhanced for both read only mode and read write mode oppia/oppia#23133

Merged

5 tasks

Revise instructions for adding prod screenshots

39d0139

Updated instructions for adding production screenshots in acceptance tests to clarify the process and improve readability.

mon4our assigned seanlip and unassigned imchristie Sep 29, 2025

mon4our removed the request for review from imchristie September 29, 2025 01:31

seanlip approved these changes Sep 29, 2025

View reviewed changes

seanlip merged commit c5571ec into develop Sep 29, 2025
3 checks passed


		16. Go to the prod-desktop-screenshots/prod-mobile-screenshots folder depending on your failure

		17. Paste the image we split.


		17. Paste the image we split.

		18. Check the name of the image from the artifact. It will be {screenshot_name}-diff.png


		20. Rename the pasted image to {screenshot_name}-snap.png

		21. Check that the correct image got replaced, commit and push your changes!

Document steps for updating prod screenshots in tests #469

Document steps for updating prod screenshots in tests #469

Uh oh!

Conversation

mon4our commented Aug 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mon4our commented Sep 2, 2025

Uh oh!

seanlip left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

seanlip commented Sep 26, 2025

Uh oh!

mon4our commented Sep 29, 2025

Uh oh!

seanlip left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

mon4our commented Aug 31, 2025 •

edited

Loading