lib-classifier: refactor single image viewer with VisXZoom

[mcbouslog]

Package

• lib-classifier

Linked Issue and/or Talk Post

• closes Add mouse wheel zoom to image subject viewers #5109

Describe your changes

• create new single image subject viewer
• create storybook for new single image subject viewer
• rotate
• invert
• add VisXZoom for pan and zoom (includes mouse wheel zoom)
• drag
• drawing / InteractionLayer
• placeholder image
• check new single image subject viewer in test project
• check new single image subject viewer with other viewers that use single image viewer
• fix broken tests
• add new tests
• update related READMEs (i.e. Subject Viewer, likely others)
• review mobile/pinch handling new functionality will be investigated and potentially a separate PR

Notes

• see existing VisXZoom notes - https://github.com/zooniverse/front-end-monorepo/tree/master/packages/lib-classifier/src/components/Classifier/components/SubjectViewer#pan-and-zoom
• ZoomingScatterPlot uses VisXZoom: https://github.com/zooniverse/front-end-monorepo/blob/master/packages/lib-classifier/src/components/Classifier/components/SubjectViewer/components/ScatterPlotViewer/components/ZoomingScatterPlot/ZoomingScatterPlot.js#L206-L224

How to Review

Helpful explanations that will make your reviewer happy:

• What Zooniverse project should my reviewer use to review UX?
  • test project with single and multi image subjects, drawing task:
    • staging/lib-classifier: https://local.zooniverse.org:8080/?project=1693&workflow=3838
    • staging/app-project: https://local.zooniverse.org:3000/projects/markb-panoptes/test-project-1-mb-fem-lab
• What user actions should my reviewer step through to review this PR?
  1. load subject with image(s)
  2. pan, zoom, rotate, invert, and reset image
  3. draw marks on the image, repeat step 2 with drawn marks
• Which storybook stories should be reviewed?
  • mainly http://localhost:6006/?path=/story/subject-viewers-singleimageviewer--default, but also DataImageViewer, FlipbookViewer, ImageAndTextViewer, MultiFrameViewer, and SeparateFrames stories
  • SeparateFrames stories onKeyDown (i.e. + to zoom in) doesn't work, but it does in the linked test project, not sure why
• Are there plans for follow up PR’s to further fix this bug or develop this feature?

Checklist

PR Creator - Please cater the checklist to fit the review needed for your code changes.
PR Reviewer - Use the checklist during your review. Each point should be checkmarked or discussed before PR approval.

General

• Tests are passing locally and on Github
• Documentation is up to date and changelog has been updated if appropriate
• You can yarn panic && yarn bootstrap or docker-compose up --build and FEM works as expected
• FEM works in all major desktop browsers: Firefox, Chrome, Edge, Safari (Use Browserstack account as needed)
• FEM works in a mobile browser

General UX

Example Staging Project: i-fancy-cats

• All pages of a FEM project load: Home Page, Classify Page, and About Pages
• Can submit a classification
• Can sign-in and sign-out
• The component is accessible
  • Can be used with a screen reader BBC guide to VoiceOver
  • Can be used from the keyboard WebAIM guide to keyboard testing
  • It is passing accessibility checks in the storybook

New Feature

• The PR creator has listed user actions to use when testing the new feature
• Unit tests are included for the new feature
• A storybook story has been created or updated
  • Example of SlideTutorial component with docgen comments, and its story

Refactoring

• The PR creator has described the reason for refactoring
• The refactored component(s) continue to work as expected

[coveralls]

coverage: 75.769% (+0.1%) from 75.621%
when pulling bd4aec2 on lib-classifier_mouse-scroll-zoom
into e68e2c6 on master.

[eatyourgreens]

There's an undocumented feature, used by Seabird Watch and Penguin Watch, which you should add to your manual drawing tool tests.

When you're making a large number of marks, you don't want to be constantly switching between zoom mode and drawing mode. For this reason, you can control pan-and-zoom with the keyboard, from a focused drawing mark.

To see this in action, try placing marks, while zooming and panning with the keyboard, at either of the following links:

• https://www.zooniverse.org/projects/penguintom79/penguin-watch/classify
• https://frontend.preview.zooniverse.org/projects/penguintom79/penguin-watch/classify/workflow/21738

You should be able to mark a penguin, then zoom and pan from the keyboard while continuing to mark penguins on the zoomed picture (on crowded pictures, you really need to zoom in, so this feature isn't optional.)

Screen.Recording.2024-11-22.at.10.15.02.mov

This PR breaks that keyboard accessibility for drawn marks.

[eatyourgreens]

One last comment: on this branch, drawn marks scale up with the image when you zoom in. Drawn marks should ignore the scale transformation and stay the same size at all zoom levels (see #6066 for one attempt to fix this for stroke widths. You can also style SVG shapes so that they are independent of scale.)

[mcbouslog]

Thank you @eatyourgreens , the comments and videos are very helpful! To summarize I think you've identified 3 issues:

1. marks with drag and move handles aren't scaled and working as expected
2. keyboard pan and zoom should be enabled in annotation mode
3. marks aren't properly scaled zoomed in or out

I'll report back on progress.

[eatyourgreens]

Thank you @eatyourgreens , the comments and videos are very helpful! To summarize I think you've identified 3 issues:

• marks with drag and move handles aren't scaled and working as expected

You're welcome. For that first one, I can also reproduce it with the point tool, without drag handles. Drop a point on the image somewhere, then drag and move it while zoomed in. The pointer movement, within the subject viewer, isn't properly mapped to the image. It looks like the pointer coordinates are still scaled to the subject viewer dimensions, not the new, larger image dimensions. Dragging only works when the subject viewer and image are the same size. Could be a missing transformation somewhere on the <rect> element that acts as the drawing canvas. Maybe it isn't resizing to match the image size?

[mcbouslog]

I can recreate the noted 2 and 3 issues, but I'm having trouble recreating 1 (marks with drag and move handles aren't scaled and working as expected)? I'm using Chrome v131, but I don't think it'd be browser specific.

ZoomInAndDrawRectangle.mp4

[eatyourgreens]

Sorry, I forgot to say that my video was Firefox 132. SVG implementations do vary between browsers, unfortunately.

EDIT: I think the drawing tools are only using bits of the SVG 2 spec that are well-supported across the major browser engines.

[eatyourgreens]

I just tried a few different browsers. Scaling and dragging seems to be ok in Chrome or Safari, but broken in Firefox.

[eatyourgreens]

This might be relevant. Dragging interactions get the current transformation matrix from a svg element stored in SVGContext here:

front-end-monorepo/packages/lib-classifier/src/plugins/drawingTools/components/draggable/draggable.js

Line 51 in 1862ade

const { canvas } = useContext(SVGContext)

Creating a mark here gets the CTM from the DrawingCanvas component in the InteractionLayer component, which is an SVG rect that handles all the pointer events that create marks.

front-end-monorepo/packages/lib-classifier/src/components/Classifier/components/SubjectViewer/components/InteractionLayer/InteractionLayer.js

Line 88 in 1862ade

? svgPoint.matrixTransform(canvas.current?.getScreenCTM().inverse())

These refs don’t point to the same element, so that could account for differences in behaviour between creating a mark and dragging a mark. They probably should be using the same element and transformation matrix. I don't think I've noticed this before. At least, I can't find an issue for it, and I'm sure I would have opened an issue for something like this.

[eatyourgreens]

Update: dragging a zoomed-in mark is broken in Firefox on the production release. I'll open an issue. It's not a bug that's been introduced on this branch. The reason is that the draggable decorator is getting the screen CTM from the wrong DOM node, so not transforming pointer coordinates correctly.

• Drawing is broken for zoomed-in images in Firefox #6490

Update to the update:

• very quick fix for that broken ref, and some code cleanup, in fix(lib-classifier): canvas context for SVG drawing tools #6491.

[mcbouslog]

A brief update on a few issues previously noted:

0. classification.metadata.subject_dimensions broken, reversion of previous bug - I think fixed per c58b452 and 7eb30ef and 0a2d54f
1. marks with drag and move handles aren't scaled and working as expected - fixed with fix(lib-classifier): canvas context for SVG drawing tools #6491
2. keyboard pan and zoom should be enabled in annotation mode - fixed with 0193a44
3. marks aren't properly scaled zoomed in or out - fixed with refactor(classifier): use native browser SVG scaling for drawing tools #6066

[mcbouslog]

I think this is ready for another review 😵‍💫 !

I've been using https://local.zooniverse.org:3000/projects/markb-panoptes/test-project-1-mb-fem-lab?env=staging&demo=true to test various workflows, image sizes, and drawing tools (see workflow descriptions).