In-depth user guidance within the GUI

[1j01]

I want a tutorial/guide/setup/help system built into the UI, with explanatory graphics, maybe even some animations.

Coach user on:

• Granting permissions
  • Camera access (for head tracking)
  • In desktop app on macOS, Accessibility permission (for moving the mouse)
  • I've seen some nice permission prompts
• Troubleshooting camera access
  • Another application may be using it
    • I give a friendly error message for this
    • Tell how to check for other programs using the camera (How to describe this? Well if your camera has a hardware indicator light, it should be on if the camera is in use. )
  • Try unplugging and plugging it back in
  • Make sure you can use your camera with another application (but close this application before trying to get it to work in here again)
  • Installing (and maybe running?) guvcview can magically fix a webcam not showing up (worked for my Logitech C920 when it wouldn't show up in applications even after restart, but was listed in lsusb) (source)
  • Correct camera
    • enumerateDevices
• Disabling camera autofocus maybe?
• Positioning the camera and yourself
  • Above or below the screen is fine but you should be centered so the pointer doesn't move left/right too much when you want it to go up or down
    • In particular, you should be in line with the camera, such that your face appears head-on when looking comfortably at the center of the screen
      • A guide could show your head rotation
      • Calibration for an off-center camera should be possible (or explicitly using your head rotation instead of a projected position)
  • If the camera is above, leaning forward generally moves the pointer down
  • If the camera is below, leaning forward generally moves the pointer up
• Tilting your head or moving your head both move the pointer
• Lighting
  • Recommend good front lighting (it's good to be near a window (which is in front of you, not behind), but you'll
  • Illustrate good/bad environmental setups
  • Detect bad lighting conditions and report to the user (Warn about poor lighting conditions, or no head detected #26)
• "Calibration" via simply moving your head to the edges of the screen (it's not a specially defined gesture, it's just implicit in the fact that there are boundaries)
• Choosing settings (sensitivity etc.)
  • If you move yourself or your camera, you may want to adjust the sensitivity.
  • If you're further away from the camera, you'll want a higher sensitivity.
    • Would it make sense to scale this to your head size in the camera? Maybe not with the inaccurate face tracker, but with the face tracker... but you probably wouldn't want it to switch setting scaling schemes suddenly
    • It could detect if your head size significantly changes (and is stable for a period of time) from what it has been (stably for a period of time), and alert you, suggesting changing the setting, maybe even suggesting a value