Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Keyboard Navigation #3193

Open
wants to merge 44 commits into
base: master
Choose a base branch
from
Open

Keyboard Navigation #3193

wants to merge 44 commits into from
+1,059 −133

Conversation

Venkata-Sai-Vishwanath-robo
Copy link
Contributor

@Venkata-Sai-Vishwanath-robo Venkata-Sai-Vishwanath-robo commented Jul 7, 2024 •
edited
Loading

This PR improves the keyboard accessibility of the App Inventor by introducing the following features:

  • Navigation in the project explorer: this enables the user to open projects, expand or collapse folders, and sort projects by name or date, using the keyboard.
  • Navigation in dropdown menus: this allows the user to open and close the dropdown with the help of the enter key and navigate the dropdown items using the arrow keys.
  • Navigation in Dialog boxes: this allows the user to cycle through the widgets inside a dialog box without letting the focus escape out of the dialog.
  • Component addition: this allows the user to add a component to the app's workspace from the component palette by pressing the enter key and also allows the user to add components into a specific arrangement they can by having the required arrangement selected in the Source Structure Explorer. The same function could also be done by double-clicking the palette item.
  • Accessing property editors with keyboard: this allows the user to open the popup attached to the property editor by pressing the enter key and the options in the popup are accessible with tab and arrow keys, upon closing the focus remains at the same spot.
  • The visibility of UI elements when they are focused has been enhanced.
  • Moving Components: this allows the user to move the selected component by pressing the alt + ↓/↑. This feature is not perfect and will require some changes.
  • Keyboard shortcuts have also been added which are listed below.

Keyboard Shortcuts

Action Key Combination
Focus component search box /
Focus Components tree T
Focus Hidden components checkbox V
Focus Properties Panel P
Switch between Designer and Block editor Ctrl + Alt
Rename Component Alt + N
Reset Connection Alt + Shift + R
Refresh Companion Screen Alt + R
Open/Close Backpack Alt + B
Switch between Inline and External Inputs Alt + I
Add comment to a block Alt + K
Expand/Collapse block Alt + O
Navigate up the blocks Alt + S
Navigate down the blocks Alt + W
Zoom in Alt + +
Zoom out Alt + -
Re-center Alt + G
Open Shortcuts Dialog Alt + ?

@AppInventorWorkerBee
Copy link
Collaborator

Can one of the admins verify this patch?

@Venkata-Sai-Vishwanath-robo Venkata-Sai-Vishwanath-robo marked this pull request as ready for review July 8, 2024 16:53
@SusanRatiLane SusanRatiLane self-assigned this Jul 9, 2024
@SusanRatiLane
Copy link
Contributor

Issues from testing Sep 4, using Mac Firefox:

  1. I was unable to get the backpack to open using the shortcut.
  2. Highlighting of trees is not consistent in Neo. Often, only the bottom border of the tree is highlighted when the tree has focus

SusanRatiLane
SusanRatiLane reviewed Dec 5, 2024
View reviewed changes
appinventor/appengine/src/com/google/appinventor/client/editor/youngandroid/YaFormEditor.java
@@ -939,6 +945,88 @@ public void onResetDatabase() {
}
}


private void registerKeyDownListeners () {
RootPanel.get().addDomHandler(new KeyDownHandler() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice to add some comments here summarizing what you're doing with the KeyDownHandler, e.g. what keystrokes you're checking for on what parts of the UI. It's easy for that to get lost in the wall of code.

SusanRatiLane
SusanRatiLane requested changes Dec 5, 2024
View reviewed changes
Copy link
Contributor

@SusanRatiLane SusanRatiLane left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs a few updates to line up with our ui refactor.

SusanRatiLane
SusanRatiLane requested changes Dec 18, 2024
View reviewed changes
appinventor/appengine/src/com/google/appinventor/client/actions/ShowShortcutsAction.java Outdated Show resolved Hide resolved
appinventor/appengine/src/com/google/appinventor/client/actions/ShowShortcutsAction.java
db.hide();
private native void shortcutKeyHandler(ShowShortcutsAction action) /*-{
$wnd.document.addEventListener("keydown", function (event) {
if (event.altKey && event.key === "?") {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change seems to have broken Alt-? shortcut for Macs. I'm pretty sure it worked before. I may turn up more information in further testing.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More insight into the Alt - ? on Mac: It looks like if I use either Alt-/ or Alt-shift-/ (i.e. ?), the / is the only key that is being taken seriously. My focus is moving to the component search bar rather than opening the help popup.

Is there any chance that there are circumstances where / is being checked before this check for Alt-/ or Alt-shift-/

(I will avoid adding a question mark to my sentence because this comment is confusing enough.)

appinventor/appengine/src/com/google/appinventor/client/explorer/SourceStructureExplorer.java Outdated Show resolved Hide resolved
appinventor/appengine/src/com/google/appinventor/client/OdeMessages.java Show resolved Hide resolved
appinventor/appengine/src/com/google/appinventor/client/actions/ShowShortcutsAction.java Show resolved Hide resolved
appinventor/appengine/war/static/css/Ya.css
@@ -834,6 +843,10 @@ div.StatusPanel {
background-color: #d2e0a6;
}

.gwt-Tree-focused {
outline: solid 2px #000000;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be changed to a color variable

SusanRatiLane
SusanRatiLane requested changes Dec 19, 2024
View reviewed changes
Copy link
Contributor

@SusanRatiLane SusanRatiLane left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is very close. I may be able to get it on our test server before the holidays. I can't quite promise that yet, but hopefully.

appinventor/appengine/src/com/google/appinventor/client/OdeMessages.java Show resolved Hide resolved
appinventor/appengine/src/com/google/appinventor/client/OdeMessages.java Outdated Show resolved Hide resolved
appinventor/appengine/src/com/google/appinventor/client/OdeMessages.java Show resolved Hide resolved
appinventor/appengine/src/com/google/appinventor/client/OdeMessages.java
"<tr><td>Rename Component</td><td>Alt + N</td></tr>" +
"<tr><td>Reset Connection</td><td>Alt + Shift + R</td></tr>" +
"<tr><td>Refresh Companion Screen</td><td>Alt + R</td></tr>" +
"<tr><td>Open/Close Backpack</td><td>Alt + B</td></tr>" +
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alt(Opt)-B also does not seem to work for me on either Firefox or Chrome. I'm still not sure what makes these key combinations distinctive.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Alt + B shortcut was implemented in a way so that it works only when the user clicks on the blockly workspace to make it active, this was done because if the shortcut was pressed in the designer view then the backpack flyout opens but in a weird way where the blocks are outside of the flyout.

@ewpatton ewpatton added the GSOC Changes made as part of GSOC label Jan 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ewpatton ewpatton ewpatton left review comments

@SusanRatiLane SusanRatiLane SusanRatiLane requested changes

Assignees

@SusanRatiLane SusanRatiLane

Labels
affects: master GSOC Changes made as part of GSOC pull request: needs reply to review
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@Venkata-Sai-Vishwanath-robo @AppInventorWorkerBee @SusanRatiLane @ewpatton