Setup

Begin by cloning the blockly repo and checking out the accessibility-hackathon branch of Blockly. This contains some sample code to get us started.

git clone https://github.com/google/blockly
git checkout accessibility-hackathon

Most of our changes are going to be in the demos/keyboard_nav directory.

  • In your choice of browser, open blockly/demos/keyboard_nav/index.html
  • In your choice of code editor, open the blockly repo you checked out.

Create A New Cursor

The Cursor controls how a user navigates around the different blocks, fields, inputs and stacks on the workspace. To create a new cursor you can either start by copying an existing cursor or creating a new object that inherits from Blockly.Cursor. A cursor can be set on any workspace by calling workspace.setCursor(cursor). We won't go through the entire process of creating a custom cursor here. Instead we'll modify the one in demos/keyboard_nav.

BasicCursor

The BasicCursor in demos/keyboard_nav/basic_cursor.js does a pre order traversal on the AST and only allows the user to go forward or backward through the entire tree. In the BasicCursor, the in ('D' by default) and out ('A' by default) methods just call the prev ('W' by default) and next ('S' by default) methods . If you test it out by selecting the BasicCursor from the first dropdown, you'll notice that there are two nodes for each connection between blocks. We are going to modify the BasicCursor so that we skip over these extra nodes.

First, let's add the skip behavior to the prev method.

/** @override */
Blockly.BasicCursor.prototype.prev = function() {
  var curNode = this.getCurNode();
  if (!curNode) {
    return null;
  }
  var newNode = this.getPreviousNode_(curNode);

  if (newNode) {
    // If the new node is a next/input connection and it is connected to
    // something, skip it.
    if (newNode.getType() == Blockly.ASTNode.types.NEXT
        || newNode.getType() == Blockly.ASTNode.types.INPUT) {
      if (newNode.getLocation().targetConnection) {
        // If there is no other node after newNode then stick with our current
        // node
        newNode = this.getPreviousNode_(newNode) || newNode;
      }
    }
    this.setCurNode(newNode);
  }
  return newNode;
};

Now add the same functionality to next, making sure to use getNextNode_ instead of getPreviousNode_. Test out the change by refreshing your browser page and choosing BasicCursor from the dropdown again.

There are lots of other ways to tweak the cursor. Here are a few ideas to get you started!

  • Create a cursor that traverses all inline inputs in a line.
  • Create a cursor that traverses blocks as if they were lines of text (prev and next move between statements, in and out move along a row of inputs/fields).
  • Navigates by block with 'W' and 'S' and by field with 'A' and 'D'.
  • Create a cursor that will center the workspace on the current block as the cursor moves.

Key Mappings

Change the Key Mapping

Let's change disconnecting two blocks on a workspace from using the letter ‘X’ to using ‘Shift X’. This can be added to the index.html file found under blockly/demos/keyboard_nav/index.html.

// Create a serialized version of shift + 'x'
var key =  Blockly.user.keyMap.createSerializedKey(
    Blockly.utils.KeyCodes.X,[Blockly.user.keyMap.modifierKeys.SHIFT]);
// Change the mapping for the disconnect action
Blockly.user.keyMap.setActionForKey(key, Blockly.navigation.ACTION_DISCONNECT);

Add A New Action

Next let's add a collapse action to the key map. Start by defining a new action and adding it to the keymap.

// Create a new action for collapsing
var collapseAction = new Blockly.Action('Collapse', 'Toggle collapse on the current block.');
// Add it to the keymap using shift + 'c'
var key =  Blockly.user.keyMap.createSerializedKey(
    Blockly.utils.KeyCodes.C,[Blockly.user.keyMap.modifierKeys.SHIFT]);
Blockly.user.keyMap.setActionForKey(key, collapseAction);


Once we have added our action to the key map we then need to decide what to do when the user hits that key. For this we are going to need to modify the navigation.js file. In the workspaceOnAction method we are going to add a case to the switch statement for the collapsed action: 

  case 'Collapse':
    var curNode = Blockly.getMainWorkspace().getCursor().getCurNode();
    var block = Blockly.navigation.getSourceBlock_(curNode);
    if (block) {
      block.setCollapsed(!block.isCollapsed());
    }
    return true;

Note: In the future this will likely become an API for registering actions on a workspace.

Other ideas:

  • Collapse all the blocks on the workspace, instead of just collapsing a single block.
  • Create an action for going to the top of the stack of blocks.
  • Create an action for switching between stacks on the workspace.
  • Add a way to delete the block the cursor is currently on.

Add a Key Mapping Menu

Everyone has different setups that work for them, so it is really important to give users the ability to change the key mappings to work with their current setup. The page in demos/keyboard_nav/index.html has some basic UI to change the mapping, but it's pretty basic.

Other ideas:

  • Add a tab stop (example if you go to google calendar and hit tab a menu appears).
  • Create an accessible dialog that will capture keystrokes and allow users to choose which action goes with what keystroke (good for multiple languages).

Add Audio Feedback

By default, all navigation-related logs, warnings, and errors are printed to the console. You can override this to provide custom behavior by defining Blockly.Navigation.loggingCallback. This function will be called in Blockly.Navigation.log, Blockly.Navigation.warn, and Blockly.Navigation.error and passed a string with either ‘log’, ‘warn’, or ‘error’ and the log message.

Let's add updates to an aria live region for all logs while using keyboard navigation.

  Blockly.navigation.loggingCallback = function(type_str, msg) {
    var loggingElement = document.getElementById('loggingFeedback');
    if (type_str === 'log') {
      // Log the message.
      loggingElement.innerText = msg;
    } else if (type_str === 'warn') {
      // Do something to warn the user.
      loggingElement.innerText = msg;
    } else if (type_str === 'error') {
      // Do something to tell the user there was an error.
      loggingElement.innerText = msg;
    }
  }

Other ideas:

  • Have different sounds play when a user encounters an error vs a warning.
  • Only read out warnings or errors.
  • Create some kind of visual effect when an error occurs (block fades, cursor flashes, etc).