Grid-Coding: An Accessible, Efficient, and Structured Coding Paradigm for Blind and Low-Vision Programmers

Md Ehtesham-Ul-Haque, Syed Mostofa Monsur, and Syed Masum Billah

The ACM Symposium on User Interface Software and Technology (UIST)

PDF / 30s preview / Video / Talk / GitHub / Live Demo

Challenges Blind and Low-Vision (BLV) Programmers Face

Text editors' unstructured code representation and unrestricted editing are not well-suited for BLV programmers. Without structure, BLV programmers find it difficult to determine where they are in the code, which leads to editing code at incorrect locations.

Design Goal: Represent code using a familiar structure to support easier understanding and error-free editing.

Whitespace Characters (e.g., SPACE, TAB) are meaningless for BLV programmers. They have to manually count them to determine the current scope or level in the code, which is slow, tedious, and error-prone.

Design Goal: Represent whitespace characters meaningfully.

BLV programmers need to move their cursor to a different part of the code frequently to seek contextual information. During this process, they lose their cursor position and have difficulty returning to their editing location.

Design Goal: Provide contextual information within the line to prevent frequent movement.

How Grid-Coding Functions to Overcomes the Challenges

Structured Representation

Grid-Coding represents code using a grid or a spreadsheet to structure. Each row represents a code statement, and each column represents a code level. With the rows and columns, the code is converted to a 2D space with a finite number of cells, allowing BLV programmers to pinpoint the location of their editing.

Meaningful Cell Types

The cells in the grid are categorized into three distinct types, as shown on the right.

The Statement Cells contain the editable statements in each row, indicated by an input box within the cell.
The Indentation Cells represent the visual indentations that always appear before the Statement Cells in a row. We add meaning to these cells by providing information such as within if and within for so that they are no longer mere visual gaps to separate the levels in the code. We also add bright colors to these cells so low-vision users can easily distinguish them.
The Padding Cells are the whitespaces that appear after the Statement Cells in a row. They ensure the grid is uniform and facilitate navigation. Unlike Indentations Cells, these cells contain no text, further differentiating these two types of whitespaces.

Inline Context

With the help of Indentation cells, Grid-Coding encodes the context of a statement within the row. As shown in the left figure, a BLV user can find out the context of the print statement at line 12 just by checking the indentation cells in the same row, instead of moving the cursor upwards to find it out. As a result, losing cursor position and returning to editing location is less likely.

Two Distinct Modes

Grid-Coding supports two distinct modes - (a) Navigation and (b) Edit mode. In Navigation mode, users can navigate the whole grid with four array keys, inspect the overall structure, find context, and identify appropriate code levels. Users can go to Edit mode by pressing Enter on a valid statement cell to write their code. They do not deal with whitespaces and only focus on the language-specific syntax in the statement cell, helping them to manage indentation in the code.

Designing Grid-Coding: Iterative Prototyping + Participatory Design

Approach

Collected initial design guidelines from prior work
Built a web-based prototype of Grid-Coding using HTML, CSS, and JavaScript following WCAG guidelines and ARIA authoring practices
Recruited 28 BLV participants through mailing lists
Shared the URL with the participants to use the prototype for daily coding tasks and provide feedback
Iteratively refined the prototype based on the feedback for 2 months

Iterative Refinement: Key Design Changes

1. Removing header row

The initial prototype contained headers in the first row (e.g., line number, level 1, level 2, …)
Screen readers consider the header row as row 1, code starts from row 2
Participants suggested removing the header as the line numbers mismatch between the text and the grid editor

2. Manually writing block statements

The initial design only allowed writing block statements with the auto-complete option to prevent errors
Participants preferred writing block statements manually and as naturally as possible

Writing an if block with an auto-completion feature. (a) Typing ‘i’ at row 2 provides a list of suggestions for auto-completion including the option, if block. (b) Selecting the if block from the list and pressing Enter creates a skeleton for the if block. The input box at row 2 for writing the condition of the if block is focused. (c) Writing the condition of the if block at row 2. (d) After pressing Enter, the focus is switched to the Statement cell for writing the body at row 3. The Indentation cell is modified to contain the condition. (e) Writing inside the body of the if block at row 3.

Writing an if block manually. (a) Writing the condition of the if block at row 2. (b) After pressing Enter, the focus is switched to the Statement cell for writing the body at row 3. The Indentation cell is modifed to contain the condition. (c) Writing inside the body of the if block at row 3

3. Changing Indentation Levels

Initial design assigned Tab and Shift + Tab to change indentation levels
Participants found these shortcuts to be inconsistent with Excel
They preferred going to navigation mode and checking for indentation cells to make an informed decision to change indentation

Going out of a block. (a) Creating a new row inside the block. (b) Pressing Esc to go to Navigation Mode. (c) Finding the Indentation cell where the new level will start. (d) Pressing Enter to replace the Indentation cell with a Statement cell.

Modified Error Message

Participants were confused by the original error message from the Python Compiler
The visual indicator was confusing and participants misunderstood it
We rearranged the error message in two lines

Modified error message. a) Original error output from a Python interpreter. The output contains redundant information such as a visual indicator at line 3. (b) Modified error output in Grid Editor. Only useful information is provided to users in a consistent format, such as the error message, the line number of error, and the error content.

Final Usability Evaluation: Within-subjects Study

Recruited 12 BLV participants
Two conditions - (C1) Text Editor and (C2) Grid Editor
Three Measured Tasks:
- T1: Code reading and context understanding
- T2: Error Correction
- T3: Code writing
Performance Metrics
- Completion Time (all tasks)
- Comprehension Accuracy (T1)
- Number of Errors (T3)
Data Analysis
- Transcribed and analyzed the video recordings of participants to investigate programming behavior
- Observed approaches and strategies to solve each task
- Noticed keyboard use by following their cursor movements and listening to screen readers’ audio to determine navigation strategies
- Analyzed participants’ comments while performing the tasks to understand their reaction towards the editors

Quantitative Analysis

We employed non-parametric bootstrapping with R = 1000 iterations. We also reported the mean difference as a sample effect size and Cohen’s d as a standardized measure of effect size.

Performance of T1: Completion Time and Accuracy

Completion Time and Accuracy of T1. On the left, X-axis indicate the task completion time in seconds, the less the better. The center dot indicates the mean or average completion time and the two wings indicate the 95% confident interval. From this graph, a participant took 80 seconds on average compared to 175 second of the baseline. In other words, participants were 54% faster in Grid coding, and which had a large effect. On the right, X-axis indicates the task accuracy in percentage, the more the better. Participants were 100% accurate in grid editor compared to 64% of the baseline. This result also had a large effect.

Performance of T3: Number of Errors

Number of Indentation and Non-indentation errors of T3. On the left, X-axis indicates the number of indentation errors per block, less is better. Participants made 0.92 indentation errors per block in text editor. In grid editor, this the number is 0.09, which is almost close to 0 (90% fewer than the baseline). This result had a large effect. Similarly, for non-indentation errors on the right, the number is 0.29 with grid editor compared to 0.97 with the text editor. So, grid editor had 70% fewer non-indentation errors per block, and the effect was large.

Qualitative Analysis

We performed a thematic analysis with initial coding to analyze the qualitative data. While performing certain tasks, we observed remarkably similar

and novel navigation patterns in Grid-Coding. Some key observations are shown below.

Three cell types aid code navigation

Up/Down on a Statement cell
Right on an Indentation cell (statement at higher level)
Left on a Padding cell (statement at a lower level)
Participants quickly skipped Indentation and Padding cells

Grid allowed multiple navigation strategies

(purple) Navigating only the statements of a particular level, such as level 1 or level 3
(red) Exploring from the maximum level to find the most nested statements and findings its context
(green) Going to a specified line using the line number column, suc as from line 9 to line 2

Observations

Finding context of a statement was easier for all participants
- Participants could check only the indentation cells without moving out of a line
- Similar context finding in text editor required counting the number of spaces/tabs in front of a statement
Substituting whitespace with meaningful indentation cell helped participants maintain their indentation while editing code
- Participants did not need to bother about the default number of indentations
- Writing at any cell ensured that they are writing at a correct level
- Writing at incorrect level was common in text editor
Writing block statements with both manual and auto-complete option was preferred by the participants
- Manually writing block statements was natural and similar to writing block statements in a text editor
- Auto-complete option was preferred by beginner participant (P1), as he did not need to worry about syntax errors
- Auto-complete was preferred by expert users when writing a block statement they do not use frequently (e.g., try-except block)
Participants could easily go out of indentation without making errors
- In text editor, participants made errors while going out of indentation

An Inspiring Insight

''Think about coding as walking a road. For a sighted person, he can see the road, but for a blind person, the road is dark. As a result, he may fall. Errors are also like falling down. But if I have something to hold on to while walking, it will help me walk the road more easily. The way I see it, Grid Editor gives me that structure I can hold on to. I always know which line and level I am. Not only that, I have the nesting information like within if, within for in the same line. So, I am always aware of the structure and my position in it. This awareness is the most important source of not making any errors.''

Design Reflection: Extending Grid-Coding for C/Lisp Style Languages

Extension of Grid-Coding for Java (C/Lisp-style) programming language. (Left) A sample Java code written in Visual Studio Code. (Right) Representation of the Java code in a grid. All Grid-Coding principles, such as each row representing a single line, each column representing a level (except for the frst column representing line numbers), and all three cell types are also present. Two additional semantic cells - open paren and close paren - have been used to replace the curly braces that enclose a scope in Java. These markers always appear at the same level for BLV users to fnd them while traversing a level. Further, as Java always contains functions within a class, we added two additional semantics - within * class and within * function in the Indentation cells to distinguish them from other block statements. Block statements have the same Indentation cells as before for understanding context.

Citation

BibTex

@inproceedings{ehtesham2022grid,

title={Grid-coding: An accessible, efficient, and structured coding paradigm for blind and low-vision programmers},

author={Ehtesham-Ul-Haque, Md and Monsur, Syed Mostofa and Billah, Syed Masum},

booktitle={Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology},

pages={1--21},

year={2022}

}

APA

Ehtesham-Ul-Haque, M., Monsur, S. M., & Billah, S. M. (2022, October). Grid-coding: An accessible, efficient, and structured coding paradigm for blind and low-vision programmers. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology (pp. 1-21).

Page updated

Google Sites

Report abuse