Braille Generation

MAIDR incorporates a Braille mode that represents the plot using Braille symbols. This allows users with visual impairments to explore and interact with the plot using a refreshable Braille display. To achieve this, our system translates the plot's visual elements and data points into a corresponding tactile representation using Braille patterns. For different plot types, such as bar plot, boxplot, heatmap, and scatter plot, MAIDR employs unique encoding strategies tailored to effectively convey the data distribution, patterns, and trends. These tactile encodings range from using distinct Braille characters to represent value ranges, to using characters that visually resemble the corresponding sections of a plot. By providing a comprehensive Braille representation for various plot types, MAIDR enables users with visual impairments to gain a deeper understanding of the underlying data and its insights. For multiline braille display setup, see Multiline Braille Display Support.

Bar plot

In the Braille representation of a bar plot, data values are encoded as Braille characters based on their relative magnitude within the plot. Low values are denoted by Braille characters that have dots only along the bottom, while high values are indicated by characters that have dots along the top. Given the four height levels of Braille, the encoding is as follows:

• ⣀ represents values 0 to 25%
• ⠤ represents 25% to 50%
• ⠒ represents 50% to 75%
• ⠉ represents 75% to 100%

This tactile encoding allows users to easily differentiate between the various value ranges in the bar plot, facilitating their understanding of the data distribution and its underlying trends.

Multiline Displays

Bar plots use a single-line representation. On multiline braille displays, the bar plot appears on the first line and the remaining lines are unused.

Heatmap

In the Braille representation of a heatmap, values are depicted based on their relative magnitude within the plot, much like the approach used for bar plots and scatter plots. Low values are denoted by Braille characters with dots only along the bottom, high values are represented by characters filled with dots, and blank or null values are indicated by empty spaces. With three height levels of Braille, the encoding is as follows:

• ⠤ represents values from 0% to 33%
• ⠒ represents values from 33% to 66%
• ⠉ represents values from 66% to 100%
• "⠀" (braille space) represents null or empty values
• "⢳" represents a row separator

Multiline Displays

In multiline braille displays, all rows of the heatmap are represented simultaneously. Horizontally, the height of the braille encoding from left to right represents the value of each cell in the corresponding row. Vertically, each line of the braille display corresponds to a different row of the heatmap, allowing users to perceive the distribution of values across the entire heatmap at once.

In single-line braille displays, use the up and down arrow keys to move between rows. The braille representation updates to show the current row.

Box plot

The Braille representation of a boxplot uses Braille characters that visually resemble the corresponding sections of the boxplot. An example of such braille may look like ⠂ ⠒⠒⠒⠒⠒⠒⠿⠸⠿⠒ . The size of each section is denoted by the number of Braille characters used. The sections are encoded as follows:

• ⠂ represents lower outlier and upper outlier(s)
• ⠒ represents the left or right whiskers
• ⠿ represents the second or third quartiles
• ⠸⠇ represents the 50% midpoint (median)
• blank spaces represent empty spaces. Blank spaces represent empty spaces. At the beginning of a line, blank space acts as a positional offset to ensure multiple box plots align correctly to the same numerical scale.

We also impose some overarching rules:

1. Each section must be represented with at least one braille character, assuming they have some positive length.
2. Differences or equalities in whiskers and quartiles must be upheld. That is, if the min and max whisker are of equal length, they must have the same number of braille characters, or if they're different, the number of characters must be different.
3. A set character always represents zero length sections, such as outliers and the median. ⠂ in the case of outliers, ⠸⠇ in the case of the median.
4. Spatial Alignment: When comparing multiple box plots, the leading blank space represents the distance from the global minimum to the start of the specific dataset. This allows users to perceive relative differences in value by comparing the "indentation" of each plot.

This tactile encoding enables users to discern the various components of the boxplot, allowing them to comprehend the data distribution, detect outliers, and identify central tendencies and dispersion within the dataset.

To generate the braille, we use an algorithm that generates a distribution of characters based on a given proportional distribution and a specified total number of characters in the user's braille display. This can be described mathematically as follows:

ci = round(n * pi), for i = 1, 2, 3, ..., k c_i = round((n - C) * p_i), for i = 1, 2, 3, ..., k

Where

• n: Total number of characters (integer)
• C: Total number of length 0 characters to offset the total characters (outliers and median) (integer)
• p_i: Proportional distribution of each category i, where i ∈ {1, 2, 3, ..., k} (real numbers, 0 ≤ p_i ≤ 1, and the sum of all p_i equals 1)
• c_i: Number of characters for each category i (integer)

The process is as follows in the code:

1. We first convert our data set for a particular boxplot to an array of lengths.
2. We then assign the single required character to each section.
3. We also note connected sections, such as min and max.
4. We then normalize and allocate all remaining characters according to their proportional distribution, making sure to add extra characters where needed to keep differences or equalities.

As an example, consider a boxplot with the following distribution: [10, 0, 20, 40, 30, 0, 30, 60, 50, 30, 0, 10, 0], with types [blank space, outlier, larger blank space, large min whisker, moderate sized lower quartile, the median, moderate sized upper quartile, another larger max whisker, a large blank space, an outlier, a small blank space, then another outlier], and a braille display length of 33. We would produce braille that looks like so:

⠂ ⠒⠒⠒⠒⠿⠿⠿⠸⠇⠿⠿⠿⠒⠒⠒⠒⠒⠒ ⠂ ⠂

Multiline Displays

Multiline braille displays represent grouped boxplots, such as horizontal boxplots and vertical boxplots with multiple groups, by displaying each group on a separate line of the braille display. Each line corresponds to a different group, allowing users to compare the distributions of multiple groups simultaneously. The same encoding principles for boxplots apply to each line, with Braille characters representing the various sections of the boxplot for each group.

Single-line braille displays represent grouped boxplots by allowing users to navigate vertically between groups using the up and down arrow keys. As the user navigates, the braille representation updates to show the boxplot for the current group, enabling users to explore each group's distribution one at a time.

Scatter plot

In the Braille representation of a scatter plot, the encoding is performed only for the line layer (layer 2). Stand alone scatterplots without a line layer are not represented in braille. The representation of the line layer is similar to that used for bar plots, wherein data values are represented as Braille characters based on their relative magnitude within the plot. Low values are denoted by dots along the bottom, while high values are indicated by dots along the top. With four height levels of Braille, the encoding is as follows:

• ⣀ represents values from 0% to 25%
• ⠤ represents values from 25% to 50%
• ⠒ represents values from 50% to 75%
• ⠉ represents values from 75% to 100%

Multiline Displays

When the grid navigation rotor is activated, the braille representation of the scatter plot changes to highlight the number of points in each grid cell. The representation is similar to that of a heatmap, where the number of points in each cell is represented by the height of the Braille character.

In multiline braille displays, all cells are represented simultaneously. Horizontally, the height of the braille encoding from left to right represents the number of data points in the corresponding grid cell. Vertically, each line of the braille display corresponds to a different row of the grid, allowing users to perceive the distribution of points across the entire grid at once.

In single-line braille displays, the user can navigate vertically with the up and down arrow keys to move between rows of the grid, and the braille representation updates to show the number of points in each cell of the current row.

Segmented Bar Plots

Stacked bar, dodged bar, and normalized stacked bar all share the same system:

In the braille representation of segmented bar plots, braille depends on where you are. There are typically multiple levels to a segmented bar plot, and as you move (Up and Down arrow keys) between levels, the braille changes to represent your current level. At the top, there is also a Summary pseudo level of all levels added together, and a Combined pseudo level of each level separately.

• Regular level: Braille appears similar to a bar plot, with braille values corresponding to the size of the level's value for this point.
• Summary level: Same as regular level, but values now reflect the combined size of all levels' values for this point.
• Combined level: Similar to heatmap, where there are groups of magnitudes for each point separated by a ⢳ character. The first group has braille characters for each level for the first point, then a separator, then the second group has braille characters for each level in the second point, then a separator, and so on.

Multiline Displays

In multiline braille displays, all levels are represented simultaneously. Horizontally, the height of the braille encoding from left to right represents the size of the level's value for a particular point. Vertically, each line of the braille display corresponds to a different level, allowing users to perceive the distribution of values across all levels at once.

In single-line braille displays, use the up and down arrow keys to move between levels. The braille representation updates to show the current level.

Violin Plot

Violin plots have two layers, each with their own braille representation:

Violin KDE (Density Curve)

The braille representation for the KDE layer uses the same encoding as bar plots, based on the density value at each point along the curve. Low density values use bottom dots, high density values use top dots:

• ⣀ represents values from 0% to 25%
• ⠤ represents values from 25% to 50%
• ⠒ represents values from 50% to 75%
• ⠉ represents values from 75% to 100%

The braille string represents the density profile of the current violin from bottom to top of the curve.

Violin Box (Summary Statistics)

The braille representation for the box layer is identical to the standard box plot encoding described above. It uses the same characters and proportional allocation algorithm to represent whiskers, quartiles, median, and outliers.

Multiline Displays

In multiline braille displays, each violin is represented on its own line using the KDE or box encodings above, depending on the selected layer. Horizontally, the braille characters represent the density profile and summary statistics along the value axis for that violin. Vertically, each line corresponds to a different violin, allowing users to compare categories simultaneously. In single-line braille displays, use the up and down arrow keys to move between violins; the braille representation updates to the current violin.

Line plot

In the Braille representation of a line plot, braille is nearly identical to the above bar plot: data values are encoded as Braille characters based on their relative magnitude within the plot. Low values are denoted by Braille characters that have dots only along the bottom, while high values are indicated by characters that have dots higher up.

Multiline Displays

In multiline braille displays, all data series are represented simultaneously. Horizontally, the height of the braille encoding from left to right represents the value of the series for a particular point. Vertically, each line of the braille display corresponds to a different series in the plot, allowing users to perceive the distribution of values across all series at once.

In single-line braille displays, the user can navigate vertically with the up and down arrow keys to move between lines, and the braille representation updates to show the values for the current line.

Multiline Braille Display Support

By leveraging the two-dimensional nature of multiline braille displays, MAIDR can represent multiple lines of a plot simultaneously, allowing users to perceive the distribution of values across all lines at once. This is particularly beneficial for plots with multiple groups or categories, such as grouped boxplots or line plots with multiple lines, and it also applies to scatter plot grid navigation.

Setup

Users can set up multiline braille display support by following these steps:

1. Open any plot in MAIDR.
2. Press Ctrl+, (Command+, on macOS) to open the settings dialog.
3. Press Tab to reach the Braille Display options.
   • If your braille display model is listed, choose "Single line" or "Multi-line", then tab to the Single-Line Display or Multi-Line Display dropdown and select your braille display model.
   • If your braille display model is not listed, choose "Configure manually", then enter Braille Display Size (cells per row) and Braille Display Lines (rows). Set the number of lines to 1 for a single-line display.
4. Press Alt+S to save and close the settings dialog.