Using p5 with a screen reader

This page introduces some of the screen reader friendly features of the p5js web editor, it provides an overview of accessible outputs, and examples and tutorials on how to get started with p5 when using a screen reader.

p5.js is a library that starts with the original goal of Processing –to make to make coding accessible for artists, designers, educators, and beginners– and reinterprets it for the web using the metaphor of a software sketchbook with a set of drawing functionality. With p5.js we are able to draw in the canvas. The canvas is perhaps the most inaccessible element in HTML. The reason for this is simple: the canvas behaves just like a physical canvas, once you put paint on it, it covers up what’s behind it, making it hard to understand how the pixels comprise shapes and elements. Within the p5.js web editor it is possible to access the content of the canvas using a screen reader through text, a spatial table, and sound outputs.

What is the p5.js Web Editor

The p5.js web editor is a essentially a web page where you can type code in p5.js and run the code to view the output. It allows us to code in and for the browser. The web editor has features that make it screen reader friendly. You can access the p5 web editor here.

Pairings

Please note - the p5.accessibility library is currently not supported on MacOS.

Using the p5.js web editor with a screen reader works better when you have one of the following screen reader/browser/operating system pairings:

NVDA and Firefox
JAWS and Chrome

Outputs

With a screen reader we can access the content of the canvas through the following outputs:

Plain Text Output

This output describes the visual content present on the canvas in plain text.

Table Output

The table output laids out the content of the canvas in the form of a table based on the spatial location of each element.

Sound Output

This mode explains the movement of the objects present in the canvas. Top to bottom movement is represented by a decrease in frequency and left to right through panning.

How to select an output:

In the p5.js web editor there are two ways to select the accessible outputs that we want to make available.

When we press the “play sketch, button” accessible to screen readers the “plain-text output” will automatically be available.
If we want to access other outputs we should do the following:
- First, stop the sketch by pressing the “stop sketch, button” and go to “Settings”.
- Within settings we must find the “Accessibility” tab.
- In the “Accessibility” tab and under the “Accessible text-based canvas” section we can choose our preferred outputs.

Within “Settings” and under the “Accessibility” tab it is also possible to activate a lint warning sound that should make debugging easier. In “Settings” and under the “General Settings” tab we can increase the size of the font and select a high contrast theme. After choosing our settings we can close the “Settings” window and go back to coding and playing our sketch.

It is possible to add these accessible outputs to p5 sketches outside of the p5.js web editor by including the p5.accessibility.js library. You can learn more about the library here.

Keyboard Shortcuts

There are also ways to turn on the accessible outputs using keyboard shortcuts. Once we are on the code editor area we can use the following shortcuts (we have listed the shortcuts for Windows computers, if you are using Mac OS please use the command key instead of control):

Shift + Control +1 - Turns on ALL the accessible output options, including audio
Shift + Control +2 - Turns off ALL the accessible output options
Control + Enter - This is used to run a sketch. If you already have accessibility options turned on, they will show up as well
Shift + Control + Enter - This is used to stop a sketch.

The full list of shortcuts is also available in the p5 web editor, under Keyboard Shortcuts in the Help and Feedback section of the menu

Lessons

Lesson 1: Hello World

In this section we go through some basic functions in p5 and look at how the screen reader will read the output. There are two main functions we will use in our programs. The setup() function runs once, and is typically used for initialization, or for creating a program that does not need a loop running code repeatedly. The draw() function runs over and over again, and is used to perform a certain action repeatedly and for animation. Now we can open the editor: alpha.p5js.org and look for the code edit area. If you notice, there’s some code there already. Whenever you open the editor by default it will create a setup() function and a draw() function. However, for our first "Hello World" program we will take a step back and delete everything. Once we have a blank editor, we create a setup() block and add one line:


        function setup() {
          createCanvas(200,200);
        }

The createCanvas() function creates a canvas for us to draw or display the output. The canvas is the space on which the output is “drawn” or displayed. The code above creates a canvas of width and height 200 pixels. Since there is no movement, we will take a look only at the text and table outputs.

Text output for Lesson 1

Your output is a 200 by 200 white canvas containing the following 0 object

Table output for Lesson 1

white canvas is 200 by 200 of area 40000 contains 0 objects

As expected, the outputs do not have any details apart from that of the canvas. Now, let’s move on to the next lesson where we add some basic shapes.

Lesson 2: Basic Shapes

In this example, we will draw an ellipse on the top left, and a rectangle on the bottom right of the canvas of size 500 by 500 pixels. First, let's create the canvas and draw a shape in the setup function. Feel free to copy the code to the editor and run it.


        function setup() {
          createCanvas(500, 500);
          ellipse(50,50,20,20);
        }

This code creates a canvas of 500 by 500 pixels and then draws an ellipse. The first 2 numbers we pass to the ellipse indicate where the center of the ellipse will be on the canvas. In this case, that is 50 and 50 pixels. The next 2 numbers indicate the width and height of the ellipse. In this case, they are both 20 pixels, implying we want to draw a circle. It is important to note that the top left corner of the canvas is point 0,0 pixels and the bottom right corner of the canvas is point 500,500 pixels.

Now, we can also type this code a bit differently. We can create the canvas in the setup() function and draw the shape in the draw() function.


        function setup() {
          createCanvas(500, 500);
        }
        function draw() {
          ellipse(50,50,20,20);
        }

In the previous case, the ellipse is drawn ONCE in setup and never again. But, in this case, the ellipse is drawn again and again in the canvas. Since, the numbers passed to the ellipse are the same each time, it doesn’t make a difference. However, if we wanted to change it’s position each time and make it move, then we would have to do it in draw.

NOTE: remember, that everything in the canvas is drawn one over the other.

In this case the output will be following:

Text output for Lesson 2

The overview for the text output contains “Your output is a 500 by 500 white canvas containing the following 1 object:”.

This description is followed by a list of elements where the shape, color, position, and area of each element are described (in this case: “white ellipse at top left covering 0.13% of the canvas” ). Each element can be selected to get more details. A table of elements is also provided. In this table, shape, color, location, coordinates and area are described (example: “white ellipse location=top left coordinates =50x,50y area=0.13%”).

Table output for Lesson 2

The overview for the table output contains “white canvas is 500 by 500 of area 250000 Contains 1 objects - 1 ellipse”

This is followed by a table that describes the content spatially. It is a 10 by 10 table structure where each element is placed on a cell of the table depending on its position. Within each cell an element the color and type of shape of that element are available (example: “white ellipse”). These descriptions can be selected individually to get more details. A list of elements where shape, color, location, coordinates and area are described (example: “white ellipse location=top left coordinates =50x,50y area=0.13%”) is also available.

You can try the basic shapes example on the p5 editor

Lesson 3: Color

In these examples, we look at different ways to add color to our sketch First, let’s look at ‘background()’. We can use this function to add a background color to the canvas. Copy the below code to try it out.


        function setup() {
          createCanvas(500, 500);
          background(255,0,0);
        }

Here, we have given the color in RGB format. In this format we can describe colors by saying how much, Red, Green and Blue the color has. RGB values must be integers between 0 and 255 with 0 being no color, and 255 highest amount of a certain color. The outputs will now contain “Red (255,0,0) canvas”.

Now we can give color to our shapes using the function, fill().

Copy the below code to try it out:


        function setup() {
          createCanvas(500, 500);
          background(255,0,0);
        }

        function draw() {
          fill(200,50,8);
          ellipse(50,50,20,20);
          fill(20,250,8);
          ellipse(250,250,20,20);
          fill(20,150,250);
          ellipse(450,450,20,20);
        }

Now, in the plain text and grid output, we will notice that each shape is prepended with the color of the shape.

Text output for Lesson 3

In the text output the overview is as follows “Your output is a 500 by 500 red(255,0,0) canvas containing the following 3 objects”

After this we see the list of objects, with their color name and the RGB value. For example - “crimson(200, 50, 8) ellipse at top left covering 0.13% of the canvas” Each object contains a link that when selected provides more details.

Grid output for Lesson 3

The overview is as follows “red(255,0,0) canvas is 500 by 500 of area 250000 contains 3 0bjects - 3 ellipse”

And in the table, we will find the objects in the grids corresponding to their location. “crimson(200,50,8) ellipse” and “green(20,250,8) ellipse”

You can try the color example on the p5 editor

Instead of using Red, Green and Blue values, we can also pass just one value from 0 to 255. This is known as grayscale. It means that Red, Green and Blue will take the same value and the result will be between black and white. 0 being black, and 255 white.

Lesson 4: Text

With p5, it is also possible to draw text on the canvas! We can add text to the canvas using the text() function. This function requires us input three values inside the parentheses: the text we want to display written between quotation marks, followed by the x and y position in pixels to display the text in the canvas.

Copy and try out the below code.


        function setup() {
          createCanvas(500, 500);
          background(200);
        }

        function draw() {
          fill(255,0,0);
          text("hello world!", 50,50)
        }

Note that fill() also affects text() and that the text we want to write has to be written between quotation marks. Now, let’s go through the output to see what it says.

Text output for Lesson 4

The overview is as follows : “Your output is a 500 by 500 grey(200,200,200) canvas containing the following 1 object”

Then there is a list of the objects. (For example - “hello world!” (red(255,0,0)) at top left of the canvas

Grid output for Lesson 4

The overview is as follows: “grey(200,200,200) canvas is 500 by 500 of area 250000 contains 1 object - 1 text”

And in the table you will see the actual text and it’s color in the appropriate location (example - “hello world! (red(255,0,0))”

You can try the text example on the p5 editor

Exercise: Bar Graph

Now that we have covered how to draw basic shapes and write text on the canvas, let’s look at how we can create a bar chart with them.

Using rectangles and some numbers, we can create bar graphs. Since we expect the entire sketch to be static, we can do the whole thing in the setup function

Let’s start with a sample dataset that we want to depict.

Animals in the park:

Dogs - 260
Cats - 300
Hamsters -100
Rabbits - 50
Spiders - 10

First we create a canvas:


        function setup() {
        createCanvas(500,500);
        }

Now we can add some rectangles that represent each animal type. Notice that we will be adding comments in our code. Comments are lines in our code that are not executed and can be notes to ourselves. They start with // in the beginning of the line. We will create 5 rectangles, one for each animal type.

Remember that when we draw a rectangle, the first 2 values indicate the top left of the rectangle. If we want our bar graph to start on the left of our canvas, we need to calculate the numbers based on that. If we decide to make each bar with 50 pixels height, and to make the width represent the number of animals, for our first rectangle that represents the number of dogs the width would be 260 because there are 260 dogs. Then the values for this rectangle would be (0,0,260,50).


        function setup() {
        	createCanvas(500, 500);
        	//We will use the fill() function to add color to the first bar of our chart.
        fill(255,0,0);
        //In this case our color is 255 red, 0 green, 0 blue.
        	rect(0, 0, 260, 50);
        }

Now let’s add the corresponding text:


        function setup() {
        	createCanvas(500, 500);
        	fill(255,0,0);
        	rect(0, 0, 260, 50);
        	text('Dogs - 260', 0, 75);
        	//Remember that the function text() requires the text, and the x, and y values for its position
        }

And voila! We just added the first bar on our graph chart.

Text output for Bar Graph 1

Your output is a 500 by 500 white canvas containing the following 2 objects:

red(255, 0, 0) rect at top left covering 5.20% of the canvas
Dogs-260(red(255, 0, 0)) at top left

Now lets add rectangles to represent 300 cats, 100 hamsters, 50 rabbits, and 10 spiders. Remember that the values for the function rect() in our case should be: rect(0, y position, number of animals, 50).


        function setup() {
        	createCanvas(500, 500);
        	fill(255,0,0);
        	rect(0, 0, 260, 50);
        	text('Dogs - 260', 0, 75);
        	rect(0, 100, 300, 50); // here the second value, y position, is 100 so that we can draw the second bar under the first bar.
        	text('Cats - 300', 0, 175);
        	rect(0, 200, 100, 50);
        	text('Hamsters - 100', 0, 275);
        	rect(0, 300, 50, 50);
        	text('Rabbits - 50', 0, 375);
        	rect(0, 400, 10, 50);
        	text('Spiders - 10', 0, 475);
        }

The text output would be:

Text output for Bar Graph 2

Your output is a 500 by 500 white canvas containing the following 10 objects:

red(255, 0, 0) rect at top left covering 5.20% of the canvas
Dogs-260(red(255, 0, 0)) at top left
red(255, 0, 0) rect at top left covering 6.00% of the canvas
Cats-300(red(255, 0, 0)) at top left
red(255, 0, 0) rect at top left covering 2.00% of the canvas
Hamsters-100(red(255, 0, 0)) at top left
red(255, 0, 0) rect at top left covering 1.00% of the canvas
Rabbits-50(red(255, 0, 0)) at top left
red(255, 0, 0) rect at top left covering 0.20% of the canvas
Spiders-10(red(255, 0, 0)) at top left

Now we can add different colors to each bar:


        function setup() {
        	createCanvas(500, 500);
        	fill(255,0,0);
        	rect(0, 0, 260, 50);
        	text('Dogs - 260', 0, 75);
        	fill(0,255,0);
        	rect(0, 100, 300, 50);
        	text('Cats - 300', 0, 175);
        	fill(0,0,255);
        	rect(0, 200, 100, 50);
        	text('Hamsters - 100', 0, 275);
        	fill(200,100,0);
        	rect(0, 300, 50, 50);
        	text('Rabbits - 50', 0, 375);
        	fill(0,200,200);
        	rect(0, 400, 10, 50);
        	text('Spiders - 10', 0, 475);
        }

Our bar graph is ready! Here is the output:

Text output for Bar Graph 3