Interactivity

The screen forms a bridge between our bodies and the realm of circuits and electricity inside computers. We control elements on screen through a variety of devices such as touch pads, trackballs, and joysticks, but the keyboard and mouse remain the most common input devices for desktop computers. The computer mouse dates back to the late 1960s, when Douglas Engelbart presented the device as an element of the oN-Line System (NLS), one of the first computer systems with a video display. The mouse concept was further developed at the Xerox Palo Alto Research Center (PARC), but its introduction with the Apple Macintosh in 1984 was the catalyst for its current ubiquity. The design of the mouse has gone through many revisions in the last forty years, but its function has remained the same. In Engelbart's original patent application in 1970 he referred to the mouse as an "X-Y position indicator," and this still accurately, but dryly, defines its contemporary use.

The physical mouse object is used to control the position of the cursor on screen and to select interface elements. The cursor position is read by computer programs as two numbers, the x-coordinate and the y-coordinate. These numbers can be used to control attributes of elements on screen. If these coordinates are collected and analyzed, they can be used to extract higher-level information such as the speed and direction of the mouse. This data can in turn be used for gesture and pattern recognition.

Keyboards are typically used to input characters for composing documents, email, and instant messages, but the keyboard has potential for use beyond its original intent. The migration of the keyboard from typewriter to computer expanded its function to enable launching software, moving through the menus of software applications, and navigating 3D environments in games. When writing your own software, you have the freedom to use the keyboard data any way you wish. For example, basic information such as the speed and rhythm of the fingers can be determined by the rate at which keys are pressed. This information could control the speed of an event or the quality of motion. It's also possible to ignore the characters printed on the keyboard itself and use the location of each key relative to the keyboard grid as a numeric position.

The modern computer keyboard is a direct descendant of the typewriter. The position of the keys on an English-language keyboard is inherited from early typewriters. This layout is called QWERTY because of the order of the top row of letter keys. It was developed for typewriters to put physical distance between frequently typed letter pairs, helping reduce the likelihood of the typebars colliding and jamming as they hit the ribbon. This more than one-hundred-year-old mechanical legacy still affects how we write software today.

Mouse Data

The variables mouseX and mouseY (note the capital X and Y) store the x-coordinate and y-coordinate of the cursor relative to the origin in the upper-left corner of the display window. To see the actual values produced while moving the mouse, run this program to print the values to the screen:

When a program starts, the mouseX and mouseY values are 0. If the cursor moves into the display window, the values are set to the current position of the cursor. If the cursor is at the left, the mouseX value is 0 and the value increases as the cursor moves to the right. If the cursor is at the top, the mouseY value is 0 and the value increases as the cursor moves down. If mouseX and mouseY are used in programs without a draw() or if noLoop() is run in setup(), the values will always be 0.

The mouse position is most commonly used to control the location of visual elements on screen. More interesting relations are created when the visual elements relate differently to the mouse values, rather than simply mimicking the current position. Adding and subtracting values from the mouse position creates relationships that remain constant, while multiplying and dividing these values creates changing visual relationships between the mouse position and the elements on the screen. In the first of the following examples, the circle is directly mapped to the cursor, in the second, numbers are added and subtracted from the cursor position to create offsets, and in the third, multiplication and division are used to scale the offsets.

To invert the value of the mouse, subtract the mouseX value from the width of the window and subtract the mouseY value from the height of the screen.

The variables pmouseX and pmouseY store the mouse values from the previous frame. If the mouse does not move, the values will be the same, but if the mouse is moving quickly there can be large differences between the values. To see the difference, run the following program and alternate moving the mouse slowly and quickly. Watch the values print to the screen.

Draw a line from the previous mouse position to the current position to show the changing position in one frame and reveal the speed and direction of the mouse. When the mouse is not moving, a point is drawn, but quick mouse movements create long lines.

Use the mouseX and mouseY variables with an if structure to allow the cursor to select regions of the screen. The following examples demonstrate the cursor making a selection between different areas of the display window. The first divides the screen into halves, and the second divides the screen into thirds.

Use the logical operator && with an if structure to select a rectangular region of the screen. As demonstrated in the following example, when a relational expression is made to test each edge of a rectangle (left, right, top, bottom) and these are concatenated with a logical AND, the entire relational expression is true only when the cursor is inside the rectangle.

This code asks, "Is the cursor to the right of the left edge and is the cursor to the left of the right edge and is the cursor beyond the top edge and is the cursor above the bottom?" The code for the next example asks a set of similar questions and combines them with the keyword else to determine which one of the defined areas contains the cursor.

Mouse buttons

Computer mice and other related input devices typically have between one and three buttons; p5 can detect when these buttons are pressed with the mouseIsPressed and mouseButton variables. Used with the button status, the cursor position enables the mouse to perform different actions. For example, a button press when the mouse is over an icon can select it, so the icon can be moved to a different location on screen. The mouseIsPressed variable is true if any mouse button is pressed and false if no mouse button is pressed. The variable mouseButton is LEFT, CENTER, or RIGHT depending on the mouse button most recently pressed. The mouseIsPressed variable reverts to false as soon as the button is released, but the mouseButton variable retains its value until a different button is pressed. These variables can be used independently or in combination to control the software. Run these programs to see how the software responds to your fingers.

Not all mice have multiple buttons, and if software is distributed widely, the interaction should not rely on detecting which button is pressed.

Keyboard data

p5 registers the most recently pressed key and whether a key is currently pressed. The boolean variable keyIsPressed is true if a key is pressed and is false if not. Include this variable in the test of an if structure to allow lines of code to run only if a key is pressed. The keyIsPressed variable remains true while the key is held down and becomes false only when the key is released.

The key variable stores a single alphanumeric character. Specifically, it holds the most recently pressed key. However, it is not guaranteed to be case sensitive. To get the proper capitalization, it is best to use it within keyTyped(). For non-ASCII keys, use the keyCode variable. The key can be displayed on screen with the text() function (p. 150).

The key variable may be used to determine whether a specific key is pressed. The following example uses the expression key=='A' to test if the A key is pressed. Note the use of double quotation marks or single quotation marks has no influence on the program as long as you are consistent. The logical AND symbol, the && operator, is used to connect the expression with the keyIsPressed variable to ascertain that the key pressed is the uppercase A.

The previous example works with an uppercase A, but not if the lowercase letter is pressed. To check for both uppercase and lowercase letters, extend the relational expression with a logical OR, the || relational operator. Line 9 in the previous program would be changed to:

        if ((keyIsPressed == true) && ((key == 'a') || (key == 'A'))) {
      

Coded keys

Because each character has a numeric value as defined by the ASCII table, the value of the keyCode variable can be used like any other number to control visual attributes such as the position and color of shape elements. For instance, the ASCII table defines the uppercase A as the number 65, and the digit 1 is defined as 49.

In addition to reading key values for numbers, letters, and symbols, p5 can also read the values from other keys including the arrow keys and the Alt, Control, Shift, Backspace, Tab, Enter, Return, Escape, and Delete keys. The variable keyCode stores the BACKSPACE, DELETE, ENTER, RETURN, TAB, ESCAPE, SHIFT, CONTROL, OPTION, ALT, UP_ARROW, DOWN_ARROW, LEFT_ARROW, RIGHT_ARROW. If you're making cross-platform projects, note that the Enter key is commonly used on PCs and UNIX and the Return key is used on Macintosh. Check for both Enter and Return to make sure your program will work for all platforms (see code 12-17).

Events

A category of functions called events alter the normal flow of a program when an action such as a key press or mouse movement takes place. An event is a polite interruption of the normal flow of a program. Key presses and mouse movements are stored until the end of draw(), where they can take action that won't disturb drawing that's currently in progress. The code inside an event function is run once each time the corresponding event occurs. For example, if a mouse button is pressed, the code inside the mousePressed() function will run once and will not run again until the button is pressed again. This allows data produced by the mouse and keyboard to be read independently from what is happening in the rest of the program.

Mouse events

The mouse event functions are mousePressed(), mouseReleased(), mouseClicked(), mouseMoved(), mouseDragged(), mouseOver(), and mouseOut():

• mousePressed() - Code inside this block is run one time when a mouse button is pressed
• mouseReleased() - Code inside this block is run one time when a mouse button is released
• mouseClicked() - Code inside this block is run once after a mouse button is pressed and released over the element
• doubleClicked() - Code inside this block is run once after a mouse button is pressed and released over the element twice
• mouseMoved() - Code inside this block is run one time when the mouse is moved
• mouseDragged() - Code inside this block is run one time when the mouse is moved while a mouse button is pressed
• mouseOver() - Code inside this block is run once after every time a mouse moves onto the element.
• mouseOut() - Code inside this block is run once after every time a mouse moves off the element

The mousePressed() function works differently than the mouseIsPressed variable. The value of the mouseIsPressed variable is true until the mouse button is released. It can therefore be used within draw() to have a line of code run while the mouse is pressed. In contrast, the code inside the mousePressed() function only runs once when a button is pressed. This makes it useful when a mouse click is used to trigger an action, such as clearing the screen. In the following example, the background value becomes lighter each time a mouse button is pressed. Run the example on your computer to see the change in response to your finger.

The following example is the same as the one above, but the gray variable is set in the mouseReleased() event function, which is called once every time a button is released. This difference can be seen only by running the program and clicking the mouse button. Keep the mouse button pressed for a long time and notice that the background value changes only when the button is released.

Similarly, the gray variable is set in the mouseClicked() event function, which is called once after a mouse button has been pressed and then released. Browsers handle clicks differently, so this function is only guaranteed to be run when the left mouse button is clicked. To handle other mouse buttons being pressed or released, use mousePressed() or mouseReleased().

It is generally not a good idea to draw inside an event function, but it can be done under certain conditions. Before drawing inside these functions, it's important to think about the flow of the program. In this example, squares are drawn inside mousePressed() and they remain on screen because there is no background() inside draw(). But if background() is used, visual elements drawn within one of the mouse event functions will appear on screen for only a single frame, or, by default, 1/60th of a second. In fact, you'll notice this example has nothing at all inside draw(), but it needs to be there to force P5 to keep listening for the events. If a background() function were run inside draw(), the rectangles would flash onto the screen and disappear.

The code inside the mouseMoved() and mouseDragged() event functions are run when there is a change in the mouse position. The code in the mouseMoved() block is run at the end of each frame when the mouse moves and no button is pressed. The code in the mouseDragged() block does the same when the mouse button is pressed. If the mouse stays in the same position from frame to frame, the code inside these functions does not run. In this example, the gray circle follows the mouse when the button is not pressed, and the black circle follows the mouse when a mouse button is pressed.

The .mouseOver() function is called once after every time a mouse moves onto the element. In this example, the diameter of the ellipse increase by 10 every time the mouse moves onto the canvas.

The .mouseOut() function is called once after every time a mouse moves off the element. Similar to the above example, the diameter of the ellipse increase by 10 every time the mouse moves out of the canvas.

Wheel Events

The .mouseWheel() function is called once after every time a mouse wheel is scrolled over the element. This can be used to attach element specific event listeners. The function accepts a callback function as argument which will be executed when the wheel event is triggered on the element. The event.deltaY property returns negative values if the mouse wheel is rotated up or away from the user and positive in the other direction. The event.deltaX does the same as event.deltaY except it reads the horizontal wheel scroll of the mouse wheel. On OS X with "natural" scrolling enabled, the event.deltaY values are reversed.

In this example, an event listener is attached to the canvas element, and function changeSize() would run when scrolling is performed on canvas. By using the event.deltaY variable, scrolling up on canvas would increase the diameter of the ellipse and scrolling down would decrease the diameter. If scrolling is performed anywhere in any direction, the background is going to be darker.

Key events

Each key press is registered through the keyboard event functions keyPressed(), keyTyped() and keyReleased():

• keyPressed() - Code inside this block is run one time when any key is pressed
• keyTyped() - Code inside this block is run one time when a key is pressed, but action keys such as Ctrl, Shift, and Alt are ignored. The most recent key pressed will be stored in the key variable.
• keyReleased() - Code inside this block is run one time when any key is released

Each time a key is pressed, the code inside the keyPressed() block is run once. Within this block, it's possible to test which key has been pressed and to use this value for any purpose. If a key is held down for an extended time, the code inside the keyPressed() block might run many times in a rapid succession because most operating systems will take over and repeatedly call the keyPressed() function. The amount of time it takes to start repeating and the rate of repetitions will be different from computer to computer, depending on the keyboard preference settings. In this example, the value of the boolean variable drawT is set from false to true when the T key is pressed; this causes the lines of code to render the rectangles in draw() to start running.

When ASCII keys are pressed, the character is stored in the key variable. However, this variable does not distinguish between uppercase and lowercase when used with the keyPressed() function. For this reason, it is recommended to use keyTyped(). When used with the key variable this function will recognize case.

Each time a key is released, the code inside the keyReleased() block is run once. The following example builds on the previous code; each time the key is released the boolean variable drawT is set back to false to stop the shape from displaying within draw().

Event flow

As discussed previously, programs written with draw() display frames to the screen sixty frames each second. The frameRate() function is used to set a limit on the number of frames that will display each second, and the noLoop() function can be used to stop draw() from looping. The additional functions loop() and redraw() provide more options when used in combination with the mouse and keyboard event functions. If a program has been paused with noLoop(), running loop() resumes its action. Because the event functions are the only elements that continue to run when a program is paused with noLoop(), the loop() function can be used within these events to continue running the code in draw(). The following example runs the draw() function for about two seconds each time a mouse button is pressed and then pauses the program after that time has elapsed.

The redraw() function runs the code in draw() one time and then halts the execution. It's helpful when the display needn't be updated continuously. The following example runs the code in draw() once each time a mouse button is pressed.

Cursor icon

The cursor can be hidden with the noCursor() function and can be set to appear as a different icon or image with the cursor() function. When the noCursor() function is run, the cursor icon disappears as it moves into the display window. To give feedback about the location of the cursor within the software, a custom cursor can be drawn and controlled with the mouseX and mouseY variables.

If noCursor() is run, the cursor will be hidden while the program is running until the cursor() function is run to reveal it.

Add a parameter to the cursor() function to change it to another icon or image. Either load and use image, or use the self-descriptive options are ARROW, CROSS, HAND, MOVE, TEXT, and WAIT.

These cursor icons are part of your computer's operating system and will appear different on different machines.