Blog

Jun 8, 2017  mode_comment

Exploring HCI with p5.js and Arduino (Part II - p5.js)

Back

As technology evoles to play a more substantial role in our lives and society, the field of Human-Computer Interaction (HCI) has been gaining more attention. As I was searching for inspiring project topics, I ran into Arduino and p5.js and discovered the cool things you could do when you combine the two. In this blog, I will talk about how I used Arduino and the JavaScript library p5.js to physically interact with my computer (the browser).



1. p5.js - installation and setup


Welcome back! In this post, I will explain how to set up things on the p5.js end so that we can start interacting with our browser with Arduino.

Before anything, let's download both p5.js and the p5.js serial port library.

Once you have downloaded and unzipped the p5.js zip file, there are key files you need to become familiar with:

  • p5.js or p5.min.js
  • Inside the folder empty-example: index.html
  • Inside the folder empty-example: sketch.js



Just like d3.js, p5.js is a visualization JavaScript library that can be loaded and used to draw on the browser.

Let's load all the necessary files in our html document.

<!DOCTYPE html>
<head>
    <script src="p5.js"></script>
    <script src="sketch.js"></script>
</head>
<body>
    <!--rest of our code.. -->
</body>

In the above code, I added both the p5.js library and the js file that will draw on our browser. Make sure that the file path after src= correctly points to the respective files.



2. p5.js - coding our program


Now, we have to write some code in sketch.js. Open the file with your favorite text editor and let's write some code!
Blank sketch.js file
If you thought that this layout seems familiar, you are indeed correct! The function setup for sketch.js is very similar to the Arduino setup. Just like the Arduino, the setup function is only called once to initialize the canvas. The function draw then runs repeatedly.

The first thing we need to do to draw on the browser is to set up a canvas (this is very similar to d3).

function setup() {
    createCanvas(400, 400);
}

function draw() {

}


The p5 function for creating a canvas is createCanvas(); with the width and height as the arguments to the function.

Now that we set up the canvas, let's write some code to draw. In this example, I will be simulating a random brownian walk -- it's a type of walk where the moving entity steps in a random direction with a bounded step size.

// define global variables used in sim
var oldx, oldy, x, y;
// this is the variable we will use to communicate with the Arduino
var inData;

// initialize -- set up canvas, define initial x,y to be middle of canvas
// give an initial value to oldx and oldy
function setup() {
    createCanvas(400, 400);
    x = width / 2;
    y = height / 2;
    oldx = x;
    oldy = y;
}

function draw() {
    // define random direction with bounded step size
    dx = random(-inData, inData);
    dy = random(-inData, inData);

    // displace particle by dx and dy
    x += dx;
    y += dy;

    // constrain the coordinates to remain inside canvas
    x = constrain(x, 0, width);
    y = constrain(y, 0, height);

    // connect paths
    stroke(255, 255, 0);
    line(oldx, oldy, x, y);

    // update oldx, oldy
    oldx = x;
    oldy = y;
}


Hopefully the comments I made in the code snippet explain what the code is doing. Some things to note, however, are some p5 functions I am using. When using p5, width and height are built-in variables that store the width and height of the canvas. The next function to note is random(). This function generates a random number between the 2 arguments you provide.

The next function I want to note is constrain(). This function contrains the value to be in between the 2 arguments you provide. I call stroke() to define the line color and finally call line(oldx, oldy, x, y) to draw a path between the steps we take.

Note that I am using the variable inData to control the step sizes. In the next section, I'll explain how to communicate with the Arduino board using inData.



3. p5.js - establishing a serial connection with Arduino


Let's now establish a serial connection with our Arduino board. Before we code anything, make sure that you have installed the p5.js serial port library.

The link I provided is the p5.serialcontrol app GUI. Simply running this app will open the server and will list the ports you can communicate with from the computer. Make sure that you have the p5.serialport.js file is in your project folder. Also, make sure to add <script language="javascript" type="text/javascript" src="p5.serialport.js"></script> to your index.html file.

Now we are set to communicate -- at least on the server side. We need to write some additional code to tell p5 how to communicate with the server. To do so,

// define global variables used in sim
var oldx, oldy, x, y;
// this is the variable we will use to communicate with the Arduino
var inData;
var serial;
var portName = "/dev/cu.usbmodem1421";

// initialize -- set up canvas, define initial x,y to be middle of canvas
// give an initial value to oldx and oldy
function setup() {
    createCanvas(400, 400);
    x = width / 2;
    y = height / 2;
    oldx = x;
    oldy = y;

    // open a serial connection
    serial = new p5.SerialPort();       // make a new instance of the serialport library
    serial.on("list", printList);  // set a callback function for the serialport list event
    serial.on("connected", serverConnected); // callback for connecting to the server
    serial.on("open", portOpen);        // callback for the port opening
    serial.on("data", serialEvent);     // callback for when new data arrives
    serial.on("error", serialError);    // callback for errors
    serial.on("close", portClose);      // callback for the port closing

    serial.list();                      // list the serial ports
    serial.open(portName);              // open a serial port
}

function draw() {
    // define random step size
    dx = random(-inData, inData);
    dy = random(-inData, inData);

    // displace particle by dx and dy
    x += dx;
    y += dy;

    // constrain the coordinates to remain inside canvas
    x = constrain(x, 0, width);
    y = constrain(y, 0, height);

    // connect paths
    stroke(255, 255, 0);
    line(oldx, oldy, x, y);

    // update oldx, oldy
    oldx = x;
    oldy = y;
}

function serverConnected() {
    println("connected to server.");
}
function portOpen() {
    println("the serial port opened.")
}
function serialEvent() {
    inData = Number(serial.read());
}
function serialError(err) {
    println("Something went wrong with the serial port. " + err);
}
function portClose() {
    println("The serial port closed.");
}


Alright, we wrote a lot of additional lines of code to tell p5 how to make a serial connection to the server.

First, I defined variables serial and portName. serial will be used to instantiate a serial connection and portName will store the port name we will be communicating with. The port name can be found when you fire up the p5 serialport GUI. The port name will be similar to what I have with the last 4 digits slightly different.

Now in our setup function, we will open the serial connection with new p5.SerialPort(); and the lines following are callback functions (that need to be coded) that will invoke the function passed in as an argument when a serial event happens. The serial events are the first arguments given, such as list, connected, open. Referring to the functions defined at the end of the code, these callback functions will simply print to console to communicate what the server is currently doing. These print statements are needed for debugging purposes.

Finally (!), we will open the serial port and connect to the server by serial.open(portName);.

So what is exactly happening? If you look at this code serial.on("data", serialEvent);, whenever the serial connection receives new data, the function serialEvent() will be invoked. In the serialEvent() function, our variable inData is being updated to read the signal from the Arduino board. Since the input type is a string, we need to convert it to a number by inData = Number(serial.read());



4. It's the end!


Now we are set! It's time to play around with our Arduino and watch what we can do to our drawing. In my experience, maintaining the serial connection was a little unstable so IF something goes wrong, make sure to check whether or not the serial connection is properly opened.

This is what my final p5 sketch looks like.



I tried to make the step sizes change drastically so that it is easy to see I am changing the step size with my Arduino. That is it for the blog series! Hope it was helpful and hope you have some fun with this :)


About

I am a computational scientist finishing my PhD at U of Penn. I picked up programming coming into graduate school and after years of computational research, I'm amazed by what data can do. I love to use data analytics to find trends, which when exposed, empower people to make informed decisions about the world they live in. I'm also the co-founder of Penn Data Science Group.