Psychophysics with GUIs
Abstract
This chapter pursues dual goals. First, we want to build on the data collection within a psychophysical paradigm approach. Second, and more importantly, this chapter will introduce the concept of a graphical user interface (GUI) within MATLAB® and demonstrate its gainful use.
Keywords
GUI; guide; inspector; psychophysics
9.1 Goals of This Chapter
This chapter pursues dual goals. First, we want to build on the data collection with the psychophysical methods that were introduced in the last chapter. Second, and more importantly, this chapter will introduce the concept of a graphical user interface (GUI) within MATLAB® and demonstrate its gainful use.
9.2 Introduction and Background
A surprisingly large number of scientists pride themselves on their coding skills as a considerable source of self-esteem and identity. For them, it is bad enough that they are using a high-level interpreted language like MATLAB in the first place. They surely wouldn’t be caught dead using a GUI on top of that, as giving up the command line would likely constitute a deadly blow to their street cred.
Nevertheless, there are legitimate uses for GUIs. These reasons are largely the same ones that made GUIs catch on in the community at large. Briefly put, they make things more accessible, and they require less user knowledge to operate. This is neatly illustrated by the success of Microsoft Windows, which made the use of computers conceivable for a mass audience that couldn’t realistically be expected to learn how to profitably interact with a command line.
To illustrate this point within a MATLAB context, the following (true) example should suffice. I was once involved in a long-distance collaboration involving a question that was of theoretical interest to me. They had specialized equipment and would collect the data, but they were not trained in the arts of MATLAB, so we decided that I would do the data analysis. So far, so good. What I didn’t realize is that there was a rather large amount of data files collected under a daunting number of experimental conditions that didn’t seem to be organized or denoted in any way that made sense to me. Worse, they didn’t seem to be able to communicate the structure to me, and on top of everything else, the organization of these data files seemed to keep changing. As you will learn in some of the later chapters, the proper organization of data is absolutely crucial when attempting to analyze complex datasets. I couldn’t just send them the code so that they could adapt to their use, because they didn’t have MATLAB, nor did they know how to use it. To make a long story short, after a few miserable and drawn-out attempts, the collaboration failed. It was my fault. What I should have done instead (short of physically going there, which I couldn’t do, due to other commitments) was to write a self-contained GUI that encapsulated the data analysis itself. Such a GUI can be deployed on any machine, even if it doesn’t have MATLAB, and—if properly set up—can be operated by anyone, even without any MATLAB knowledge. Put differently, I should have used a MATLAB GUI to create a data browser (and analyzer), then given it to those who understood the structure of the dataset (because they collected it). Instead of struggling with an impossible analysis, I should have invested my time in creating a purpose-built GUI. Luckily, you can learn from my mistake. This example serves—at least to me—as a very vivid cautionary tale of why one disregards GUIs at one’s peril. They do have legitimate uses.
9.3 GUI Basics
GUI stands for graphical user interface. It is therefore redundant to say “GUI interface.” GUIs were pioneered by Xerox in the 1960s, along with the mouse. They were introduced to public use with the Apple Macintosh in 1984 and to widespread public use with Windows. In terms of MATLAB, early GUI functionality was introduced to MATLAB with version 4 in 1992.
For most purposes, the use of GUIs is not indicated. It takes effort to make them, and it is overkill if only the programmer will ever use the program, e.g., in most data analysis cases in research. This is particularly true if speed is of the essence, e.g., if you are rushing to meet a deadline. The two most common use cases that do warrant the construction of a GUI are:
• If the end user is not the programmer and not expected to know the intricacies of the program, e.g., for teaching, data collection, or analysis (as in the case of my ill-fated collaboration).
• If the program involves a lot of flags and parameters that need to be customized with every run. Instead of doing this on the command line, it is easier to press buttons (and not forget one). This is good if you, for instance, write a data browser or a simulation.
Having now discussed the conceptual history and defined use cases for GUIs, the obvious question is: How to make one?
This question is best answered not in the abstract, but with a practical example; which is why we cover that in the next section.
9.4 Using a GUI to Track an IP Address
In popular culture, GUIs are, perhaps unsurprisingly, steeped in mystery. For instance, in an episode of “CSI: New York,” a character states, during an urgent crisis, that she will “create a GUI interface using Visual Basic to see if she can track an IP address” (http://goo.gl/0Dxv0). While this is plainly ridiculous on many levels (see use cases, previous section), this will do as a simple starter example that will allow you to grasp the basic functionality. Let’s see if we can create a GUI (not a GUI interface!) to track an IP address. And we’ll do it using MATLAB, not Visual Basic.
The first thing to note is that while you can, in principle, build a GUI by hand and from scratch, no one really does this anymore since MathWorks introduced the function guide.
GUIDE stands for “GUI design environment,” and it provides exactly that, an editor that allows you to create GUIs in a point-and-click fashion. It allows you to quickly create functional GUIs, automatically taking care of most of the plumbing. In MATLAB, GUIs consist of a figure with associated code. The shell of all of this is created by guide, you are simply expected to add the functionality. You add elements like buttons, sliders, and lists in the figure, and you spell out what should happen once the users interact with these elements in the code. Let’s try it.
After you type guide, you will be prompted with a quick start dialog window; see Figure 9.1. Note: This GUI was created on a Mac. If you use a PC, your directory structure will show backslashes, not slashes.
For now, please opt to create a new GUI, use the default “Blank GUI” template, and click “OK.”
Once you do that, a figure opens that will allow you to build your GUI (see Figure 9.2). At this point, nothing is on it, but note in the bottom right the indication of a “current point” vector, which denotes the x and y position of your mouse cursor on the figure, as well as the “Position” vector. You can resize the figure by dragging the black rectangle on the bottom right corner of the gray figure canvas. Make sure to keep the screen dimensions of the machine that you want to deploy the GUI in mind. It would be annoying if the GUI you create so painstakingly wouldn’t fit on the target screen.
On the left is a palette of tools you can insert in the figure, containing buttons and the like. Click on the button icon (“OK,” below the arrow), and draw its outline on the canvas. Once you are done, it should look something like Figure 9.3.
The button reads “Push Button”; let’s change that. Double-click on the button. This brings up the “Inspector.” In this case, the inspector indicates that it is inspecting the uicontrol pushbutton1. You can also invoke this menu by typing “inspector” in the command line. You do need to give the handle of the uicontrol as an argument. For now, simply scroll down and take note of the different properties. At this point, the tag “pushbutton1” and the “String” are most relevant. The tag is how we will later address the button. We click on the text next to the string, change it to “TRACK!,” and click OK; see Figure 9.4. Uicontrols are user interface objects such as buttons, sliders, and the like. They have a great many properties that can be set. They can be created both by dragging them onto the canvas within the guide, or programmatically. For now, we’ll focus on the guide approach.
Now add an “Edit Text” control. After adding it, change its background color to white (via the inspector) and take note of its tag. Your figure should now look something like Figure 9.5.
Now add another “Edit Text” control, and label them with Static text controls as “Host” and “IP,” as in Figure 9.6.
As you can see, the aesthetics of this GUI leave a lot to be desired, but we’ll fix that later. For now, let’s focus on functionality.
Click on the green arrow to run the GUI. If you do this for the very first time, you will be polled for a name. Call it iptracker. After you enter a filename, two things will open: The GUI as it will look to the end user and the code that was created by GUIDE; see Figure 9.7.
The code looks complicated. It seems like GUIDE already created over a hundred lines of code. But we’ll look at that soon. First, click on the buttons. Nothing will happen. You can enter text into the text controls, but that’s it. Clearly, we need to add functionality to our program.
That’s what we’ll do now. Close the running instance of the GUI, which should leave you with the GUI editor and the code. All existing code is stored in iptracker.m, and was created automatically by GUIDE. Analyzing the code, there are eight functional parts. First, a lengthy comment section explains the iptracker function (in principle). You can edit this if you want to. Then comes initialization code, which you should not edit until you know what you are doing; changes here can break the GUI. After that comes a function that executes at the opening of the GUI, before you can see it. Do not concern yourself with this either at this point. Then comes a function that handles potential output to the command line. Of particular interest for our purposes are the functions that come after that. Two of them, “pushbutton1_Callback” and “pushbutton2_Callback,” govern what happens if the respective button is pressed. To make this explicit: If a given button (identified by the tag, which you can change via the inspector) is pressed, the code in the corresponding callback function is executed. The remaining functions govern the creation and updating of the two edit text boxes. These are the guts of the GUI. We need to define what the code should be doing by editing the “callback” functions. Speaking more generally, every time you invoke the uicontrol by moving a slider or pressing a button, the code within the associated “callback” function is executed. On a sidenote, you can add uicontrols and associated callback functions to any figure, without formally creating a GUI if you want the user to be able to interact with the figure directly, e.g., allowing to change line styles on the fly.
Now, we are in a position to create the right functionality in the right places. There are actually only a few key structures and functions that govern the behavior of the GUI. These are:
• handles: this is the name of the structure that contains all data that is passed around within the GUI as its elements.
• get: this function is used to get the value of a GUI element.
• set: this function is used to set the value of a GUI element.
• guidata(hObject,handles): invoking this updates the handles data structure.
And that’s basically it. The rest is details and commentary.
We now add the following code to the track callback function, which executes when the track button is pressed. Basically, we implement a three-step process. First, we read in the value of the “host” text field, then we use a special Java function provided by MATLAB (getHostAddress) to resolve the IP address. Third and finally, we put output this value into the ip text field, as follows:
handles.temp = get(handles.host,‘String‘); %Reading in the string in "host" text field,
%putting it into temp.
ipaddress = char(getHostAddress(java.net.InetAddress.getByName(handles.temp))); %Resolving the IP address.
set(handles.ip,‘String‘,ipaddress); %Updating the string field of the ip object with
%the ip address.
At the end of this exercise, your function should look like Figure 9.8.
Now execute your code and try it. Input a URL into the host field, then click the track button (see Figure 9.9).
It works! Congratulations, you just executed your first MATLAB function from a GUI, and you tracked an IP address in the process.
Naysayers might complain that we didn’t actually track an IP address, we just resolved one. OK, fine. Luckily, we anticipated this in the design of our program.
We’ll use similar but slightly different logic as before, (as we now need to use functions to retrieve our own IP). To do that, add the following code to the callback function that corresponds to the “own machine” button (see Figure 9.10):
ipaddress = char(getHostAddress(java.net.InetAddress.getLocalHost)); %Get own IP
%address.
set(handles.ip,‘String‘,ipaddress); %Updating the string field of the ip object with %
%the ip address.
set(handles.host,‘String‘,‘This machine‘); %Updating the string field of the host
%object.
We taste sweet success yet again. We tracked an IP address (in real time!), even if it was our own. You can even toggle back and forth between tracking URLs and your own IP.
Now that you understand the basic mechanics of GUIs, we can move on to something more exciting, like psychophysics.
9.5 Using a GUI for Psychophysics
We won’t reinvent the wheel here. If temporal precision is an issue and if you want to do advanced psychophysics, you should use the Psychophysics Toolbox or MGL, as explained in more detail in Appendix B. Nevertheless, you can use GUIs to nicely collect psychophysical data, and maybe even add a button to calculate thresholds, and so on.
As you will also learn in Appendix B, there is a MATLAB compiler that allows you to deploy a GUI without needing the machine of the end user to have MATLAB installed. This adds versatility if you need to collect data in the field.
People are very used to GUIs by now. Most participants in your experiments won’t know much about MATLAB, but they will know how to use a GUI (if you designed it right).
For educational purposes, we won’t start with a fresh GUI, but will continue in medias res. Let’s add some things to the existing GUI that you will need for psychophysics.
The first thing to do is to resize it (make it bigger) to accommodate our changes.
Then, add an axes element (you can add as many axes as you want, but one will do for now), a slider element, another two buttons (“START” and “HAPPY”), and another text edit field (tagged brightness), roughly as shown in Figure 9.11.
As you know by now, these elements don’t do anything yet, so we have to add the functionality by adding code. Once we execute the program by clicking on the green arrow, MATLAB will create the necessary wrappers/callbacks for us. We just have to fill them.
Appropriately, we’ll start with the functionality of the “START” button. The point of this button will be to initialize the display in “axes.” Put this code into the callback function for the start button:
%Create a dark background with one spot of random brightness
%Create a matrix with zeros, in 3 dimensions
actual_brightness = randi([0 255],1);
%Pick a random integer as a luminance value from 0 to 255.
actual_brightness = actual_brightness./255;
handles.X(250:259,200:209,:) = actual_brightness;
%Assign it to a 9×9 pixel square.
imagesc(handles.X,[0 255]); %Image it, scaled.
axis off; %Take off axes labels, etc.
handles.actbright = actual_brightness; %Put the actual brightness in the handles %structure.
Now every time you execute the code, a bright square will be displayed on a dark background. Every time you press start again, a new, random brightness is picked (see Figure 9.12).
Now for the slider. The idea is that the participant can dial the brightness of a comparison square up and down. The goal is to match the brightness of the square set by MATLAB (once START is pressed).
Add this code to the slider function. It executes every time the slider is moved. The logic is that we first get the slider value, then add it to the matrix:
handles.bright = get(handles.slider1,‘Value‘); %Getting the slider value
handles.X(250:259,300:309,:) = handles.bright; %Assign it to another 9×9 pixel
%square next to the other one.
imagesc(handles.X,[0 255]); %Image it, scaled.
Run the code. Something weird will happen. In my case, the whole screen turns blue every time I move the slider (see Figure 9.13). Not quite the outcome I was hoping for. What is going on?
Bugs like these are hard to track down. In this case, it helps to remember what we discussed earlier in terms of the functions that implement virtually all basic GUI operations. The individual functions that make up the GUI don’t by themselves have access to variables in other functions. They only do so via the handles structure. And that is only updated if the function guidata is invoked. This means there is an easy fix here. The slider function did not have access to the X matrix we created in the start button function, as handles hadn’t yet been updated. To remedy this, add this code at the end of the start button code (and for good measure, put it at the end of the slider code as well):
That did it; see Figure 9.14. The user can adjust the brightness of the right patch at will, by moving the slider.
We now need to assign an end condition, a condition that allows the user to indicate that he is happy with the match and ready to move on to the next trial. That’s where the “HAPPY” button comes in.
Add this code to the function that executes when the happy button is pressed:
handles.diff = abs(handles.bright-handles.actbright).*255; %Calculate the absolute
%difference between the values
set(handles.brightness,‘String‘,num2str(handles.diff)); %Put it in the text field tagged
%"brightness"
guidata(hObject, handles); %Don’t forget to update. Save your work.
It calculates the difference between the two brightness values, and outputs it in the text field we haven’t used yet (see Figure 9.15).
Now, if you were adventurous, you could add code that starts a new trial in this very same function. You could also add code that calculates thresholds on the press of a button, you could display the data on the screen and prettify the design of this figure, etc.; you get the idea.
This is as far as we’ll go for now. You can write GUIs of arbitrary complexity with hundreds of elements and multiple pages with the principles we covered here. If you are interested in more details and, in particular, how to build GUIs by hand (without using guide), we refer you to Smith (2006), although the book is somewhat dated by now.
Congratulations, you did “CSI: New York” one better. Not only did you create a GUI that tracks an IP address in real time, the same GUI also allows you to collect psychophysical data at the same time. Impressive.
9.6 Project
This one is very straightforward. Put the psychophysics task that you created in the last chapter into a GUI! Make sure to add a button that allows you to calculate thresholds at the end. It doesn’t have to calculate IP addresses, so start with a fresh GUI.