Lecture 12: Views and Graphical User Interfaces
GUI Views Part1 Lecture Videos.
GUI Views Part2 Lecture Videos.
12.1 Introduction to views
The primary responsibility of the view is to display information to the user of the application. In an ideal design, the view gets the data from the model (directly or indirectly), but does not have the ability to directly change the data inside the model.
A view can support simple console-based output, as well as complicated graphical outputs using a graphical user interface. In this lecture we investigate the design of the view, its communication with the other parts of an application and its implementation.
The design of a view follows the canonical interface-implementation template. In the traditional MVC architecture, the controller acts as the client for both the model and the view. Therefore the view should offer an interface that lets the controller call operations on the view. The design process for this is similar to that of the model:
Determine what the controller needs to be able to tell the view to do. This may include providing it with relevant data to display, telling the view to take specific actions at specific times, etc.
Verify that the view interface is complete: is the controller able to complete all its usage scenarios using only methods specified in the view interface?
Implement the view.
One complication may arise depending on the complexity of the view. A view may be implemented as a graphical user interface. We think of a graphical user interface as being able to both display information to the user, as well as offer ways for the user to specify input. However taking inputs is the controller’s responsibility, so the challenge is to implement the view and its communication with the controller so that responsibilities are divided appropriately.
12.2 Event-driven programming
12.2.1 Introduction
A simple standalone program in Java begins execution with the main method. The main method is a set of instructions, possibly calling other methods of other objects. The program executes these instructions line-by-line, and when the last line has been executed, the program ends.
Contrast this with the typical behavior of a program with a graphical user interface (GUI). When such a program is started, it completes any setup procedures, and then waits. When the user interacts with the program (e.g. by clicking a button, selecting a menu item, typing text, etc.) the program springs into action, performs certain tasks and then goes back to waiting. Such a program usually does not even end until the user explicitly signals so. Thus programs with GUIs seem to be fundamentally reactive in nature. In more technical terms, each action by the user is called an event, to which the program reacts by executing some instructions. Programs with GUIs follow an event-driven model, where the program’s actions depend on generated events. In the case of GUIs these events are mostly generated by user actions, but in more general programs, events could be generated without any user actions as well (e.g. the program auto-saves a file after a fixed time interval, the program auto-checks email periodically, etc.).
The design and implementation of event-driven programs involves two steps: identifying which events the program should react to, and what that reaction should be. In the case of GUIs, this translates to: create the GUI layout and determine which user interaction the program should react to, and write the reactions as snippets of code.
12.2.2 Callbacks
The concept of a callback is similar to giving someone your phone number. You would like them to call you if they need your help, but they decide when to call you. Many customer service platforms offer the ability for the caller to leave a phone number, so that they can call the caller whenever the caller is the next in line. This avoids the caller to be on hold for an indeterministic amount of time.
Reactions to events can be written as functions. These functions are called when the program is run, but since the events are asynchronous (e.g. the program cannot predict when the user will click a button), the design resorts to a different mechanism. We provide these functions to the part of the program that generates an event, with the following intent: when the program is run, and you generate an event, call this function. In this role, these functions are called callback functions. In the case of graphical user interface, we provide callbacks for the button clicks, menu item selections, etc. to which we would like the program to react.
12.3 Graphical user interfaces: basic design and implementation
Do Now!
Please download and set up the above MVCExample program and follow along.
The MVCExample above shows a simple program with a graphical user interface. We start from a simplistic but flawed design, and then mold it into the traditional model-view-controller design. Finally we add some more features and redesign the program.
Some commonly used terms in GUIs (they follow closely terms related to actual windows):
Frame: this is what we think of as a “window”. In its canonical form, it comes with a title bar (possibly with a caption and icon) along with the three buttons to minimize, maximize and close the window respectively.
Pane: the area inside the borders of the frame is referred to as the pane. The pane usually contains all the components of the GUI.
Panel: the pane may be divided into smaller regions. These regions are referred to as panels.
Container: a general-purpose entity that is capable of containing other things within it (e.g. a button can contain text or an icon, a menu contains menu items, etc.
Component: a general-purpose entity that provides some functionality (and usually fires events). The same item (e.g. button) can function as both a container and a component).
This example uses the Java Swing library. Most graphical user interface frameworks follows similar design patterns, so the knowledge gained by using one transfers easily to others, even across programming languages (for example, all of them use callbacks in some form).
12.3.1 A basic GUI design
The BadDesignButFunctional module contains a complete working program. The main method creates a simple model, and a simple view, and then relinquishes control to the view. This example contains a simple model (IModel interface and Model implementation) that offers all necessary operations. It also defines a view with a IView interface. These methods represent the operations that the view must offer: get the string typed by the user, set the caption on the label and clear the string typed by the user.
To create a window with a frame in Java Swing, we use the JFrame class. The recommended way to customize the frame to the needs of our program is to write a subclass of JFrame. The constructor of this class constructs the visual layout of the window. Some details about the code in the constructor:
Always call the constructor of JFrame: it contains important initialization.
setSize creates a frame with a specific size.
setDefaultCloseOperation determines the behavior when the “close-window” button is clicked. Options are to close the application, close the window but not the application, or do nothing.
getContentPane().setBackground(...) changes the background color of the (pane of the) window.
setVisible sets the window to be visible. Often the creation of a window and its display are two separate operations. This allows the designer to ensure that the window (and what it shows) is fully initialized before making it visible to the user.
pack packs the window to tightly enclose its contents. This makes it easy to create a GUI without manually calculating its desired size.
We use the JLabel, JTextField and JButton classes to represent a text-label, text field for input and a button respectively. We give each button a unique name (called an action command). A JButton object generates an ActionEvent-type event when it is clicked. The corresponding callback is in the form of an object that implements the ActionListener interface. The actual callback function exists as the only method inside this interface: actionPerformed.
A simple way to implement this would be for the view class to implement the ActionListener interface itself.
class JFrameView extends JFrame implements ActionListener { public JFrameView(String caption, IModel model) { ... echoButton = new JButton("Echo"); // Create a button, echoButton.setActionCommand("Echo Button"); // set its command, echoButton.addActionListener(this); // set the callback, this.add(echoButton); // and add it to the UI exitButton = new JButton("Exit"); // ditto, for another button exitButton.setActionCommand("Exit Button"); exitButton.addActionListener(this); this.add(exitButton); ... } @Override public void actionPerformed(ActionEvent e) { switch (e.getActionCommand()) { case "Echo Button": ... case "Exit Button": ... } } ... }
A second design is to segregate the two buttons. Since we have two buttons that implement two separate features (echo the string, and exit the program) we write two ActionListener objects.
class JFrameView extends JFrame { public JFrameView(String caption, IModel model) { ... echoButton = new JButton("Echo"); // Create a button, echoButton.setActionCommand("Echo Button"); // set its command, echoButton.addActionListener(new EchoButtonListener()); // set the callback, this.add(echoButton); // and add it to the UI exitButton = new JButton("Exit"); // ditto, for another button exitButton.setActionCommand("Exit Button"); exitButton.addActionListener(new ExitButtonListener()); this.add(exitButton); ... } private class EchoButtonListener implements ActionListener { @Override public void actionPerformed(ActionEvent e) { //action for the echo button } } private class ExitButtonListener implements ActionListener { @Override public void actionPerformed(ActionEvent e) { //action for the exit button } } ... }
In reality, our design would be a mix of these approaches: group together buttons with similar functionality in one callback, and have separate callbacks for separate groups of buttons.
The design of BadDesignButFunctional has two critical flaws. The first flaw is that the view directly accesses the model, which means it may mutate the model. This does not obey the separation of concerns of the view and the model (the UI team can now make the entire program work incorrectly). The second flaw is that it conflates the notion of a view and a controller. It does this by adding the callback logic inside the view itself. The ramification of this design is that when we attempt to replace this view with another interface, we must replicate the logic of these callbacks as well. A correct application of the model-view-controller architecture would minimize or eliminate this replication.
12.3.2 A MVC-compliant design
The BasicMVC example refactors the same application as above, to follow a more traditional MVC design. This example contains a controller. The controller effectively acts as the callback for the buttons in the view.
The main method creates the model, view and controller and passes the first two to the third one before relinquishing control to the controller. Thus the view no longer has direct access to the model, unlike before. During initialization, the controller passes itself as the listener for all the view’s buttons. The effect of this design is that when the program is run and the button is clicked, a method inside the controller is called. Thus the controller gets control over what to do next.
public class JFrameView extends JFrame implements IView { public JFrameView(String caption) { // NOTE: No model! ... echoButton = new JButton("Echo"); // NOTE: No action listener echoButton.setActionCommand("Echo Button"); this.add(echoButton); exitButton = new JButton("Exit"); exitButton.setActionCommand("Exit Button"); this.add(exitButton); ... } public void setListener(ActionListener listener) { echoButton.addActionListener(listener); // Rather adding *this* as a listener, exitButton.addActionListener(listener); // add the provided one instead. } ... } public class Controller implements ActionListener { public Controller(IModel m, IView v) { this.model = m; //the controller has the model this.view = v; view.setListener(this); //controller tells view which listeners to use view.display(); } @Override public void actionPerformed(ActionEvent e) { switch (e.getActionCommand()) { case "Echo Button": ... // same code as before, but now case "Exit Button": ... // it's extracted out of the view } } }
12.3.3 Adding keyboard inputs
We now enhance our application from before with the following features:
Pressing the ’D’ key toggles the color of the echoed text between black and red.
Holding down the ’C’ key makes the echoed text upper-case, and releasing it reverts back to the original case.
Both features require our GUI to respond to keyboard events. Note that our program already supports keyboard input, in the form of a text field (JTextField). For these features we want certain keys to respond with features, not merely echo the characters on the screen. To do this, we must:
Identify an object that listens to key events.
Add it to the GUI somewhere so that it always listens for these key events (similar to how we added an ActionListener to a button to listen to ActionEvents created by the button).
Implement the above functionality for the specific key presses.
A keyboard key can generate many events (a key was pressed, a key was released, a key was typed (a combination of press and release)). Our features above hint that we need this level of granularity. All of these are examples of a KeyEvent. We need callbacks for these events, which are in the KeyListener interface. In other words, we must implement a KeyListener object and connect it to a part of the GUI so that it listens to these events. We choose to connect the object to the frame itself.
We note that only one of the above features requires a new method in the view interface (to toggle color). The other feature can be implemented by converting the string from the model to upper case before sending it to the view.
/** * The interface for our view class */ public interface IView { void setListeners(ActionListener clicks, KeyListener keys); //NOTE: the second listener /** * Toggle the color of the displayed text. This is an explicit view operation because this is * something that only the view can control */ void toggleColor(); }
We can implement this idea by having our controller implement the KeyListener interface.
public class Controller implements ActionListener, KeyListener { private IModel model; private IView view; public Controller(IModel m, IView v) { model = m; view = v; v.setListeners(this, this); // This controller can handle both kinds of events directly } ... @Override public void keyTyped(KeyEvent e) { switch (e.getKeyChar()) { case 'd': //toggle color view.toggleColor(); //NOTE: method added in view interface break; } } @Override public void keyPressed(KeyEvent e) { switch (e.getKeyCode()) { case KeyEvent.VK_C: //caps String text = model.getString(); text = text.toUpperCase(); view.setEchoOutput(text); break; } } @Override public void keyReleased(KeyEvent e) { switch (e.getKeyCode()) { case KeyEvent.VK_C: //caps String text = model.getString(); view.setEchoOutput(text); break; } } }
Similar to action listeners, the controller now passes the key listener to the view as well. When the keys are pressed, the appropriate method in the controller is called to implement the logic.
There is a subtlety with keyboard events. Key events are captured only when the part of the user interface that the key listener is added to, is in focus. We can bring something in focus by making it focusable, and calling its requestFocus method. Although we can start the program by giving the frame the focus, subsequent events may change the focus. For example, when the user clicks on a button, it gets the focus. The key events will no longer work, unless the frame gets the focus back. In order to be able to restore focus from the controller, we add a new method in the view interface, and then call it whenever a button is pressed.
/** * The interface for our view class */ public interface IView { ... /** * Reset the focus on the appropriate part of the view that has the keyboard listener attached to * it, so that keyboard events will still flow through. */ void resetFocus(); ... } public class JFrameView extends JFrame implements IView { ... @Override public void resetFocus() { this.setFocusable(true); this.requestFocus(); } ... } public class Controller implements ActionListener, KeyListener { ... @Override public void actionPerformed(ActionEvent e) { switch (e.getActionCommand()) { //read from the input text field case "Echo Button": ... //NOTE: set focus back to main frame so that keyboard events work view.resetFocus(); break; case "Exit Button": System.exit(0); //NOTE: no need to set focus, as the program is ending break; } } ... }
12.3.4 Configuring/changing keyboard shortcuts easily
We notice that all the key-relevant callbacks above follow the same pattern: “if key-x generated this event, do function-y”. As more key shortcuts are supported, these methods are expected to grow in size quickly while retaining the same pattern. Also there is no easy way to change the keyboard shortcuts while still offering all the features (e.g. toggling color should now use ‘T’ instead of ‘D’).
We note that the response to each key event (called function-y above) is a set of instructions. We can encapsulate each of them in its own method. Since they represent a set of instructions, this method need not take any arguments, and need not return anything. Thus we can unify all such methods as objects of an interface that contains such a method signature. We use the existing Runnable interface for this purpose.
We now store a table with each row of the form key-x -> runnable-to-be-executed. This can be stored as a Map object, one for each type of key event (pressed, released, typed). The keyTyped (also keyPressed and keyRelease) now have the same canonical structure:
public void keyTyped(KeyEvent e) { if (keyTypedMap.containsKey(e.getKeyChar())) keyTypedMap.get(e.getKeyChar()).run(); }
Thus, the switch statement reduces to a lookup inside a map. Moreover, we can change the key shortcuts by simply making a different map during initialization!
The KeyboardMaps example illustrates this design. We factor the key listener out of the controller and into its own class. We then provide the class with our maps, and implement the listener methods inside it.
public class KeyboardListener implements KeyListener { private Map<Character, Runnable> keyTypedMap; private Map<Integer, Runnable> keyPressedMap, keyReleasedMap; /** * Set the map for key typed events. Key typed events in Java Swing are characters */ public void setKeyTypedMap(Map<Character, Runnable> map) { keyTypedMap = map; } ... /** * This is called when the view detects that a key has been typed. Find if anything has been * mapped to this key character and if so, execute it */ @Override public void keyTyped(KeyEvent e) { if (keyTypedMap.containsKey(e.getKeyChar())) keyTypedMap.get(e.getKeyChar()).run(); } ... }
The controller “prepares” these maps, creates the above object and finally passes it to the view.
public class Controller { private IModel model; private IView view; public Controller(IModel m, IView v) { this.model = m; this.view = v; configureKeyBoardListener(); ... } /** * Creates and sets a keyboard listener for the view. In effect it creates snippets of * code as a Runnable object, one for each time a key is typed, pressed and released, only * for those that the program needs. * * Last we create our KeyboardListener object, set all its maps and then give it to the view. */ private void configureKeyBoardListener() { Map<Character, Runnable> keyTypes = new HashMap<>(); Map<Integer, Runnable> keyPresses = new HashMap<>(); Map<Integer, Runnable> keyReleases = new HashMap<>(); keyPresses.put(KeyEvent.VK_C, new MakeCaps()); //NOTE: see below keyReleases.put(KeyEvent.VK_C, new MakeOriginalCase()); //NOTE: see below ... KeyboardListener kbd = new KeyboardListener(); kbd.setKeyTypedMap(keyTypes); kbd.setKeyPressedMap(keyPresses); kbd.setKeyReleasedMap(keyReleases); view.addKeyListener(kbd); //NOTE: view takes each type of listener separately } class MakeCaps implements Runnable { public void run() { String text = model.getString(); text = text.toUpperCase(); view.setEchoOutput(text); } } class MakeOriginalCase implements Runnable { public void run() { String text = model.getString(); view.setEchoOutput(text); } } class ExitButtonAction implements Runnable { public void run() { System.exit(0); } } }
We can apply this idea to the action listeners for the buttons as well.
public class Controller { private IModel model; private IView view; public Controller(IModel m, IView v) { this.model = m; this.view = v; configureKeyBoardListener(); configureButtonListener(); } private void configureButtonListener() { Map<String,Runnable> buttonClickedMap = new HashMap<String,Runnable>(); ButtonListener buttonListener = new ButtonListener(); buttonClickedMap.put("Echo Button",new EchoButtonAction()); buttonClickedMap.put("Exit Button",new ExitButtonAction()); buttonListener.setButtonClickedActionMap(buttonClickedMap); view.addActionListener(buttonListener); //NOTE: view takes each type of listener separately } class EchoButtonAction implements Runnable { public void run() { String text = view.getInputString(); //send text to the model model.setString(text); //clear input textfield view.clearInputString(); //finally echo the string in view text = model.getString(); view.setEchoOutput(text); //set focus back to main frame so that keyboard events work view.resetFocus(); } } class ExitButtonAction implements Runnable { public void run() { System.exit(0); } } }
We note that the Runnable interface has only one method, and as such can be specified as a lambda function. The KeyboardMapsWithLambdaJava8 example achieves the same design, but using lambda functions to produce more terse code.
12.3.5 High-level coupling of controller and view
All of the above designs have the following limitations:
The controller communicates with the view in a manner that reveals how the view is implemented. For example, the controller assumes that the text echoing input is supported by the view with a button (or something else that specifically generates an ActionEvent). Furthermore some of its methods assume view-specific details (e.g. the action commands of the buttons). What if the view changes the buttons’ names, or instead uses a pop-up dialog box?
Either the controller itself depends on view-specific interfaces (like ActionListener or KeyListener, or it uses other objects that depend on them. As a result, view-specific details leak out of the view.
The result of these limitations is that changing the view implementation may cause changes in the controller, or other helper classes. How can we remove this limitation?
We start by capturing the high-level capabilities of our program, that the view should expose:
Echo on (some part of) the view a string that was provided by the user
Toggle the color of the text shown by (some part of) the view
Display the text shown by (some part of) the view in upper case
Restore the case of the text displayed by (some part of) the view
Exit the program
We encapsulate each of the above features as a callback function, in a common interface.
public interface Features { void echoOutput(String typed); void toggleColor(); void makeUppercase(); void restoreLowercase(); void exitProgram(); }
We implement these callback functions in a class. Since the controller should get control in response to any event, the controller can implement them itself, or it sets them up in another object (similar to KeyboardListener in Configuring/changing keyboard shortcuts easily). Then the controller provides an object that implements the above interface to the view (instead of actual listeners).
public class Controller implements Features { private IModel model; private IView view; public Controller(IModel m) { model = m; } public void setView(IView v) { view = v; //provide view with all the callbacks view.addFeatures(this); } ... } /** * The interface for our view class */ public interface IView { ... void addFeatures(Features features); //NOTE: this replaces addListeners(..) }
The view defines its own listeners, and then connects each listener to the appropriate callback function.
@Override public void addFeatures(Features features) { //connect echoOutput callback to the clicking of the echo button echoButton.addActionListener(l->features.echoOutput(input.getText())); //NOTE: connect exitProgram to the clicking of the exit button exitButton.addActionListener(l->features.exitProgram()); this.addKeyListener( new KeyListener() { @Override public void keyTyped(KeyEvent e) { if (e.getKeyChar()=='d') { //NOTE: connect toggleColor callback to typing 'd' features.toggleColor(); } } @Override public void keyPressed(KeyEvent e) { if (e.getKeyCode()==KeyEvent.VK_C) { //NOTE: connect makeUppercase callback to pressing 'c' features.makeUppercase(); } } @Override public void keyReleased(KeyEvent e) { if (e.getKeyCode()==KeyEvent.VK_C) { //NOTE: connect restoreLowercase callback to releasing 'c' features.restoreLowercase(); } } } ); }
When a button is clicked, the listener attached to the button calls the appropriate callback, thereby giving control to the controller.
Suppose we wanted to add a new button to toggle the color of the text (in addition to using the ’d’ key). We would add it to the GUI, and then add the following line to the above addFeatures method.
public void addFeatures(Features features) { ... //NOTE: connect toggleColor to the clicking of the toggle button toggleButton.addActionListener(l->features.toggleColor()); ... }
This change was made without changing anything in the controller! Thus it illustrates how the communication between the controller and the view is independent of the view-specific details of which events are generated and which listeners are used.