{ 2007 06 16 }

The misused mouse, part 2: A proposal for a nearly mouseless interface.

Since I said the mouse needed to be seriously re-examined as the primary device for interacting with the user-interface (see my previous entry), it’s only fair that I give an example of a better way to do it. In this entry I explore one possible way to minimally change the interface to almost remove the mouse entirely, without increasing the difficulty of learning how to use software.

(Note: Click on Images for a full-size view.)
Original OOo Screenshot
(Here we have an unaltered screenshot of the experimental subject.)

First step: rip out essentially all of the traditional controls. That means drop-downs, buttons, and menus. Notable exceptions include the scroll bars and status bar (both of which provide excellent and frequently needed feedback like what the open file is, and where in the document the user is). Also, I’m going to take some liberties with the status bar to pull out some of the more cryptic (and rarely referenced) information in favor of somewhat more relevant data.

Original OOo Screenshot
(The closest thing to a decapitation of an application you’ll see.)

Second step: sit a user down (possibly with a close supply of anti-anxiety medication for those less comfortable with change), and tell them that if they want to “Control” the application, they need to press the “Control” key (great name for that key, huh?). When they do, overlay the application window with something like the following:

Design proposal for mouseless GUI
(Okay, so I’m not a graphic designer, but I bet there are a few around who could pretty this up.)

Notes on the sketch: (1) Yes, this is a lot few functions than OpenOffice writer has. I’m just trying to present proof that all the icons and the most used part of the menu can readily be represented this way. Comprehensive feature lists are better represented by my menu-replacement sketch below. However, the idea is that that should be rarely needed. If it’s used with any frequency, the application designer anticipated the user needs poorly. (2) I know some of the key-bindings are less than intuitive. I blame the 3am restarting of the whole design thanks to a bug that trashed my last design (followed by the same bug killing it a second time at 6am).

Now, there are some subtleties to the design. First, there could be two ways to access the dialog—tapping control, alt, or whatever could toggle the reference screen on until the modifier is tapped again, or, if the user holds down one of those modifiers, the reference screen disappears as soon as it’s released. This makes the use of the control key much more accessible for those of us who haven’t moved it from it’s instant-carpel-tunnel-inducing location at the very edge of what an average-sized hand can reach.

Next, commands can be put in bold if they’ve been used recently. (The definition of “recently” was the subject of extensive debate when I was working with highlighting recently changed items in my last project. I’ll leave “recent” undefined for lack of true resolution of that question for me.) Microsoft’s “adaptive” menu system (also known as “Help! Where did half my menu go?”) tries to address the same problem of adapting to user’s usage patterns. This, however, is a much better way to speed finding of common commands. It doesn’t shuffle items around or hide them, (both of which confound the user’s ability to memorize the interface and wreaks havoc on users trying to use someone else’s copy of a program).

Now, imagine the user’s thought sequence as they try to enter a command. “Hm. I need to save. Hit ‘Control,’ save… ah, ’s.’” Imagine that a few dozen times, and it starts to sound a lot like studying flashcards. For free, just by using the interface! Within weeks (assuming fairly sporadic usage), a user has memorized the shortcuts to all their common commands, obviating even looking as they execute them. Daily users could be fully proficient in even uncommonly-used combinations within days, with the pop-up screen little more than a flicker (at which point the user could set a delay for its appearance or turn it off entirely to keep it from getting in the way).

Of course, there are some potential difficulties with the design. The most glaring is the amount of seeming screen clutter when the user first sees the reference screen. However, the current system requires the person to memorize that the picture of a floppy disk means “Save the file I’m working on, without changing the name” versus the floppy disk with red lines on it, which means “Save the file I’m working on with a new name,” or look at long lists of words which (sometimes) lack any visual cues at all (in the menu system). Careful layout can also minimize this problem. There’s also the relatively computationally expensive nature of the reference screen, being a translucent (possibly dynamically-generated) pop-up. Thankfully all current versions of windowing systems (X Windows (for Linux/BSD/etc.), Windows, and MacOS X) support hardware accelerated eye candy like translucence. It’s not even essential to have it look so fancy, and caching makes generation nearly free.

Now, software like OpenOffice.org tends to have features packed in by the pound, so my button-replacement fairs poorly with those. It would be entirely reasonable to say that menus can stay. They do take minimal screen estate, and can be used with a keyboard.

I’m not a fan of half-measures, though. The menu is hard enough to navigate with a keyboard to encourage the user to reach for the mouse I’m trying to obviate. Consequently, it could be replaced by the following design (again, click for larger versions):

Menu Replacement Design part 1

Now, the user accesses the “Menu System” (which, curiously now looks more like a menu than before) by tapping the “Alt” key. The above screen appears over the application. The user then types a few letter from the command they’re looking for (the letters can be from anywhere in the command, and several groups of letters can be searched for by separating them by spaces). Once the user starts typing, entries not matching the string(s) are greyed out, as are columns without any matches. After typing the three letters “sav” the screen looks like the next mock-up (note the underlining of the matching part of the name):

Menu Replacement Design part 2

Once the list has been trimmed down enough, the user selects the item they want by hitting tab (or moving with the arrow keys) until the one they want is selected (selection not shown here) and hitting enter. If there is only one match, it is automatically selected so the user need only hit enter once the list has been fully reduced.

This system again draws on the same strengths as the pop-up control window (which I won’t enumerate again), but allows vastly denser commands.

There are several shortcomings of these last two mock-ups (which is fine, they’re not final designs, just proof-of-concepts). First, the OpenOffice.org’s menu system has been copied here, but I excluded the sub-menus for time considerations. Second-layer menus could easily be implemented by restarting the process with the sub-menu’s options once the name of a sub-menu is selected. Also, this design excludes the helper icons that OpenOffice.org has (again, time considerations—cropping, cleaning and placing scores of icons is just too slow work), as well as check marks beside the entries that are toggles. The latter problem is because I didn’t leave myself enough room for those, and they aren’t important enough to the idea of the system to warrant a complete re-layout.

Final thoughts: I hope this design has given you an idea of how easy a re-design of the GUI concept would be, and how powerful common input devices like the keyboard can be when used correctly. HCI/User Experience designers, don’t just live with the status quo.

Update: Jeff Atwood wrote about Vista’s new, more keyboard-friendly, start menu, which it seems, is only a few tweaks away from being comparable to my menu replacement design. Unfortunately, I, being an exclusive GNU/Linux user, I don’t see much of the recent Redmond or Cupertino products.

Posted by Clay Barnes on Saturday, June 16th, 2007, at 7:28 am, and filed under hci, interface design, usability, user experience, user interface.

Follow any responses to this entry with the RSS 2.0 feed.

You can post a comment, or trackback from your site.