One of the best examples of combining language with a visual interfaces is the ability in OS X to search the control panel for keywords, and it will suggest which panels you are interested in. Don’t know where to configure DHCP? Type “DHCP” into the search and it will present you with the correct tab of the correct control panel.

Going back to the Emacs example, since it is entirely based on text and language, searching is trivial.

It’s fair to argue a language-based interface probably will not work as well for something like Photoshop, although even in that case there are a lot of hidden layers (both literally and figuratively) and abstraction in how you work with the application that are not necessarily best served by purely graphical push-button UI.

So I think there is a visual vs. language dynamic going on here — similar to recognition vs. recall dynamic mentioned by a previous poster.

Like anything else, both “search” and “memorization” based interfaces can be done well or done poorly. However, I think it is a false dichotomy, because most good user interfaces allow the user to start with search and work their way towards memorization.

One thing that is important to consider is the “distance” between commands or modes. A memorization based interface encourages a “flat” command space, where you can access all possible commands or values with just a few keystrokes. Hierarchical search based interfaces traditionally tend to bury those same commands under several levels of menus, icons and dialog boxes which require multiple mouse clicks to access.

Where the flat command space really wins, though, is that it is actually easier to search. By typing in part of the command name and/or annotating commands with a full text description with additional keywords, the user can often find the commands they need in a simpler and more straightforward way than hunting through menus.

A downside to the flat command space is that it often harder to discover new commands. Where in a hierarchical-search based interface the user will often stumble upon other functionality in their search for some particular command, the ability to access commands more efficiently actually reduces this aspect of discovery.

The best example of a searchable memorization-based interface is Emacs. I know of no other program where one can have 400+ different files, buffers and views open, and yet be able to switch between them with only three or four keystrokes. At the same time, I can go into the help text and use the standard text search features to quickly find the commands I am looking for. The GUI version of Emacs also has the usual windows, buttons and dialog boxes, to help new users get up to speed. It is a “scalable” user interface. We study how to make computers scale up to process large amounts of information, why not study how to design interfaces that scale up to allow you to work with huge amounts of information?

This distinction strikes me as being very similar to the familiar “recall versus recognition” issue in HCI. Is there something more to it that I’m missing?

http://minimaldesign.net/blog/web-design/behaviors-and-ui-design

You might be interested, and I’m definitely curious to see what you think about it. Small warning: I respectfully doubt the usefulness of your article on there… But I’m very open to discussion and definitely not looking for a fight!

Regarding _only two_, I wonder if that includes in some subtle way or perhaps overlooks the concept of spatial mapping and motion, which brings back into consideration the valid uses of the mouse (pointing device).

As a simple example, while there may be keyboard bindings to perform scrolling (and drawing), the mouse offers some advantages in it’s role as the architypical pointing device.

However, there certainly remain some mouse behavior things still looking for solutions. Here’s one I thought you might be interested in:

Scrollbars on long documents are often almost totally useless (as a positioning tool) because the viewport moves way too much for the smallest mouse drag motion. At least for me, the scrolling operation is so disorienting that I specificaly avoid using it.

When the document is less than ‘a few’ times the length of the viewport, the scrollbar paradigm works just marvelously, I would say.

I’m thinking that when the document length is above some limit (relative to viewport length), there should be an alternate (visually distinguished) mouse _thumb_ surrounding the usual one. The usual one shrinks as the document grows, but the alternate one might be thought of as position within the surrounding ‘few viewports-worth’ of document, and has a minimum size. The alternate thumb would be designed to scroll more slowly, as if the document were shorter. It is left as an excercise for the reader to resolve the details of moving the alternate thumb, resyncing alternate with traditional thumb, continuing to scroll at boundaries, etc.

I suppose it is also useful and necessary to look at gesture mechanisms and mousewheel behavior. I believe Microsoft Vista has some changes in this area worth examining, as well.

Keep up the good work!

Regards,
..jim

As for your suggestion that I distinguish between windowed and command-line interfaces, the point of the entry was not that there were no other ways to differentiate designs, but rather that memorization versus search is on that is generally overlooked, and that it could be argued to be one of the most fundamental decisions.

Along this line of thinking, one important difference is that when you search for help in a GUI, it is all in menu popups or help page popups that disappear (or at least can be hidden), so you can get back to what you were doing after you have finished searching for the action. In CLI interfaces, the help output is interspersed with the real work.

For this distinction, it might be better to distinguish windowed interfaces vs. command line interfaces. There are windowed interfaces to character terminals, where different rectangular areas of the screen are dedicated to different purposes. Many operating systems have character based text editors, such as emacs or vi, which divide the character terminal screen into different windows without being graphical.

Another important difference might be that a GUI divides commands into a hierarchy of categories (the tree of menus), whereas CLI help is often not so hierarchical.