Let the mouse still work, as well as for clicking the icons in the HUD, but this design should naturally encourage users to migrate quickly to the keyboard.

I also wanted to extend this to button and tab labels, so that users can conveniently select the button they want without looking for an accel (which may not even be specified by the programmer), and without having to “tab” through buttons on the window. But I don’t know how to integrate it with the menu search.

I am astonished at how most commenters have misunderstood your proposal and its implications. Most are completely off-topic. The proposed UIs (Clay’s and mine) are orthogonal to keyboard control of windows, program invocation and file search. It is a way to issue commands to an application. You can’t do that with Ion, Spotlight, Quicksilver, Enso, “start menu” search, or a “run” dialog. Unlike ribbons, KeyCue, search under the Help menu, and easy access to standard menus, its incremental search avoids hierarchy. This avoids having to hunt for the item in the menu hierarchy, or remember where itis.

It removes the burden of learning key combinations, the reason why regular shortcuts are rarely used. It supports exploration. And it works for all kinds of apps — desktop apps, web apps and niche apps — and benefits all users — occasional and frequent, beginner and “advanced”, visual, auditory and kinesthetic. But it is much more useful for native apps because users spend much less time using Web apps.

We are not trying to remove the mouse entirely, just use the best device for each task. The keyboard skill required for the tasks described is much less than the mouse skill, even with a trackpoint. Invoking a pie menu and selecting an item requires much more coordination, even if the mouse is constrained. You have to make sure you click on “empty space” and that you select the right item; this reauires you to watch the screen and requires more coordination. And, you cannot habituate across apps, unlike a command like “Alt+save as” in the new UI, which means the same in all apps that have it. All Gimp’s commands cannot fit on a pie menu anyway, nor are there enough mouse button combos to go round. Mouse gestures also require a lot of skill and remembering (how do I invoke italic font?), and are less efficient for tasks described in the post. Typing shortcut commands like “sav3″ instead of “save as” is much harder to remember and explain, and this is too big a price to pay for saving several keystrokes. And using the keypad to direct the mouse pointer misses the point altogether. The Optimus keyboard is expensive, only works on one computer, cannot accomodate all functions of a large app, and requires mode switching for different apps’ diffrent key combos.

Unlike a toolbar, the command list can show all the functions the app supports, including their icons and descriptions. And, users can specify the command they want, instead of hunting for it. I have not seen evidence that people remember images seen better than words heard, but I have that people visually distinguish words better than images when there are more than a dozen or so. Anyway, the former is irrelevant because the new UI includes icons, and because icons alone are imprecise and ambigouous — see http://www.humanized.com/weblog/2007/06/25/the_end_of_an_icon/ for example.

The problem that different apps have different key combos is solved by use of commands. Instead of Control+Shift+s or ALT+F-A, type “Alt+save as”, which will execute the same command in all apps. Of course, app shortcuts should be standardized, but changing apps requires more work, coordination and cooperation among developers. Customization is not a solution either, it carries problems.

It would be great to re-implement the whole system “correctly”, like Archy (http://www.raskincenter.org), but that is a lot of up-front effort, and throws out everything that works already.

@Alex - It is not good to have different overlays (fast commands versus all commands) because that interferes with habituation.

@Paul Lalonde, @mace - how do Plan9 and Acme GUIs help efficiency?

@Sherman - Having one hand on the mouse does not work for typing, such as emails and papers. I go further than Clay — the keyboards is also better for selecting text and positioning the cursor during text editing.

What argument are you talking about?

But are you familiar with the Plan 9 From Bell Labs[0], a son of Unix if you will. The original Unix ideas have been taken further, for example “everything is a stream” (including the elements of the GUI which is marvellous).

Plan 9 is more of a laboratory project, though some enthusiasts actually use it too. The interface of the Acme text editor is fascinating, a GUI for heavyusers. Well, the whole project worth a good look. :)

[0] http://netlib.bell-labs.com/plan9/