Windows 7 NUI: Stepping Beyond the GUI

(Follow me on Twitter: UX Evangelist)

There's a vision in Microsoft which was recently discussed at MIX '08 by Daniel Makoski (Interaction Design Manager, Microsoft Surface) called Natural User Experience (NUE). The vision of NUE is to, "deliver intuitive, seamless experiences that unfold through natural human input." Part of the NUE is the Natural User Interface (NUI), which is the interface a user would interact with. Microsoft Surface is a prime example of a NUI. Think of NUI as the following: CLI -> GUI -> NUI (and if you want to take it a step further, which I won't delve into at the moment because we won't be seeing or hearing of it for quite a long while) -> OUI

CLI (Command Line Interface) = (Think textual, such as the command prompt or DOS)
GUI (Graphical User Interface) = (The Windows UI)
NUI (Natural User Interface) = (Think of physical interaction with a GUI)
OUI (Organic User Interface) = (Spend some time researching here)

Important: NUI is NOT a replacement for a GUI but moreover an enhancement allowing a person to physically interact via touch with what they see. Windows 7 will have its GUI but the multi-touch functionality and usability can be thought of as its NUI.

If you’re reading this article, there’s no doubt you’ve seen - or at least heard of - the Windows 7 multi-touch demonstration from D6 not even a week ago. Well, my fellow UX friend, Mr. Long Zheng, did a little write-up discussing the notably-different task bar in the demoed build of Windows 7. As noted by Long, the task bar is superficially called the, "Superbar." Well, while watching Daniel Makoski's MIX '08 presentation, I noticed a futuristic Windows UI in a healthcare concept video. Since Microsoft seems to use fairly up-to-date internal mock-up UI images to aid in the creation of future concept videos and how they see Windows, I thought this would be appropriate to post simply to stir up a little bit of conversation, if nothing else. The whole clip (weighing in at just over 4 minutes) is actually extremely cool, so I recommend watching the whole thing but if you're only interested in the task bar, the notable times to reference are 0:55 - 1:00, 2:43, and 2:55 - 3:00.

Video: Microsoft's Future Healthcare Vision


Here are a few video stills of said notable moments (I tried to get a hold of a high-resolution version of this video so as to have some better-looking stills, but no dice. Click on the image below to see it in full-resolution):





In the stills, you will notice four orbs in the left-hand side of the task bar:




Of the four orbs, I can definitely make out the left-most, which is the start button, obviously, and the right-most, which contains a grayed-out Internet Explorer icon. It looks like the 2nd orb starting from the left has a picture icon in it and the 3rd orb has an envelope icon. Start button, pictures, email, internet... That sounds pretty handy. Seeing this image brings some things to mind: Are the other orbs customizable? Can more orbs be added? What colors would they use for the other orbs when clicking on them? What does the small circular button to the right of the 4th orb represent?

In addition to the orbs, we can see that search is a priority due to the search bar residing in the task bar. There doesn't appear to be a whole heck of a lot going on in the system tray next to the time, outside of what appears to be a show/hide button. Aside from the noted, the height of the task bar resembles the "Superbar" in Windows 7 more than it does Vista's task bar. Does the 4-orb task bar represent ideas floating around for Windows 7's "Superbar," or is this just another prototype task bar with no real usability at heart? Likewise, just how old is this mock-up, anyway? That green start button with the white Windows flag feels more MSX than XDR/WEX, if you Microsoft geeks catch my drift.

Now, all of this content from a Surface presentation having to do with Windows? Absolutely. Microsoft Surface + Windows 7 = Windows Surface? What a cool idea. Think of the possibilities of a Windows NUI. So far, Microsoft has only demonstrated the following applications: Photo, Paint, Digital Earth, and Piano. If you're a programmer, what types of applications would YOU code with multi-touch in mind? (Some of you, I don't even want to know... lol)

Important: Do not confuse multi-touch for touch! Some of you may be thinking, "Hasn't Microsoft had touch around since XP/Tablet PC?" True, they've had touch but not multi-touch, where you can interact with different locations on a screen at the same time. I know most of you understand this but for those who were simply thinking in terms of "touch," now you know. While the idea of multi-touch certainly isn't new, the idea of multi-touch within an operating system, such as Windows, IS. Likewise, while you would need actual hardware to take advantage of such features, hopefully the cost of said technology won't be through the roof by the time Windows 7 hits the market.

Now take a look at what is around a 2-minute clip of Daniel discussing some of the early prototypes for integrating a Natural User Interface into Windows. He talks about how the evolution of exploring a range of interfaces that would make a computer easier to access went from influencing features that ended up in Windows Starter Edition to their realization that their interface was much better-suited for touch application rather than mouse interaction. Included here are the screenshots/prototype images you will see in the clip:



Video: Microsoft NUI (Natural User Interface) Prototypes









In closing, you may still be wondering just what in the world all of this "NUI stuff" has to do with Windows 7. Well, when Microsoft found it fitting to ONLY demonstrate Windows 7's multi-touch integration, everything pertaining to the usability and interface of Microsoft Surface (and similar touch efforts) suddenly became incredibly relevant in the world of Windows. Windows 7 clearly isn't about eye-candy that you can look at and click around in... It's about eye-candy you can look at and naturally interact with! Sure, we're all familiar with the idea of a NUI in portable devices like an iPhone, but bringing such functionality to a desktop/laptop and having it become mainstream means the potential for truly refreshing computing laying in the not-so-distant future. I find this very exciting; regardless of how long the writing has been on the wall!

Material Referenced:

Daniel Makoski's Powerpoint Presentation: Download (7MB)
Daniel Makoski's Full Video Presentation: Download (100MB)
D6 Media: Visit

-Stephen

PS - Saving the best for last, here I went and spent all this time writing this article when I could've simply summed it all up in one simple phrase: Microsoft copies OSX! (Yes, I'm being sarcastic, for those of you who missed the fact that this is a Microsoft-centric blog) lol. Oh, how I hope Microsoft pulls something out of their hat to the extent of shutting the mouths of the hip 'n trendy Apple fanboys and fangirls all us Windows users are just DYING to be like! After all, I know I want MY company to be represented by an actor who played the role of opening a college where kids can essentially sit on their duffs and figure out what to be when they grow up allllll on their own! ;) (Yes, I did like that movie - it's called "Accepted" - but point made and no offense to the Apple users out there who have the ability to, umm... think and be individuals without mindlessly falling victim to a marketing ploy that makes you believe you're being an individual by being a part of? lol) /rant