Apple working on 3D Mac OS X user interface

nfoav8or · #1 12-11-2008, 07:28 PM

Apple Insider, December 11, 2008 (read through for illustrations):

"A series of Apple patent filings published this week reveal the Mac maker has spent a considerable amount of time outlining a new multi-dimensional interface for Mac OS X that would make better use of screen real estate by increasing the number of virtual surfaces capable of housing application and interface elements.

The most extensive of those filings is labeled "Multi-Dimensional Desktop" and was submitted to the United States Patent and Trademark Office back in June of last year, around the same time the company took the wraps off the feature set for its upcoming release of Mac OS X 10.6 Snow Leopard.

None of the latest interface enhancements described in the June patent are present in current pre-release builds of Snow Leopard, however, suggesting they'll be candidates for inclusion in versions of Mac OS X that would surface in years to come.

Generally speaking, the filings depict a 3D interface by which side walls, a top, and a floor all protrude from a back surface that resembles today's two-dimensional Mac OS X desktop. A few examples also suggest a radical departure from traditional interface design by which the Mac OS X menubar would be removed from the top of the screen and thrown into a stack or floating element.

An illustrated description of each of Apple's proposed design changes can be found below:

In the example above, Apple portrays an exemplary multidimensional desktop environment that includes a back surface like today's traditional two-dimensional desktop environment, which includes the Mac OS X menu bar and desktop surface. Protruding from this surface would be one or more side surfaces capable of housing object receptacles, including a "floor" that would contain the Mac OS X Dock and icon stacks positioned in the background. Other side surfaces would intersect with the floor, forming a three-dimensional box, though Apple says a "top" surface may not always be necessary.

Most of the diagrams in the filing show the multi-dimentional Finder with side surfaces, or "walls," as shown above. As is the case with the floor surface, object receptacles like the Dock can be placed along the walls at certain perceived depths. The example above shows the Dock displaced behind the 3D icon stacks sitting on the floor, but could also be arranged closer to the viewing surface so that it displays in front of the icon stacks.

The 3D icon stacks in the multi-dimensional Finder would fan outwards as they do in today's version of the Mac OS X but would not necessarily be tied to a linear path. They could instead fan in a three-dimensional arch towards a central region of the display, producing the perception of depth.

Alternatively, a short stack of icons could instead expand into a two dimensional matrix that would also display behind other elements such as the Dock. Apple adds that the stack items could corresponding to the categories of online buddies and music, which could be controlled contextually. For example, a stack item can automatically appear when the Finder receives a notification that an event associated with another user that is designated as an "online buddy" has occurred, such as an "online buddy" logging onto a network.

Apple in another example goes on to explain that the floor can be vast and sprawling, with only a portion of the surface displayed at any given time. "For example, upon a selection of the surface, [...] the surface can be scrolled in the directions indicated by the arrows," the company said. "The floor surface can include a scroll ingress and a scroll egress in which a scroll direction transitions from the scroll ingress to the scroll egress."

More specifically, the intersections of the two side walls may define a scroll ingress and a scroll egress for a left-to-right scroll direction, or the left edge and the right edge of the reflection region may define a scroll ingress and a scroll egress for a left-to-right scroll direction.

During a scroll, the stack items can be displaced from the floor surface by fading from view or by a failing effect. Meanwhile, application windows can be dragged to one of the side walls, where they'd appear in thumbnail form, or as stacks of windows. The windows could be restored to their traditional form by dragging them back to the back surface (Desktop) or through a simple double-click.

The side walls could also be colored to emphasize the back surface and the reflection regions of the interface. For example, the side surfaces could be black in color, or respective patterns, colors, or images can be rendered on each side surface. Other differentiation schemes including color schemes and image schemes can also be applied, Apple said.

A lighting aspect could also be employed to generate an illumination effect from window thumbnails or widgets placed on the side walls. For example, the illumination effect could comprise a simulated sunbeam emanating from one of the window thumbnails or widgets on the left wall. The illumination effect could change according to local environmental states. For example, it could track across the surfaces according to a local time, or its intensity could be modulated by local weather conditions that are received over the network -- high intensity for sunny days, low intensity for overcast days and during the early evening. Each element on the side walls could also cast its own shadow on subsequent interface elements to emphasize a depth aspect and an overall user interface relationship.

The above example shows how stack items on the floor can move in response to the addition of a new stack item. As the new stack item drops onto the surface, the stack items already on the surface can move in response to the displacement induced by the new stack item, shifting those stack items forwards, backwards, to the right or left.

"In one implementation, a maximum number of stack items can be displayed on the surface. If the addition of a new stack item causes the number of displayed stack items to be exceeded, then a stack item nearest a surface intersection can be displaced from the surface," Apple explained. "For example, if the maximum number of stack items to be displayed is four, then [one of the stacks] can continue to move to the edge of the surface, where [it would be] displaced, e.g., fades from view, atomizes, etc."

Another example shows a three-dimensional Finder that lacks a traditional menubar on the back surface. In this case, a stack item could be utilized to access menu items traditionally found at the top of the Mac OS X dektop. Alternatively, selection of the menu stack item could throw a traditional menubar back into place at the top of the screen, or display menu items on the back surface, or one of the walls.

The same diagram shown above also depicts a running application by which one or more application control elements -- such as palettes -- can be displayed on one or more side walls. For example, a tool bar or layers palette can be displayed on the side walls when Photoshop is running.

The above diagram expands on the concept of having application control elements reside on the walls of a multi-dimensional Finder by portraying those elements as 3D icons, which would expand their functionality. For example, each three-dimensional function icon could include a separate function command on each surface, and each three-dimensional function icon could be be rotated, positioned, and manipulated through the use of an input device, such as a mouse.

Additional 3D icons could be added to the surface by use of a menu, while removing one of the 3D icons can cause the remaining icons to "fall" in a downward direction on the surface wall. Apple said 3D icons can also be used to transition between work spaces, or to transition between a 2D Finder and a 3D Finder.

Windows in a 3D Mac OS X interface can also be dragged or displaced across one or more surfaces. For example, windows can transition out of view by sliding across the back surface and side walls. Side surfaces could also be used to implement a deletion characteristic.

nfoav8or · #2 12-11-2008, 07:28 PM

"For example, the last access time for icons and other system object representations can be monitored. If the last access time for an icon or other system object representation exceeds a first threshold, the icon or other system object representation can be automatically transitioned to the surface implementing the deletion characteristic," Apple said. "Additionally, if the last access time for the icon or other system object representation located on the surface exceeds a second threshold, the icon or other system object representation can be automatically deleted from view."

One final multidimensional desktop environment described in the filing includes an arcuate back surface with side surfaces defined by arcuate regions having curvature intersections.

"Other multidimensional desktop environment geometries can also be used. For example, in one implementation, the multidimensional desktop environment can conform to a tetrahedron-shaped environment in which a front surface of the tetrahedron defines a viewing surface, and the remaining three surfaces define a left surface, a bottom surface, and a side surface," Apple said. "In another implementation, the multidimensional desktop environment can conform to a triangular environment, in which one axis of the triangle defines the viewing surface and the remaining two sides of the triangle define a left surface and a right surface."

The 54-page filing is credited to Apple employees Imran Chaudhri, John Louch, Christopher Hynes, Timothy Bumgarner, and Eric Peyton."

All I have to say is wow! I thought the Apple interface was great to begin with, but if they are going to make it more 3D with all these options, I'm in love. I don't think I'd ever go back to windows now.

oneshot · #3 12-19-2008, 09:16 AM

This seems like something that can be completely amazing if they do it just right but if they don't it can end up a complete mess. Kind of looks like something Microsoft would attempt and completely screw up. No offense to Microsoft. It will be really interesting to see how this turns out if Apple uses this idea.

schady4 · #4 12-19-2008, 10:27 PM

there is an application in beta right now called bump top that utilizes the features shown in the pictures, although it's windows only right now there was a mac coming soon button on the page.

nfoav8or · #5 12-19-2008, 10:33 PM

but if this is a feature that is built-in to the OS then (in theory) it should work better than any add-on app.

Plus I'm not so sure windows could pull this off and still look good (does it ever anyways? haha, jk)

schady4 · #6 12-20-2008, 12:53 AM

http://www.youtube.com/watch?v=M0ODskdEPnQ
this is the url for the video for that bump top program. cool idea but way too buggy right now.
i agree that having an integrated 3d desktop into the os would run much faster than an app

nfoav8or · #7 12-20-2008, 01:01 AM

right now this "bump-top" looks like it is only in development for the tablet windows PCs... Not sure though.

schady4 · #8 12-20-2008, 01:07 AM

nope the software works with trackpads on laptops and regular mice for desktops, i played around with it for a bit, it's just too much for my laptop to handle. It would freeze or crash when folderizing items on the desktop. Anything associated with administrative functions in windows would cause a halt also... visually it was to cool to see things on the desktop organized the way they were.

nfoav8or · #9 12-20-2008, 01:47 AM

hmmm... interesting

too bad about the freezes. It doesn't look that hard to learn all the special movements required to get things the way you want them... it just looks like it would be easier with a tablet PC or touch-screen...

I'm hoping the Apple variant won't freeze up or cause KP's on my Hackintosh

lanceomni · #10 12-22-2008, 05:33 PM

How about a 3d gui where your eyes control which elements are in focus. The computer measures the point of focus "xy" and the diameter of your pupal and does calculations to determine your "z" focus. Maybe even measure the ambient light and that of the monitor to adjust the actual depth of field. Bright monitor=smaller pupal=greater the depth of field. That would make for some intense 3d gaming

Maybe in 50 years