The US Patent & Trademark Office recently published a patent application from Microsoft that reveals that they have their eye on a future 3D Glasses-Free User Interface for a possible full range of devices including smartphones, a next generation Xbox controller and smart TVs. Microsoft's invention focuses on implementing a newly advanced omni-spatial gesturing input system that includes specialized gesture sensors. The new gesturing technology is intended to provide users with a superior means of executing hover based gesturing on future devices that are accurately delivered on both regular and 3D user interfaces.
Microsoft's Patent Background
Traditionally, a user interacts with a handheld device in a limited volume of space, usually on a display surface of the device. For example, a smart phone with a touch-sensitive display allows a user to interact with the device and the user interface in the volume of space immediately adjacent to the display screen. Limiting user interaction to an almost planar area that is between a user interface and the user, limits the types of interactions available to the user and may introduce usability problems, such as occlusion of the user interface. Additionally, limiting an area of user interaction to a relatively two-dimensional scope prevents the user from manipulating objects presented in the user interface in a natural and intuitive manner.
Microsoft's Omni-Spatial Gesture Input Solution
One of Microsoft's latest inventions relate to systems, methods, and computer storage media (including holographic media) for detecting user gestures in a space surrounding a device. In particular, aspects may include detecting a user gesture in front of a handheld device with a first sensor and also detecting a user gesture in the back of the handheld device with a second sensor. As a result of detecting the gestures, which may be non-device-contacting gestures, a user interface may be updated to reflect an input based on the gestures.
An exemplary technology used in devices may include a controlled refracted optical input mechanism that transfers an image as presented to the display surface (e.g., a user's hand hovering in the positive z-axis space) to one or more edges of the display.
In another area of the Microsoft's patent filing they state that some of the same technology that is used in their "Wedge" product could be incorporated/utilized into certain aspects of this new gesturing system.
Furthermore, Microsoft states that the device noted above in patent FIG. 3 (#300) is further comprised of one or more gesture sensors. A gesture sensor is a sensing component that is capable, at least in part, to detect a gesture as a user input.
Four gesture sensors are depicted on the front surface of the device in FIG. 3 as well as five gesture sensors that reside on the back of the a device such as a tablet computer. Patent Bolt has highlighted the gesture sensors in yellow for easy identification.
Yet Microsoft goes on to point out that that it should be understood that any number--including none--may be implemented in this system. Meaning that the gesture-sensor(s) may actually be incorporated right into the display itself which very much sounds like they're PixelSense technology.
As a side note, Microsoft revealed their breakthrough technology called PixelSense back in January 2011; a technology that basically allows the display to act as a large camera that could actually "see" what's touching its surface. You could see a demo video of it here.
More Details about the Gesture Sensor as a Camera
Microsoft states that it is contemplated that a gesture sensor may utilize existing sensing technology (e.g., a camera built into the device). For example, it is contemplated that one or more waveguides may be built into the device to distribute the "pixels" able to be captured by a camera in a pattern that captures a desired space (e.g., a hemispherical volume extending outward from the device in a positive axial and/or a negative axial direction).
Because one or more techniques may extend the area captured beyond the intended range of the camera, it is contemplated that time-multiplexing the pixels may be utilized to provide a desired resolution.
Therefore, a gesture sensor may be an optical-based technology (e.g., camera), a capacitive-based technology, a resistance-based technology, a thermal-based technology, an ultrasonic-based technology, a pressure-based technology, and the like.
Various combinations of the technologies (and similar technologies) may be implemented in combination. For example, it is contemplated that a proximity sensing technology (e.g., capacitive) may be used for gestures in a first direction of the z-axis and a second technology (e.g., optical) may be utilized in a second direction of the z-axis. Further, it is contemplated that a variety of sensing technologies may be utilized in a common z-axis space (e.g., positive, negative) to detect a gesture as an intended input.
Gesture sensors may include sensors capable of detecting non-contact gestures, contact gestures, and sensors capable of detecting an internal pose/orientation of the device. For example, non-contact gesture sensors may include cameras utilizing adapted lenses. In an exemplary aspect, the adapted lens may provide a distributed-angle perspective, such as a fish-eye lens.
A Possible new Microsoft "Kinection"
Another point of interest is that Microsoft is contemplating that the gesture sensor may capture one or more portions of depth information. For example, a depth camera (e.g., a camera used in conjunction with the Kinect available from the Microsoft Corporation of Redmond, Wash.) may be utilized to detect a user gesture as an intended input, especially when the device is in context with a next generation game controller which is mentioned specifically below.
Microsoft Contemplates a Glasses-Free 3D User Interface
In an exemplary embodiment, it is contemplated that the display 803 is functional for providing a UI that is perceivable by a user as a three-dimensional UI. For example, the display may be partnered with one or more lenses that are positioned between the display 803 and a user's eye(s). The combination of the display and the lenses may provide a three-dimensional perspective, as is known in the art. Other technologies are contemplated for providing a three-dimensional visual experience (e.g., hologram, glasses-free three-dimensional displays, The Wedge with adapted visual output for three-dimensional viewing).
Microsoft states that an aspect of the present invention contemplates utilizing non-contact sensors (e.g., depth camera, visible light camera, IR camera, capacitive, ultrasonic, and the like), contact sensors, and/or device pose/orientation sensors (e.g., accelerometers, magnetometers, gyros, GPS, electro-magnetic sensors, and the like) in any combination to identify an orientation, gesture, and/or intent of the device and/or a user. For example, contact sensors, non-contact sensors, and internal pose/orientation sensors may identify what hand of a user is maintaining the device, if the device is moving, if the user is providing a gesture, and/or the like.
In addition to the new touch sensors coming to traditional computers such as desktops and notebooks the technology will apply to future set top boxes and others such as those specifically listed below:
For TV: Microsoft states that "the input may be intended to manipulate an object presented by a display of the device or even an external display (e.g., television, monitor)."
For Xbox: Additionally, Microsoft states that the device may be a video game controller that is manipulated by a user to affect one or more objects displayed by an external display. Based on the first input (e.g., a rotation, a pinching, an elongating gesture), an object of the UI may be manipulated with an appropriate transformation (e.g., rotating the object, compressing the object, stretching the object).
For Smartphones & Tablets: And lastly, Microsoft states that in an additional exemplary embodiment, it is contemplated that a UI is presented as part of the device itself, such as with a mobile communications device (e.g., mobile phone).
Microsoft's patent application which was filed under serial number 250532 back in Q3 2011 was published earlier this month by the US Patent and Trademark Office. Considering that this is a patent application, the timing of such a product to market is unknown at this time.
A Note for Tech Sites covering our Report: We ask tech sites covering our report to kindly limit the use of our graphics to one image. Thanking you in advance for your cooperation.
Patent Bolt presents a detailed summary of patent applications with associated graphics for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for full and accurate details. Revelations found in patent applications shouldn't be interpreted as rumor or fast-tracked according to rumor timetables. About Comments: Patent Bolt reserves the right to post, dismiss or edit comments.