There are a few high-tech consumer side companies that are currently experimenting with various forms of 3D applications. These applications may one day be implemented into our desktop user interfaces or applied to video gaming in conjunction with advanced video glasses and/or helmets. The race is definitely on for gaming developers to push gaming to the next level so that users could be excited by viewing holographic imagery and/or experience elevated haptics so as to draw us further into their insanely creative worlds. In our latest game-front patent report we'll take a peek at one of Microsoft's latest projects/inventions that could one day allow a future Kinect system to project imagery onto to a surface in front of the gamer. The imagery, in the form of virtual objects, could be things that the gamer may need to assemble, upgrade or explore in order to survive and/or advance to the next level. At the end of the day it's great to see that Microsoft is pushing the technology that's behind tomorrow's gaming, yet realistically, it's probably a decade away before it ever sees the light of day.
Microsoft Invents 3D Interactive System for Games and Beyond
One of Microsoft's latest inventions relate to an interactive system (elsewhere noted as an interaction system) that will allow a user to create and interact with 3D virtual objects. In one implementation, the interaction system operates by a capturing a depth image of a physical object placed on, or in proximity to, a surface. For example, the physical object may comprise of an inanimate object, a body part, or combination thereof. The interaction system simultaneously captures a video image of the physical object. The interaction system then generates a 3D virtual object based on the depth image and the video image.
The 3D virtual object is a virtual counterpart of the physical object. The interaction system then uses a 3D projector (such as a projector for generating stereoscopic images) to project the 3D virtual object on, or in proximity to, the surface. The patent filing describes one of the components of this system being the Microsoft's Kinect controller noted in patent FIG. 2 below which includes an illumination source as well as depth and video cameras.
In patent FIG.2 noted above, Microsoft presents one example of the interaction system on a tabletop surface. The workspace noted as patent point # 202 could take on the dimensions of an invisible cube as noted in the patent figure. The projection system will project items into this cubed area that could be part of a game.
3D Glasses
Microsoft's patent filing states that the user will sit on one side of the interactive surface looking down onto the interactive surface from an angle. The user will wear the 3D viewing device 114, such as shutter glasses, to view the stereoscopic content projected onto the interactive surface by the projector or Kinect controller.
In Microsoft's patent FIG. 3 noted above we're able to see a more realistic depiction of one manner of use of the interaction system shown in FIG. 2. In fact in this scenario, the interactive system is being used in connection with an interactive table surface which would likely be associated with Microsoft's PixelSense. The user sits on one side of the interactive surface 102, wearing the 3D viewing device 114 interacting with objects projected on the surface of the table.
The Physics Simulator
The system also includes a physics simulator as presented in patent FIG. 1. In some cases, the physics simulator 124 may apply simulation effects that are modeled based on the laws of physics. These effects cause the 3D virtual objects to behave in the same manner as physical objects. In addition, or alternatively, the physics simulator could apply simulation effects that adopt nonrealistic dynamics, providing, for example, exaggerated or fanciful effects that may be appropriate for game environments.
For example, the physics simulator could determine that the user has used his or her finger to poke a 3D virtual object with a certain perceived force (e.g., based on the speed at which the user applies his or her finger to the 3D virtual object). Based on principles of Newtonian mechanics, the physics simulator 124 could apply a movement to the 3D virtual object which is proportional to the force. If the user applies the force to a corner of the 3D virtual object, the physics simulator could apply a spinning motion to the 3D virtual object. A modeled amount of friction will eventually bring the 3D virtual object to rest.
If the 3D virtual object is a soft body rather than a rigid body, the physics simulator could apply a deformation to the 3D virtual object which is proportional to the amount of force applied thereto. If the user applies a pinching gesture to the 3D virtual object with a thumb and finger, the physics simulator will assume that the user intends to grasp the 3D virtual object, and so on.
Texturing
According to one another illustrative feature, the interaction system could apply the texture using a projective texturing technique. This approach may take into consideration at least: a direction of attention of the user; a position of the 3D projector; and the geometry of a scene on which the 3D virtual object is to be projected (as represented in the depth image). By virtue of this technique, the interaction system could provide a natural-looking 3D virtual object (from the perspective of the user), even on an irregular real surface, such as the user's cupped hand.
According to another illustrative feature, the interaction system could present the 3D virtual object on, or in proximity to, the surface in a mirrored relationship to its counterpart physical object. This allows the user to gain an understanding of the correlation between the physical object and its virtual counterpart, without interfering with the physical object.
The new interaction system that Microsoft is developing will go beyond gaming applications. The patent filing definitely gives you the impression that the system could be also be used as a form of advanced CAD application. For example, Microsoft states that "a user could apply the interaction system to successively construct a composite model based on 3D virtual object building blocks, ultimately derived from one or more physical objects." Once in virtual mode, the user will be able to interact with the object so as to manipulate the object in any manner (rotate, spin, pinch etc).
Microsoft's patent application was originally filed in Q1 2011 and published by the US Patent and Trademark Office in Q3 2012. Realistically, this is likely a longer term project – meaning it's not going to be ready for many years. The good news however, is that Microsoft's research into creating a virtual objects system is well underway for the desktop. These advancements could later be used to create holographic-like images using Microsoft's Kinect.
Note to Referring Sites: We ask that referring sites limit the use of our graphics to a maximum of two per report. Thank you for your cooperation.
Notice
The Patent Bolt blog presents a detailed summary of patent applications with associated graphics for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for full and accurate details. Revelations found in patent applications shouldn't be interpreted as rumor or fast-tracked according to rumor timetables. About Comments: Patent Bolt reserves the right to post, dismiss or edit comments.
