It's no secret that Microsoft is working on a series of potential Xbox peripherals in the form of a Head Mounted Display (HMD) or glasses. In March 2012 we first broke the news that Microsoft filed for a patent relating to projector eyewear in the form of a helmet or glasses depending on the game or application. In June of that year we covered the optics behind their future eyewear and we covered this subject again in January 2013. And to make it real, we reported that Microsoft filed for a trademark application for "Xbox SmartGlass" covering "audiovisual devices" in April. Yesterday the US Patent and Trademark Office published Microsoft's latest patent in this project and our report extensively covers some of the sensors that Microsoft is considering for their eyewear. The second aspect of their latest invention involves a new invitational system for multiplayers who will own the new eyewear. Invitations will be simplified for those with the new eyewear. Gamers will be able to enter a new virtual area where you'll be able to interact with potential players. The invites could be simplified with natural audio invites instead of filling out forms and you'll be able to see the player that will be joining your game. All in all, if you're an Xbox fan and especially a multiplayer fan, you'll enjoy our latest report.
Microsoft's Patent Background
Online multiplayer games are typically played across multiple hardware devices in various locations. Such games may associate a particular user to a hardware device via an abstraction, and make a connection through the abstraction to the device. Additionally, some multiplayer games may be played through a hardware device, such as a gaming console, that is typically located in a fixed location, such as an entertainment room in a home.
To invite others to join a multiplayer game, a user may access and work through a menu system with their hardware device to send an invitation to another player. Using such a menu system may be time-consuming and may involve the user focusing on the tasks associated with sending an invitation. Such a menu system may also offer a stilted and limited user experience that discourages a user from making spontaneous invitations.
Additionally, if the user is actively playing a multiplayer game, using such a menu system to send invitations mid-game may take the user away from the game action.
Further, where the user's hardware device is located in a fixed location, the user's ability to make face-to-face invitations in other locations may also be limited.
Microsoft Invents New Multiplayer Invitational System Requiring a Head Mounted Display
Microsoft's latest invention addresses the issues outlined above with a multiplayer game invitation system that includes a user head-mounted display device and related methods for inviting a potential player to participate in a multiplayer game are provided.
In one example, a method may include receiving user voice data from a user and determining that the user voice data is an invitation to participate in a multiplayer game. Eye-tracking information, depth information, facial recognition information, potential player head-mounted display device information, and/or potential player voice data may be received. The invitation from the user may be associated with the potential player using the eye-tracking information, the depth information, the facial recognition information, the potential player head-mounted display device information, and/or the potential player voice data.
The method may include matching a potential player account with the potential player. The method may further include receiving an acceptance response from the potential player, and joining the potential player account with a user account associated with the user in participating in the multiplayer game.
Microsoft's patent FIG. 1 shown above is a schematic view of one embodiment of a multiplayer game invitation system for visually augmenting an appearance of an interaction environment as seen through a display in a head-mounted display (HMD) device. The multiplayer game invitation system includes a potential player invitation program.
Microsoft states that their future Head Mounted Display or Glasses will be able to be used with the Xbox but also potentially for future tablets, desktops, notebooks/ultrabooks, smartphones and other future devices.
Transparent OLED Display based Glasses
Microsoft's patent FIG. 2 shows an example of an HMD device in the form of a pair of wearable glasses that include a transparent display. The transparent display may be configured to enable a user to view a real-world object in the physical environment through one or more partially transparent pixels that are displaying a virtual object representation.
In one example, the transparent display noted as #54 may include image-producing elements located within lenses (#204 – e.g. a see-through OLED display).
As another example, the transparent display may include a light modulator on an edge of the lenses. In this example, the lenses may serve as a light guide for delivering light from the light modulator to the eyes of a user.
The glasses may also include various systems and sensors such as an eye-tracking system noted as patent point #58 that utilizes at least one inward facing sensor noted as #208 of patent FIG. 2 above. The inward facing sensor may be an image sensor that is configured to acquire image data in the form of eye-tracking information from a user's eyes. Provided the user has consented to the acquisition and use of this information, the eye-tracking system may use this information to track the position and/or movement of the user's eyes. The eye-tracking system may then determine where and/or at what person or object the user is looking.
In another example, the inward facing sensor may capture retinal scan information from a user's retina. Provided the user has consented to the acquisition and use of this information, such information may be used to identify the user wearing the glasses.
Optical Sensor System
The glasses may also include an optical sensor system, noted as patent point #62 of patent FIG. 1, which utilizes at least one outward facing sensor #212, such as an optical sensor. The outward facing sensor may detect movements within its field of view, such as gesture-based inputs or other movements performed by a user or by a person within the field of view.
The outward facing sensor may also capture image information, such as facial recognition information, and depth information from a physical environment and real-world objects within the environment. For example, the outward facing sensor may include a depth camera, a visible light camera, an infrared light camera, and/or a position tracking camera.
In some examples, the outward facing sensor may include one or more optical sensors for observing visible spectrum and/or infrared light from the ambient lighting conditions in the physical environment.
In other examples, the glasses may include facial recognition capabilities via one or more still or video cameras. To detect a facial image of a person, the glasses and/or computing device may use any suitable face detection technologies and/or algorithms including, but not limited to, local binary patterns (LBP), principal component analysis (PCA), independent component analysis (ICA), evolutionary pursuit (EP), Elastic Bunch Graph Matching (EBGM), or other suitable algorithm or combination of algorithms.
As noted above, the glasses may include depth sensing via one or more depth cameras. Time-resolved images from one or more of these depth cameras may be registered to each other and/or to images from another optical sensor such as a visible spectrum camera, and may be combined to yield depth-resolved video.
In some examples, a depth camera may take the form of a structured light depth camera configured to project a structured infrared illumination comprising numerous, discrete features (e.g., lines or points). The depth camera may be configured to image the structured illumination reflected from a scene onto which the structured illumination is projected. A depth map of the scene may be constructed based on spacings between adjacent features in the various regions of an imaged scene.
In other examples, a depth camera may take the form of a time-of-flight depth camera configured to project a pulsed infrared illumination onto a scene. This depth camera may be configured to detect the pulsed illumination reflected from the scene. Two or more of these depth cameras may include electronic shutters synchronized to the pulsed illumination. The integration times for the two or more depth cameras may differ, such that a pixel-resolved time-of-flight of the pulsed illumination, from the source to the scene and then to the depth cameras, is discernible from the relative amounts of light received in corresponding pixels of the two depth cameras. The HMD device 200 may also include an infrared projector to assist in structured light and/or time of flight depth analysis.
The Glasses Recognize Motion & Gesture Controls
Microsoft notes that gesture-based and other motion inputs from the user and/or persons in the physical environment may also be detected via one or more depth cameras in the glasses. For example, the outward facing sensor may include two or more optical sensors with known relative positions for creating depth images. Using motion results from these optical sensors with known relative positions, such depth images may evolve over time.
In one example, the multiplayer game program noted in patent FIG. 1 above as #34 may include a 3D modeling system that uses such input to generate a virtual environment that models the physical environment that is captured.
Motion sensors may also be employed as user input devices, such that a user may interact with glasses via gestures of the neck and head, or even of the body. Non-limiting examples of motion sensors include an accelerometer, a gyroscope, a compass, and an orientation sensor, which may be included as any combination.
Position Sensors with GPS and/or Cellular
Microsoft states that the glasses may also include a position sensor system (#66 of FIG. 1) that utilizes on or more motion sensors (#216 of FIG. 1) to enable position tracking and/or orientation sensing of glasses, and determine a position of the HMD device within a physical environment.
As one example, the position sensor system may be configured as a six-axis or six-degree of freedom position sensor system. This example position sensor system may, for example, include three accelerometers and three gyroscopes to indicate or measure a change in location of the glasses within three-dimensional space along three orthogonal axes (e.g., x, y, z), and a change in an orientation of the HMD device about the three orthogonal axes (e.g., roll, pitch, yaw).
The position sensor system may support other suitable positioning techniques, such as GPS or other global navigation systems. For example, the position sensor system may include a wireless receiver (e.g., a GPS receiver or cellular receiver) to receive wireless signals broadcast from satellites and/or terrestrial base stations. These wireless signals may be used to identify a geographic location of the glasses.
Microphones for Voice Recognition Commands
Microsoft's additionally notes that the glasses may also include one or more microphones which may receive audio input from a user and/or audio input from one or more persons in a physical environment around the user. Such audio input may include, for example, commands, requests, casual speaking, singing, whistling, etc. Additionally or alternatively, one or more microphones separate from the glasses may be used to receive audio input.
In other examples, audio may be presented to the user via one or more speakers integrated in the glasses. Such audio may include, for example, music, instructions, and/or other communication from the multiplayer game program, the potential player invitation program or other sources.
Multiplayer Game Invitation System
Microsoft's patent FIG. 3 noted below is a schematic view of a user in an interaction space using the multiplayer game invitation system of FIG. 1; patent FIG. 4 is a schematic view of the interaction space of FIG. 3 as seen through the glasses transparent display worn by the user.
You get the impression that this "interaction space" is their version of a holodeck, where you could meet other gamers wanting to join a game. As you could see above the potential player is noted as #308 and is wearing the same virtual glasses as the one making the invitation. So you're going to be able to see the person you could be playing a game with due to the face recognition software built into glasses. The invitation could be accomplished verbally, bypassing any form to fill out. Patent FIG. 4 also shows that you and your contact in the "interaction space" can choose a game to play with a pull down menu. Below you can see the flow chart related to the multiplayer invitational process.
Microsoft filed their patent application in Q1 2012. Considering that this is a patent application, the timing of such a product to market is unknown at this time.
A Note for Tech Sites covering our Report: We ask tech sites covering our report to kindly limit the use of our graphics to one image. Thanking you in advance for your cooperation.
Patent Bolt presents a detailed summary of patent applications with associated graphics for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for full and accurate details. Revelations found in patent applications shouldn't be interpreted as rumor or fast-tracked according to rumor timetables. About Comments: Patent Bolt reserves the right to post, dismiss or edit comments.