A 2012 patent reveals that Sony is working on a new 3D Controlling System for differing markets and levels of gaming. Some have written about this patent before but have missed the boat entirely. Sony's proposed 3D controller will add great depth to video gaming in general and multiplayer gaming in particular. Sony is even thinking of adding 3D video-centric glasses to the mix over time. On another level, it's simply a potential Nintendo Wii and Xbox Kinect Killer. At the end of the day, if you love 3D video gaming and especially multi-player gaming – then you won't want to miss out on our report.
Sony's Patent Overview
Generally, methods and systems are described for multi-player video games and other interactive video presentations for which augmented video is presented on a user's mobile device display based on the relative position of another user. A user could hold up his device and see an avatar, vehicle, game marker, target crossbars, or other object in the place of where the other user is sitting. In some embodiments, the other user's avatar on the display could move, look, etc. in the same manner as the other user's physical movements. For example, if the other user turns toward the first user, the display will show the avatar turning toward him.
In some embodiments, the users could be located in different rooms across town, but their avatars are rendered on their respective mobile device's screens as if their avatars were seated next to each other in the same room. A common reference point for each of the players could be the center of his or her fixed display.
Sony explains that the mobile device used as a 3D controller could be a Playstation Portable, a pair of 3D glasses with integrated display or other unnamed devices.
3D Controller Could also be 3D Glasses
As noted above, one of the future options that Sony's lists as an advanced 3D controller is that of 3D Glasses. Interestingly enough, Sony released a patent on such advanced glasses in a patent shortly after the 3D controller patent came to light. Graphics from that Sony patent are illustrated below.
The sides of the head mounted display system or 3D glasses, incorporates two cameras that are shown above as 111A and 111B. In context with Sony's 3D controller concept, the cameras would be associated with the head tracking attributes of Sony's new gaming system. Sony's initial patent on 3D glasses doesn't provide much information regarding its use with gaming. Though as the project advances, I'm sure that more patents with new details will surface. But for now, it sure sounds like it could be a lot of fun.
Initial Calibration of Sony's Next Generation 3D Controller
Sony's patent FIG. 1 shown below illustrates a first user holding a mobile device at a first position in space relative to a fixed display such as their television. A reference frame is defined with origin 104 at the center of fixed display102 and a polar/cylindrical angle of zero projecting perpendicularly from the plane of the screen. Game player 106 (P1) holds the mobile device (108) in front of him. The point at the top middle rear of the mobile device is noted as the reference point (110).
Why is this important? Because the initial set up of the 3D controller is designed to figure out the exact distances between the advanced controller and your television. This is done so that the controller will be able to move independently from the general view of what's taking place in the game on your television.
Examples of the Controller's Independence from the Regular Gaming Action
Using the coordinates representing the position and view direction, the mobile display is able to be used as a secondary display to `look around` the virtual environment. In a war game, for instance, a player driving a virtual tank will be able to quickly swing their 3D controller to the left to see enemy troops that are outside of the view of their fixed display or television. Additionally or alternatively, the new 3D controller will be able to be used as virtual binoculars to, for instance, better resolve figures in the distance that might be a threat to you in the game. You'll be able to use the virtual binoculars independent of your television's view of the game.
That's a wild idea. Sometime you want to keep the camera still on what's ahead in your game but still want to be able to look around for ammo, team mates, and health and so forth. This will allow you to that.
Advanced Multiplayer Interactivity
Sony's patent FIG. 7 shown below illustrates a virtual relative direction and range from a first game player to a second game player. The second opposing player 206 could be made to appear as a mirror or 180.degree rotated image through the center of the television (or other display). Using vector addition and subtraction, the opposing player appears to be across from the first player 106. If the opposing player steps forward or move toward the left of his screen (see figure), the display on player 106's mobile device 108 will show the opposing player's avatar move right.
This mirrored movement could be useful to simulate games in which players play across from one another, such as tennis, handball, chess, etc. This could be used by players in the same room with the same, central television or by players in different rooms with their own displays. This type of multiplayer interactivity goes far beyond the abilities of Nintendo's Wii or Microsoft's Kinect.
In Sony's patent FIG. 8 noted above, we see a screen view of an avatar in the virtual direction and range from the first to the second players as shown in FIG. 7. As the second player 206 physically moves in front of his television, player 106 (not shown in this figure) sees avatar 814 representing the second player move across the display on the integrated display of his mobile device 108. If the mobile device is moved, the tennis court, avatar and other elements of the view move oppositely so that it appears that the virtual world is inertially stabilized with respect to the real, physical world.
Sony's new Gaming Camera System Could be used for Facial and Head Tracking
About Sony's Off-Board Camera System: In Sony's patent FIG. 9 shown below we could see an off-board camera system for tracking the position of a mobile device. The Video camera (920) sits in a convenient, fixed position atop the television and tracks the mobile device 908 using infrared, radio frequency, visible light, or other suitable methods.
Sony's Camera could also be enabled to track faces as is known in the art. Facial tracking technology could work to directly determine the position and view direction of a player's head, eyes, nose, etc. A camera on the mobile device could also be used to track the player's head.
The PlayStation connects to camera 920 and television via a wired connection while connecting to the 3D controller via wireless port 924. The camera may output the position of tracked objects or raw video to the PlayStation that processes the raw video to determine the position, velocity, etc. of tracked objects.
The PlayStation could send the coordinates of the tracked objects to the mobile device along with the determined view direction of the mobile device which could then use the coordinates and view direction to render an avatar in the correct position on its screen.
In some embodiments, the wireless link 926 could be used to send remote control-like commands to the video display. For example, a cellular phone could be used to turn up or down the volume on a television.
About Sony's On-Board Camera System: In Sony's patent FIG. 10 we see an on-board camera system for tracking the position of a mobile device. The mobile device noted as #1008 includes video camera 1020. The video camera tracks the rectangular screen of a television, markers on the screen, or markers off the screen, such as infrared sources as known in the art.
The markers on the screen could be in the corners of the television screen and be rendered at a predetermined frequency so that mobile device could positively track the screen. The markers could include bar codes, two-dimensional codes, or be modulated in time to send information from the TV to the mobile device. A camera on the mobile device facing the player could also be used to track the player's head.
The position of the mobile device could be used as an input to a video game. For example, a user could pace around his living room floor, marking locations where she will have her battleships for a virtual board game of Battleship.
In other embodiments, the mobile device could automatically determine its position and view direction using a Global Positioning System (GPS) receiver, accelerometer-based inertial system, mechanical or solid-state gyroscope, electronic magnetic compass, radio frequency triangulation, and/or other methods known in the art.
Another Aspect of the 3D Controller is Better Understood through Sony's Invisimals
Trying to follow Sony's patent regarding patent figures 4 and 5 is a little difficult o convey. To get a superior understanding of these patent figures, you could think of them describing an advanced version of Sony's Invisimals. Check out the first video on that linked page to see what Sony is conveying in this segment of the patent. It's not what the heart of the patent is all about, but it has its place in the overall market for this advanced 3D controller system.
The current version of Invisimals uses a clunky camera attachment whereas the proposed next generation portable Playstation will have the camera integrated into the device. Of course, Sony's "Invisimals" game is designed for a younger audience – which shouldn't be confused with the more advanced games that are mentioned earlier in the report.
Sony's new controller will offer you that ability. Sony's main patent applicationregarding the 3D controller was originally filed in Q3 2010 and published in Q1 2012.
Notice: Patent Bolt presents a detailed summary of patent applications with associated graphics for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for full and accurate details. Revelations found in patent applications shouldn't be interpreted as rumor or fast-tracked according to rumor timetables. About Comments: Patent Bolt reserves the right to post, dismiss or edit comments.
