According to a new report posted today by Tricia Duryee of AllThingsD, Microsoft's Xbox holds a commanding 56 percent of the current generation of gaming consoles sold today. Microsoft states that the Xbox has been the best-selling videogame console for 22 straight months. That certainly puts a lot of pressure on Sony that was once the console leader. A series of new Sony patents have come to light of late that tell us that Sony's engineering teams are looking at wide spectrum of new technologies that could advance their platform in the coming years. Today's report focuses in on two new Sony PlayStation controllers that integrate both a camera and advanced nerve sensors.
The Evolution of the Gaming Interface
To provide some of their invention background, Sony begins by discussing the pros and cons of various forms of gaming interfaces and the need to add a new one to the mix. Sony states that Keyboard interfaces are good for entering text, but less useful for entering directional commands. Joysticks and mice are good for entering directional commands and less useful for entering text. Camera-based interfaces are good for tracking objects in two-dimensions, but generally require some form of augmentation (e.g., use of two cameras or a single camera with echo-location) to track objects in three dimensions. Microphone-based interfaces are good for recognizing speech, but are less useful for tracking spatial orientation of objects. Touch-based interfaces provide more intuitive interaction with a computer program, but often experience latency issues as well as issues related to misinterpreting a user's intentions. It would be desirable to provide an interface that supplements some of the interfaces by analyzing additional characteristics of the user during interaction with the computer program.
A given user of a computer program may exhibit various activity levels in the nervous system during interaction with the computer program. These activity levels provide valuable information regarding a user's intent when interacting with the computer program. Such information may help supplement the functionality of those interfaces described above.
Advancing Sony's PS Controller & Beyond
Looking at Sony's patent FIG. 2 below we're able to see a PlayStation (PS) controller configured to measure nerve activity levels from the user's hands. While the nerve sensors are seen in the patent filing as pertaining to a future PS controller, Sony on several counts claims the invention may extend to different hardware such as a computer mouse, keyboard, joystick, steering wheel, or other interface device. Furthermore, nerve sensors may be included on the case of a hand-held computing device such as a tablet computer or smartphone.
Sony's filing outlines all of the aspects of the standard PS game controller as we know it today and then adds some new features such as a built in camera and nerve sensors that could you could see highlighted in yellow as feature 213. Although the illustration limits the nerve sensors to four, Sony makes it clear that there could be nerve sensors built into the right and left triggers noted about as patent point # 211 or elsewhere in the final product.
Sony states that the built-in camera may be configured to track the position of the fingers with respect to the controller or the acceleration of the fingers. The camera provides supplemental data used to help more accurately determine the relationship between the user's body parts and the components of the controller.
New Wireless Stress Sensor & Special Ring
In Sony's patent FIG. 3A shown below we're able to see an "alternative component" that could be used with either a PS controller and/or handheld device.
Patent FIG. 3A illustrates a wireless stress sensor that is configured to be positioned around a ring (302) which could be placed on a user's finger. The wireless stress sensor (303) measures nerve activity levels of the finger during operation of the computer program (such as a game) by correlating electrical resistance induced by the finger to a nerve activity level.
The wireless stress sensor may additionally include a spring element, noted as patent point # 305 below, which may activate the stress sensor when the user's finger flexes. This spring may also provide supplemental information (e.g., force with which finger is pushing a button on the controller) to facilitate determination of a relationship between the user's finger and the controller.
It is noted that embodiments Sony's invention include implementations that utilize "wearable" nerve sensing devices located on wearable articles other than the ring-based sensor depicted in FIG. 3A. Some other non-limiting examples of wearable nerve sensing devices include nerve sensors that are incorporated into wearable articles such gloves or wrist bands or necklace or Bluetooth headset or a medical patch.
Such wearable nerve sensing devices could be used to provide information to determine if a user is interacting with a virtual user interface that may only be visible to the user. For example, a user could interact with projected or augmented virtual user interfaces by using these wearable nerve sensors to determine when a user is pressing a virtual button or guiding a virtual cursor.
Sony's patent FIG. 3B illustrates an example in which the ring of FIG. 3A is used in conjunction with a hand-held device having a touch interface. The device could be a portable game device, portable internet device, cellular telephone, personal digital assistant, tablet or similar device. The touch interface 307 could be a touch pad, which acts as an input device. Alternatively, the touch interface may be a touch screen, which also acts as both a visual display and an input device.
In a second interrelated patent we're able to get a better look at what Sony is describing above in terms of a new touchscreen controller. Sony's patent FIG. 9 shown below illustrates a user viewing a scene in a game on the gaming controller that would also control the action of the game on the users HDTV via touch controls. The controller looks a little like a modified PSP 3000 system, but one that includes touchscreen capabilities.
Tracking Gamer Eye Gazes During a Game
In this section, Sony describes the use of Gaze-based technologies for games. Sony states "by detecting the nerve or muscle activities at different locations on a user's fingers or arms, it's possible to implement fine control of the touch interface. By way of example and not by way of limitation, the handheld device may include a camera that looks back at the user's face to track the user's eye gaze, e.g., using images from a camera that faces the user."
The filing goes on to state that "Alternatively, gaze may be tracked using an infrared source that projects infrared light towards the user in conjunction with a position sensitive optical detector (PSD). Infrared light from the source may retroreflect from the retinas of the user's eyes to the PSD. By monitoring the PSD signal it is possible to determine the orientation of the user's eyes and thereby determine eye gaze direction. Tracking the user's eye gaze could be used to enhance manipulation of objects displayed on the user's touch screen."
The patent filing states that one way to obtain a user's eye gaze direction involves a pair of glasses (See 409 of FIG. 4 below) and a camera (see 407 of FIG. 4 below). The glasses may include infrared light sensors. The camera is then configured to capture the infrared light paths emanating from the glasses and then triangulate the user's eye gaze direction from the information obtained.
This aspect of the patent was covered in greater detail in a recent report of ours titled "Future Sony Gaming May use Gaze, Gesture & Brainwave Controls." The technology is already way past the experimental Phase.
An Alternative PS Controller Design
On a last note, we point to an alternative concept that Sony is playing with that stems from that second patent application that we pointed to earlier. In that patent filing, Sony points to a PS controller (as shown below) that includes electrodes 138a and 138b for detecting bio-electric signals from the user. The bio-electric signals process biometric data that could be used as an input for an interactive video game.
The Pressure is all on Sony
In September we revealed one of Microsoft's wild next generation concepts for Kinect working with the Xbox and over the last few months the US Patent Office has revealed that Microsoft's engineering teams working on Xbox is pushing the pedal to the medal to find the right combination of gaming experiences that will excite the customer base. You could find a number of those patents covering such things as video headsets and wearable controllers here: One, two, three, four and five.
So without a doubt, the pressure is on Sony to take their gaming platform to the next level quickly before they fall too far behind the curve. For consumers, it's can't get any better. Although it may take a few years before some of these exciting features actually materialize, the fact is that when they do, it'll allow us to tap into new sensory experiences that will put us closer to the action than ever before. As far as I'm concerned, it can't come quick enough.
Sony's patent application was originally filed in Q2 2011 and published in Q3 2012. Marketing timing of anything found in this report is unknown at this time.
Note that technological revelations revealed in Intellectual Property filings are not to be interpreted as rumor. Furthermore, fictitious rumor site timetables associated with this report should be dismissed.
NOTICE: The Patent Bolt blog presents a detailed summary of patent applications with associated graphics for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for full and accurate details. About Comments: Patent Bolt reserves the right to post, dismiss or edit comments.
