The US Patent and Trademark Office have published a patent application from Microsoft that reveals that the software giant has their eye on a next generation Video Conferencing system. The technology that will make this system unique from any of their competitors is that the video camera associated with the new software will be trained to follow the user's body within a given space in a living room for instance. So as you move about, the camera will be able to follow you and keep you in focus and in the center of the frame. This certainly looks like it could be a winner for Microsoft, but only time will tell.
A Problem with Video Conferencing Today
Video communication systems such as video conferencing systems and video chat systems provide multiple participants located at different locations the ability to communicate with each other. Video communication systems may utilize web cameras and microphones to capture and transmit audio/video content to other users, in real time, over a network. However, video communication typically requires a user to be stationery by being in front of the camera while communicating with a remote user at a remote computing device. Any motion by the user typically results in the user no longer being optimally viewed by the camera and by the remote user at the remote computing device.
Microsoft's Body Tracking System for Superior Video Conferencing Experiences
Microsoft's invention relates to a method and system by which a user participating in a video chat application with a remote user at a remote computing device can move within a field of view of a camera connected to the user's computing device and still be optimally viewed by the camera and by the remote user.
In one embodiment, a position of the head, neck and shoulders of the user is automatically tracked by the camera as the user moves to different locations within the field of view, thereby eliminating the necessity of the user to be stationery and in front of the camera while interacting with the video chat application. A sub-frame of pixels containing the position of the head, neck and shoulders of the user in a first location within the field of view is identified and displayed to the remote user at the remote computing device. The user's movement to a next location within the field of view is detected by automatically tracking the position of the head, neck and shoulders of the user in the field of view. A next sub-frame of pixels identifying a position of the head, neck and shoulders of the user in the next location is identified and displayed to the remote user at the remote computing device.
Microsoft's patent FIGS 9A to 9D illustrate screenshots of a user's movement to different locations within a field of view of a video conferencing system's camera that is built into devices such as a TV, Microsoft's Kinect , Xbox or the like.
The System uses an Advanced Body Scanning System
Microsoft's video conferencing system is likely to be an extension of Kinect as it uses the same technology that scans the body for interactive video games for the Xbox. In a video conferencing system, the body scan is used to ensure that the camera is tracking the user's body movements so as to keep the user in the center of the camera's frame at all times.
Microsoft states that the skeletal model of the user may be generated by scanning information such as bits or pixels associated with the user to determine one or more body parts associated with the user. In one example, the user may be isolated and a bit mask may be created to scan for one or more body parts associated with the user.
Microsoft discusses how the camera system will automatically track different movements such as lateral or depth every few seconds. The skeletal model that the system creates with a body scan can be adjusted and updated in real time.
The Creepy yet Necessary Skeletal Scan
In Microsoft's patent FIG. 8 we see an example of a skeletal model or mapping (#840) representing a scanned human target. According to one embodiment, the skeletal model may include one or more data structures that may represent a human target as a three-dimensional model. Each body part may be characterized as a mathematical vector defining joints and bones of the skeletal model.
As noted above, the skeletal model includes joints n1-n18. The bones and joints may collectively make up a skeletal model which allows the video conferencing to accurately move the camera to keep the user centered in the frame at all times. The whole idea of body scan is a little creepy. I've seen this many times while investigating several Xbox/Kinect patents and always wondered if this scan is actually healthy. Perhaps that'll be a subject to tackle another day.
The Evidence Points to this Being a Future Application for Kinect
For now, you could check Microsoft's Kinect news report from March that discusses how Bodymetrics, a London-based pioneer in 3D body-mapping, is using Kinect to scan a consumers body in just 5 seconds. Bodymetrics is now trying out their new system at Bloomingdale's.
Microsoft reported that "Bloomingdale's, a leader in retail innovation, has one of the largest offerings in premium denim from fashion-forward brands like J Brand, 7 for all mankind, Citizens and Humanity, AG, and Paige. The Bodymetrics services allow customers to get their body mapped and find jeans that fit and flatter their unique shape from the hundreds of different jeans styles that Bloomingdale's stocks."
The very same body scanning capabilities of Kinect as noted above could be utilized in a future Microsoft Video Conferencing Application. In February we covered another Kinect related report and noted that Microsoft was in advanced talks with the likes of Vizio and Sony to incorporate Kinect directly into future HDTVs. Like our opening graphic suggests, Microsoft is likely focusing their Video Conferencing system for the living or family room where it would be used the most. And having built right into a future HDTV only makes sense.
The good news is that Microsoft has a solid history with Kinect products and their new Kinect web site shows us how it's being adopted outside of the home as well as slowly working itself into future PC's. So this isn't a vaporware concept but rather one that is likely to emerge in the coming years.
Microsoft's video conferencing app will make video conferencing more natural if the camera is designed to move with the targeted user when needed. Considering that they've already pulled this off seemlessly for Xbox gaming, it's easy to why this should work seemlessly with their next generation video conferencing system as well. It's really just an extension of an existing proven technology.
Microsoft's patent application was originally filed in Q4 2010 and published by the US Patent and Trademark Office this month. Darren Apfel, one of the engineers who developed the new system, has worked on the Xbox and Windows Live projects.
Notice
The Patent Bolt blog presents a detailed summary of patent applications with associated graphics for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for full and accurate details. Revelations found in patent applications shouldn't be interpreted as rumor or fast-tracked according to rumor timetables. About Comments: Patent Bolt reserves the right to post, dismiss or edit comments.
