Whether Star Trek or Halo played any role in influencing Microsoft's engineers, I don't know. But what I do know is that Microsoft has been working on an advanced wearable computer system since at least 2008 without the press getting any wind of it. The heart of Microsoft's patent relates to a wearable electromyography-based computer controller which could be worn on or incorporated into an armband, wristwatch, shirt, pair of gloves or even eyeglasses. The wearable computer will be able to control devices such as a music player, television, game console and much more. Although wearable computers may take some time to come to market, it's clear that they're going to play a role in the next wave of technology that tech companies will be tackling – and Microsoft is off to good start.
Two of the Basics behind a Wearable Electromyography Based Controller
A "Wearable Electromyography-Based Controller" provides a physical device worn by or otherwise attached to a user that directly senses and decodes electrical signals produced by human muscular activity using surface Electromyography (sEMG) sensors. The resulting electrical signals provide a muscle-computer interface for use in controlling or interacting with one or more computing devices or other devices coupled to a computing device.
Human-computer interfaces (HCl) have been primarily implemented by monitoring direct manipulation of devices such as mice, keyboards, pens, dials, touch sensitive surfaces, etc. However, as computing and digital information becomes integrated into everyday environments, situations arise where it may be inconvenient or difficult to use hands to directly manipulate an input device. For example, a driver attempting to query a vehicle navigation system might find it helpful to be able to do so without removing his or her hands from the steering wheel. Further, a person in a meeting may wish to unobtrusively and perhaps invisibly interact with a computing device. Unfortunately, the general assumptions described above with respect to the setup and use of conventional EMG sensors and signal measurement tend to make the use and setup of conventional EMG systems impractical for typical HCl purposes which allow a user to control and interact with computing systems, applications, and attached devices.
Overview of the Wearable Electromyography-Based Controller
Microsoft's invention relates to a "Wearable Electromyography-Based Controller" which could implemented into various device form factors and worn by the user or temporarily attached to the user's body. In combination with associated initialization and configuration software and user interface techniques, the Wearable Electromyography-Based Controller provides a human computer interface (HCl) device that allows the user to control and interact with computing systems and attached devices via electrical signals generated by the movement of the user's muscles following initial automated self-calibration and positional localization processes. In other words, the Wearable Electromyography-Based Controller provides a user wearable muscle-computer interface (muCI).
In general, the Wearable Electromyography-Based Controller includes one or more integrated Electromyography (EMG) sensor nodes. The EMG sensor nodes within the Wearable Electromyography-Based Controller measure muscle electrical activity for use in muscle-computer interaction applications. However, unlike conventional Electromyography (EMG) measurement systems, the Wearable Electromyography-Based Controller described herein requires only general positional placement on the user's body. In fact, this general placement of the Wearable Electromyography-Based Controller is enabled by including more EMG sensors than are expected to be necessary to measure muscle electrical activity. An automated positional localization process is then used to automatically identify and select a subset of some or all of the sensor nodes that are in an appropriate position to collect muscle electrical signals corresponding to particular user gestures or movements.
More specifically, because various embodiments of the Wearable Electromyography-Based Controller includes an excess of EMG sensors, the initial positional localization process allows the overall system to self-select a set of one or more appropriate sensor nodes within the Wearable Electromyography-Based Controller in order to capture appropriate muscle electrical signals for controlling and interacting with computing systems and attached devices.
The Scope of Devices Associated with Wearable Electromyography-Based Controllers
As noted above, the Wearable Electromyography-Based Controller could be implemented in various forms, including wearable devices or articles of clothing. For example, the Wearable Electromyography-Based Controller could be implemented as an armband, a wristwatch, eyeglasses (with sensors integrated into the frame), a shirt, gloves, or other article of clothing worn by the user, or any other physical device or collection of devices worn by the user. Further, it should also be understood that a user could wear multiple Wearable Electromyography-Based Controllers, with each such Wearable Electromyography-Based Controller being used to interact with either the same or a different computing device or application.
Further, it should also be understood that a user could wear multiple Wearable Electromyography-Based Controllers, with each such Wearable Electromyography-Based Controller being used to interact with either the same or different computing devices, applications, or other attached devices.
For example, in patent FIG. 2 shown below the EMG sensor nodes of the Wearable Electromyography-Based Controller are placed in a simple band which is worn around the users forearm in order to sense muscle activity associated with specific finger and hand gestures.
According to Microsoft's filing, once the armband has been placed, as muscles contract in the forearm, the EMG sensor nodes (215, 220, 225, 230, 235 and 240) will detect the corresponding electrical activity and transmit the signals. The armband 200 may transmit raw signals or it may have a processor to perform some initial signal processing. The armband will be able to recognize particular user gestures and/or motions and transmit signals indicating that particular actions corresponding to those gestures of motions are to be performed in video games, for instance.
To clarify, we see that under Patent Point #139 Microsoft states that system will work with game consoles or video games operating on such consoles …"
Yet in the big picture, Microsoft notes that it should be understood that the Wearable Electromyography-Based Controller could in fact provide users with a "universal" input mechanism that could be used to control any computing device, the applications running on computing devices, electronic or mechanical devices coupled to a computing device, or any other electronic device (television, radio, appliance, light switch, etc.) having an appropriate infrastructure or interface for receiving input from a wired or wireless controller.
It should also be appreciated that the control and interface capabilities provided by the Wearable Electromyography-Based Controller are potentially invisible in the sense that a user wearing one or more such controllers could remotely interact with various devices without anyone else being able to see or hear any overt actions by the user to indicate that the user is interacting with such devices. This could apply to controlling a music player, a pan-tilt-zoom camera, a home automation system, a game console, television and beyond.
Similarly, the Wearable Electromyography-Based Controller could also be used to provide control of electromechanical prosthetic devices such as prosthetic hands, arms, legs, etc., by performing particular motions or gestures which in turn cause specific muscles of the user to generate electrical signals that are then used to activate one or more predetermined motions in the prosthetic device.
The system will also implement various forms of haptics which could include the playing of audible feedback include playing a tone, sound clip, music, speech, or other audio output when a command has been successfully entered or executed. Conversely, visual, and/or audible feedback may also be used to provide negative feedback to indicate that a command wasn't successfully entered or executed in response to a particular gesture or motion by the user.
Overview of Program Modules
Microsoft's patent FIG. 1 shown below provides an exemplary architectural flow diagram that illustrates program modules for implementing various embodiments of the Wearable Electromyography-Based Controller.
Sets of Individual Sensor Nodes Including Sound
Microsoft notes that the techniques described in their filing could also be used with one or more sets of individual EMG sensor nodes, such that a wearable device is not necessary. In this case, the individual EMG sensor nodes may be placed on various positions on the user's body without regard for precise location. General examples of such embodiments are illustrated in patent FIG. 3 below which shows sets of individual sensor nodes 300, 320, 340 or 360, placed in various positions on various parts of the user's body, including the user's arm, leg, chest and head, 310, 330, 350 and 370, respectively.
Additionally, we see an example of using a separate sound source in patent FIG. 8 that's on the surface of the user's forearm 810. As illustrated by FIG. 8, sound waves 820 (illustrated as curved broken lines) radiate out from the sound source 800, and impinge on various sensor nodes 830 which then report to a sound hub. Microsoft adds that patent FIG. 8 illustrates the use of a separate sound source 800 placed by the user, the sound source may be implemented by using individual microphones in each sensor node 830 as a speaker device, or by integrating one or more speakers or sound transducers into the Wearable Electromyography-Based Controller (such as, for example an armband, wristwatch, glove, article of clothing, etc.).
Exemplary Operating Environments
The Wearable Electromyography-Based Controller described in Microsoft's patent filing is operational for interfacing with, controlling, or otherwise interacting with numerous types of general purpose or special purpose computing system environments or configurations, or with devices attached or coupled to such computing devices.
For example, patent FIG. 9 below provides a simple diagram which illustrates a plurality of wireless sensor nodes (905, 910, 915, 920, 925, 930, and 935) acting either as individually placed sensors or as sensor nodes embedded in a device such as the armband described earlier in patent FIG. 2. As illustrated in FIG. 9, these wireless sensor nodes are variously in communication with each other and one or more devices, including a wristwatch 940, a digital media player 945, a PC-type notebook computer 950, and a cell phone 955. Note also that FIG. 9 illustrates the wristwatch acting as a "hub" in the case, as a wireless intermediary between one or more of the sensor nodes and the digital media player 945.
Microsoft also lists a number of general purpose computers may be implemented in concert with a Wearable Electromyography-Based Controller, as follows: personal computers, server computers, hand-held computing devices, laptop or mobile computers, communications devices such as cell phones and PDA's, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, video media players, in-vehicle computing systems (e.g., automotive computer system), etc.
Microsoft states that programming one of these controllers will be extremely easy and that there could be a voice activated system to guide users through the startup process. Though something tells me that it could be trickier than they make it out to be, but who knows; maybe they'll get it right out of the starting gate. Only time will tell.
For more information on this invention, see Microsoft's patent application 20120188158.
Note to Referring Sites: We ask that referring sites limit the use of our graphics to a maximum of two per report. Thank you for your cooperation.
Notice
The Patent Bolt blog presents a detailed summary of patent applications with associated graphics for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for full and accurate details. Revelations found in patent applications shouldn't be interpreted as rumor or fast-tracked according to rumor timetables. About Comments: Patent Bolt reserves the right to post, dismiss or edit comments.
