News Column

"Preserving Audio Data Collection Privacy in Mobile Devices" in Patent Application Approval Process

July 9, 2014



By a News Reporter-Staff News Editor at Telecommunications Weekly -- A patent application by the inventors Grokop, Leonard Henry (San Diego, CA); Narayanan, Vidya (San Jose, CA); Dolter, James W. (San Diego, CA); Nanda, Sanjiv (Ramona, CA), filed on February 21, 2014, was made available online on June 26, 2014, according to news reporting originating from Washington, D.C., by VerticalNews correspondents.

This patent application is assigned to QUALCOMM Incorporated.

The following quote was obtained by the news editors from the background information supplied by the inventors: "Mobile devices are incredibly widespread in today's society. For example, people use cellular phones, smart phones, personal digital assistants, laptop computers, pagers, tablet computers, etc. to send and receive data wirelessly from countless locations. Moreover, advancements in wireless communication technology have greatly increased the versatility of today's mobile devices, enabling users to perform a wide range of tasks from a single, portable device that conventionally required either multiple devices or larger, non-portable equipment.

"For example, mobile devices can be configured to determine what environment (e.g., restaurant, car, park, airport, etc.) a mobile device user may be in through a process called context determination. Context awareness applications that perform such context determinations seek to determine the environment of a mobile device by utilizing information from the mobile device's sensor inputs, such as GPS, WiFi and BlueTooth.RTM.. In many scenarios, classifying audio from the mobile device's microphone is highly valuable in making context determinations, but the process of collecting audio that may include speech can raise privacy issues."

In addition to the background information obtained for this patent application, VerticalNews journalists also obtained the inventors' summary information for this patent application: "Techniques disclosed herein provide for using the hardware and/or software of a mobile device to obscure speech in the audio data before a context determination is made by a context awareness application using the audio data. In particular, a subset of a continuous audio stream is captured such that speech (words, phrases and sentences) cannot be reliably reconstructed from the gathered audio. The subset is analyzed for audio characteristics, and a determination can be made regarding the ambient environment.

"In some embodiments, a method of privacy-sensitive audio analysis is presented. The method may include capturing a subset of audio data contained in a continuous audio stream. The continuous audio stream may contain human speech. The subset of audio data may obscure content of the human speech. The method may include analyzing the subset of audio data for audio characteristics. The method may include making a determination of an ambient environment, based, at least in part, on the audio characteristics.

"Embodiments of such a method may include one or more of the following: The subset of audio data may comprise a computed function of the continuous audio stream having a lesser number of bits than is needed to reproduce the continuous audio stream with intelligible fidelity. The subset of audio data may comprise a plurality of audio data segments, each audio data segment comprising data from a different temporal component of the continuous audio stream. The method may include making a determination of an identity of a person based, at least in part, on the audio characteristics. The plurality of audio data segments may comprise between 30 ms to 100 ms of recorded audio. Each temporal component of the continuous audio stream may be between 250 ms to 2s in length. The method may include randomly altering an order of the plurality of audio data segments before analyzing the subset of audio data. Randomly altering the order of the plurality of audio data segments may be based, at least in part, on information from one of: a Global Positioning System (GPS) device, signal noise from circuitry within a mobile device, signal noise from a microphone, and signal noise from an antenna.

"In some embodiments, a device for obscuring privacy-sensitive audio is presented. The device may include a microphone. The device may include a processing unit communicatively coupled to the microphone. The processing unit may be configured to capture a subset of audio data contained in a continuous audio stream represented in a signal from the microphone. The continuous audio stream may contain human speech. The subset of audio data may obscure content of the human speech. The processing unit may be configured to analyze the subset of audio data for audio characteristics. The processing unit may be configured to make a determination of an ambient environment, based, at least in part, on the audio characteristics.

"Embodiments of such a device may include one or more of the following: The subset of audio data may comprise a computed function of the continuous audio stream having a lesser number of bits than is needed to reproduce the continuous audio stream with intelligible fidelity. The subset of audio data may comprise a plurality of audio data segments, each audio data segment comprising data from a different temporal component of the continuous audio stream. The processing unit may be configured to make a determination of an identity of a person based, at least in part, on the audio characteristics. Each of the plurality of audio data segments may comprise between 30 ms to 100 ms of recorded audio. Each temporal component of the continuous audio stream may be between 250 ms to 2s in length. The device wherein the processing unit is further configured to randomly altering an order of the plurality of audio data segments before analyzing the subset of audio data. Randomly altering the order of the plurality of audio data segments may be based, at least in part, on information from one of: a Global Positioning System (GPS) device, signal noise from circuitry within a mobile device, signal noise from the microphone, and signal noise from an antenna.

"In some embodiments, a system for determining an environment associated with a mobile device is presented. The system may include an audio sensor configured to receive a continuous audio stream. The system may include at least one processing unit coupled to the audio sensor. The processing unit may be configured to capture a subset of audio data contained in the continuous audio stream, such that the subset of audio data obscures content of human speech included in the continuous audio stream. The processing unit may be configured to analyze the subset of audio data for audio characteristics. The processing unit may be configured to make a determination of an ambient environment, based, at least in part, on the audio characteristics.

"Embodiments of such a system may include one or more of the following: The system may include a network interface configured to send information representing the subset of audio data via a network to a location remote from the mobile device. The at least one processing unit may be configured to make the determination of the ambient environment at the location remote from the mobile device. The subset of audio data may comprise a plurality of audio data segments, each audio data segment comprising data from a different temporal component of the continuous audio stream.

"The at least one processing unit may be configured to make a determination of an identity of a person based, at least in part, on the audio characteristics. Each of the plurality of audio data segments may comprise between 30 ms to 100 ms of recorded audio. Each temporal component of the continuous audio stream may be between 250 ms to 2s in length. The processing unit may be further configured to randomly alter an order of the plurality of audio data segments before analyzing the subset of audio data.

"In some embodiments, a computer program product residing on a non-transitory processor-readable medium is presented. The non-transitory processor-readable medium includes processor-readable instructions configured to cause a processor to capture a subset of audio data contained in a continuous audio stream. The continuous audio stream may contains human speech. The subset of audio data may obscure content of the human speech. The processor-readable instructions may be configured to cause the processor to analyze the subset of audio data for audio characteristics. The processor-readable instructions may be configured to cause the processor to make a determination of an ambient environment, based, at least in part, on the audio characteristics.

"Embodiments of such a computer program product may include one or more of the following: The subset of audio data may comprise a computed function of the continuous audio stream having a lesser number of bits than is needed to reproduce the continuous audio stream with intelligible fidelity. The subset of audio data may comprise a plurality of audio data segments, each audio data segment comprising data from a different temporal component of the continuous audio stream. The processor-readable instructions may be configured to cause the processor to make a determination of an identity of a person based, at least in part, on the audio characteristics. Each of the plurality of audio data segments may comprise between 30 ms to 100 ms of recorded audio. Each temporal component of the continuous audio stream may be between 250 ms to 2s in length. The processor-readable instructions may be configured to randomly alter an order of the plurality of audio data segments before analyzing the subset of audio data. The processor-readable instructions for randomly altering the order of the plurality of audio data segments is based, at least in part, on information from one of: a Global Positioning System (GPS) device, signal noise from circuitry within a mobile device, signal noise from a microphone, and signal noise from an antenna.

"In some embodiments, a device for obscuring privacy-sensitive audio is presented. The device may include means for capturing a subset of audio data contained in a continuous audio stream represented in a signal from a microphone. The continuous audio stream may contain human speech. The subset of audio data may obscure content of the human speech. The device may include means for analyzing the subset of audio data for audio characteristics. The device may include means for determining an ambient environment, based, at least in part, on the audio characteristics.

"Embodiments of such a device may include one or more of the following: The means for capturing the subset of audio data may be configured to capture the subset of audio data in accordance with a computed function of the continuous audio stream having a lesser number of bits than is needed to reproduce the continuous audio stream with intelligible fidelity. The means for capturing the subset of audio data may be configured to capture the subset of audio data such that the subset of audio data comprises a plurality of audio data segments, each audio data segment comprising data from a different temporal component of the continuous audio stream. The means for determining the ambient environment may be configured to make a determination of an identity of a person based, at least in part, on the audio characteristics. The means for capturing the subset of audio data may be configured to capture the subset of audio data such that each of the plurality of audio data segments comprises between 30 ms to 100 ms of recorded audio.

"Items and/or techniques described herein may provide one or more of the following capabilities, as well as other capabilities not mentioned. Obscuring of the content of speech that may be included in an audio stream used for a context determination while having little or no impact on the accuracy of the context determination. Utilizing a relatively simple method that can be executed in real time, using minimal processing resources. Including an ability to upload a subset of audio data (having obscured speech) to help improve the accuracy of models used in context determinations. While at least one item/technique-effect pair has been described, it may be possible for a noted effect to be achieved by means other than that noted, and a noted item/technique may not necessarily yield the noted effect.

BRIEF DESCRIPTION OF THE DRAWINGS

"An understanding of the nature and advantages of various embodiments may be facilitated by reference to the following figures. In the appended figures, similar components or features may have the same reference label. Further, various components of the same type may be distinguished by following the reference label by a dash and a second label that distinguishes among the similar components. If only the first reference label is used in the specification, the description is applicable to any one of the similar components having the same first reference label irrespective of the second reference label.

"FIG. 1 is a simplified block diagram of basic components of a mobile device configured to support context awareness applications, according to one embodiment.

"FIGS. 2a-2c are visualizations of processes for capturing sufficient audio information to classify the ambient environment of a mobile device without performance degradation, while helping ensure privacy of speech.

"FIGS. 3a and 3b are flow diagrams of methods for providing the functionality shown in FIGS. 2b and 2c.

"FIG. 4 is a graph illustrating results of an analysis computing an upper bound on the probability of a speech recognizer reconstructing n-grams of words, from audio data resulting from certain processing methods described herein."

URL and more information on this patent application, see: Grokop, Leonard Henry; Narayanan, Vidya; Dolter, James W.; Nanda, Sanjiv. Preserving Audio Data Collection Privacy in Mobile Devices. Filed February 21, 2014 and posted June 26, 2014. Patent URL: http://appft.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=HITOFF&u=%2Fnetahtml%2FPTO%2Fsearch-adv.html&r=1375&p=28&f=G&l=50&d=PG01&S1=20140619.PD.&OS=PD/20140619&RS=PD/20140619

Keywords for this news article include: Machine Learning, Emerging Technologies, QUALCOMM Incorporated, Government Agencies Offices and Entities.

Our reports deliver fact-based news of research and discoveries from around the world. Copyright 2014, NewsRx LLC


For more stories covering the world of technology, please see HispanicBusiness' Tech Channel



Source: Telecommunications Weekly


Story Tools






HispanicBusiness.com Facebook Linkedin Twitter RSS Feed Email Alerts & Newsletters