News Column

"Image Processing Device, Object Selection Method and Program" in Patent Application Approval Process

July 15, 2014



By a News Reporter-Staff News Editor at Information Technology Newsweekly -- A patent application by the inventors FUKUCHI, Masaki (Tokyo, JP); MATSUDA, Kouichi (Tokyo, JP); SUTO, Yasuhiro (Tokyo, JP); OI, Kenichiro (Tokyo, JP); GUO, Jingjing (Tokyo, JP), filed on February 26, 2014, was made available online on July 3, 2014, according to news reporting originating from Washington, D.C., by VerticalNews correspondents.

This patent application is assigned to Sony Corporation.

The following quote was obtained by the news editors from the background information supplied by the inventors: "The present invention relates to an image processing device, an object selection method and a program.

"The number of electronic devices which users should operate at home or office is increasing year by year. Further, the functions of one electronic device are becoming increasingly diversified. Accordingly, a burden on a user who operates such electronic devices is getting heavier. In light of this, techniques for enabling the operation of a plurality of electronic devices with use of one remote control device have been developed. However, in order to achieve integration of remote control functions into one device, it is required to provide a means of identifying which device a user intends to operate when there are a plurality of remotely operable devices to be controlled in the environment where the user is located.

"For example, Japanese Patent Application Laid-Open No. 11-345024 proposes a remote control device that recognizes a remote control target device and details of control from a speech of a user by employing speech recognition technology."

In addition to the background information obtained for this patent application, VerticalNews journalists also obtained the inventors' summary information for this patent application: "However, in the recognition of a remote control target device and details of control from a speech, recognition process becomes complicated when there are a plurality of same type devices, especially in distinguishing between them. For example, when television sets are located both in a living room and a room next to the living room, an intended television set cannot be specified appropriately unless a modifier is added, such as 'the smaller' TV or TV in 'the living room'. Further, when a word that specifies a target device could not be recognized, there is a possibility that another device (another television or a device of another type) is turned off despite that a user intended to turn off one television.

"Further, although a remote control target device and the like could be recognized from a gesture of a user, for example, it is difficult to distinguish between a plurality of devices that are located at positions which are lined up along the user's line of sight (or that are located in close proximity to each other) only with a gesture.

"In light of the foregoing, it is desirable to provide novel and improved image processing device, object selection method and program capable of distinguishing between a plurality of electronic devices and easily selecting one as a target of remote control.

"According to an embodiment of the present invention, there is provided an image processing device including: a data storage unit that stores object identification data for identifying an object operable by a user and feature data indicating a feature of appearance of each object; an environment map storage unit that stores an environment map representing a position of one or more objects existing in a real space and generated based on an input image obtained by imaging the real space using an imaging device and the feature data stored in the data storage unit; and a selecting unit that selects at least one object recognized as being operable based on the object identification data, out of the objects included in the environment map stored in the environment map storage unit, as a candidate object being a possible operation target by a user.

"The image processing device may further include an image output unit that generates an output image for allowing a user to recognize a position on a screen of the candidate object selected by the selecting unit from the input image and outputs the generated output image to a display device.

"The environment map storage unit may further store imaging position data indicating a position of the imaging device determined based on the input image and the feature data, and a position of each candidate object in the output image is calculated based on the environment map and the imaging position data stored in the environment map storage unit.

"The image processing device may further include a user interface for allowing a user to specify an object to be operated, out of candidate objects.

"The user interface may determine an object located at a center or near the center of the output image at a time when user input is detected, out of candidate objects, as the object to be operated.

"When a plurality of candidate objects are located at the center of the output image when user input is detected, the user interface may determine the candidate object with a smallest size in the output image, out of candidate objects, as the object to be operated.

"The feature data may include shape data indicating a three-dimensional shape of each object, and the image output unit may generate the output image by generating a frame surrounding each candidate object on the screen based on the shape data and superposing the generated frame on the input image.

"The image processing device may further include: the imaging device and the display device, and an imaging plane of the imaging device and a screen of the display device are arranged to face opposite directions from each other.

"According to another embodiment of the present invention, there is provided an image processing method in an image processing device including a data storage unit that stores object identification data for identifying an object operable by a user and feature data indicating a feature of appearance of each object, the method including steps of: generating an environment map representing a position of one or more objects existing in a real space based on an input image obtained by imaging the real space using an imaging device and the feature data stored in the data storage unit; and selecting at least one object recognized as being operable based on the object identification data, out of the objects included in the generated environment map, as a candidate object being a possible operation target by a user.

"According to another embodiment of the present invention, there is provided a program for causing a computer for controlling an image processing device including a data storage unit that stores object identification data for identifying an object operable by a user and feature data indicating a feature of appearance of each object to function as: an environment map generating unit that generates an environment map representing a position of one or more objects existing in a real space based on an input image obtained by imaging the real space using an imaging device and the feature data stored in the data storage unit; and a selecting unit that selects at least one object recognized as being operable based on the object identification data, out of the objects included in the environment map generated by the environment map generating unit, as a candidate object being a possible operation target by a user.

"According to the embodiment of the present invention described above, it is possible to provide an image processing device, an object selection method and a program capable of distinguishing between a plurality of electronic devices and easily selecting one as a target of remote control.

BRIEF DESCRIPTION OF THE DRAWINGS

"FIG. 1 is a schematic diagram for illustrating an overview of an image processing device according to an embodiment;

"FIG. 2 is a block diagram illustrating an example of a configuration of an image processing device according to an embodiment;

"FIG. 3 is a block diagram illustrating an example of a detailed configuration of an environment map generating unit according to an embodiment;

"FIG. 4 is a flowchart illustrating an example of a flow of a self-position detection process according to an embodiment;

"FIG. 5 is an illustrative diagram for illustrating a feature point set on an object;

"FIG. 6 is an illustrative diagram for illustrating addition of a feature point;

"FIG. 7 is an illustrative diagram for illustrating an example of a prediction model;

"FIG. 8 is an illustrative diagram for illustrating an example of a configuration of feature data;

"FIG. 9 is a flowchart illustrating an example of a flow of an object recognition process according to an embodiment;

"FIG. 10 is an illustrative diagram for illustrating an example of a configuration of object identification data;

"FIG. 11 is a flowchart illustrating an example of a flow of a device recognition process according to an embodiment;

"FIG. 12 is a flowchart illustrating an example of a flow of a candidate object selection process according to an embodiment;

"FIG. 13 is an illustrative diagram illustrating an example of an output image generated by an image output unit according to an embodiment;

"FIG. 14 is an illustrative diagram illustrating another example of an output image generated by an image output unit according to an embodiment;

"FIG. 15 is an illustrative diagram illustrating yet another example of an output image generated by an image output unit according to an embodiment;

"FIG. 16 is an illustrative diagram illustrating still another example of an output image generated by an image output unit according to an embodiment;

"FIG. 17 is an illustrative diagram illustrating still another example of an output image generated by an image output unit according to an embodiment;

"FIG. 18 is an illustrative diagram illustrating still another example of an output image generated by an image output unit according to an embodiment;

"FIG. 19 is a block diagram illustrating an example of a hardware configuration of a general-purpose computer; and

"FIG. 20 is a block diagram illustrating an example of a configuration of an image processing device according to an alternative example."

URL and more information on this patent application, see: FUKUCHI, Masaki; MATSUDA, Kouichi; SUTO, Yasuhiro; OI, Kenichiro; GUO, Jingjing. Image Processing Device, Object Selection Method and Program. Filed February 26, 2014 and posted July 3, 2014. Patent URL: http://appft.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=HITOFF&u=%2Fnetahtml%2FPTO%2Fsearch-adv.html&r=4120&p=83&f=G&l=50&d=PG01&S1=20140626.PD.&OS=PD/20140626&RS=PD/20140626

Keywords for this news article include: Sony Corporation, Information Technology, Information and Data Storage.

Our reports deliver fact-based news of research and discoveries from around the world. Copyright 2014, NewsRx LLC


For more stories covering the world of technology, please see HispanicBusiness' Tech Channel



Source: Information Technology Newsweekly


Story Tools






HispanicBusiness.com Facebook Linkedin Twitter RSS Feed Email Alerts & Newsletters