News Column

Patent Issued for Image Processing Method, Image Pickup Apparatus, Image Processing Apparatus, and Non-Transitory Computer-Readable Storage Medium

June 19, 2014



By a News Reporter-Staff News Editor at Computer Weekly News -- A patent by the inventors Hiasa, Norihito (Utsunomiya, JP); Hatakeyama, Koshi (Tokyo, JP), filed on August 15, 2012, was published online on June 3, 2014, according to news reporting originating from Alexandria, Virginia, by VerticalNews correspondents.

Patent number 8743245 is assigned to Canon Kabushiki Kaisha (JP).

The following quote was obtained by the news editors from the background information supplied by the inventors: "The present invention relates to an image pickup apparatus that obtains a two-dimensional light intensity distribution and angle information of a ray in an object space, and an image processing method for the image obtained by the image pickup apparatus.

"Recently, an image pickup apparatus that performs a calculation using data obtained by an image pickup element and performs a corresponding digital image processing so as to output various images is proposed. Ren Ng, et al., 'Light Field Photography with a Hand-held Plenoptic Camera', 2005 Computer Science Technical Report CTSR, Todor Georgiev, et al., 'Superresolution with Plenoptic 2.0 Camera', 2009 Optical Society of America, and Aaron Isaksen, et al., 'Dynamically Reparameterized Light Fields' and ACM SIGGRAPH and pp. 297-306 (2000), disclose an image pickup apparatus that obtains the two-dimensional light intensity distribution and the angle information of the ray in the object space using 'Light Field Photography'. The two-dimensional light intensity distribution and the angle information of the ray are collectively referred to as a light field, and three-dimensional information in the object space can be obtained by obtaining the light field. According to such an image pickup apparatus, the light field is obtained and an image processing is performed after taking an image so that a focus position of the image called a refocus, a shot point of view, a depth of field, or the like can be changed.

"However, such an image pickup apparatus needs to use pixels of the image pickup element for storing the angle information of the ray, in addition to storing the two-dimensional light intensity distribution. Therefore, the spatial resolution is deteriorated with respect to an image pickup apparatus which only stores the two-dimensional light intensity distribution. Todor Georgiev, et al., 'Superresolution with Plenoptic 2.0 Camera', 2009 Optical Society of America, discloses a configuration in which a certain point on an image plane that is formed by an imaging optical system is taken by a plurality of small lenses constituting a lens array. A plurality of small images obtained like this are reconstructed so that the resolution of the reconstructed image can be improved. Such a method of improving the resolution is referred to as a 'pixel shift effect'.

"However, Todor Georgiev, et al., 'Superresolution with Plenoptic 2.0 Camera', 2009 Optical Society of America, only describes a specific focus position as a method of obtaining the effect of the super-resolution from subpixel shift. When an image is generated at a different focus position in accordance with the refocus, the effect of the super-resolution from subpixel shift is changed in accordance with the focus position and the spatial resolution is deteriorated."

In addition to the background information obtained for this patent, VerticalNews journalists also obtained the inventors' summary information for this patent: "The present invention provides an image processing method that is capable of reducing the dependency of a focus position for resolution of a refocus image so as to obtain a high-resolution refocus image.

"An image processing method as one aspect of the present invention is capable of reconstructing an input image to generate a plurality of output images that have different focus positions, and includes the steps of obtaining the input image that is an image in which information of an object space is obtained from a plurality of points of view using an image pickup apparatus that includes an image pickup element having a plurality of pixels and an imaging optical system, calculating a first position of a virtual imaging plane that corresponds to a specified focus position, setting the virtual imaging plane to a second position that is in a range of a depth of focus of the imaging optical system with reference to the first position so that a maximum value of an apparent pixel pitch that is formed by reconstructing the input image is decreased, and generating the output image in a state where the virtual imaging plane is set to the second position.

"An image pickup apparatus as another aspect of the present invention is capable of reconstructing an input image to generate a plurality of output images that have different focus positions, and includes an imaging optical system, an image pickup element that includes a plurality of pixels, a lens array configured so that rays from the same position of an object plane enter pixels of the image pickup element that are different from each other in accordance with a pupil region of the imaging optical system through which the ray passes, and an image processing portion configured to generate the output image using the image processing method for the input image obtained by the image pickup element. The lens array is disposed on an image side conjugate plane of the imaging optical system with respect to the object plane.

"An image pickup apparatus as another aspect of the present invention is capable of reconstructing an input image to generate a plurality of output images that have different focus positions, and includes an imaging optical system, an image pickup element that includes a plurality of pixels, a lens array configured so that rays from the same position of an object plane enter pixels of the image pickup element that are different from each other in accordance with a pupil region of the imaging optical system through which the ray passes, and an image processing portion configured to generate the output image using the image processing method for the input image obtained by the image pickup element. The lens array is disposed so that an image side conjugate plane of the imaging optical system with respect to the object plane is conjugate to the image pickup element.

"An image pickup apparatus as another aspect of the present invention is capable of reconstructing an input image to generate a plurality of output images that have different focus positions, and includes an imaging optical system that includes a plurality of optical systems each having a positive refractive power, at least one image pickup element that includes a plurality of pixels, and an image processing portion configured to generate the output image using the image processing method for the input image obtained by the image pickup element. The plurality of optical systems are arranged so that rays from the same position of an object plane enter pixels of the image pickup element that are different from each other in accordance with a pupil region of the imaging optical system through which the ray passes, and a pupil of the imaging optical system is a combined pupil that is obtained by combining pupils of the plurality of optical systems.

"An image processing apparatus as another aspect of the present invention is capable of reconstructing an input image to generate a plurality of output images that have different focus positions, and includes a storage portion configured to store image pickup condition information of the input image, and an image processing portion configured to generate the output image using the image pickup condition information for the input image. The image processing portion performs an image processing method including the steps of obtaining the input image that is an image in which information of an object space is obtained from a plurality of points of view using an image pickup apparatus that includes an image pickup element having a plurality of pixels and an imaging optical system, calculating a first position of a virtual imaging plane that corresponds to a specified focus position, setting the virtual imaging plane to a second position that is in a range of a depth of focus of the imaging optical system with reference to the first position so that a maximum value of an apparent pixel pitch that is formed by reconstructing the input image is decreased, and generating the output image in a state where the virtual imaging plane is set to the second position.

"A non-transitory computer-readable storage medium as another aspect of the present invention stores a process for causing an information processing apparatus to execute a method capable of reconstructing an input image to generate a plurality of output images that have different focus positions, and the method includes the steps of obtaining the input image that is an image in which information of an object space is obtained from a plurality of points of view using an image pickup apparatus that includes an image pickup element having a plurality of pixels and an imaging optical system, calculating a first position of a virtual imaging plane that corresponds to a specified focus position, setting the virtual imaging plane to a second position that is in a range of a depth of focus of the imaging optical system with reference to the first position so that a maximum value of an apparent pixel pitch that is formed by reconstructing the input image is decreased; and generating the output image in a state where the virtual imaging plane is set to the second position.

"An image pickup apparatus as another aspect of the present invention includes an imaging optical system, an image pickup element that includes a plurality of pixels, and a lens array configured so that rays from the same position of an object plane enter pixels of the image pickup element that are different from each other in accordance with a pupil region of the imaging optical system through which the ray passes. The lens array is disposed so that an image side conjugate plane of the imaging optical system with respect to the object plane is conjugate to the image pickup element, and an appropriate conditional expression is met.

"Further features and aspects of the present invention will become apparent from the following description of exemplary embodiments with reference to the attached drawings."

URL and more information on this patent, see: Hiasa, Norihito; Hatakeyama, Koshi. Image Processing Method, Image Pickup Apparatus, Image Processing Apparatus, and Non-Transitory Computer-Readable Storage Medium. U.S. Patent Number 8743245, filed August 15, 2012, and published online on June 3, 2014. Patent URL: http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PALL&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.htm&r=1&f=G&l=50&s1=8743245.PN.&OS=PN/8743245RS=PN/8743245

Keywords for this news article include: Canon Kabushiki Kaisha.

Our reports deliver fact-based news of research and discoveries from around the world. Copyright 2014, NewsRx LLC


For more stories covering the world of technology, please see HispanicBusiness' Tech Channel



Source: Computer Weekly News


Story Tools






HispanicBusiness.com Facebook Linkedin Twitter RSS Feed Email Alerts & Newsletters