No assignee for this patent application has been made.
News editors obtained the following quote from the background information supplied by the inventors: "This disclosure relates generally to image processing, and, more specifically, to image stabilization in a device that includes a camera.
"When recording a scene with a video capturing device (e.g., a video camera), the captured frames may reflect any motion of the camera caused by a shaking hand, a foot-fall while walking, or other types of disruptions. In some cases, image stabilization techniques can be applied to reduce the amount of motion exhibited in the captured video sequence and/or to reduce the effects of motion caused by the camera's rolling shutter. A variety of techniques may be used to determine the motion reflected in the scene, such as using data from a gyroscope in the camera, or generating motion vectors based on optical flow calculations.
"Regardless of how the motion data is generated, the resulting images (e.g., the frames of the video sequence) will require the application of one or more image transformations to adjust for the motion. These transformations can include any combination of offset translation, rotation, scaling, shear transformation and/or perspective transformation. Typically, such operations are applied piecewise to the images to account for the changes in motion during the frame capture due the rolling shutter. Typically, such transformations are applied to each frame of the image data after the data for that image frame is written to system memory. For example, a Graphics Processing Unit (GPU) or similar hardware may access the system memory in order to perform the piecewise transformations to different sections of the image data for each of the frames. Alternatively, software running on a CPU in the system may access the system memory in order to perform such transformations."
As a supplement to the background information on this patent application, VerticalNews correspondents also obtained the inventor's summary information for this patent application: "An image signal processor of a device, apparatus, or computing system that includes a camera capable of capturing video clips may, in some embodiments, be configured to apply non-warping and warping transformations (including image stabilization transformations) to image data received from the camera's image sensor. For example, the image signal processor may be configured to apply translation operations, horizontal or vertical scaling operations, rotations, shear transformations, and/or perspective transformations to correct for camera motion in a sequence of image frames (e.g., a tilt, roll, or rotation of the camera during image capture) or to compensate for rolling shutter effects.
"In some embodiments, the transformations may be applied in a piecewise manner to pixels of the image data that have been gathered into multiple line buffers within the image signal processor (or an output rescaling unit thereof) after the data is received, where the pixels in each line buffer correspond to a line of pixels in the input image. In some embodiments, the line buffers may be divided into vertical slices, each of which may be processed in parallel by a respective portion of the image signal processor circuitry (or the circuitry of the output rescaling unit thereof). Splitting the lines buffers into vertical slices may allow the image signal processor to perform warping transformations with larger vertical displacements than would otherwise be supported, without the need for larger buffers (or, e.g., a full frame buffer). In some embodiments, the stripes may share existing line buffers and various filter components, as the image signal processor hardware walks through the different stripes applying the appropriate transformations.
"By leveraging existing filters and scalers of the image signal processor, rather than a CPU or GPU, image stabilization may be provided that consumes half of the bandwidth and power consumed by existing techniques, in some embodiments. For example, the image signal processor may not need to write the received image data to system memory until after the image stabilization transformations have been performed. By eliminating costly memory transfers between the system memory and the CPU or GPU, the cost of implementing image stabilization for video capture (in terms of memory bandwidth and power consumption) may be reduced, as compared to exiting image stabilization methods.
BRIEF DESCRIPTION OF THE DRAWINGS
"FIG. 1 is a block diagram illustrating one embodiment of a system that performs image stabilization transformations.
"FIG. 2 and FIG. 3 are block diagrams illustrating various data paths in a system that performs image stabilization transformations.
"FIG. 4 is a flow diagram illustrating one embodiment of a method for performing image stabilization transformations.
"FIG. 5 is a block diagram illustrating one embodiment of an image signal processor.
"FIG. 6 is a flow diagram illustrating one embodiment of a method for performing piecewise warping or non-warping image transformations.
"FIG. 7 is a block diagram illustrating one embodiment of an output rescaling unit of an image signal processor.
"FIG. 8 is a flow diagram illustrating one embodiment of a method for applying piecewise warping or non-warping image transformations to vertical stripes of an image frame.
"FIGS. 9A-9B illustrate the application of the techniques described herein for performing image stabilization, according to one embodiment.
"This specification includes references to 'one embodiment' or 'an embodiment.' The appearances of the phrases 'in one embodiment' or 'in an embodiment' do not necessarily refer to the same embodiment. Particular features, structures, or characteristics may be combined in any suitable manner consistent with this disclosure.
"'Comprising.' This term is open-ended. As used in the appended claims, this term does not foreclose additional structure or steps. Consider a claim that recites: 'An apparatus comprising one or more processor units . . . .' Such a claim does not foreclose the apparatus from including additional components (e.g., a network interface unit, graphics circuitry, etc.).
"'Configured To.' Various units, circuits, or other components may be described or claimed as 'configured to' perform a task or tasks. In such contexts, 'configured to' is used to connote structure by indicating that the units/circuits/components include structure (e.g., circuitry) that performs those task or tasks during operation. As such, the unit/circuit/component can be said to be configured to perform the task even when the specified unit/circuit/component is not currently operational (e.g., is not on). The units/circuits/components used with the 'configured to' language include hardware--for example, circuits, memory storing program instructions executable to implement the operation, etc. Reciting that a unit/circuit/component is 'configured to' perform one or more tasks is expressly intended not to invoke 35 U.S.C. .sctn.112, sixth paragraph, for that unit/circuit/component. Additionally, 'configured to' can include generic structure (e.g., generic circuitry) that is manipulated by software and/or firmware (e.g., an FPGA or a general-purpose processor executing software) to operate in manner that is capable of performing the task(s) at issue. 'Configure to' may also include adapting a manufacturing process (e.g., a semiconductor fabrication facility) to fabricate devices (e.g., integrated circuits) that are adapted to implement or perform one or more tasks.
"'First,' 'Second,' etc. As used herein, these terms are used as labels for nouns that they precede, and do not imply any type of ordering (e.g., spatial, temporal, logical, etc.). For example, a buffer circuit may be described herein as performing write operations for 'first' and 'second' values. The terms 'first' and 'second' do not necessarily imply that the first value must be written before the second value.
"'Based On' or 'Dependent On.' As used herein, these terms are used to describe one or more factors that affect a determination. These terms do not foreclose additional factors that may affect a determination. That is, a determination may be solely based on those factors or based, at least in part, on those factors. Consider the phrase 'determine A based on B.' While in this case, B is a factor that affects the determination of A, such a phrase does not foreclose the determination of A from also being based on
For additional information on this patent application, see: Mills,
Keywords for this news article include: Patents, Software.
Our reports deliver fact-based news of research and discoveries from around the world. Copyright 2014, NewsRx LLC
Most Popular Stories
- Miley Cyrus Performs in Undies After Costume Goes Missing
- HBO No Go During 'True Detective' Finale
- FBI Helping Ukraine Recover Stolen Billions
- Colo. Raises $3.5 Million in Pot Revenue
- Uninsured Rate Continues to Fall
- Kim Jong Un Elected in Remarkable Unanimous Vote
- Neil Young Debuts PonoMusic This Week
- Rodman Calls It Quits With Kim Jong Un
- Growth Expected if Congress Passes Budget
- Shipwright Jobs Offered in N.C.