News Column

Researchers Submit Patent Application, "3d Mapping Using Structured Light and Formation of Custom Surface Contours", for Approval

May 22, 2014



By a News Reporter-Staff News Editor at Politics & Government Week -- From Washington, D.C., VerticalNews journalists report that a patent application by the inventor Joseph, Benjamin E. (Oakland, CA), filed on October 31, 2013, was made available online on May 8, 2014.

No assignee for this patent application has been made.

News editors obtained the following quote from the background information supplied by the inventors: "There are numerous ways to gather three-dimensional (3D) information of an object such as its shape, dimensions, and surface profile. Generally, 3D scans are used to generate a 'point cloud', which is a collection of points in a three dimensional space. Each point represents a point on the surface of the scanned object. In essence, the point cloud forms a 3D map of the object's surface.

"Cameras, image sensors, or other image recorders may be used generate the point cloud through a structured light technique. The technique involves projecting a known pattern, e.g., an array, of light onto the target object. For example, an array of horizontal bars can be projected onto a surface. The distortion of the pattern by the unevenness of the surface can then be used to determine the shape of the surface to high accuracy. Thus, when a picture (or another image sensing technique) is taken of the target object illuminated with a structured light source, it is possible to determine the shape of the object with the information in the picture.

"Structured light techniques have often utilized a method called Moire interferometry, which was applied to visualize strains on structural elements placed under a load. A target object such as a metal plate would have a series of fine parallel lines painted onto its surface, or projected on the object's surface with a bright lamp, lens system, and fine grating. The object would be loaded in tension or compression, and when the object was viewed through a reference grating, the resulting interference between the distorted object and the reference grid would indicate the displacement of the object towards or away from the viewer. The accuracy of this method is on the same order as the dimensions of the painted pattern and reference grating. Using gratings with spacing on the order of 10 micrometers, for example, features less than 10 micrometers can be accurately resolved.

"In the early 1980s, computers enabled using structured light for scanning complicated 3D objects in a completely digital fashion. As exemplified in Gasvik, (1983) 'Moire technique by means of digital image processing,' APPLIED OPTICS, 22(22):3543-48, advanced computational capabilities in computers enabled more complicated structured light techniques. For example, light patterns other than lines may be used (e.g. circles), and time-varying patterns may be used to analyze objects (especially moving objects).

"For example, U.S. Pat. No. 8,462,207 to Garcia et al. describe a method for three-dimensional mapping of an object, including projecting with a projector a set of fringes on the object and capturing an image of the object in a camera. The method further includes processing the captured image so as to detect a Moire pattern associated with the object and so as to extract depth information from the Moire pattern, and configuring the projector and the camera so that a locally unambiguous characteristic of the Moire pattern is related to a depth of the object.

"Many modern structured light techniques involve recovering a phase signal of a periodic pattern that yields the shape of the object. Whether the pattern is two-dimensional pseudo-random sequence, a sinusoidal pattern, a regular grid of dots, the basic principle is the same. While these newer techniques yield performance gains (e.g. higher resolution), these are effectively just refinements to the original Moire interferometry techniques.

"3D scanning technology that employs structured light techniques has been adapted for use in various fields of endeavor. For example, U.S. Pat. No. 8,121,718 to Rubbert et al. describes a computerized, interactive system for orthodontic treatment. The system includes a hand-held optical scanner capturing 3D information of objects, interactive computer-based treatment planning using three-dimensional tooth objects and user specified simulation of tooth movement, and appliance manufacturing apparatus, including bending machines. Similarly, U.S. Pat. No. 8,509,501 to Hassebrook et al. describes a biometrics system that captures and processes a handprint image using a structured light illumination to create a 2D representation equivalent of a rolled inked handprint.

"Recently, a number of infrared-based technologies have been developed to track motion in 3D. For example, Microsoft Corporation (Redmond, Wash.) has brought to market Kinect.RTM. motion-sensing input devices that work with Xbox 360.RTM. interactive video game consoles. Such motion-sensing devices typically employ an infrared laser projector in combination with a monochrome CMOS sensor, which captures video data in 3D under ambient light conditions. While such motion-sensing devices are well suited for tracking the movements of gamers, the devices are relatively complex in construction and may not possess sufficient imaging sensitivity for static 3D contour mapping purposes.

"Structured light techniques have so far been overlooked in a number of substantially static mapping applications. For example, 3D scanning techniques have not been widely used to produce items having a surface contour personalized for an individual, e.g., custom orthotics, ergonomic devices, etc. Instead, such items are typically made through traditional time-consuming and costly casting techniques. Often, a podiatrist will produce custom orthotics by first making a plaster cast to provide a negative impression of the patient's foot. The cast is sent to a laboratory with a prescription for recommended modifications.

"At the laboratory, a positive cast may be made by pouring plaster into the negative cast. When the plaster dries and is removed, a reproduction of the bottom of the foot is formed. Using the podiatrist's recommendations for corrections, laboratory technicians will custom-mold an orthotic made of a supportive material that incorporates the podiatrist's recommended adjustments.

"Alternatively, the laboratory may begin the production process by laser scanning the negative cast. The information may then be processed by a computer to produce the digital image on screen. After corrections are implemented, the corrected positive cast is ready to be produced.

"Advances in microelectronics have resulted in considerable improvements in the functionality of mobile devices such as smart phones, tablet computers, and notebook/laptop computers. Such portable devices have now been recognized to possess both computational resources and a high-resolution camera that may be exploited for 3D scanning purposes. However, current mobile devices cannot yet perform structured light techniques for 3D imaging.

"Thus, there exist opportunities to leverage the advances in mobile devices for 3D scanning purposes to produce items having a surface contour personalized for an individual, thereby eliminating the need for making casts."

As a supplement to the background information on this patent application, VerticalNews correspondents also obtained the inventor's summary information for this patent application: "In a first embodiment, a 3D mapping apparatus that includes a light source, a projector, and a camera. The projector is constructed to project a structured light pattern onto a target object. The projector includes an interface enabled to collects light from the light source, a grating that corresponds to the structured light pattern, and a lens interposed between the light source and the target object. The camera may be an integrated component of a portable device, e.g., a cellular phone or tablet computer with wireless networking capabilities, and may serve to capture light reflected from the target object and to generate an image therefrom. Also included is a storage medium enabled to store the image. The camera and the lens of the projector are located at a predetermined distance from each other.

"In another embodiment, a multicomponent item is provided having a desired surface contour. The item includes a first component, e.g., a stock component having a first surface and a first bonding surface, and a second component, e.g., a custom component, comprising a custom surface and a second bonding surface. The first bonding surface and second bonding surface are enabled to bond with each other. The custom surface forms at least a portion of the desired surface contour of the item. For example, the multicomponent item may form a portion or the entirety of an orthotic, medical, dental and/or ergonomic device, e.g., personalized for a human or animal.

"Methods are provided as well for producing a multicomponent item having a desired surface contour customized for an individual. Typically, a digital 3D map, optionally modified from an initial 3D image file, is obtained associated with the individual. A custom component is formed from the digital 3D map. The custom component relative is then immobilized to a stock component to form the multicomponent item. The personalized surface contour is associated at least in part with a surface of the custom component. Optionally, the custom component may be printed from the digital 3D map on a stock component to form the multicomponent item.

"The digital 3D map may be obtained in any of a number of ways. For example, a structured light pattern may be projected from an incident direction onto the individual. The digital map may be produced after receiving light reflected from the individual at an angle of greater than zero to about ninety degrees relative to the incident direction. In addition or in the alternative, the 3D map may be formed produced from a data file, modified or otherwise, associated with the individual that may not be a result of structure light pattern techniques.

BRIEF DESCRIPTION OF THE DRAWINGS

"FIG. 1 illustrates the need for parallax between a structured light source and camera for determining the depth of an object.

"FIG. 2 depicts a system utilizing structured light and a camera to determine the shape of a 3D object.

"FIGS. 3A to 3D, collectively referred to as FIG. 3, depicts an exemplary projector of the invention in combination with an exemplary portable device that may be used to carry out 3D scans. FIG. 3A depicts an integrated grating and lens assembly of the projector. FIG. 3B depicts a light pipe of the projector. FIG. 3C depicts and exemplary mobile device having an integrated camera and a flash lamp. FIG. 3D depicts how the projector components shown in FIGS. 2A and 3D may be affixed to the mobile device shown in FIG. 3 to form a 3D mapping apparatus.

"FIG. 4 provides a flow chart that exemplifies how an orthotic many be produced based on 3D scan.

"FIG. 5 depicts how background clutter may be removed from a scan of a foot

"FIG. 6 depicts exemplary key points on the scan of a human foot.

"FIG. 7 depicts how the rotation of the foot may be computed with respect to an ideal coordinate frame

"FIG. 8 depicts how parts of a foot scan not related to the production of the orthotic may be removed.

"FIG. 9 depicts how an orthotic based on foot point cloud data may be constructed, wherein some sections of the foot are ignored while others are contoured as with the shape of the foot.

"The invention and aspects thereof shown in the figures may not necessarily be depicted to scale, and certain dimensions may be exaggerated for clarity of presentation."

For additional information on this patent application, see: Joseph, Benjamin E. 3d Mapping Using Structured Light and Formation of Custom Surface Contours. Filed October 31, 2013 and posted May 8, 2014. Patent URL: http://appft.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=HITOFF&u=%2Fnetahtml%2FPTO%2Fsearch-adv.html&r=3040&p=61&f=G&l=50&d=PG01&S1=20140501.PD.&OS=PD/20140501&RS=PD/20140501

Keywords for this news article include: Patents.

Our reports deliver fact-based news of research and discoveries from around the world. Copyright 2014, NewsRx LLC


For more stories covering arts and entertainment, please see HispanicBusiness' Arts & Entertainment Channel



Source: Politics & Government Week


Story Tools






HispanicBusiness.com Facebook Linkedin Twitter RSS Feed Email Alerts & Newsletters