How big is big data? For
"Scientists use big data for everything from predicting weather on Earth to monitoring ice caps on Mars to searching for distant galaxies," said
Building Castles of Data
De Jong explains that there are three aspects to wrangling data from space missions: storage, processing and access. The first task, to store or archive the data, is naturally more challenging for larger volumes of data. The Square Kilometer Array (SKA), a planned array of thousands of telescopes in
JPL is involved with archiving the array's torrents of images: 700 terabytes of data are expected to rush in every day. That's equivalent to all the data flowing on the Internet every two days. Rather than build more hardware, engineers are busy developing creative software tools to better store the information, such as "cloud computing" techniques and automated programs for extracting data.
"We don't need to reinvent the wheel," said
"It's a win-win solution for everybody," said Mattmann.
In Living Color
Archiving isn't the only challenge in working with big data. De Jong and his team develop new ways to visualize the information. Each image from one of the cameras on
"Data are not just getting bigger but more complex," said De Jong. "We are constantly working on ways to automate the process of creating visualization products, so that scientists and engineers can easily use the data."
Data Served Up to Go
Another big job in the field of big data is making it easy for users to grab what they need from the data archives.
"If you have a giant bookcase of books, you still have to know how to find the book you're looking for," said
Sometimes users want to access all the data at once to look for global patterns, a benefit of big data archives. "Astronomers can also browse all the 'books' in our library simultaneously, something that can't be done on their own computers," said Groom.
"No human can sort through that much data," said
QuakeSim's images and plots allow researchers to understand how earthquakes occur and develop long-term preventative strategies. The data sets include GPS data for hundreds of locations in
Ultimately, the tide of big data will continue to swell, and
TNS 30BautistaJude 131018-4520439 30BautistaJude
Most Popular Stories
- Twitter Names Woman to Board
- Obamacare Doing Just Fine, Ky. Governor Says
- Rand Paul Signs up for Obamacare
- Hispanic Employment Improves in November
- How to Arm Yourself Against CryptoLocker Virus
- Thalia Gets Star on Hollywood Walk of Fame
- Trapped Florida Whales Head for Deeper Waters
- Aspen Contracting Adding 300 Jobs
- Dow Jumps 200 Points on Jobs Data
- U.S. Chamber to Run Ads in Idaho, W.Va.