By a News Reporter-Staff News Editor at Information Technology Newsweekly -- Current study results on Information Technology have been published. According to news originating from Palo Alto, California, by VerticalNews correspondents, research stated, "To remain competitive, enterprises are evolving in order to quickly respond to changing market conditions and customer needs. In this new environment, a single centralized data warehouse is no longer sufficient."
Our news journalists obtained a quote from the research from HP Laboratories, "Next generation business intelligence involves data flows that span multiple, diverse processing engines, that contain complex functionality like data/text analytics, machine learning operations, and that need to be optimized against various objectives. A common example is the use of Hadoop to analyze unstructured text and merging these results with relational database queries over the data warehouse. We refer to these multi-engine analytic data flows as hybrid flows. Currently, it is a cumbersome task to create and run hybrid flows. Custom scripts must be written to dispatch tasks to the individual processing engines and to exchange intermediate results. So, designing correct hybrid flows is a challenging task. Optimizing such flows is even harder. Additionally, when the underlying computing infrastructure changes, existing flows likely need modification and reoptimization. The current, ad-hoc design approach cannot scale as hybrid flows become more commonplace. To address this challenge, we are building a platform to design and manage hybrid flows. It supports the logical design of hybrid flows in which implementation details are not exposed. It generates code for the underlying processing engines and orchestrates their execution. But the key enabling technology in the platform is an optimizer that converts the logical flow to an executable form that is optimized for the underlying infrastructure according to user-specified objectives. In this paper, we describe challenges in designing the optimizer and our solutions. We illustrate the optimizer through a real-world use case. We present a logical design and optimized designs for the use case."
According to the news editors, the research concluded: "We show how the performance of the use case varies depending on the system configuration and how the optimizer is able to generate different optimized flows for different configurations."
For more information on this research see: Hybrid Analytic Flows - the Case for Optimization. Fundamenta Informaticae, 2013;128(3):303-335. Fundamenta Informaticae can be contacted at: Ios Press, Nieuwe Hemweg 6B, 1013 Bg Amsterdam, Netherlands.
The news correspondents report that additional information may be obtained from A. Simitsis, HP Labs, Palo Alto, CA 94304, United States. Additional authors for this research include K. Wilkinson and U. Dayal.
Keywords for this news article include: Palo Alto, California, United States, Information Technology, North and Central America
Our reports deliver fact-based news of research and discoveries from around the world. Copyright 2013, NewsRx LLC