IBM speeds high performance data analytics

Intelligent scheduling software helps clients adopt, manage Apache Spark analytics.

Friday, 3rd June 2016 Posted 8 years ago in by Phil Alsop

IBM has expanded its portfolio of software-defined infrastructure solutions with cognitive features to help clients improve the management of computing resources to achieve faster, results from data-driven applications and analytics.

The new intelligent resource and workload management software, called IBM Spectrum Computing is designed to make it easier for organizations to extract full value from data to accelerate performance-intensive analytics or machine learning. This technology can be used across industries to help sequence genomes for improved cancer treatment, or for engineers to design a championship-winning Formula One race car or bankers to personalize financial services to attract new customers.

IDC estimates the market for software-defined infrastructure – where the data center is managed, provisioned and automated by software regardless of compute, storage or network components -- to reach nearly $51 billion by 2019, growing at a rate of 23.8 percent from 2014 to 2019. This makes it the fastest-growing segment of the infrastructure market, according to IDC.

The IBM Spectrum Computing platform offers new cognitive, resource-aware scheduling policies that help increase the utilization of existing compute resources, controlling costs while speeding results for high performance computing, big data analytics, new generation applications and open source frameworks, such as Hadoop and Apache Spark.

IBM Spectrum Computing assists organizations with consolidating data center infrastructure and sharing resources across on-premise, cloud or hybrid environments -- and includes three new software products.

Software to Achieve Faster, More Predicable Insights

· Designed to speed analysis of data – IBM Spectrum Conductor works with cloud applications and open source frameworks, speeding time to results by enabling increasingly complex applications to share resources, all while protecting and managing data throughout its lifecycle.

· Integrates Apache Spark – IBM Spectrum Conductor with Spark simplifies the adoption of Apache Spark, an open source big data analytics framework, while delivering up to 60 percent faster analytical results.

· Accelerates research and design – IBM Spectrum LSF is a comprehensive workload management software featuring flexible and easy to use interfaces to help organizations accelerate research and design by up to 150 times while controlling costs through advanced resource sharing and improved utilization.

“Supermicro Total Rack Solutions, combined with IBM Spectrum Computing delivers the industry’s most comprehensive, software-defined infrastructure optimized for complex cognitive analytics applications,” said Charlie Wu, General Manager, Rack Solutions at Supermicro. “Working with IBM, we have integrated our latest server, storage and networking solutions with IBM Spectrum Conductor with Apache Spark and IBM Spectrum LSF to accelerate deployment of scalable, high-performance analytics infrastructure. Our collaborative efforts enable extraction of more predictable results and insight across hybrid cloud environments.”

IBM Spectrum Conductor was developed over two years through the collaboration of IBM developers and clients focused on accelerating next-generation analytics. The software manages multiple applications at one time ensuring allocation of resources to achieve faster time to results. Highly-efficient, multi-tenant scheduling allows for data and resource sharing without compromising availability or security.

Recognizing the vital role of open source software to the technical community, IBM intends to contribute a key component of IBM Spectrum Conductor to further advance the adoption of Apache Spark by data scientists and developers.

“Data is being generated at tremendous rates unlike ever before, and its explosive growth is outstripping human capacity to understand it, and mine it for business insights,” said Bernard Spang, vice president, IBM Software Defined Infrastructure. “At the core of the cognitive infrastructure is the need for high performance analytics of both structured and unstructured data. IBM Spectrum Computing is helping organizations more rapidly adopt new technologies and achieve greater, more predictable performance.”

IBM Spectrum LSF delivers comprehensive workload and resource management capabilities for high-performance research, design and simulation applications. Ease of use is improved through an enhanced mobile user interface, improved reporting and workload visibility. Significant performance enhancements offer five-times greater throughput and up to three-times higher scalability than previous IBM Platform LSF versions.

“While races may be won by drivers at the wheel, designing a championship-winning Formula One car requires the ongoing efforts of technology to utilize the power of available IT resources,” said Matt Cadieux, CIO, Red Bull Racing. “IBM Spectrum LSF helps us achieve excellent performance for our most demanding compute- and data-intensive applications. We can get more accomplished with fewer resources, reducing infrastructure and administration, and speeding new race car designs.”