In a sign that the worlds of big data and government-owned high performance computing centers are beginning to converge, the Department of Energy’s Lawrence Livermore National Laboratory and IBM announced that they are joining forces to help boost the competitiveness of U.S. industries in the global economy.
The announcement drew the attention and praise of Sen. Dianne Feinstein (D-Calif.) during a Capitol Hill briefing June 27, during which Feinstein stressed the growing importance of high performance computing and data analytics in the U.S.
Joining a chorus of industry proponents, who see big data as the next “new natural resource,” Feinstein hailed the joint effort, and the fact that for the first time since November 2009, a U.S. supercomputer “is the fastest in the world – and one of the most energy efficient,” she said.
“The new collaboration between the Lawrence Livermore National Laboratory and IBM is an excellent example of using the technical expertise of both the government and the private-sector to spur innovation and investment in the U.S. economy,” she said.
Under a recently concluded agreement, IBM and LLNL have formed a collaboration called Deep Computing Solutions aimed at helping American industry harness the power of supercomputing to better compete in the global marketplace. The joint effort will take place within LLNL’s High Performance Computing Innovation Center in Almeda County, Calif., and make use of two supercomputers:
One is called Sequoia, the world’s fastest supercomputer as of June 2012, an IBM BlueGene/Q system which can process 16.32 quadrillion mathematical computations per second (or petaflops). Sequoia, however, is primarily used for classified work by the Energy Department’s National Nuclear Security Administration.
The other is called Vulcan, a five-petaflop system (and the world’s 48th fastest), which uses a new 24-rack IBM Blue Gene/Q system and will be available beginning this summer for non-classified work.
Dr. David McQueeney, vice president of software, IBM Research, (pictured with Feinstein above) said the significance of the announcement lies in the increasing intermingling of scientific disciplines, which use high performance computing typically to conduct simulations, and big data analytics, where analysts are attempting to discern useful patterns from large volumes of data in real time.
Until now, high performance systems have remained generally beyond the reach of industry because its deployment requires access to special expertise and systems like those at LLNL.
Deep Computing Solutions will directly address the accessibility problem that currently limits development and deployment of advanced computing solutions by commercial organizations, according to IBM research and LLNL officials.
Everyone agrees there is an explosion of data taking place, said McQueeney. The challenge now is “a lot of the data is being captured, but not being analyzed and made available to decision makers. Leaders sense that they are operating with insufficient information and see a huge opportunity if they can master the tsunami of data.”
McQueeney said that government high performance computing systems “are fundamentally changing the way we work.”
“We see HPC and big data on a converging path,” he said, particularly as massive volumes of data from sensors, satellites, video and other instruments begin to overwhelm our ability to store and process it.
It will become increasingly necessary to “bring computation to the data instead of the data to the computing systems,” McQueeney said. That’s in large part because data processing demands are exploding in four dimensions simultaneously, he said, in terms of:
- Volume, as we move from routinely processing terabytes of data (thousands of gigabytes) to exabytes (billions of gigabytes);
- Velocity, where the need to process and respond to streaming data is measured milliseconds;
- Veracity, where the uncertainty, inconsistency, and ambiguities of data has become a growing variable in itself;
- Variety, as more and more data is unstructured (such as social media and multimedia).
By introducing HPC and other tools to the traditional process of analyzing data, it will become possible to gain “a much better ability to be directive, rather than corrective,” he suggested.
“Instead of depending only on human insights, we can augment our understanding with applied semantics,” he said. “Today, the best we can say about a system, is that it’s as efficient as it can be, rather than whether it is optimized to be the best it could be,” he said.
For those in the financial field, or medicine, energy and other industries, that could also mean “Rather than monthly risk assessments, we can do sub-second responses,” he said.
The goal of the new Deep Computing Solutions collaboration, according to IBM research and LLNL officials, is aimed at trying to help a range of American industries accelerate the development of new technologies, products and services in such fields as applied and renewable energy, biology, materials science, fabrication, manufacturing, data management and informatics.
Details on how organizations would begin to tap into the new Deep Computing Solutions collaboration were not made available. And while IBM and LLNL have a long string of award-winning science and technology innovations to their credit, questions remain about how readily readily LLNL will be able to accommodate the likely demands by industry for LLNL’s high performance computing time and expertise – and how quickly that will translate into greater global competitiveness.
| How HPC Transforms Data Usage | |
| Traditional | New Approach |
| Instinct & intuition | Fact-driven |
| Corrective | Directive |
| Years, months, weeks | Hours, minutes, seconds |
| Human insight | Applied semantics |
| Decision support | Action support |
| Efficient | Optimized |