Canada’s most powerful research supercomputer, Niagara, is now available to researchers of all disciplines across the country. Located at University of Toronto and supported by the university’s high-performance computing division SciNet, the system is open to all Canadian university researchers. Niagara is part of a system of national advanced research computing infrastructure, which includes Cedar at Simon Fraser University, Arbutus at the University of Victoria, and Graham at University of Waterloo. Together, these systems provide Canadian researchers with the tools to compete globally using big data and high-performance computing systems.
Advanced research computing is the backbone of Canadian innovation. Niagara will give researchers the computing power they need to study and find solutions to some of the world’s biggest challenges. It will enable large-scale computation and simulation required for artificial intelligence, climate change research, ocean modelling, genomics, astrophysics, and other disciplines using big data research to fuel discovery.
This new system is jointly funded by the Canada Foundation for Innovation, the Government of Ontario, and the University of Toronto.
Niagara is the fastest computer system in the country and is able to run a single job across all 60,000 cores thanks to a high-performance network which interconnects all the nodes. Niagara is highly energy efficient and is configured specifically to accelerate innovation.
The cluster is an end-to-end Lenovo solution with 1,500 ultra-dense ThinkSystem SD530 compute nodes, providing more than three petaflops of processing power, supported by 12 petabytes of storage. Mellanox EDR InfiniBand is used to create an industry-first Dragonfly+ network topology featuring adaptive routing to provide the high-speed low-latency communications necessary for large-scale full-system simulations. Burst-buffer technology from Excelero helps improve performance for data-intensive work loads. The system leverages Lenovo Ethernet for cluster management.
Niagara is named after the iconic Niagara Falls; an Ontario landmark that symbolizes incredible power, discovery, and awe.
To download a copy of the full media release, please click here.
Facts about Niagara
Cores
Cores are the basic processing unit of all computers, including laptops and personal computers which may have between two and four cores. Niagara has a whopping 60,000 compute cores. The unique feature of Niagara is that it is configured in such a way as to allow computation simultaneously on all 60,000 cores.
Nodes
A node is another name for server. It is the basic building block of a cluster and runs a single operating system and contains multiple cores, 40 per node on Niagara, that all share the same memory. Niagara has 1,500 nodes. The previous GPC had almost 4,000 nodes and half as many cores.
Memory and Storage
Each node has 192 Gigabytes of memory (12 times as much as on a high-end laptop), for a total of 288TB memory available to use during calculations. In terms of disk space, Niagara has 12 PB of storage available for storing data from calculations. A GB is 109 bytes, a TB is 1012 bytes, and a Petabyte is 1015 or a million-billion bytes. Or thought of another way, Niagara has 12,000,000 GB of storage.
Power
For its size, Niagara is incredibly power efficient. The previous University of Toronto supercomputer consumed around 1000 kW of power. Niagara will use about 650 kW and has 10 times the performance, making it 15 times more energy efficient. This is a power savings of roughly 300 average family homes.
Network
In the data centre alone, there are over 40km of fibre optic network cables linking all of the 1500 nodes with high-speed, low-latency connections. This high speed InfiniBand interconnect with its Dragonfly+ network is one of the key features that differentiates Niagara from other systems in Canada, in that it makes it possible to use all 60,000 cores together to solve a single problem simultaneously.
Flops
FLOPs or Floating Point Operations per second for Niagara is theoretically 4.61 PF (PF = Peta Flop, Peta is 1015 or million billion).
Researcher quotes
“As an initial demonstration of full system capability, Niagara is currently being employed to perform a heroic computation of the interaction between the tides raised in the oceans by the gravitational attraction of the sun and moon and ocean bottom topography at very high spatial resolution. The accurate description of this interaction is at the heart of improving the skill of the ocean component of the climate models we employ to make global warming projections.”
–Prof. W. Richard Peltier, FRSC is University Professor and Professor of Physics at the University of Toronto, Director of the Centre for Global Change Science, PI of the Polar Climate Stability Network, and Scientific Director of SciNet.
—
“The large parallel capability of Niagara enables world-leading precision cosmological simulations incorporating neutrinos. This draws talent from across the world, focusses their research strengths, and brings visibility to the research results”
–Dr.Ue-Li Pen, Professor and Director, Canadian Institute for Theoretical Astrophysics, University of Toronto; Senior Fellow, CIFAR.
—
“The technological solutions for biggest challenges facing humankind are being driven by a combination of scientific know-how, data science, and supercomputing. In this digital economy, research and innovation competitiveness is highly dependent upon the high-performance computing infrastructure. In this regard, the installation of Niagara, Canada’s newest computing facility, fulfills a critical need and will allow my research group to computationally design and discover new materials for promoting sustainable energy, transportation, and health in Canada and worldwide.”
–Dr. Chandra Veer Singh, PhD, PEng is Erwin Edward Hart Endowed Associate Professor of Materials Science and Engineering, and Associate Chair for Research in the Department of Materials Science and Engineering at University of Toronto.
—
“As is recognized by many countries and increasingly in Canada, advanced computing underpins a great deal of work in science and engineering. Consequently, access to world-class computing power is essential to enable research to address important societal issues in a timely manner. For example, Canada’s new advanced research computer, Niagara, is crucial for my research investigating next generation aircraft configurations with improved fuel efficiency and reduced environmental impact.”
–Dr. David Zingg PhD, FCASI, AFAIAA, FCAE is University of Toronto Distinguished Professor of Computational Aerodynamics and Sustainable Aviation; Director, Centre for Research in Sustainable Aviation; Director, Centre for Computational Science and Engineering, University of Toronto Institute for Aerospace Studies
