Entry № 041-3 / V-3594 · 0:00 synced

Unboxing Canada's BIGGEST Supercomputer!

Linus Tech Tips@LinusTechTips2.6M viewsJul 2, 201715:52
Source
YT
Views
2.6M
Subscribers
16.8M
Critic
?
Audience
?

0 up · 0 down · 0 ratings

Promos

27,000 Intel Xeon Cores, 190 TERABYTES of RAM, and 64 PETABYTES of storage lie within this crazy datacenter called Cedar. Let's check it out!! Thanks to SFU and Compute Canada for allowing us to visit this INCREDIBLE facility. Learn more about Cedar at geni.us Squarespace sponsor link: Visit squarespace.com and use offer code LTT for 10% off Savage Jerky sponsor link: Use offer code LTT to save 10% on Savage Jerky at geni.us Buy Nvidia GPUs Amazon: geni.us Newegg: geni.us Discuss on the forum: linustechtips.com Our Affiliates, Referral Programs, and Sponsors: linustechtips.com Linus Tech Tips merchandise at designbyhumans.com Linus Tech Tips posters at crowdmade.com Production gear: geni.us twitter.com @LinusTech Intro Screen Music Credit: Title: Laszlo - Supernova Video Link: youtube.com iTunes Download Link: itunes.apple.com Artist Link: soundcloud.com Outro Screen Music Credit: Approaching Nirvana - Sugar High youtube.com Sound effects provided by freesfx.co.uk

Start
AI OverviewDefault language

The video introduces Cedar, Canada’s largest supercomputer, and proceeds to unveil its scale and capabilities in a hands-on tour. The host emphasizes Cedar’s role across Canada for high impact research topics like human genomics, bioinformatics, particle physics, and materials science, with a total investment exceeding 16 million dollars from the federal government through the Canadian Foundation for Innovation. The initial segment focuses on the scale of Cedar, detailing 27,000 Intel Xeon cores, 190 terabytes of RAM, and 64 petabytes of storage, complemented by 584 GPUs and a power draw around 560 kilowatts. A key point is Cedar’s remarkable energy efficiency, achieving a PUE of 1.07, which the video teases will be explained later in more depth. The tour then moves to the high availability racks, where dual power supplies, battery backups, and diesel generators underscore the system’s redundancy for mission-critical workloads such as networking and management. As the discussion shifts to connectivity, the host highlights the network backbone, including dual 100 gigabit connections through Vancouver and dual 40 gigabit links via Surrey, ensuring robust failover in case of any single fiber disruption. The video then dives into Cedar’s core networking design, introducing 48 port omni-path switches arranged in an island topology, with dedicated lines from each compute island to core switches to guarantee both bandwidth and failover. Throughout, the host contrasts the archival tape library with faster storage options, and explains how the general storage land feeds the active data needs, while a high-performance Data Direct Network storage appliance provides sustained throughput for data-intensive workloads. The compute layer is illustrated in detail, showing multiple node types optimized for different research tasks, from base compute nodes with dual Xeon CPUs to memory-optimized and GPU-heavy configurations. Finally, the video covers the mechanical and cooling infrastructure that keeps Cedar running efficiently, describing a large scale heat exchanger system, evaporative cooling, and a modular cooling tower setup that enables future expansion. The overall message is that Cedar is not just powerful hardware, but a carefully engineered ecosystem enabling Canada’s researchers to tackle some of the world’s most challenging scientific problems.”,

Topics · technology · data_center · high_performance_computing · science_and_research

Questions answered

What is the total storage capacity of Cedar and how is it organized for different workloads?
Cedar houses 64 petabytes of storage in total, with a portion dedicated to general purpose storage in 10 petabytes currently and planned expansion to 20 petabytes. This is complemented by a 40 gigabit link to the rest of the system and an IBM tape library for archival storage. The archival storage is slower and used for data not accessed frequently, while the general storage area feeds active research projects, and a dedicated high-performance storage appliance provides sustained throughput for intensive tasks.
How does Cedar achieve cooling and why is it important for performance?
Cooling is achieved with a large scale heat exchanger at the rear doors of the racks, front-to-back air flow, and evaporative cooling in the cooling towers upstairs. The system uses room temperature air that is cooled through the heat exchanger, enabling efficient cooling without vast chiller usage. This cooling approach is crucial because the GPU nodes and other high-power components draw substantial energy, and effective cooling allows maintaining performance while keeping power and operational costs manageable.