High performance computing linux

Linux clusters: High-performance computing

This SearchEnterpriseLinux.com guide explores the many uses of Linux clusters in the enterprise. Find out why Linux clusters can be your company’s best bet for handling storage, applications, high-volume transactions and more.

Linux clusters are not a one-trick pony, so don’t dismiss them as scientific lab tools only. This SearchEnterpriseLinux.com guide explores the many uses of Linux clusters in the enterprise. Find out why Linux clusters can be your company’s best bet for handling storage, applications, high-volume transactions and more. Take a spin though these articles, tutorials, tips, tools, white papers and expert Q&As. Drop me us an e-mail to let me know what other learning guides you’d like to see on SearchEnterpriseLinux.com.

TABLE OF CONTENTS Basics High-performance computing clusters Application clusters Storage clusters Load-balancing clusters High availability clusters Clustering on blade servers Cluster configuration, management and administration Virtualization and clusters Other open source cluster projects

Glossary (SearchEnterpriseLinux.com, powered by Whatis.com)

Other basic resources

• Q&A: Linux beats cluster barriers, and Microsoft follows (SearchEnterpriseLinux.com)
• Q&A: Emic CEO: Do you really need a cluster? How to find out (SearchEnterpriseLinux.com)
• Linux clustering information center
  LCIC is a central repository of links and information regarding Linux clustering, in all its forms.
• White paper: OSDL Cluster Architecture (OSDL)
• Tip: Linux cluster security: False alarms and real threats
  Find out whether security threats to your Linux servers are real or imagined and determine how to set up your Linux server for clusters in this tip.
• Tip: Linux clusters vs. grids
  Can Linux clusters outperform grid computing? Expert Ken Milberg compares them — covering Beowulf, HPC, Linux Virtual Server and the Globus Toolkit for grids — and chooses a winner.
• Tip: SUSE Linux administration: How to set up a cluster file system
  Setting up OCFS2 as a shared file system in a cluster environment helps when multiple nodes have to write to the same file system concurrently. Learn how to do it with SLES10.

High-performance computing clusters

• Cluster how-to
  How to set up a high performance Linux cluster.
• Linux Clusters Institute
  The LCI site provides links to education and advanced technical training for the deployment and use of Linux-based computing clusters to the high-performance computing community worldwide.
• Article: Driving Linux cluster performance (Bio-IT World)
  Planning early and choosing the right tools can boost performance by 27%, according to experts cited here.
• Linux HPC cluster how-to
  This is a specific description of how a lab’s cluster is set up. It details the data center aspects and the desktop, laptop and public server aspects of building a cluster.
• Article: The Linux Business Case: Clusters (CIO Information Network)
  This article discusses the characteristics of high-performance applications that make them a good fit for Linux clusters.
• Q&A: Tips for maximizing HPC cluster productivity (HPC Wire)
  Tom Quinn, Director of Government Business Development at Linux Networx, provides tips on maximizing cluster productivity.
• Build a heterogeneous cluster with coLinux and openMosix (IBM.com)
  An experimental case study combines coLinux with an openMosix-enabled kernel to build a hybrid cluster.
• SGI Partners to Launch ‘Bright’ Linux Clusters (IT Jungle)
  Clusters that behave like, are preconfigured like, and sold like monolithic systems are called «bright clusters,» and supercomputer maker Silicon Graphics is partnering to target this fast-growing market with its Linux-based Altix machines.
• Article: Dell inks HPC cluster deal with Cray
  This story describes how Dell plans to bolster its HPC Linux cluster business by partnering with Cray Inc., the Seattle-based supercomputer company.

• Beowulf.org
  Project page for the open source project. Beowulf.org is a collection of resources for the expanding universe of users and designers of Beowulf class cluster computers. These enterprise systems are built on commodity hardware deploying Linux OS and open source software.
• Beowulf cluster FAQ (Aspen Systems)
• Beowulf how-to
  This document describes step by step instructions on building a Beowulf cluster.
• Q&A: Beowulf founder: Linux holds its own in high-performance computing
  Beowulf founder Don Becker reflects on the legacy of Beowulf and its future in this interview. He also takes a hard look at Cray Canada CTO Paul Terry’s claims that Linux clusters can’t hold their own in HPC situations.
• Article: Scyld’s new cluster has Linux distro, Ganglia inside (SearchEnterpriseLinux.com)
  Scyld Software has added performance monitoring capabilities and hardware and customer support in the newest version of its Scyld Beowulf Series 29.

• Case study: Los Alamos Linux cluster to keep watch on nukes (SearchEnterpriseLinux.com)
  While the eagle gets the glory, the penguin is quietly standing guard over America’s nukes.
• Rocks Linux and Sun desktops ‘supersize’ scientists’ images (SearchEntepriseLinux.com)
  Biomedical researchers use Linux to assist in viewing huge datasets on a PC.
• Rocks Linux cluster: 64-bit Teraflop Cluster installed in Tromso Norway’s first 64-bit Teraflop cluster was installed at the University of Tromso.
• Article: Dell joins UT’s $38M supercomputer project (Austin Business Journal)
  About 300 Dell servers are being used in a computing cluster that gives scientists and engineers at University of Texas at Austin the power of more than 3 trillion computer operations per second.

Whether or not to use HPC clusters

• Q&A: Linux clusters for HPC – Why and why not?
  What’s keeping high-performance computing users away from Linux clusters? Should they reconsider?
• Expert advice: What are the advantages of moving to a Linux HPC cluster? (Ken Milberg, SearchEnterpriseLinux.com expert)
• Q&A: Cray CTO: Supercomputers outshine Linux clusters in HPC, Part 1 (SearchEnterpriseLinux.com)
  Linux clusters can not offer the same price-performance as supercomputers, according to Paul Terry, chief technology officer of Burnaby, British Columbia-based Cray Canada. In this interview, Terry explains that assertion and describes Cray’s new Linux-based XD1 system, which will be priced competitively with other types of high-end Linux clusters.
• Q&A: Cray Canada CTO answers IT pros’ HPC questions (SearchEnterpriseLinux.com)
  Is high-performance computing only used in scientific labs? In this interview, Cray Canada chief technology officer Paul Terry answers this question, offers advice on how to choose the right HPC system responds to questions from SearchEnterpriseLinux.com’s readers.
• Q&A: Linux clusters give HPC price-performance (SearchEnterpriseLinux.com)
  High-performance computing (HPC) users are switching from supercomputers to Linux clusters, and enterprise IT shops are getting keen on that idea, too. In this interview, PathScale exec Alex Rublowsky explained why HPC on Linux makes sense. Also, find out which types of enterprise IT shops are checking out and what’s ahead for HPC Linux clusters.
• Linux clusters can have their day vs. supercomputers (SearchEnterpriseLinux.com)
  Linux Networx marketing executive Eric Pitcher makes the case for Linux clusters against supercomputers.
• Filling in the gaps among Linux clusters (SearchEnterpriseLinux.com)
  Is the price right for a Linux cluster? Would it meet all of your company’s needs? Get the scoop in part two of this interview with Linux Networx executive Eric Pitcher.

• Tutorial: Data sharing clusters (Red Hat)
  This is a Red Hat tutorial on data sharing clusters for storage, grid and Web applications.
• Q&A: Emic CEO: Open source platforms to get more support (SearchEnterpriseLinux.com)
  What is an application cluster? Get this CEO’s definition, as well as his descriptions of other types of clusters and the future of clustering and open source software.
• MySQL Cluster FAQ (MySQL)
  MySQL Cluster combines the open source database with a parallel-server, fault tolerant clustering architecture.
• Resource: Oracle RAC 10g home page (Oracle)
  This guide to Oracle RAC 10g includes links to tutorials and white papers.
• White Paper: Introduction to application deployment for Real Application Clusters (Stanford.edu)
  Tips on implementing Oracle 9i features with Real Application Clusters.
• Case study: MTN’s prepaid value added services now running faster and more reliably (IT Web)
  For MTN, the Oracle9i Real Application Clusters’ built-in server redundancy offers clear advantages, the most compelling of which is that average fail-over time is 10 to 15 seconds compared to four to six minutes for traditional processor fail-over.
• Case study: 9i RAC is the ticket for StubHub (SearchEnterpriseLinux.com)
  Online ticket broker StubHub has no time for downtime and apparently has no qualms about putting its bottom line in Linux’s hands.
• Case study: Tractor Supply takes cluster route out of Win-Tel (SearchEnterpriseLinux.com)
  Tractor Supply needed a scaleable, redundant replacement for a pay-as-you-grow WinTel system.
• Case study: Clustering eases Talk America’s replication woes (eWeek)
  Talk America Holdings Inc., a phone and high-speed Internet access provider and an early adopter of Oracle Corp.’s 10g technology, has untangled itself from 50 Informix databases down to two Oracle databases running on an RAC cluster—a choice that’s jazzed up its querying capabilities and helped it wipe its hands of a data replication mess.
• Expert advice: Setting up a cluster with Linux and Oracle (Ken Milberg, SearchEnterpriseLinux.com expert)
• Novell partner guide: Emic Application Cluster (Novell)
  A description of this type of application cluster and the certifications for it.
• Q&A: Emic CEO: Do you really need a cluster? How to find out (SearchEnterpriseLinux.com)
  Emic CEO Eero Teerikorpi offers dos and don’ts for evaluating clusters and defines and compares types of clusters.

• Article: Lowering storage and file-serving costs with network attached storage (NAS) clusters (Source)
  NAS clustering can enable a scalable, fault-tolerant, file-serving cluster for thousands of users, Web clients or computing nodes.
• Article: NAS clustering beats 1GB barrier (SearchStorage.com)
  Clustering low-cost Linux servers together with a network file system saves significant costs over purpose-built NAS filers; that much is a no-brainer. Now, advocates of NAS clustering argue that the performance of these offerings matches or even outperforms traditional NAS appliances.
• Case study: Windows flops in online retail project (SearchEnterpriseLinux.com)
  Nowadays, more and more Java-based retail apps are being deployed on Linux. So, when Datavantage had to choose a platform for its new Stored Value gift card transaction-processing solution, Windows wasn’t the only option.
• Article: NetApp to reshape its storage architecture (SearchStorage.com)
  NetApp’s Storage Grids will parallel an important trend in the server world: the move away from large symmetric multi-processing (SMP) systems, and toward large server farms made up of commodity Linux servers. That trend is quite evident today in the technical computing space, where NetApp intends to find its first customers for the Storage Grids technology.
• Article: PolyServe thinks outside the box with NAS cluster (SearchEnterpriseLinux.com)
• Case study: Company bypasses NAS, chooses Linux cluster (SearchEnterpriseLinux.com)
• Expert advice: How do I back up Linux clustering file systems? (Sam Greenblatt, SearchEnterpriseLinux.com expert)

• Home page: Linux Virtual Server Project
  The Linux Virtual Server is a highly scalable and highly available server built on a cluster of real servers, with the load balancer running on the Linux operating system. The architecture of the server cluster is fully transparent to end users, and the users interact as if it were a single high-performance virtual server.
• Home page: Ultra Monkey
  Ultra Monkey is a project to create load-balanced and highly available network services, such as a cluster of Web servers that appears as a single Web server to end users.
• White paper: Craft a load-balancing cluster with ClusterKnoppix (IBM)
  Using Knoppix-based LiveCDs, build your own supercomputing Linux cluster.
• Article: Load balancing gives Web traffic a green light (SearchCIO.com)
• Resource: Load balancing Linux clusters (LCIC.org)

High availability clusters

• Home page: High-Availability Linux Project (Linux-ha.org)
  The basic goal of the High Availability Linux project is to provide a high-availability (clustering) solution for Linux which promotes reliability, availability and serviceability through a community development effort.
• White Paper: Introduction to clustering on IBM eServer OpenPower 710 (IBM)
  This article first introduces some general clustering concepts and software and then provides examples of how the OpenPower 710 can be used in the two most common types of clusters: high performance clusters and high availability clusters.
• Home page: Kimberlite HA cluster project site
  The collaboration site for Kimberlite cluster development. Kimberlite is an open source project for high availability clustering on Linux. The page offers links to documentation and more information.
• Resource: SGI Linux FailSafe FAQ (SGI)
  Linux FailSafe provides a clustered HA application platform in software.

Clustering on blade servers

• Article: Verari’s blades look sharp (Sign On San Diego)
  Verari and other blade server makers believe the advantages of blades will usher in a major change in the way large corporations nationwide meet their computer needs.
• Article: BladeRunner hunts the competition
  Information about Linux-exclusive vendor Penguin Computing’s new blade server, BladeRunner. (SearchEnterpriseLinux.com)
• Review: Penguin Computing’s BladeServer (eWeek)

Cluster configuration, management and administration

• Q&A: Beowulf founder: Virtualization hot on Linux clusters
  Virtualization is creating a buzz in the IT industry today, but that buzz fizzles when virtualization on Linux clusters is mentioned. That’s a shame, says Beowulf Project co-founder Donald Becker, because clusters offer a no-clutter virtualization option.
• White paper: What is CSM? (IBM)
  Find out more about Cluster Systems Management (CSM) software, which provides a distributed system management solution that allows a system administrator to set up and maintain a cluster of nodes that run the Linux operating system.
• Home page: System Imager
  This is the open source project site for SystemImager, software that automates Linux installs, software distribution, and production deployment for Internet server farms, database server farms, high performance clusters, computer labs, and corporate desktop environments.
• Article: Scali releases new products for multi-platform clusters (Linux HPC.org)
  This is an update on Scali’s high performance clustering management product, called Scali Manage.
• Home page: Maui Scheduler
  The Maui Scheduler helps admins manage large-system resources. Maui was designed to batch schedule and launch MPI-based parallel jobs, and can also schedule and run your own custom message-passing jobs.
• Home page: The Cluster Command and Control (C3) tool suite
  This is the project page for The Cluster Command and Control (C3) tool suite, which implements a number of command line based tools that have been shown to increase system manager scalability by reducing time and effort to operate and manage the cluster.
• Tip: Distributed computing, clusters and high-availability solutions
  In this tip, clusters and servers expert Don Becker describes basic rules for making network boot servers more reliable — like avoiding multicasts — and explains the Mosix approach to cluster architecture. He also suggests ways to avoid data loss when consolidating servers.
• Expert Response: Consolidation and common sense on the server side
  A Unix-to-Linux expert describes educational resources about distributed computing, clusters and high-availability solutions for a user who is considering a move to Linux from Unix.
• Tip: Mastering MySQL: Drivers, clustering and remote access
  MySQL expert Scott Noyes describes the tricky aspects of clustering with MySQL and upgrading from MySQL 4.0 to MySQL 5.0.
• Tip: Setting up MySQL replication for Linux server clusters
  MySQL facilitates Linux server clustering for load balancing and fault tolerance. Create a master/slave configuration with full data replication using the commands in this tip.

Virtualization and clusters

• Article: Beowulf founder: Virtualization hot on Linux clusters
  Beowulf Project co-founder Donald Becker explains how Linux clusters can make for a no-clutter virtualization option.
• Tip: Configuring Xen HA with Heartbeat for SUSE Linux Enterprise Server
  A Heartbeat cluster with SUSE Linux Enterprise can be used to support virtual machines. Get instructions for creating a SAN for high availability storage for Xen from an expert.
• Tip: CTO clears air about clusters vs. virtualization, blades vs. racks and more
  Scali CTO Hakon Bugge discusses how to combine server clusters with virtualization technology and offers ways to develop a good virtualization strategy in this interview.

Other open source cluster projects

• Home page: OpenSSI (Single System Image) Clusters for Linux
  The OpenSSI project is a comprehensive clustering solution offering a full, highly available SSI environment for Linux.
• Home page: Cluster infrastructure for Linux
  This project is developing a common infrastructure for Linux clustering by extending the Cluster Membership and Internode Communication Subsystems from HP’s NonStop Clusters for Unixware code base.
• Resource: Cluster Project Page
  This page provides links to a number of Linux cluster development projects.
• Home page: Open Cluster Group
  The Open Cluster Group is an informal group of people dedicated to making cluster-computing practical for high performance computing.
• Home page: OSCAR: Open Source Cluster Application Resources
  OSCAR is an open source project for the development of a fully integrated, easy to install, small-to-midsized high performance clustering solution.

Источник