Overview of Microsoft HPC Pack 2019

Learn how to evaluate, set up, deploy, maintain, and submit jobs to a high-performance computing (HPC) cluster that is created by using Microsoft HPC Pack 2019. HPC Pack allows you to create and manage HPC clusters consisting of dedicated on-premises Windows or Linux compute nodes, part-time servers, workstation computers, and dedicated or on-demand compute resources that are deployed in Microsoft Azure.

Based on where the compute resources are located, Microsoft HPC Pack can be categorized into three cluster Modes:

Cluster Mode	Highlights	Topology
HPC Pack On-premises Get started with HPC Pack On-premises	— Supports Windows and Linux compute nodes — Advanced job scheduling and resource management — Proved and scale-tested capabilities — Free of charge — Easy to extend to hybrid
HPC Pack Hybrid Get started with HPC Pack Hybrid	— Burst to cloud to handle peaks in demand or special projects — Automate the deployment of Windows and Linux Azure VMs — Use your current HPC scheduler or HPC Pack — Pay only for what you use
HPC Pack IaaS Get started with HPC Pack IaaS	— Deploy a cluster all in the cloud, on demand — Use your current scheduler or HPC Pack — Readily shift existing applications to the cloud — Use templates, scripts, and gallery images to deploy on demand

Microsoft also has cloud-born HPC Scheduler Service called Azure Batch. You can either use Azure Batch directly or you can use HPC Pack as your scheduler and have your job to burst to azure batch.

Follow below links to start with your HPC Pack:

Windows HPC Server 2008

Windows HPC Server 2008, released by Microsoft in September 2008, is the successor product to Windows Compute Cluster Server 2003. Like WCCS, Windows HPC Server 2008 is designed for high-end applications that require high performance computing clusters (HPC stands for High Performance Computing). This version of the server software is claimed to efficiently scale to thousands of cores. It includes features unique to HPC workloads: a new high-speed NetworkDirect RDMA, highly efficient and scalable cluster management tools, a service-oriented architecture (SOA) job scheduler, and cluster interoperability through standards such as the High Performance Computing Basic Profile (HPCBP) specification produced by the Open Grid Forum (OGF).

In June 2008, a system built collaboratively with the National Center for Supercomputing Applications (NCSA) and Microsoft ranked #23 on the Top500 list, a ranking of the world’s fastest supercomputers, with a LINPACK score of 68.5 teraflops. The NCSA supercomputer uses both Windows Server HPC and Redhat Enterprise Linux 4. That ranking has since dropped to #73 as of November 2009 rankings.

In the November 2009 rankings published by Top500.org, a Windows HPC system built by the Shanghai Supercomputer Center achieved 180.6 teraflops, a peak performance that placed the Dawning 5000A system at #19 on the list of the world’s fastest supercomputers. However, as of November 2009, Windows HPC only has about 1% (5 out of 500) of the market of the 500 most powerful supercomputers, with about 1.5% of total gigaflops, with Linux dominating the rankings with 89.20% and Unix second place with 5%. The other two systems in the top 100 that can run Windows HPC do so only part of the time [1] [2] .

Contents

Windows HPC Server 2008 R2

Windows HPC Server 2008 R2 will be the second release based on Windows Server 2008 R2. The release candidate is available on Microsoft’s websites for evaluation and includes some improvements and new features. The OS is in beta development stage and there is a CTP (community technology preview) program running. [3]

www.makeuseof.com

Follow MUO

What Is Windows Server and How Is It Different From Windows?

What make Windows Server different from regular versions of Windows? Learn more about Windows Server.

If you’re a regular computer user, you may have only come across the consumer-facing editions of Windows. But did you know that Microsoft also publishes an entire Windows Server line of its operating system?

Let’s take a look at the differences between Windows Server and normal Windows. We’ll see what Windows Server includes, what it leaves out, and why it’s so different.

What Is Windows Server?

Image Credit: Ana Mercedes Gauna/Wikimedia Commons

In case you’ve never heard of Windows Server, we’ll first explain what it is. Essentially, Windows Server is a line of operating systems that Microsoft specifically creates for use on a server. This means in almost all cases, Windows Server is used in business settings.

Microsoft has published Windows Server under that name since Windows Server 2003 launched in April 2003. However, even before this, server versions of Windows were available. For instance, Windows NT 4.0 was available in both workstation (for general use) and server flavors.

In almost all cases, normal users don’t need to worry about Windows Server. You won’t find it on the shelf in stores or accidentally download it from Microsoft when you mean to get the standard version. But it’s still interesting to learn about.

Windows Server vs. Regular Windows: The Basics

With just a quick glance, you might have trouble telling the difference between Windows Server and normal versions of Windows. The desktop looks the same, including the Taskbar, desktop icons, and Start button.

As it turns out, every Windows Server release corresponds to a consumer version of Windows. Windows Server 2003, for instance, is the server version of Windows XP. Current versions include Windows Server 2016, which is based on the Windows 10 Anniversary Update, and Windows Server 2019.

Because Windows Server and Windows share a code base, you can perform many of the same functions on both. You can download and install programs like browsers and photo editors on Windows Server, and many Windows basics like Notepad are included in Windows Server.

However, the two versions have more differences than similarities. Let’s review some of them.

Windows Server Has Enterprise Management Software

Because Windows Server is intended for businesses, it includes plenty of enterprise software. Below are a few roles that a server can perform thanks to these tools:

• Active Directory: Active Directory is a user management service that allows a server to act as a domain controller. Instead of logging into a local computer, the domain controller handles all user account authentication. See our explanation of Windows domains for more on this.
• DHCP: DHCP, or Dynamic Host Configuration Protocol, is a protocol that lets a server automatically assign IP address to all devices on the network. At home, your router probably handles this. But in a business setting, IT staff can take advantage of the greater DHCP functionality in Windows Server.
• File and Storage: Having a file server for your company is another common use. This allows you to keep important data in a central location and set permissions to control who can access what.
• Print Services: If a business has dozens of printers across the building, it’s a waste of time for IT staff to configure them individually for each new workstation. Setting up a print server allows you to easily map printers to computers and reduce redundant work.
• Windows Update Services: Often, businesses don’t want all Windows Updates to come through right away. By setting up a server as a Windows Update controller, you can route all workstation updates through that server and configure specific rules for how they should work.

These are only a few of the server roles Windows Server can handle. Often, a company will have more than one server and split the above roles across multiple devices.

Standard copies of Windows don’t include these capabilities out of the box. You can install some third-party tools to replicate some of this functionality, but it won’t be as robust.

Windows Server Has Fewer Hardware Limitations

Image Credit: Robert/Flickr

Most people don’t worry about the maximum amount of RAM they can put in their computer. Windows 10 Pro allows you to install up to a huge 2TB of RAM. However, the majority of users don’t have more than 32GB of RAM in their systems, so even installing 1TB of RAM would be insane.

Knowing this, can you believe that Windows Server supports up to 24TB of RAM? It also allows you to use up to 64 CPU sockets, which is much greater than the two sockets that Windows 10 Pro supports.

This might sound ridiculous, but there are good reasons for these high hardware caps. A server can power important functionality for hundreds of people in a business, so it often needs to be extremely powerful. For instance, a server running dozens of virtual machines needs loads of RAM in order to keep them all running smoothly at the same time.

Windows Server Doesn’t Include Extraneous Features

As you’d expect, Windows Server retains power user features like the Command Prompt and other administrative tools. However, Server editions of Windows strip out a lot of the quality-of-life features that Windows 10 includes.

For example, in Windows Server 2016 and 2019, you won’t see Microsoft Edge, the Microsoft Store, Cortana, and other handy Windows 10 features. It doesn’t bundle in apps like Your Phone, and you can’t activate the Linux terminal on Windows server, either. Server OSes also don’t let you log in with a Microsoft account. Because they’re designed for enterprise use, you don’t need these consumer-facing tools on a server OS.

Additionally, some apps check to see if you’re using Windows Server before you install them. In some cases, the app won’t work on a server version of Windows.

Windows Server is also much more locked down by default. It uses Internet Explorer as the default browser, but the security settings are much more restrictive than usual. This makes sense, as having a server compromised would be catastrophic compared to just a workstation.

Windows Server Has Different Pricing

As you may expect from a business-oriented product, Windows Server doesn’t come cheap. It’s much more expensive than a consumer version of Windows, and comes in various flavors depending on your needs.

Microsoft’s Windows Server 2019 pricing page gives an idea of what you might pay for the software. Depending on how many people will access the server, you also need to pay for CALs (Client Access Licenses) to use the services legally.

Businesses historically installed Windows Server on a physical onsite server, which has much greater hardware capabilities than a workstation, as discussed above. However, you also have the option of running Windows Server in a cloud service like Azure.

This allows you to offload the burden of maintaining a physical server to a cloud provider like Microsoft. In addition, this enables companies to spread the cost of upgrading out over a subscription instead of paying all at once for a new physical server. What works best depends on the specific organization’s needs.

Now You Understand Windows Server

In the end, though Windows Server and regular Windows share common code and look similar, they’re for totally different uses.

Consumer editions of Windows 10 are designed for maximum usability and don’t include software made for enterprise use. Meanwhile, Windows Server isn’t concerned with looking pretty. Its purpose is to run many services reliably that company users need.

What is High-Performance Computing?

High-performance computing (HPC) is the ability to process data and perform complex calculations at high speeds. To put it into perspective, a laptop or desktop with a 3 GHz processor can perform around 3 billion calculations per second. While that is much faster than any human can achieve, it pales in comparison to HPC solutions that can perform quadrillions of calculations per second.

One of the best-known types of HPC solutions is the supercomputer. A supercomputer contains thousands of compute nodes that work together to complete one or more tasks. This is called parallel processing. It’s similar to having thousands of PCs networked together, combining compute power to complete tasks faster.

Why is HPC important?

It is through data that groundbreaking scientific discoveries are made, game-changing innovations are fueled, and quality of life is improved for billions of people around the globe. HPC is the foundation for scientific, industrial, and societal advancements.

As technologies like the Internet of Things (IoT), artificial intelligence (AI), and 3-D imaging evolve, the size and amount of data that organizations have to work with is growing exponentially. For many purposes, such as streaming a live sporting event, tracking a developing storm, testing new products, or analyzing stock trends, the ability to process data in real time is crucial.

To keep a step ahead of the competition, organizations need lightning-fast, highly reliable IT infrastructure to process, store, and analyze massive amounts of data.

How does HPC work?

HPC solutions have three main components:

To build a high-performance computing architecture, compute servers are networked together into a cluster. Software programs and algorithms are run simultaneously on the servers in the cluster. The cluster is networked to the data storage to capture the output. Together, these components operate seamlessly to complete a diverse set of tasks.

To operate at maximum performance, each component must keep pace with the others. For example, the storage component must be able to feed and ingest data to and from the compute servers as quickly as it is processed. Likewise, the networking components must be able to support the high-speed transportation of data between compute servers and the data storage. If one component cannot keep up with the rest, the performance of the entire HPC infrastructure suffers.

What is an HPC cluster?

HPC use cases

Deployed on premises, at the edge, or in the cloud, HPC solutions are used for a variety of purposes across multiple industries. Examples include:

• Research labs . HPC is used to help scientists find sources of renewable energy, understand the evolution of our universe, predict and track storms, and create new materials.
• Media and entertainment . HPC is used to edit feature films, render mind-blowing special effects, and stream live events around the world.
• Oil and gas . HPC is used to more accurately identify where to drill for new wells and to help boost production from existing wells.
• Artificial intelligence and machine learning. HPC is used to detect credit card fraud, provide self-guided technical support, teach self-driving vehicles, and improve cancer screening techniques.
• Financial services. HPC is used to track real-time stock trends and automate trading.
• HPC is used to design new products, simulate test scenarios, and make sure that parts are kept in stock so that production lines aren’t held up.
• HPC is used to help develop cures for diseases like diabetes and cancer and to enable faster, more accurate patient diagnosis.

NetApp and HPC

The NetApp ® HPC solution features a complete line of high-performance, high-density E-Series storage systems. A modular architecture with industry-leading price/performance offers a true pay-as-you-grow solution to support storage requirements for multi-petabyte datasets. The system is integrated with leading HPC file systems, including Lustre, IBM Spectrum Scale, BeeGFS, and others to handle the performance and reliability requirements of the world’s largest computing infrastructures.

E-Series systems provide the performance, reliability, scalability, simplicity, and lower TCO needed to take on the challenges of supporting extreme workloads:

• Performance . Delivers up to 1 million random read IOPS and 13GB/sec sustained (maximum burst) write bandwidth per scalable building block. Optimized for both flash and spinning media, the NetApp HPC solution includes built-in technology that monitors workloads and automatically adjusts configurations to maximize performance.
• Reliability. Fault-tolerant design delivers greater than 99.9999% availability, proven by more than 1 million systems deployed. Built-in Data Assurance features help make sure that data is accurate with no drops, corruption, or missed bits.
• Easy to deploy and manage. Modular design, on-the-fly (“cut and paste”) replication of storage blocks, proactive monitoring, and automation scripts all add up to easy, fast and flexible management.
• Scalability. A granular, building-block approach to growth that enables seamless scalability from terabytes to petabytes by adding capacity in any increment—one or multiple drives at a time.
• Lower TCO . Price/performance-optimized building blocks and the industry’s best density per delivers low power, cooling, and support costs, and 4-times lower failure rates than commodity HDD and SSD devices.

Continue reading

Meet the critical needs of your high performance computing environment in Azure