Christian Saborío’s Blog

While on Bogotá, I had the honor of speaking to a lot of interested people in Windows HPC Server 2008. Recently I have received some questions which I thought I would post here with the hope of helping someone in the future.

Question #1: If I am building my cluster, do ALL machines have to be 64-bit?

Answer: The general requirement of Windows HPC Server 2008 is that the hardware must be 64-bit. This applies for both the compute nodes and the head nodes. So basically, you cannot have a 32-bit machine take the role of a compute node, router node, or head node in a Windows HPC Server 2008 cluster (or Windows Compute Cluster 2003 Edition, for that matter).

Question #2: I am planning on building a small cluster and I would like to know the # of required machines and their characteristics to build a cluster

Answer: A compute cluser, by definition, is a head node, a scheduling mechanism, a compute node, and a queue. Based on this definition, a head that servers the purpose of being a compute node as well is in good theory, a cluster (although I would not expect it to show up in the Top 500 list anytime soon

If you would like to mess around a bit with Windows HPC Server 2008, I would recommend at least 4 machines including the head node. This can give you a great experience testing deployment, configuration, troubleshooting, and so on. As far as the requirements, they are the same as any Windows Server 2008 x64 edition: at least 512 MB of RAM and and x64 compatible processor. If you are planning on using a private network, then you will need at least two NICs.

Question #3: What would be the best connection to test my cluster?

Answer: It all depends, but for starters, you can use GigE to set up a test topology and it should work rather well. If you are planning on doing some heavy duty MPI development, then an MPI network (e.g. Infiniband) should do the trick for you; but remember, a high speed network will only help you if you are running MPI applications.

Hope this helps!

During our LATAM Jumpstart conferences, we usually referred to the Top 500 and kept being secretive on the results for Windows HPC Server 2008 as we announced that something good was coming. Well, the wait is over, and if you head over to Volker’s Blog, you can get a great summary on this great Microsoft accomplishment!

Microsoft campus is huge. The following maps are a great way to get around:

Main Campus: http://members.microsoft.com/careers/mslife/locations/images/campus_map.jpg
West Campus: http://members.microsoft.com/careers/mslife/locations/images/west_campus.jpg
North Campus: http://members.microsoft.com/careers/mslife/locations/images/north_campus.jpg

Just in case the links die someday:

Main Campus

West Campus

North Campus

One of the attendees from the Costa Rican Airlift event inquired on how he could get to test drive Windows Server HPC Server 2008. Would he need to download a trial version of the HPC OS? Would he need to buy Windows Server 2008 in order to test drive it? The good news is that you do not have to shell out a dime (at this point) if you want to test drive any of these technologies:

• Windows Hyper-V
• Windows HPC Server 2008

How, you ask? Here is what you need to do.

First, go and download a trial version of Windows Server 2008 64-bit. Yes, 64-bit - neither Hyper-V of HPC 2008 works with 32-bit editions of the software. You can use enterprise, datacenter, or standard. I’ve done all my tests with Enterprise.

Once you download the ISO, burn it and install Windows Server. One of the things that I like about these trial versions is that you do not need to input a key in order to test them. When you get to this screen:

Just click Next and click on NO when asked if you would like to enter a key.

This will give you 60 days of trial version. If you are running out of time, then you can extend the 60 day trial time 2 more times (for a total of 180 days).

Ok, that takes care of the base OS / host. You can now install Hyper-V and run your own VMs to run a virtual cluster.

Ever since I started working a bit more on Windows HPC Server 2008, I knew that I had to learn the basics of WCF (Windows Communication Foundation) and pronto. I was lucky I had bought a great book a while back that was currently stacked alongside a minion of books waiting to be read. The book is called “Learning WCF” by the good fellows at O’Reilly. This has been an amazing source of information as of now.

I also found some great .Net Framework 3.0 labs (virtual) from the following site:

http://msdn.microsoft.com/en-us/virtuallabs/aa740389.aspx

The labs are a great place to start since you have to suffer creating everything from scratch, which in my experience is the best way to learn. I am not buying the whole virtual lab thing yet though, reasons include: not being able to save your session or extend the time, network lag, very small window to work with, etc. So I downloaded the PDF from the lab (MSDN Virtual Lab: The Fundamentals of Programming the Windows Communication Foundation) and tried it on my VM running Visual Studio 2008.

The bad thing is that the labs are a bit outdated, so even though they will work on the virtual lab environment, you will be using deprecated methods, attributes, and so on, which is not good at all. Running the labs in VS2008 had a bunch of issues that took a while to track down and fix. In case you want to carry out the lab with Visual Studio, I have posted the PDF I used that contains some notes on what you need to change to make things work. The lab is definitely worth carrying out if you want to learn the basics of WCF:

http://www.yousendit.com/transfer.php?action=batch_download&batch_id=RXNqQ1ZyTERFc0kwTVE9PQ

Now, it is time to understand and work with the WCF Broker Nodes in Windows HPC Server 2008

While doing some cleaning in one of my VMs, I found a script that created 30 user accounts automatically in an active directory. It had 30 accounts being created, and when I created it was racing against time so did not had the chance to create a script that would iterate. Here is the updated script:

Run this in your AD and you will have the accounts created for you. This is very useful in lab environments. Notice that in the script, every account will be a domain admin, research a bit more on the net group command to understand how to add to other groups instead.

BTW, here is a link on for loops on batch files: http://www.ss64.com/nt/for.html

Or was putting these two fellows right next to each other something completely irrational:

I know you will receive confirmation on whether you want to format and so on, but I really don’t see any relation in having them side by side. Call me naive, but I just don’t see how destroying information on a disk and ejecting a disk is related in any way whatsoever.

I recently had to reinstall Windows 2008 on my HPC demo laptop. I thought this was going to be a great way to test the Hyper-V export features from Windows 2008 Hyper-V. At this point my head node (a VM) has over 10 snapshots (over 15 GB of snapshots!!) , which are very different from one another. If this worked, then I would definitely think this product is ready for production use.

Unfortunately, I did not get to that point. I followed the standard procedure for installing Hyper-V:

1. Do a Windows Update
2. Install the Hyper-V Role
3. Reboot

When I was booting my VMs, I got an error message about not being able to restore the saved state. I discarded the saved state and tried again. It booted fine but when it was booting, the following caught my attention:

Beta???? Hyper-V is currently in Released Candidate, and when you boot a VM, it displays “Release Candidate” in green. Needless to say, when my VM booted, the Integration Services did not work and pretty much nothing worked the way I wanted to. Why did Windows Update given me a Beta version? I got info on the version of Hyper-V and found the following:

I have no clue what version number is RC, so I unistaled Hyper-V and downloaded the KB windows update file that has the Hyper-V role:

Upon installation, rebooting, adding role, I got the version number and found the following:

*SIGH* - same version. I then created a VM, started it and this is what I got:

Don’t ask why and I won’t lie

About a week ago I started delivering the LATAM Jumpstart event, which is a 1-day Microsoft event (3 day in Brasil) in which we get to showcase the features of Windows HPC Server 2008 (version 2 of Windows Compute Cluster 2003).  So far, it has been a great experience in both terms of showcasing the new and improved features of the product as well as meeting with very nice and cool people that use or are planning on using this product.

The first event was in Bogota, Colombia - where I must say, are the nicest people I have ever met in my life.   People from Bogota are extremely polite and very very helpful.  The event went well despite the NVidia driver on my MacBook Pro crapping out 30 minutes before the presentations.  I managed to get the machine back online through a series of black magic spells and internal cussing.   It would have been terrible if I had not been able to use the machine as all the demos were running in Hyper-V, and getting everything back together would have taken quite some time.   We had about 60 people show up, which was a great number.   Most of these folks had never used CCS in the past, so it was a great way to let them know what they have been missing

The last 3 days, I’ve been here with Phil Pennington, developer evangelist at Microsoft, delivering a 3-day training/session kind of thing in Sao Paulo.   I had no idea this city was so big, I was quite impressed.   Our attendees in Sao Paulo were really great.   During the labs, we realized that ALL the machines had been imaged from the same source without being syspreped, and that is a very very bad thing when you are dealing with active directory.   The HPC pack could not communicate well with the DC, so we had to carry out a series of tasks such as syspreping ALL the machines and creating a new DC so that things were normal again.  All this while, our attendees were very patient about the whole deal.

Right now I packing up to go to the airport and catch my flight to Santiago, where I will try and get to know a bit better over the weekend.  The bad thing about these events is that they are very close one from the other, so it has been literally impossible to get to know any of the cities I have been to so far.   Oh well, maybe next time we’ll manage to plan it somewhat differently.

First off, bookmark this page, it has a lot of info on Caspol:

http://blogs.msdn.com/shawnfa/archive/2004/12/30/344554.aspx

I am currently setting up some demos to show in an HPC Jumpstart event in Bogota Colombia in 3 days. One of the demos includes using MPI.Net (an amazing implementation of MPI), and I was running into serious problems trying to get it to work.

When you run binaries with Windows Server HPC 2008, you need a place that will hold the binaries so that each worker process can make a run them from there. An obvious solution to this is to use network shares so that anyone with the right set of permissions - in this case the worker nodes, can run the binaries.

If you ever run a .Net assembly from a local share, you will be greeted a with System.SecurityException that will cause your application to go down in flames. In Windows Server 2003, you could easily modify these settings so that you could change the policy level and let all .Net assemblies un UNC paths execute without any type of restriction.  This was done by the .Net Configuration pane in Administrative Tools.

I tried to find the .Net Configuration pane in Windows Server 2008 and it was nowhere to be found, it’s gone. I looked for it under server manager, tried installing new roles to see if it showed up and NADA. There goes my demos, I thought

In the midst of pulling out my hair, Phil suggested to try Caspol, a utility included with the 2.0 .Net framework that allows you to change security settings for assemblies. The following did the trick:

c:\Windows\Microsoft.NET\Framework64\v2.0.50727\CasPol.exe -q -m -ag 1.2 -url file://headnode/scratch/04-ManagedAPI/03-PingPong/* FullTrust

This allowed me to run the binary from\\headnode\scratch4-ManagedAPI3-PingPong

I then used the most excellent utility from Windows HPC Server 2008 Clusrun to run a batch file on ALL the nodes of my cluster to set the security settings for all of my demos…sweet:

HEADNODE -> Finished
————————————————————————————————————————

C:\Users\administrator.HPC>c:\Windows\Microsoft.NET\Framework64\v2.0.50727\CasPol.exe -q -m -ag 1.2 -url file://headnode/scratch/04-ManagedAPI/01-Hello/* FullTrust
Microsoft (R) .NET Framework CasPol 2.0.50727.1434
Copyright (c) Microsoft Corporation. All rights reserved.

Added union code group with “-url” membership condition to the Machine level.
Success

…