Wednesday, 21 December 2011

Compiling Rmpi and doMPI for Windows HPC’s MSMPI

For 32-bit only:

To get the Rmpi and doMPI packages working on Windows HPC, using Micrsoft’s MSMPI libraries:

PKG_CFLAGS   = -I"C:\Program Files\Microsoft HPC Pack 2008 SDK\Include" -DMPI2 -DWin32 "-D__int64=long long"
PKG_LIBS     = -L"C:\Program Files\Microsoft HPC Pack 2008 SDK\Lib\i386" -L"C:\Program Files\Microsoft HPC Pack 2008 SDK\Lib\amd64" -lmsmpi

  • Open an administrative command prompt (so that your R installation can be updated) and do the following:

cd <folder above the extracted packages>

set path=c:\Rtools\bin;c:\Rtools\MinGW\bin;c:\Rtools\MinGW64\bin;%PATH%

"c:\Program Files\R\R-2.14.0\bin\R.exe" --vanilla CMD INSTALL --build Rmpi

"c:\Program Files\R\R-2.14.0\bin\R.exe" --vanilla CMD INSTALL –build doMPI

  • Copy the zip files created, Rmpi_0.6-0.zip and doMPI_0.1-5.zip, to your cluster head node and pass the file paths to R’s install.packages function together with “foreach”.
  • library(doMPI) should then state that it has loaded Rmpi.

For 64-bit:

That all worked for i386 (32-bit), but I got an access violation when trying to load the Rmpi package for x64. So instead, I built the Rmpi sources using the Visual C++ 2010 compiler (some fixing required), and dropped the new dlls over the top of the ones installed in the R library folder.

Using parallel foreach on Windows HPC

MSMPI (Microsoft’s implementation of MPI) doesn’t support spawning (at least not in the 2008 R2 version), so you need to use the non-spawning method:

library(doMPI)

cl <- startMPIcluster()

registerDoMPI(cl)

… use of foreach and %dopar% …

closeCluster(cl)

mpi.quit()

Finally, you queue the job using mpiexec on R (or a batch file that calls R).

To run one worker per core:

Set the job resource type to core, and set the minimum and maximum number of cores for the task to the number of cores on your worker nodes.

Use the command mpiexec –n * myRrunner.bat

To run one worker per node:

Set the job resource type to node, and set the minimum and maximum number of nodes for the task to the number of nodes in your cluster.

Use the command mpiexec –cores 1 myRrunner.bat

Wednesday, 14 December 2011

Playing 24-bit wav files from DVD-R on the Sony BDP-S370

In an earlier post, I mentioned that the Sony BDP-S370 can play back 24-bit 44.1Khz LPCM audio if it’s stored as a wav file on a USB memory stick.

Just to add to that, the BDP-S370 can play back PCM (wav files) in 16-bit and 24-bit, 44.1Khz, 48Khz and 96Khz (if not more).

Although it won’t play back wav files via UPnP / DLNA, another option is to burn a DVD-R containing the audio files. This has worked for wav files, and I expect it’ll work for MP3 too.

As usual the S370 is picky about the format it will support, it didn’t like the data disc I burned using the standard Windows 7 DVD-ROM stuff from explorer.

Instead, I used oscdimg.exe with the –u option to generate a UDF image as an ISO file, which I then burned with the Windows 7 right-click burn option. oscdimg is documented here, and is available as a free download as part of the Windows Automated Installation Kit (Windows AIK).

Monday, 12 December 2011

“bootmgr is missing” installing Windows HPC 2008 R2 Compute Nodes

Creating a node template from a Windows Server 2008 R2 HPC edition disc, and then PXE booting the cluster nodes seems to leave the cluster node unbootable.

It seems that the default diskpart.txt creates a single partition that the Server 2008 WIM doesn’t initialize correctly.

If instead, you use one of the now unbootable cluster nodes and perform a GUI install into that single partition, you can create a WIM from that node to use with a new compute node template.

Trying to use the GUI install from an unpartitioned disk won’t work, because it overrides your partition choice and creates extra boot / system partitions. If you instead choose to install into the single partition that the diskpart stage created during the failed node install, you’ll get a Windows install that you can use to create your “Golden Image” from, following the instructions on technet: How to Capture a “Master” Compute Node Image Using Node Templates

Monday, 5 December 2011

Extending Wi-fi network range to wireless and wired devices

I recently wanted to extend the (woeful) range of the Sky router to get better Wi-fi signal for devices upstairs. I didn’t want to have to run a cable from the router to a new access point, but I thought it would be a bonus to be able to connect wired devices on the new ‘satellite’ network.

It turns out that this is possible with a cheap box from Edimax, the EW-7228APN. It can act as a wireless access point with better coverage (with the same security settings as the main router, but a different SSID) and also connect wired devices and send the traffic wirelessly to the main router.

Setup isn’t obvious though because the documentation isn’t great, so here’s the steps:

  1. From a command prompt, type “ipconfig” and note the ip address and gateway (for a default Sky Broadband setup the ip address will be 192.168.0.something, gateway 192.168.0.1)
  2. Visit the sky router config page in your browser (at 192.168.0.1) and note the wireless security settings.
  3. Connect the access point to the network port of a PC with the supplied cable.
  4. Change the TCP/IPv4 settings of the PC to use the static address 192.168.2.2 subnet mask 255.255.255.0 and leave the gateway blank
  5. Use your web browser to visit 192.168.2.1 to configure the access point, the user name is “admin” and the default password is “1234”.
  6. In the System Utility section, set the IP address to be on your subnet, e.g. 192.168.0.50 (you might want to change the sky router settings to not give out the .50 IP address in its DHCP), and the gateway address to be the address from your ipconfig. Apply but don’t reboot yet.
  7. In the Basic Settings section, choose Repeater, and click Site Survey. Choose your sky router. You may need to choose Repeater again when it returns to the Basic Settings page. Make up a new SSID for the access point. Apply but don’t reboot again.
  8. Go to the Security page and set the security settings to be the same as your Sky router. My router is set to WEP for the benefit of some old-school Nintendo DS Lites, so I set WEP and set the key length and hex codes.
  9. At this point I could apply and reboot. I then needed to change the PC’s TCP/IPv4 settings to be 192.168.0.1 (not the original address) to log back in to the access point at 192.168.0.50 and check everything was still set OK.
  10. Finally, I disconnected the cable and connected wirelessly using the new access point’s SSID and the sky router’s password/key. To do this with Windows 7 I had to create a new connection manually, otherwise I was asked for credentials (no idea what that’s about).
  11. I then set the PC back to dynamically acquire a TCP/IPv4 address, and connected via the wire to check that the PC got a DHCP address on the home network.

Result – the wireless devices have their Wi-fi signal bounced from the access point to the Sky router, but with better coverage, and any wired devices connected to the access point have their traffic sent wirelessly to the router too.

Phew.