My Doc's Got No Nodes

Looking for a punchline since 2002

Compiling Rmpi and doMPI for Windows HPC’s MSMPI

Mark

For 32-bit only

To get the Rmpi and doMPI packages working on Windows HPC, using Micrsoft’s MSMPI libraries:

PKG_CFLAGS   = -I"C:\Program Files\Microsoft HPC Pack 2008 SDK\Include" -DMPI2 -DWin32 "-D__int64=long long"
PKG_LIBS     = -L"C:\Program Files\Microsoft HPC Pack 2008 SDK\Lib\i386" -L"C:\Program Files\Microsoft HPC Pack 2008 SDK\Lib\amd64" -lmsmpi
  • Open an administrative command prompt (so that your R installation can be updated) and do the following:
cd <folder above the extracted packages>

set path=c:\Rtools\bin;c:\Rtools\MinGW\bin;c:\Rtools\MinGW64\bin;%PATH%

"c:\Program Files\R\R-2.14.0\bin\R.exe" --vanilla CMD INSTALL --build Rmpi

"c:\Program Files\R\R-2.14.0\bin\R.exe" --vanilla CMD INSTALL –build doMPI
  • Copy the zip files created, Rmpi\_0.6-0.zip and doMPI\_0.1-5.zip, to your cluster head node and pass the file paths to R’s install.packages function together with “foreach”.
  • library(doMPI) should then state that it has loaded Rmpi.

For 64-bit

That all worked for i386 (32-bit), but I got an access violation when trying to load the Rmpi package for x64. So instead, I built the Rmpi sources using the Visual C++ 2010 compiler (some fixing required), and dropped the new dlls over the top of the ones installed in the R library folder.

Using parallel foreach on Windows HPC

MSMPI (Microsoft’s implementation of MPI) doesn’t support spawning (at least not in the 2008 R2 version), so you need to use the non-spawning method:

library(doMPI)
cl <- startMPIcluster()
registerDoMPI(cl)

… use of foreach and %dopar% …

closeCluster(cl)
mpi.quit()

Finally, you queue the job using mpiexec on R (or a batch file that calls R).

To run one worker per core:

  • Set the job resource type to core, and set the minimum and maximum number of cores for the task to the number of cores on your worker nodes.
  • Use the command mpiexec –n \* myRrunner.bat

To run one worker per node:

  • Set the job resource type to node, and set the minimum and maximum number of nodes for the task to the number of nodes in your cluster.
  • Use the command mpiexec –cores 1 myRrunner.bat

Playing 24-bit wav files from DVD-R on the Sony BDP-S370

Mark

In an earlier post, I mentioned that the Sony BDP-S370 can play back 24-bit 44.1Khz LPCM audio if it’s stored as a wav file on a USB memory stick.

Just to add to that, the BDP-S370 can play back PCM (wav files) in 16-bit and 24-bit, 44.1Khz, 48Khz and 96Khz (if not more).

Although it won’t play back wav files via UPnP / DLNA, another option is to burn a DVD-R containing the audio files. This has worked for wav files, and I expect it’ll work for MP3 too.

As usual the S370 is picky about the format it will support, it didn’t like the data disc I burned using the standard Windows 7 DVD-ROM stuff from explorer.

Instead, I used oscdimg.exe with the –u option to generate a UDF image as an ISO file, which I then burned with the Windows 7 right-click burn option. oscdimg is documented here, and is available as a free download as part of the Windows Automated Installation Kit (Windows AIK).

“bootmgr is missing” installing Windows HPC 2008 R2 Compute Nodes

Mark

Creating a node template from a Windows Server 2008 R2 HPC edition disc, and then PXE booting the cluster nodes seems to leave the cluster node unbootable.

It seems that the default diskpart.txt creates a single partition that the Server 2008 WIM doesn’t initialize correctly.

If instead, you use one of the now unbootable cluster nodes and perform a GUI install into that single partition, you can create a WIM from that node to use with a new compute node template.

Trying to use the GUI install from an unpartitioned disk won’t work, because it overrides your partition choice and creates extra boot / system partitions. If you instead choose to install into the single partition that the diskpart stage created during the failed node install, you’ll get a Windows install that you can use to create your “Golden Image” from, following the instructions on technet: How to Capture a “Master” Compute Node Image Using Node Templates

Extending Wi-fi network range to wireless and wired devices

Mark

I recently wanted to extend the (woeful) range of the Sky router to get better Wi-fi signal for devices upstairs. I didn’t want to have to run a cable from the router to a new access point, but I thought it would be a bonus to be able to connect wired devices on the new ‘satellite’ network.

It turns out that this is possible with a cheap box from Edimax, the EW-7228APN. It can act as a wireless access point with better coverage (with the same security settings as the main router, but a different SSID) and also connect wired devices and send the traffic wirelessly to the main router.

Setup isn’t obvious though because the documentation isn’t great, so here’s the steps:

  1. From a command prompt, type ipconfig and note the ip address and gateway (for a default Sky Broadband setup the ip address will be 192.168.0.something, gateway 192.168.0.1)
  2. Visit the sky router config page in your browser (at 192.168.0.1) and note the wireless security settings.
  3. Connect the access point to the network port of a PC with the supplied cable.
  4. Change the TCP/IPv4 settings of the PC to use the static address 192.168.2.2 subnet mask 255.255.255.0 and leave the gateway blank
  5. Use your web browser to visit 192.168.2.1 to configure the access point, the user name is “admin” and the default password is “1234”.
  6. In the System Utility section, set the IP address to be on your subnet, e.g. 192.168.0.50 (you might want to change the sky router settings to not give out the .50 IP address in its DHCP), and the gateway address to be the address from your ipconfig. Apply but don’t reboot yet.
  7. In the Basic Settings section, choose Repeater, and click Site Survey. Choose your sky router. You may need to choose Repeater again when it returns to the Basic Settings page. Make up a new SSID for the access point. Apply but don’t reboot again.
  8. Go to the Security page and set the security settings to be the same as your Sky router. My router is set to WEP for the benefit of some old-school Nintendo DS Lites, so I set WEP and set the key length and hex codes.
  9. At this point I could apply and reboot. I then needed to change the PC’s TCP/IPv4 settings to be 192.168.0.1 (not the original address) to log back in to the access point at 192.168.0.50 and check everything was still set OK.
  10. Finally, I disconnected the cable and connected wirelessly using the new access point’s SSID and the sky router’s password/key. To do this with Windows 7 I had to create a new connection manually, otherwise I was asked for credentials (no idea what that’s about).
  11. I then set the PC back to dynamically acquire a TCP/IPv4 address, and connected via the wire to check that the PC got a DHCP address on the home network.

Result – the wireless devices have their Wi-fi signal bounced from the access point to the Sky router, but with better coverage, and any wired devices connected to the access point have their traffic sent wirelessly to the router too.

Phew.

Comments

And thanks to Benjamin for helping me with this

Anonymous - Dec 5, 2011

And thanks to Benjamin for helping me with this.

Sony BDP-S370 and HDCD

Mark

(Or playing back 44.1Khz 24 bit lpcm on the S370)

I wasted quite a bit of time trying to work out what kind of container to put HDCD-decoded audio into to get it to play on the BDP-S370.

The short story is this:

  • The audio should be on a local USB drive (not DLNA). I haven’t looked into DVD-R yet.
  • The S370 will play 44.1Khz 20-bit from a 24-bit wav file (like HDCD decodes), but the decoder I used had made a wav that the Sony player didn’t like (nor did Windows Media player). Doing a conversion in foobar2000 to a wav with the same bit depth and no resampling made a new 24-bit wav that played fine. I checked the actual bit depth with eac3to and found it to be 17-20 bits as expected.
  • DVD/DAT style audio at 48Khz will also play fine, as will Hi-Res audio at96Khz/24-bit. To get 96Khz through my amp (the STR-DH520) I had to set the “Audio Out” setting to “Amp” (not “TV + AMP”), the same was true for playing back SACDs in 5.1.

I might look into higher than 96Khz, or storing the audio on optical discs next. If I’m bored enough :-)

Equipment / Test disc used:

Incidental Mention:

Previous 4 of 8 Next