I am wondering if anyone could provide me with guidance on how to set up multi-locale on a cluster of Raspberry Pi -- a procedure or walk through or even just hints would be welcome.
Chapel compiles successfully on a single node and I can run all the tutorial code. I re-made chapel with the suggested environment variables in place for multi-locale but I just can't get it to work.
The output from printenv looks like this:
CHPL_HOME: /home/mpiu/chapel/chapel-1.23.0 *
script location: /home/mpiu/chapel/chapel-1.23.0/util/chplenv
CHPL_TARGET_PLATFORM: linux64
CHPL_TARGET_COMPILER: gnu
CHPL_TARGET_ARCH: aarch64
CHPL_TARGET_CPU: aarch64 *
CHPL_LOCALE_MODEL: flat
CHPL_COMM: gasnet *
CHPL_COMM_SUBSTRATE: udp *
CHPL_GASNET_SEGMENT: everything
CHPL_TASKS: qthreads
CHPL_LAUNCHER: amudprun
CHPL_TIMERS: generic
CHPL_UNWIND: none
CHPL_MEM: jemalloc
CHPL_ATOMICS: cstdlib
CHPL_NETWORK_ATOMICS: none
CHPL_GMP: gmp
CHPL_HWLOC: hwloc
CHPL_REGEXP: re2
CHPL_LLVM: none
CHPL_AUX_FILESYS: none
My environment variables look like this:
CHPL_HOME=/home/mpiu/chapel/chapel-1.23.0
CHPL_TARGET_CPU=aarch64
CHPL_COMM_SUBSTRATE=udp
CHPL_COMM=gasnet
GASNET_WORKERIP=192.168.1.0
GASNET_SPAWNFN=S
GASNET_SSH_CMD=ssh
GASNET_SSH_OPTIONS=-o LogLevel=Error -x
GASNET_SSH_SERVERS=oc2-a oc2-b
GASNET_MASTERIP=192.168.1.146
Using this example code:
use Memory;
for loc in Locales do
on loc {
writeln("locale #", here.id, "...");
writeln(" ...is named: ", here.name);
writeln(" ...has ", here.numPUs(), " processor cores");
writeln(" ...has ", here.physicalMemory(unit=MemUnits.GB, retType=real), " GB of memory");
writeln(" ...has ", here.maxTaskPar, " maximum parallelism");
}
And compiling it like this:
chpl --no-local -o printLocales printLocales.chpl
When I run the executable:
./printLocales -nl 2
I get the following error:
bash: line 1: 22782 Illegal instruction env 'AMUDP_SLAVE_ARGS=1,192,168,1,146,159,52,192.168.1.0' '/home/mpiu/Code/chapel/printLocales_real' '-nl' '1' '-E' 'SHELL=/bin/bash' '-E' 'GASNET_WORKERIP=192.168.1.0' '-E' 'CHPL_HOME=/home/mpiu/chapel/chapel-1.23.0' '-E' 'CHPL_TARGET_CPU=aarch64' '-E' 'GASNET_SPAWNFN=S' '-E' 'PWD=/home/mpiu/Code/chapel' '-E' 'LOGNAME=mpiu' '-E' 'XDG_SESSION_TYPE=tty' '-E' 'MANPATH=/home/mpiu/chapel/chapel-1.23.0/man:' '-E' 'HOME=/home/mpiu' '-E' 'LANG=en_US.UTF-8' '-E' 'GASNET_SSH_CMD=ssh' '-E' 'GASNET_SSH_OPTIONS=-o LogLevel=Error -x' '-E' 'GASNET_SSH_SERVERS=oc2-a oc2-b' '-E' 'CHPL_COMM_SUBSTRATE=udp' '-E' 'SSH_CONNECTION=192.168.1.47 53342 192.168.1.146 22' '-E' 'GASNET_MASTERIP=192.168.1.146' '-E' 'CHPL_COMM=gasnet' '-E' 'XDG_SESSION_CLASS=user' '-E' 'TERM=xterm-256color' '-E' 'USER=mpiu' '-E' 'SHLVL=1' '-E' 'XDG_SESSION_ID=c3' '-E' 'XDG_RUNTIME_DIR=/run/user/1002' '-E' 'SSH_CLIENT=192.168.1.47 53342 22' '-E' 'LC_ALL=en_US.UTF-8' '-E' 'PATH=/home/mpiu/chapel/chapel-1.23.0/bin/linux64-aarch64:/home/mpiu/chapel/chapel-1.23.0/util:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games' '-E' 'MAIL=/var/mail/mpiu' '-E' 'SSH_TTY=/dev/pts/0' '-E' 'OLDPWD=/home/mpiu' '-E' '_=./printLocales'
I have tried many different things (including using the MPI conduit as well), I just can't figure out what I am doing wrong. If it matters, MPI works fine on this cluster.
( An original error one-liner slightly re-formatted for faster readability )
bash: line 1: 22782 Illegal instruction \
env \
'AMUDP_SLAVE_ARGS=1,192,168,1,146,159,52,192.168.1.0' \
'/home/mpiu/Code/chapel/printLocales_real' \
'-nl' '1' \
'-E' 'SHELL=/bin/bash' \
'-E' 'GASNET_WORKERIP=192.168.1.0' \
'-E' 'CHPL_HOME=/home/mpiu/chapel/chapel-1.23.0' \
'-E' 'CHPL_TARGET_CPU=aarch64' \
'-E' 'GASNET_SPAWNFN=S' \
'-E' 'PWD=/home/mpiu/Code/chapel' \
'-E' 'LOGNAME=mpiu' \
'-E' 'XDG_SESSION_TYPE=tty' \
'-E' 'MANPATH=/home/mpiu/chapel/chapel-1.23.0/man:' \
'-E' 'HOME=/home/mpiu' \
'-E' 'LANG=en_US.UTF-8' \
'-E' 'GASNET_SSH_CMD=ssh' \
'-E' 'GASNET_SSH_OPTIONS=-o LogLevel=Error -x' \
'-E' 'GASNET_SSH_SERVERS=oc2-a oc2-b' \
'-E' 'CHPL_COMM_SUBSTRATE=udp' \
'-E' 'SSH_CONNECTION=192.168.1.47 53342 192.168.1.146 22' \
'-E' 'GASNET_MASTERIP=192.168.1.146' \
'-E' 'CHPL_COMM=gasnet' \
'-E' 'XDG_SESSION_CLASS=user' \
'-E' 'TERM=xterm-256color' \
'-E' 'USER=mpiu' \
'-E' 'SHLVL=1' \
'-E' 'XDG_SESSION_ID=c3' \
'-E' 'XDG_RUNTIME_DIR=/run/user/1002' \
'-E' 'SSH_CLIENT=192.168.1.47 53342 22' \
'-E' 'LC_ALL=en_US.UTF-8' \
'-E' 'PATH=/home/mpiu/chapel/chapel-1.23.0/bin/linux64-aarch64:/home/mpiu/chapel/chapel-1.23.0/util:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games' \
'-E' 'MAIL=/var/mail/mpiu' \
'-E' 'SSH_TTY=/dev/pts/0' \
'-E' 'OLDPWD=/home/mpiu' \
'-E' '_=./printLocales'