Examples and Notes

Login to Data Star: 

 ssh -l your_user_name dslogin.sdsc.edu

File transfer to/from DataStar: 

To copy the file "pi.c" from a CS machine to the DataStar:
 
          scp pi.c your_SDSC_username@dslogin.sdsc.edu:pi.c 

To copy the file pi.c from DataStar to the CS domain machines: 

          scp pi.c your_UCSB_username@linux.engr.ucsb.edu:pi.c

which will copy the "pi.c" file to the machines both in the "CS" and
the "ENGR" domain, as all cs110b students have also "Engineering"
accounts [with the same password].

Compiling Serial Programs

Serial version 
 Compile your programs with xlc, xlC, xlf90 and xlf95 (these are 
"wrapper" programs which call the actual compiler). Unless the user 
uses the -c option, these invoke the linkage editor 
to produce a single executable:

      xlc [options] file.c 	C
      xlf90 [options] file.f 	Fortran 90

Example:
      % 
      % xlc -o serial serial.c
      %

Compiling Shared Memory (Thread-based) Programs

Thread-based version 
 IBM's Parallel Environment (PE) is designed for the development 
and execution of parallel programs on DataStar. Parallel 
Operating Environment (POE) is the user interface to PE. The PE 
has several compiler scripts which link the program startup and 
shared memory libraries. Compile your programs with the following 
POE compiler scripts xlc_r, and xlf90_r (these automatically link 
with the appropriate thread-safe libraries):

      xlc_r [options] file.c	C
      xlf90_r [options] file.f	Fortran90 

Example:
      % 
      % xlc_r -o threads threads.c
      %

Compiling Message-Passing (MPI) Programs

MPI version 
 For message-passing programs, the Parallel Environment (PE) has 
several compiler scripts which link the startup and message 
passing libraries. Compile your programs with mpxlc, and mpxlf90 
(these automatically link with the appropriate MPI libraries):

      mpcc [options] file.c		C
      mpxlf90 [options] file.f		Fortran90

Example[see Running INTERACTIVE]:
      % 
      % poe exec-file -nodes 2 -tasks_per_node 2 -rmpool 1 -euilib us -euidevice sn_all 
      %

Parallel Tools

Using HPM

To use the HPM utility, compile the program: 
      %
      %mpcc -o parpi  parpi.c -lm  
      %

and run HPM with "-a" option. With this flag, a single file performance file is 
generated for all tasks.
      %
      %poe hpmcount -a -o hpm_out parpi -nodes 1 -tasks_per_node 4 -rmpool 1 -euilib us -euidevice sn_all 
      %

or run HPM with "-g 60" option. With this option, we get counts of cycles, instructions, 
and FP operations (including divides, FMA, loads, and stores).
      %
      %poe hpmcount -a -g 60 -o hpm_out parpi -nodes 1 -tasks_per_node 4 -rmpool 1 -euilib us -euidevice sn_al
      %

Using PERF

To use the  PERF utility, compile the program, using the "-g" option and linking with
the MPI-TRACE libraries. Also, make sure your code exits through MPI_FINALIZE.
      %
      %mpcc -g -o parpi-p parpi.c -L/usr/local/apps/mpitrace -lmpiprof -lm
      %

Then, run the exec file using the HPM utility:
      % 
      % poe hpmcount -a -o hpm_out parpi-p -nodes 1 -tasks_per_node 4 -rmpool 1 -euilib us -euidevice sn_all 
      %
It will produce  mpi_profile.task_number  output files.
Finaly, run in the same directory as your output files [to generate the SUMMARY file]:
      %
      % /usr/local/apps/perf/perf hpm_out > perf_summary
      %

Using IPM

To use the IPM utility, compile the program:
      %
      % mpcc -o parpi-ipm  -lm -lld parpi.c
      %

then, run the program with the command POE-IPM:
      % 
      % poe-ipm parpi-ipm -nodes 1 -tasks_per_node 4 -rmpool 1 -euilib us -euidevice sn_all 
      %

Finally, generate a Web page using the IPM_PARSE utility:    
      %
      % ipm_parse u4078.1168133946.135026.0
      %

where u4078.1168133946.135026.0 is obained by running the command POE_IPM

Setting the PATH and LICENSE for VAMPIR 

To use the VAMPIR utility, you have to be connected to dspoe.sdsc.edu and
have to set the PATH and LICENSE:
     %setenv PAL_LICENSEFILE /usr/local/apps/vampir/etc/license.dat
     %set path = ($path /usr/local/apps/vampir/bin)

Then, compile the program using the VAMPIR libraries:
     % 
     %mpcc -o parpi-v -L/usr/local/apps/vampirtrace/lib -lVT -lm -lld parpi.c
     %

Finally, run the program:
     %
     % poe parpi-v -nodes 1 -tasks_per_node 4 -rmpool 1 -euilib us -euidevice sn_all
     %

To access the VAMPIR utility, we need to run X-windows and using the command:
     %
     % vampir parpi-v.stf
     %

Using TotalView

To run TOTALVIEW, we need to compile the progam using the -g option:
      %
      % mpcc -g -o send-error send-error.c
      %

and, using the X-windows, to run:
      % 
      % totalview poe -a ./send-error -nodes 1 -tasks_per_node 4 -rmpool 1 -euilib us -euidevice sn_all 
      %
using the -a option.

Parallel Programs