Tagged: OpenMPI

El Capitan for Computational Materials Science in 2016

Every few years, I give my laptop a fresh start and remove all the debris (applications, libraries, updates) that have built up. This time I started with a clean install of Mac OS 10.11 (El Capitan).

  • Basics

The first step is to install the essentials including Dropbox, Evernote, Todoist, Xcode (with xcode-select --install), Slack, Mendeley, MS Office, gcc/gfortranPython superpack, VESTA, Transmission, Mactex, Texmaker, Unrar, VLC, Adobe Creative Suite, iTerm, Textmate, XQuartz.

  • Fortran

While it is possible to survive using gfortan and freely available maths libraries, Intel Fortran and MKL tend to be faster and better tested (easier to compile) in my experience. For non-commericial purposes Intel Composer is now free for OS X. The package installs in a few clicks, but be sure source the variables in your .bash_profile:
source /opt/intel/mkl/bin/mklvars.sh intel64
source /opt/intel/bin/ifortvars.sh intel64

Finally you will need to make the MKL fast fourier transforms (FFTs) for use in most solid-state simulation packages:
cd $MKLROOT/interfaces/fftw3xf/
sudo make libintel64 CC=gcc

The outcome:
Arons-Air-V:~ aron$ which ifort
/usr/local/bin/ifort
Arons-Air-V:~ aron$ ifort --version
ifort (IFORT) 16.0.1 20151020

  • Openmpi

To enable parallelism, I downloaded the latest source code of openmpi (1.10.1).
./configure -prefix=/usr/local/openmpi-1.10.1 CC=gcc FC=ifort F77=ifort
make
sudo make install

be patient… it can easily take 20 minutes. Finally add to your .bash_profile:
export DYLD_LIBRARY_PATH=$DYLD_LIBRARY_PATH:/usr/local/openmpi-1.10.1/lib/
export PATH=./:/usr/local/openmpi-1.10.1/bin:$PATH

The outcome:
Arons-Air-V:~ aron$ which mpif90
/usr/local/openmpi-1.10.1/bin/mpif90
Arons-Air-V:~ aron$ mpif90 --version
ifort (IFORT) 16.0.1 20151020

  • Phonopy

We use this open-source lattice-dynamics package a lot in our research. There are a few more libraries to install first:
sudo easy_install pip
pip2 install lxml
pip2 install pyyaml
export CC=/usr/local/bin/gcc

then after expanding the source code, simply type:
python setup.py install

The outcome:
Arons-Air-V:~ aron$ phonopy
_
_ __ | |__ ___ _ __ ___ _ __ _ _
| '_ \| '_ \ / _ \| '_ \ / _ \ | '_ \| | | |
| |_) | | | | (_) | | | | (_) || |_) | |_| |
| .__/|_| |_|\___/|_| |_|\___(_) .__/ \__, |
|_| |_| |___/
1.10.9

  • Phono3py

If harmonic phonons are not enough for you, then Phono3py lets you calculate phonon-phonon interactions, but it gets very computationally expensive. We need to install hdf5 (for more efficient data management):
pip2 install h5py

and lapacke for faster code. Download the latest version of lapack and:
cp make.inc.example make.inc
make lapackelib

Then you are ready to compile. Download Phono3py and modify setup3.py to link to your compiled lapacke library.
if platform.system() == 'Darwin':
include_dirs += ['/Users/aron/Documents/progs/lapack/lapack-3.6.1/lapacke/include']
extra_link_args = ['/Users/aron/Documents/progs/lapack/lapack-3.6.1/liblapacke.a']
followed by:
python setup3.py install

The outcome:
Arons-Air-V:~ aron$ phono3py
_ _____
_ __ | |__ ___ _ __ ___|___ / _ __ _ _
| '_ \| '_ \ / _ \| '_ \ / _ \ |_ \| '_ \| | | |
| |_) | | | | (_) | | | | (_) |__) | |_) | |_| |
| .__/|_| |_|\___/|_| |_|\___/____/| .__/ \__, |
|_| |_| |___/
1.10.9

  • VASP

While we use a range of electronic structure packages, VASP is the old reliable. I downloaded the latest version (5.4.1), which has streamlined the install process.
cp ./arch/makefile.include.linux_intel ./makefile.include

which needs to be modified to point to the correct compilers (here gcc, ifort and mpifort). We will also remove -DscaLAPACK from the precompiler options and set SCALAPACK = . There are now three patches/bug fixes to install:
patch -p1 < patch.5.4.1.08072015
patch -p1 < patch.5.4.1.27082015
patch -p1 < patch.5.4.1.06112015

and one fix to sort out a gcc error. To the file ./src/lib/getshmem.c add one line at the end of the include statements:
#define SHM_NORESERVE 010000

The outcome:
Arons-Air-V:test aron$ mpirun -np 4 ../vasp_std
running on 4 total cores
distrk: each k-point on 4 cores, 1 groups
distr: one band on 1 cores, 4 groups
using from now: INCAR
vasp.5.4.1 24Jun15 (build Jan 02 2016 21:20:37) complex

  • ASE

The atomistic simulation environment is a useful set of Python tools and modules. It now installs, including the gui, in two lines:
brew install pygtk
pip install python-ase

The outcome:
ase-gui

ASE

I will update with more codes and tools as I find time (posted in January; revised in July).

Mountain Lions like OpenMPI

Following the previous post on installing Fortran compilers in OSX 10.8 (Mountain Lions like Fortran), the next step is to efficiently exploit all of those lovely i7 cores for computational chemistry.

1. OpenMPI
– Forget any version that comes with XCode as you need to compile OpenMPI against your new Fortran installation.
– Download the binary from http://www.open-mpi.org/ (current version 1.6.3).
– Unzip and enter directory.
– Run “./configure –prefix=/usr/local/openmpi-1.6.3 CC=gcc FC=ifort F77=ifort”.
– Run “make”.
– Run “sudo make install”.
– Add to your .bashrc or .bash_profile:
export PATH=./:/usr/local/openmpi-1.6.3/bin:~/bin:/opt/intel/bin:$PATH
export DYLD_LIBRARY_PATH=$DYLD_LIBRARY_PATH:/usr/local/openmpi-1.6.3/lib

Result:
$ which mpif90
/usr/local/openmpi-1.6.3/bin/mpif90

2. FHI-AIMS
– A popular quantum chemistry package (from here; current version 081912).
– Update the Makefile to include:

FC = ifort
FFLAGS = -O3 -ip
F90FLAGS = $(FFLAGS)
ARCHITECTURE = Generic
LAPACKBLAS = -L/opt/intel/mkl/lib \
-I/opt/intel/mkl/include -lmkl_intel_lp64 \
-lmkl_sequential -lmkl_core
USE_MPI = yes
MPIFC = mpif90

– Run “make mpi”.
– Enjoy parallel calculations, e.g. mpirun -np 4 fhi-aims

3. VASP
– A popular materials modelling package (from here; current version 5.3.3).
– In the main src folder, “cp makefile.linux_ifc_P4 Makefile”.
– Update the Makefile to include:

FC=mpif90
FCL=$(FC)
FFLAGS = -FR -assume byterecl
OFLAG=-O3 -ip -ftz
MKL=/opt/intel/mkl
BLAS=-L/$(MKL)/lib -I/$(MKL)/include -lmkl_intel_lp64 \
-lmkl_sequential -lmkl_core -lmkl_lapack95_lp64

– Run “make”.
– Enjoy parallel calculations, e.g. mpirun -np 4 vasp