Tags:
, view all tags

About this wiki

The following best practices document aims to provide some hints and examples on how to install and configure OpenFOAM-2.0.1 released 04/08/11 on a grid based infrastructure.

Application Description

OpenFOAM (Open Field Operation and Manipulation) CFD Toolbox is a free, open source CFD software package produced by a commercial company, OpenCFD Ltd.
It has a large user base across most areas of engineering and science, from both commercial and academic organizations. OpenFOAM has an extensive range of features
to solve anything from complex fluid flows involving chemical reactions, turbulence and heat transfer, to solid dynamics and electro-magnetics.
This wiki provides information on how to build OpenFOAM in a grid-based infrastructure.

Pre-requisites

The instructions from this best practice have been tested on a:

  • Processor: Dual Core AMD Opteron(tm) Processor 275;
  • Memory: 4147300k
  • OS: Scientific Linux SL release 5.4 (Boron)
  • Compiler: GNU 4.4.0;
  • MPI: Open MPI 1.4.2.

Source Pack

The following tar-zipped gtgz source packs are available for download.

Pack File md5sum
OpenFOAM OpenFOAM-2.0.1.gtgz 0a9dfa42282a5e629523b70e2544c773
Third-Party ThirdParty-2.0.1.gtgz 4b91af77bdbd3a87d91eeccb0f596f59

Unpacking the source and installation

In a grid-based infrastructure we would recommend to install the software in the experiment software directory for the VO using sgm privileges and share this software
packages with other VO members. In this wiki we chosen to unpack this software under the $VO_GRIDIT_SW_DIR directory of the grid CE.

$ cd $VO_GRIDIT_SW_DIR
$ tar xzf OpenFOAM-2.0.1.gtgz
$ tar xzf ThirdParty-2.0.1.gtgz
$ chown -R root.root OpenFOAM-2.0.1/ ThirdParty-2.0.1/

The files unpack to produce directories OpenFOAM-2.0.1 and ThirdParty-2.0.1

$ ll OpenFOAM
total 12
drwxr-xr-x  11 sgmgridit001 sgmgridit   440 Nov 28 16:17 OpenFOAM-2.0.1
drwxr-xr-x  12 sgmgridit001 sgmgridit   768 Nov 28 15:57 ThirdParty-2.0.1

System Requirements

OpenFOAM-2.0.1 builds on many Linux distributions but the ParaView-3.10.1 version supplied in ThirdParty requires:

Both software packages can be freely downloaded for many Linux distributions.

Before to start building software, please check if the following packages have been installed on the server.

$ yum install bison 
$ yum install ncurses-devel 
$ yum install tix.x86_64 
$ yum install glibc-devel 
$ yum install flex 
$ yum install flex-devel 
$ yum install zlib-devel 
$ yum install libXt-devel 
$ yum install binutils-devel 
$ yum install gmp.x86_64 
$ yum install gmp-devel.x86_64 
$ yum install libstdc++44-devel.x86_64

Make sure you have GCC 4.4 installed on your server.

$ wget ftp://ftp.ntua.gr/pub/linux/scientificlinux/55/x86_64/SL/gcc44-4.4.0-6.el5.x86_64.rpm
$ rpm -ivh gcc44-4.4.0-6.el5.x86_64.rpm

$ wget ftp://ftp.ntua.gr/pub/linux/scientificlinux/55/x86_64/SL/gcc44-c++-4.4.0-6.el5.x86_64.rpm
$ rpm -ivh gcc44-c++-4.4.0-6.el5.x86_64.rpm

Building Qt-4.6.4

First uncompress the archive in the preferred location, then unpack it:

$ cd /opt/exp_soft/gridit/OpenFOAM/ThirdParty-2.0.1
$ wget ftp://ftp.trolltech.com/qt/source/qt-everywhere-opensource-src-4.6.4.tar.gz
$ tar zxf qt-everywhere-opensource-src-4.6.4.tar.gz
$ chown -R root.root qt-everywhere-opensource-src-4.6.4
$ cd qt-everywhere-opensource-src-4.6.4
$ ./configure
$ gmake && gmake install

Building cmake-2.8.6

$ cd /opt/exp_soft/gridit/OpenFOAM/ThirdParty-2.0.1
$ wget  http://www.cmake.org/files/v2.8/cmake-2.8.6.tar.gz
$ tar zxvf cmake-2.8.6.tar.gz
$ cd cmake-2.8.6
$ ./configure
$ gmake

Makefile changes for GNU compilers

Once you have successfully installed GCC 4.4.0, then changes these files:

$ cd /opt/exp_soft/gridit/OpenFOAM/OpenFOAM-2.0.1/wmake/rules/linux64Gcc44/

$ cat wmake/rules/linux64Gcc44/c
.SUFFIXES: .c .h
cWARN        = -Wall
cc          = gcc44 -m64   <== Change Here!
include $(RULES)/c$(WM_COMPILE_OPTION)
cFLAGS      = $(GFLAGS) $(cWARN) $(cOPT) $(cDBUG) $(LIB_HEADER_DIRS) -fPIC
ctoo        = $(WM_SCHEDULER) $(cc) $(cFLAGS) -c $$SOURCE -o $@
LINK_LIBS   = $(cDBUG)
LINKLIBSO   = $(cc) -shared
LINKEXE     = $(cc) -Xlinker --add-needed -Xlinker -z -Xlinker nodefs


$ cat wmake/rules/linux64Gcc44/c++
.SUFFIXES: .C .cxx .cc .cpp
c++WARN     = -Wall -Wextra -Wno-unused-parameter -Wold-style-cast
CC          = g++44 -m64   <== Change Here!
include $(RULES)/c++$(WM_COMPILE_OPTION)
ptFLAGS     = -DNoRepository -ftemplate-depth-100
c++FLAGS    = $(GFLAGS) $(c++WARN) $(c++OPT) $(c++DBUG) $(ptFLAGS) $(LIB_HEADER_DIRS) -fPIC
Ctoo        = $(WM_SCHEDULER) $(CC) $(c++FLAGS) -c $$SOURCE -o $@
cxxtoo      = $(Ctoo)
cctoo       = $(Ctoo)
cpptoo      = $(Ctoo)
LINK_LIBS   = $(c++DBUG)
LINKLIBSO   = $(CC) $(c++FLAGS) -shared
LINKEXE     = $(CC) $(c++FLAGS) -Xlinker --add-needed

Pre-setup requirements

$ cd /opt/exp_soft/gridit/OpenFOAM/OpenFOAM-2.0.1/etc

Configure the OpenFOAM bashrc file as follows:

  • # Location of the OpenFOAM installation
    foamInstall=/opt/exp_soft/gridit/$WM_PROJECT

  • #- Compiler: # WM_COMPILER = Gcc | Gcc43 | Gcc44 | Gcc45 | Gcc46 | Clang | Icc (Intel icc)
    export WM_COMPILER=Gcc44

  • # Location of installation
    export WM_PROJECT_INST_DIR=$FOAM_INST_DIR
    export WM_PROJECT_DIR=$WM_PROJECT_INST_DIR/$WM_PROJECT-$WM_PROJECT_VERSION

  • # Location of third-party software
    export WM_THIRD_PARTY_DIR=$WM_PROJECT_INST_DIR/ThirdParty-$WM_PROJECT_VERSION

  • # Location of user file
    export WM_PROJECT_USER_DIR=$WM_PROJECT_DIR

Checking the System

To check your system is ready to build the sources, execute the foamSystemCheck script (in the OpenFOAM-2.0.1/bin directory).
If any critical software is missing, or needs updating to a newer version, please contact the system administrator to install the required
software before proceeding to the build.

$ cd /opt/exp_soft/gridit/OpenFOAM/OpenFOAM-2.0.1/bin
$ ./foamSystemCheck 

Checking basic system...
-----------------------------------------------------------------------
Shell:           /bin/bash
Host:            grid012.ct.infn.it
OS:              Linux version 2.6.9-89.0.11.ELsmp
User:            root

System check: PASS
==================
Continue OpenFOAM installation.

Building Sources

Go to the top-level source directory $WM_PROJECT_DIR and execute the top-level build script ./Allwmake
In principle this will build everything, but if problems occur with the build order it may be necessary to update the environment variables and re-execute.

If you experience difficulties with building the source-pack, or your platform is not currently supported, please contact software support to arrange a
support contract and we will do the port and maintain it for future releases.

$ cd /opt/exp_soft/gridit/OpenFOAM/OpenFOAM-2.0.1/
$ source  etc/bashrc
$ ./Allwmake >log 2>log &

Compiling Paraview 3.10.1 and the PV3FoamReader Module

Paraview is the third-party software that we provide for graphical post-processing in OpenFOAM. Itís compilation is automated using a script called makeParaView
in the ThirdParty-2.0.1 directory. Installation of Paraview 3.10.1 requires a version of QT that is 3.6.2 or newer and cmake which is 2.8.2 or newer, so again make
sure that this is on your system.

$ cd $WM_THIRD_PARTY_DIR

Configure the QMAKE_PATH and CMAKE_PATH variable settings in the makeParaView file

  • # Set the path to the Qt-4.5 (or later) qmake if the system Qt is older
    QMAKE_PATH="/opt/exp_soft/gridit/OpenFOAM/ThirdParty-2.0.1/qt-everywhere-opensource-src-4.6.4/bin/"

  • # Set the path to cmake
    CMAKE_PATH="/opt/exp_soft/gridit/OpenFOAM/ThirdParty-2.0.1/cmake-2.8.6/bin/"

To install Paraview, execute the following:

$ ./makeParaView
$ ./makeParaView >log_paraview 2>log_paraview &

The PV3blockMeshReader and the PV3FoamReader ParaView plugins are compiled as usual for OpenFOAM utilities:

$ cd $FOAM_UTILITIES/postProcessing/graphics/PV3Readers
$ wmSET
$ ./Allwclean
$. /Allwmake 

Testing the Installation

To check your installation setup, execute the foamInstallationTest script (in the OpenFOAM-2.0.1/bin directory).
If no problems are reported, proceed to getting started with OpenFOAM; otherwise, go back and check you have installed the software correctly
and/or contact your system administrator.

Getting Started

Create a project directory within the $HOME/OpenFOAM directory named -2.0.1 and create a directory named run within it, e.g. by typing:

$ mkdir -p $HOME/OpenFOAM/OpenFOAM-2.0.1/$FOAM_RUN 

Copy the tutorial examples directory in the OpenFOAM distribution to the run directory. If the OpenFOAM environment variables are set correctly,
then the following command will be correct:

$ cp -r $FOAM_TUTORIALS $HOME/OpenFOAM/OpenFOAM-2.0.1/$FOAM_RUN 

Run the first example case of incompressible laminar flow in a cavity:

$ cd $FOAM_RUN/tutorials/incompressible/icoFoam/cavity
$ blockMesh
$ icoFoam
$ paraFoam 

Post-configuration on the LSF node

Add in the etc_bashrc file the following settings for sourcing the OpenFOAM bashrc profile

. /opt/exp_soft/gridit/OpenFOAM/OpenFOAM-2.0.1/etc/bashrc

Testing OpenFOAM in a Grid Infrastructure

This section describes a test submission of a serial OpenFOAM job on the GRIDIT Infrastructure.

JDL & script files

This is the JDL file used for testing:

[
     Type = "Job";
     JobType = "Normal";

     Executable = "/bin/bash";
     Arguments = "start_openfoam.sh";

     StdOutput = "log.out";
     StdError = "log.err";

     InputSandbox = {"start_openfoam.sh"};
     OutputSandbox = {"log.err","log.out","openfoam.log","openfoam.err"};
     Requirements = Member("VO-gridit-OpenFoam-2.0.1",other.GlueHostApplicationSoftwareRunTimeEnvironment);
     Rank = (other.GlueCEStateWaitingJobs == 0 ? other.GlueCEStateFreeCPUs : -other.GlueCEStateWaitingJobs);
]

This is the bash script sent in InputSandbox with the JDL file:

#!/bin/sh
echo "+ Running OpenFoam-2.0.1 on "`hostname -f` as `whoami`
echo;echo "+ Copying the OpenFoam example..."
cp -R $FOAM_TUTORIALS/incompressible/icoFoam/cavity $PWD
cd $PWD/cavity
echo; echo "+ Starting at "`date`

blockMesh
icoFoam >./openfoam.log 2>./openfoam.err
cp openfoam.* ../

# Testing the scratch area
cp openfoam.* $VO_GRIDIT_SW_DIR/scratch
chmod a+w $VO_GRIDIT_SW_DIR/scratch/openfoam.*
echo "+ Done at "`date`

Submission

$ glite-wms-job-submit -a -r grid012.ct.infn.it:2119/jobmanager-lcglsf-short 
                                        -e https://wmsdecide.dir.garr.it:7443/glite_wms_wmproxy_server 
                                        -o jobID 
                                         openfoam-2.0.1.jdl 

Connecting to the service https://wmsdecide.dir.garr.it:7443/glite_wms_wmproxy_server

===================== glite-wms-job-submit Success =====================

The job has been successfully submitted to the WMProxy
Your job identifier is:

https://lb-4.dir.garr.it:9000/vUAJk3Sq3xBnviahz7HSqg

The job identifier has been saved in the following file:
/home/larocca/OpenFoam-2.0.1/jobID
=====================================================================

Checking Status

$ glite-wms-job-status --noint -i jobID

====================== glite-wms-job-status Success ====================
BOOKKEEPING INFORMATION:

Status info for the Job : https://lb-4.dir.garr.it:9000/vUAJk3Sq3xBnviahz7HSqg
Current Status:     Done (Success)
Exit code:          0
Status Reason:      Job terminated successfully
Destination:        grid012.ct.infn.it:2119/jobmanager-lcglsf-short
Submitted:          Fri Dec  2 16:59:56 2011 CET
=====================================================================

Downloading Results


$ glite-wms-job-output --dir . --noint -i jobID

Connecting to the service https://wmsdecide.dir.garr.it:7443/glite_wms_wmproxy_server

===============================================================================
			JOB GET OUTPUT OUTCOME
Output sandbox files for the job:
https://lb-4.dir.garr.it:9000/vUAJk3Sq3xBnviahz7HSqg
have been successfully retrieved and stored in the directory:
/home/larocca/OpenFoam-2.0.1/larocca_vUAJk3Sq3xBnviahz7HSqg

================================================================================

References

OpenFOAM: The open source CFD toolbox
Topic attachments
I Attachment Action Size Date Who Comment
Unknown file formatlog openfoam.log manage 71.3 K 2011-12-02 - 16:14 GiuseppeLaRocca  
Edit | Attach | PDF | History: r12 | r4 < r3 < r2 < r1 | Backlinks | Raw View | More topic actions...
Topic revision: r2 - 2011-12-02 - GiuseppeLaRocca
 
  • Edit
  • Attach
This site is powered by the TWiki collaboration platformCopyright © 2008-2022 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback