diff options
| author | brodrig <brodrig@17b73243-c579-4c4c-a9d2-2d5706c11dac> | 2009-01-28 17:00:42 +0000 |
|---|---|---|
| committer | brodrig <brodrig@17b73243-c579-4c4c-a9d2-2d5706c11dac> | 2009-01-28 17:00:42 +0000 |
| commit | faada84cf965ade288cf3c3d0701339087dd94b0 (patch) | |
| tree | 8b318e276521d64cabfc033a03b292fa3c514062 /doc | |
| parent | dfb02e6849a271318ae410b66509e279f1486e6c (diff) | |
Proofreading, fixing missing commas, grammar, and typos in preface, runningCactus, and partially to ThornWriters
git-svn-id: http://svn.cactuscode.org/flesh/trunk@4528 17b73243-c579-4c4c-a9d2-2d5706c11dac
Diffstat (limited to 'doc')
| -rw-r--r-- | doc/UsersGuide/Preface.tex | 2 | ||||
| -rw-r--r-- | doc/UsersGuide/RunningCactus.tex | 361 | ||||
| -rw-r--r-- | doc/UsersGuide/ThornWriters.tex | 721 |
3 files changed, 531 insertions, 553 deletions
diff --git a/doc/UsersGuide/Preface.tex b/doc/UsersGuide/Preface.tex index aaef6053..3090ea0c 100644 --- a/doc/UsersGuide/Preface.tex +++ b/doc/UsersGuide/Preface.tex @@ -110,7 +110,7 @@ Related topics are discussed in separate documents including: Please let us know of any errors or omissions in this guide, as well as suggestions for future editions. These can be reported via our bug tracking system at \url{http://www.cactuscode.org}, or via email to -\code{cactusmaint@cactuscode.org}. Alternatively, write to us at +\code{cactusmaint@cactuscode.org}. Alternatively, you can write to us at \vskip .5cm The Cactus Team\\ diff --git a/doc/UsersGuide/RunningCactus.tex b/doc/UsersGuide/RunningCactus.tex index 07bc107b..1c420c5e 100644 --- a/doc/UsersGuide/RunningCactus.tex +++ b/doc/UsersGuide/RunningCactus.tex @@ -21,7 +21,7 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -\section{Required software} +\section{Required Software} \label{sec:required_software} In general, Cactus \emph{requires} the following set of software to function @@ -30,13 +30,13 @@ in single processor mode. Please refer to the architecture section \begin{Lentry} \item[Perl5.0] Perl is used extensively during the Cactus thorn configuration phase. Perl is available for nearly all - operating systems known to man and can be obtained at - \url{http://www.perl.org} + operating systems known to man, and can be obtained at + \url{http://www.perl.org}. \item[GNU make] The make process works with the GNU make utility (referred to as \texttt{gmake} henceforth). While other make utilities may also work, this is not - guaranteed. Gmake can be obtained from your favorite GNU site or - from \url{http://www.gnu.org} + guaranteed. Gmake can be obtained from your favorite GNU site, or + from \url{http://www.gnu.org}. \item[C] C compiler. For example, the GNU compiler. This is available for most supported platforms. Platform specific compilers should also work. @@ -44,7 +44,7 @@ in single processor mode. Please refer to the architecture section normally provided on most platforms, and many C compilers have an option to just run as a preprocessor. \item[CVS] The \textit{Concurrent Versions System} is not needed - to run/compile Cactus, but you are strongly encourage to install + to run/compile Cactus, but you are strongly encouraged to install this software to take advantage of the update procedures. It can be downloaded from your favorite GNU site. Tar files of each release are also available. @@ -54,10 +54,9 @@ in single processor mode. Please refer to the architecture section To use Cactus, with the default driver\footnote{For help with unfamiliar terms, please consult the glossary, Appendix \ref{sec:glossary}.} (\texttt{CactusPUGH/PUGH}) on multiple processors you also need: \begin{Lentry} -\item[MPI] The \textit{Message Passing Interface} +\item[MPI] The \textit{Message Passing Interface}, which provides inter-processor communication. -Supercomputing sites often supply a native MPI implementation with -which Cactus is very likely to be compatible. Otherwise there are +Supercomputing sites often supply a native MPI implementation that is very likely to be compatible with Cactus . Otherwise, there are various freely available ones available, e.g. the MPICH version of MPI is available for various architectures and operating systems at \url{http://www-unix.mcs.anl.gov/mpi/}. @@ -69,8 +68,8 @@ written in C++ you also need \begin{Lentry} \item[C++] C++ compiler. For example, the GNU compiler. This is available for most supported platforms. Platform specific compilers - should also work. Note that if a C++ compiler is available then the - \text{main()} routine in the Flesh is compiled with C++ to allow static + should also work. Note that if a C++ compiler is available, then the + \text{main()} routine in the flesh is compiled with C++ to allow static class initialisations. \end{Lentry} @@ -78,8 +77,8 @@ written in C++ you also need If you are using any thorns containing routines written in Fortran you also need \begin{Lentry} -\item[F90/F77] For routines written in Fortran 77, either an Fortran 90 or - a Fortran 77 compiler can be used. For routines written in Fortran 90 +\item[F90/F77] For routines written in Fortran 77, either a Fortran 90 or + a Fortran 77 compiler can be used. For routines written in Fortran 90, a Fortran 90 compiler is obviously required. There is a very limited set of free Fortran 90 compilers available for the different architectures. \end{Lentry} @@ -90,18 +89,18 @@ it is useful to install \begin{Lentry} \item[\texttt{ctags/etags}] These programs enable you browse through the calling structure of a program by help of a function call database. - Navigating the Flesh and arrangements becomes very easy. Emacs and + Navigating the flesh and arrangements becomes very easy. Emacs and \texttt{vi} both support this method. See \ref{sec:Appendix.tags} for a short guide to tags. \end{Lentry} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -\section{Supported architectures} +\section{Supported Architectures} \label{sec:suar} Cactus runs on many machines, under a large number of operating -systems. Here we list the machines we have compiled and verified +systems. Here, we list the machines we have compiled and verified Cactus on, including some architecture specific notes. A complete list of architectures supported by Cactus, along with more notes, can be found at @@ -118,7 +117,7 @@ be found at compilers installed. \item[\textbf{IA32}] running Linux, OpenBSD, FreeBSD, or Windows 2000/NT. Single processor mode and MPI (MPICH and LAM) supported.\\ - On Windows Cactus compiles with Cygwin. MPI (WMPI, HPVM, and MPIPro) + On Windows, Cactus compiles with Cygwin. MPI (WMPI, HPVM, and MPIPro) supported. Please read \texttt{doc/README.NT} for details. \item[\textbf{IA64}] running Linux. \item[\textbf{Macintosh PowerPC}] (MacOS X and Linux PPC) @@ -129,14 +128,14 @@ be found at \item[\textbf{NEC SX-5, SX-6}] \end{Lentry} -The following machines are only partially supported +The following machines are only partially supported, \begin{Lentry} \item[\textbf{HP Exemplar}] \end{Lentry} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -\section{Checkout procedure} +\section{Checkout Procedure} \label{sec:checkout} Cactus is distributed, extended, and maintained using the free CVS @@ -145,20 +144,22 @@ CVS allows many people to work on a large software project together without getting into a tangle. Since Cactus thorns are distributed from several repositories on the main CVS site, and from a growing number of user sites, we provide a -script, described below, on our web site for checking out the Flesh and thorns. -The Cactus web site also provides a form interface for direct download. +script, described below, on our website for checking out the flesh and thorns. +The Cactus website also provides a form interface for direct download. CVS experts who want to use raw CVS commands are directed to Appendix~\ref{sec:Appendix.cvs} for full instructions. For CVS novices, -we also summarize in this appendix basic CVS commands. +we also summarize in the appendix basic CVS commands. The space required for an installation depends on the arrangements and -thorns used. The Flesh on its own requires less than 5 MB. +thorns used. The flesh on its own requires less than 5 MB. -The script for checking out the Flesh and distribution thorns, -\texttt{GetCactus}, is available from the web site at +The script for checking out the flesh and distribution thorns, +\texttt{GetCactus}, is available from the website at -\url{http://www.cactuscode.org/download/GetCactus} +\begin{center} +\url{http://www.cactuscode.org/download/GetCactus}. +\end{center} The script takes as an argument the name of a file containing a \textit{ThornList}, @@ -168,24 +169,24 @@ that is a list of thorns with the syntax <\var{arrangement name}>/<\var{thorn name}> \end{alltt} -If no filename is given, only the Flesh is checked out. +If no filename is given, only the flesh is checked out. Optional directives in the ThornList indicate which CVS repository to fetch thorns from. The default is to take the thorns from the same repository as -the Flesh. A full description of ThornList syntax is provided in Appendix~\ref{chap:th}. -ThornLists for example applications are provided on the Cactus web site. +the flesh. A full description of ThornList syntax is provided in Appendix~\ref{chap:th}. +ThornLists example applications are provided on the Cactus website. The same script can be used to checkout additional thorns. Another script, \texttt{MakeThornList}, can be used to produce a minimal ThornList from a given Cactus par file. It needs a \emph{master} ThornList -to be copied into your \texttt{~\.cactus} directory. +to be copied into your \texttt{Cactus} directory. See \url{http://www.cactuscode.org/toolkit/makeThornList/}. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -\section{Directory structure} +\section{Directory Structure} \label{sec:dist} A fresh checkout creates a directory \texttt{Cactus} with the @@ -202,13 +203,13 @@ following subdirectories: \item[\texttt{src}] contains the source code for Cactus \item [\texttt{arrangements}] contains the Cactus arrangements. The arrangements - (the actual ``physics'') are not supplied by checking out just Cactus. + (the actual ``physics'') are not supplied by just checking out Cactus. If the arrangements you want to use are standard Cactus arrangements, or reside on our CVS repository (\texttt{cvs.cactuscode.org}), - they can be checked out in similar way to the Flesh. + they can be checked out in similar way to the flesh. \end{Lentry} -When Cactus is first compiled it creates a new directory +When Cactus is first compiled, it creates a new directory \texttt{Cactus/configs}, which will contain all the source code, object files and libraries created during the build process. Disk space may be a problem on supercomputers where home directories are small. @@ -227,19 +228,19 @@ scratch/cactus\_configs Cactus/configs/}) to the Cactus directory, if your filesystem supports soft links. \end{itemize} -Configurations are described in detail in section \ref{sec:configurations}. +Configurations are described in detail in Section \ref{sec:configurations}. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{Getting help} \label{sec:gehe} -For tracking problem reports and bugs we use GNATS, which is a bug tracking +For tracking problem reports and bugs, we use GNATS, which is a bug tracking system published under the GNU license. We have set up a web interface at -\url{http://www.cactuscode.org} which allows easy submission and browsing +\url{http://www.cactuscode.org}, which allows easy submission and browsing of problem reports. -A description of the GNATS categories which we use is provided in the appendix +A description of the GNATS categories we use is provided in appendix \ref{sec:Appendix.gnats}. % OK, there is NO emacs at the moment, because the GNATS setup is really stupid @@ -271,26 +272,26 @@ the source files, and these different configurations coexist in the architectures}. You can keep executables for multiple architectures based on a single copy of source code, shared on a common file system. -\item{} You can compare different \textit{compiler options, debug-modes}. +\item{} You can compare different \textit{compiler options}, and \texit{debug-modes}. You might want to compile different communication protocols - (e.g. MPI or Globus) or leave them out all together. + (e.g. MPI or Globus), or leave them out all together. \item{} You can have different configurations for \textit{different thorn collections} compiled into your executable. \end{enumerate} -Once a configuration has been created, by \texttt{gmake <\var{config}>} as +Once a configuration has been created by \texttt{gmake <\var{config}>}, as described in detail in the next section, a single call to -\texttt{gmake <\var{config}>} will compile the code. The first time it +\texttt{gmake <\var{config}>} will compile the code. The first time, it generates a compile \texttt{ThornList}, and gives you the chance to edit it before continuing. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -\section{Creating a configuration} +\section{Creating a Configuration} \label{sec:configurations} At its simplest, this is done by \texttt{gmake <\var{config}>}\footnote % -{A note on the Cactus make system --- if at any point it prompts you +{A note on the Cactus make system---if at any point it prompts you to enter something, the default value, which will be assumed if you simply press enter, is shown in parentheses.} % @@ -302,9 +303,9 @@ suitable for the current architecture. There are a number of additional command line arguments which may be supplied to override some parts of the procedure. -\subsection{Configuration options} +\subsection{Configuration Options} -There are three ways to pass options to the configuration process. +There are four ways to pass options to the configuration process. % from the gmake command line. \begin{enumerate} \item[1]{} @@ -318,7 +319,7 @@ There are three ways to pass options to the configuration process. Either: create a default configuration file \texttt{\$\{HOME\}/.cactus/config}. All available configuration options may be set in a default options file - \texttt{\$\{HOME\}/.cactus/config}, any which are not set will take a + \texttt{\$\{HOME\}/.cactus/config}, any option which is not set will take a default value. The file should contain lines of the form: \texttt{<\var{option}> [=] ...} @@ -344,7 +345,7 @@ There are three ways to pass options to the configuration process. \texttt{gmake <\var{config name}>-config options=<\var{filename}>} The options file has the same format as \texttt{\$\{HOME\}/.cactus/config}. - (Note these options are \emph{added} to those from the + (Note that these options are \emph{added} to those from the \texttt{\$\{HOME\}/.cactus/config} file.) \item[4]{} @@ -376,7 +377,7 @@ There is a plethora of available options. \begin{itemize} -\item {Cross Compiling} +\item {Cross compiling} If you are compiling on an architecture other than the one you are producing an executable for, you will need to pass the @@ -388,17 +389,17 @@ option, where \texttt{\var{x-x-x}} is the canonical name of the architecture you are compiling for, such as \texttt{sx6-nec-superux}; the format is \texttt{\var{processor}-\var{vendor}-\var{OS}}. -\item {Compiled Thorns} +\item {Compiled thorns} These specify the chosen set of thorns for compilation. If the thorn choice is not provided during configuration, a list containing all thorns in the \texttt{arrangements} directory -is automatically created, and the users prompted for any changes. +is automatically created, and the user is prompted for any changes. \begin{Lentry} \item [\texttt{THORNLIST}] Name of file containing a list of thorns with -the syntax \texttt{<\var{arrangement name}>/<\var{thorn name}>}, lines +the syntax \texttt{<\var{arrangement name}>/<\var{thorn name}>}. Lines beginning with \verb|#| or \texttt{!} are ignored. \item [\texttt{THORNLIST\_DIR}] Location of directory containing @@ -455,7 +456,7 @@ Flags for the preprocessor (used to generate compilation dependencies for and preprocess Fortran code). \item [\texttt{MKDIRFLAGS}] - Flags for \texttt{MKDIR} so that no error is given if the directory exists. + Flags for \texttt{MKDIR}, so that no error is given if the directory exists. \item [\texttt{LDFLAGS}] * Flags for the linker. \emph{Warning:} This variable is ignored @@ -482,7 +483,7 @@ compiler reports error messages about unrecognised \verb|#| directives. \item [\texttt{CROSS\_COMPILE}] Enables cross compilation. Available options are \texttt{yes} and \texttt{no}, the default is - \texttt{no}. To create a cross compiled configuration one must explicitly set this option to \texttt{yes}. + \texttt{no}. To create a cross-compiled configuration one must explicitly set this option to \texttt{yes}. \item [\texttt{DISABLE\_REAL16}] Disable support for the data type \texttt{CCTK\_REAL16}. The only options available are \texttt{yes} and @@ -509,7 +510,7 @@ debugging being used. \item [\texttt{F90\_DEBUG\_FLAGS}] Debug flags for the Fortran 90 compiler, their use depends on the -type of debuggin being used. +type of debugging being used. \item [\texttt{F77\_DEBUG\_FLAGS}] Debug flags for the Fortran 77 compiler, their use depends on the @@ -573,7 +574,7 @@ type of optimisation being used. \item [\texttt{F90\_PROFILE\_FLAGS}] Profile flags for the Fortran 90 compiler, their use depends on the - type of profilin being used. + type of profiling being used. \item [\texttt{F77\_PROFILE\_FLAGS}] Profile flags for the Fortran 77 compiler, their use depends on the @@ -620,7 +621,7 @@ Used to specify auxiliary libraries and directories to find them in. \item [\texttt{LIBS}] Additional libraries. This variable can also contain linker options, e.g.\ to switch between -static and dynamic linking. (Cactus adds an \verb+-l+ prefix to +static and dynamic linking. (Cactus adds a \verb+-l+ prefix to library names, but does not modify linker options.) \emph{Warning:} This variable is ignored while the compilers and linkers are autodetected. This can lead to strange @@ -628,7 +629,7 @@ errors while configuring. You can pass the additional libraries in the variable \texttt{LD} instead. \item [\texttt{LIBDIRS}] Any other library directories. -This variable can also contain linker options. (Cactus adds an +This variable can also contain linker options. (Cactus adds a \verb+-L+ prefix to library directories, but does not modify linker options.) \end{Lentry} @@ -643,8 +644,7 @@ Used to specify any additional directories for system include files. \item {Precision options} -Used to specify the precision of the default real and integer data types, -specified as the number of bytes the data takes up. Note that not all +Used to specify the precision of the default real and integer data types, by the number of bytes the data takes up. Note that not all values will be valid on all architectures. \begin{Lentry} @@ -672,19 +672,19 @@ which should be consulted for the full range of configuration options. \begin{Lentry} \item [\texttt{MPI}] * The MPI package to use, if required. Supported values are - \texttt{CUSTOM}, \texttt{NATIVE}, \texttt{MPICH} or \texttt{LAM}. + \texttt{CUSTOM}, \texttt{NATIVE}, \texttt{MPICH}, or \texttt{LAM}. \item [\texttt{HDF5}] -Supported values are \texttt{yes, no}. A blank value is taken as \texttt{no}. +Supported values are \texttt{yes}, and \texttt{no}. A blank value is taken as \texttt{no}. \item [\texttt{LAPACK}] -Supported values are \texttt{yes, no}. A blank value is taken as \texttt{no}. +Supported values are \texttt{yes}, and \texttt{no}. A blank value is taken as \texttt{no}. \item [\texttt{PETSC}] -Supported values are \texttt{yes, no}. A blank value is taken as \texttt{no}. +Supported values are \texttt{yes}, and \texttt{no}. A blank value is taken as \texttt{no}. \item [\texttt{PTHREADS}] -Supported values are \texttt{yes}. +Supported values are \texttt{yes}, and \texttt{no}. A blank value is taken as \texttt{no}. \end{Lentry} @@ -703,7 +703,7 @@ commands that it is executing. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -\subsection{Compiling with extra packages} +\subsection{Compiling with Extra Packages} \label{subsec:cowiexpa} @@ -717,7 +717,7 @@ such as MPICH, LAM, WMPI, or PACX. To compile with MPI, the configure option is -\texttt{MPI = <\var{MPI\_TYPE}>} +\texttt{MPI = <\var{MPI\_TYPE}>}, where \texttt{<\var{MPI\_TYPE}>} can take the values (entries followed by * may be specified on the configuration command line): @@ -741,13 +741,13 @@ by the options \begin{Lentry} \item [\texttt{MPICH\_ARCH}] * machine architecture. \item [\texttt{MPICH\_DIR} ] * directory in which MPICH is installed. - If this option is not defined it will be searched for. + If this option is not defined, it will be searched for. \item [\texttt{MPICH\_DEVICE}] * the device used by MPICH. If not defined, the configuration process will search for this in a few defined places. Supported devices are currently \texttt{ch\_p4}, \texttt{ch\_shmem}, \texttt{globus} and \texttt{myrinet}. - For versions of MPICH prior to 1.2.0 the devices are searched for + For versions of MPICH prior to 1.2.0, the devices are searched for in this order, for 1.2.0 you may need to specify \texttt{MPICH\_DEVICE}, depending on the installation. \end{Lentry} @@ -759,7 +759,7 @@ If \texttt{MPICH\_DEVICE} is chosen to be \texttt{globus} \end{Lentry} The Globus flavor may be chosen optionally \begin{Lentry} - \item[\texttt{GLOBUS\_FLAVOR}] * Globus flavor to build Cactus with + \item[\texttt{GLOBUS\_FLAVOR}] * Globus flavor to build Cactus with. \end{Lentry} If it is not set, the first Globus flavor found will be used. @@ -776,7 +776,7 @@ This is controlled by the variables \item[\texttt{LAM\_DIR} ] * directory in which LAM is installed. This will be searched for in a few provided places if not given. \end{Lentry} -if the \texttt{LAM} installation splits libraries and include files into different +If the \texttt{LAM} installation splits libraries and include files into different directories, instead of setting \texttt{LAM\_DIR} set the two variables \begin{Lentry} \item[\texttt{LAM\_LIB\_DIR}] * directory in which LAM libraries are installed. @@ -806,14 +806,14 @@ Use the PACX Metacomputing package (\textit{PArallel Computer eXtension},\\ \url{http://www.hlrs.de/structure/organisation/par/projects/pacx-mpi/}). This is controlled by the variables \begin{Lentry} \item[\texttt{PACX\_DIR}] * directory in which PACX is installed. - If this option is not defined it will be searched for. - \item[\texttt{PACX\_MPI}] * the MPI package PACX uses for node-local + If this option is not defined, it will be searched for. + \item[\texttt{PACX\_MPI}] * the MPI package PACX uses node-local communication. This can be any of the above MPI packages. \end{Lentry} \end{Lentry} -Note that the searches for libraries etc. mentioned above use the +Note that the searches for libraries, etc., mentioned above, use the locations given in the files in \texttt{lib/make/extras/MPI}. \subsubsection{HDF5: Hierarchical Data Format version 5} @@ -850,15 +850,15 @@ LAPACK = yes | no | <blank> [ LAPACK\_EXTRA\_LIBS = <\var{libs}> ] \end{alltt} -If \texttt{LAPACK\_DIR} is not given the configuration process will search for a +If \texttt{LAPACK\_DIR} is not given, the configuration process will search for a LAPACK library \texttt{liblapack.[\{a,so\}]} in some standard places (defined in -\texttt{lib/make/extras/LAPACK}). If \texttt{LAPACK\_DIR} is set to \texttt{no} +\texttt{lib/make/extras/LAPACK}). If \texttt{LAPACK\_DIR} is set to \texttt{no}, the LAPACK library path is assumed to be installed in a standard system location -(e.g. \texttt{/usr/lib/}) and thus the library path will not be added to the +(e.g. \texttt{/usr/lib/}), and thus the library path will not be added to the linker's command line. Because LAPACK doesn't come as a standardized system installation, there are -additional configuration variables to set the name of the lapack library +additional configuration variables to set the name of the LAPACK library (\texttt{LAPACK\_LIBS}) as well as the name (\texttt{LAPACK\_EXTRA\_LIBS}) and location (\texttt{LAPACK\_EXTRA\_LIBS\_DIRS}) of extra libraries that are required by LAPACK itself. @@ -877,33 +877,33 @@ PETSC = yes | no | <blank> [ PETSC\_ARCH\_LIBS = <\var{architecture-specific libraries}> ] \end{alltt} -If \texttt{PETSC\_DIR} is not given the configuration process will search for an +If \texttt{PETSC\_DIR} is not given, the configuration process will search for an installed PETSc package in some standard places (defined in \texttt{lib/make/extras/PETSC}). -If \texttt{PETSC\_ARCH} is not given the configuration process will choose the +If \texttt{PETSC\_ARCH} is not given, the configuration process will choose the first PETSc configuration found in \texttt{\$PETSC\_DIR/lib/libO/}. -If \texttt{PETSC\_ARCH\_LIBS} is not given the configuration process will choose -architecture-specific libraries as required by a PETSc configuration (usually +If \texttt{PETSC\_ARCH\_LIBS} is not given, the configuration process will choose +architecture-specific libraries, as required by a PETSc configuration (usually PETSc needs the LAPACK library). \subsubsection{Pthreads: POSIX threads} -To enable multithreading support within Cactus using POSIX threads +To enable multithreading support within Cactus using POSIX threads, the configure option is \texttt{PTHREADS = yes} -The configuration process will check if a reentrant C library is available +The configuration process will check if a reentrant C library is available, and adds it to the linker flags. It will also search for the system's Pthreads -library (either \texttt{libpthread} or \texttt{libpthreads}) and set +library (either \texttt{libpthread} or \texttt{libpthreads}), and set preprocessor defines necessary for compiling multithreaded code. -\subsection{File layout} +\subsection{File Layout} The configuration process sets up various subdirectories and files in the -\texttt{configs} directory to contain the configuration specific files, these +\texttt{configs} directory to contain the configuration specific files; these are placed in a directory with the name of the configuration. \begin{Lentry} @@ -933,7 +933,7 @@ automatically detected, and have thus been hand-coded for this architecture. These are the first files which should be checked or modified to suit any peculiarities of this configuration. -In addition the following files may be informative: +In addition, the following files may be informative: \begin{Lentry} \item [\texttt{fortran\_name.pl}] @@ -942,7 +942,7 @@ This is used to make some C routines callable from Fortran, and Fortran routines callable from C. \item [\texttt{make.config.deps}] -Initially empty. Can be edited to add extra architecture specific dependencies +Initially empty. It can be edited to add extra architecture specific dependencies needed to generate the executable. \item [\texttt{make.config.rule}] @@ -984,7 +984,7 @@ A scratch directory which is used to accommodate Fortran 90 modules. \end{Lentry} -\section{Building and Administering a configuration} +\section{Building and Administering a Configuration} \label{sec:buanadaco} Once you have created a new configuration, the command @@ -996,18 +996,17 @@ thorns which should be included. There is a range of \texttt{gmake} targets and options which are detailed in the following sections. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -\subsection{gmake targets for building and administering configurations} +\subsection{gmake Targets for Building and Administering Configurations} \label{sec:gmtafobuanadco} A target for \texttt{gmake} can be naively thought of as an argument -that tells it which of several things listed in the \texttt{Makefile} it -is to do. The command \texttt{gmake help} lists all \texttt{gmake} targets: +that tells it which of several things listed in the \texttt{Makefile} is to do. The command \texttt{gmake help} lists all \texttt{gmake} targets: % colon clarifies that all (config) targets are listed here \begin{Lentry} \item [\texttt{gmake <\var{config}>}] - builds a configuration. If the configuration doesn't exist + builds a configuration. If the configuration doesn't exist, it will create it. \item [\texttt{gmake <\var{config}>-clean}] removes all object and dependency @@ -1027,13 +1026,13 @@ The configuration options are stored in a file \item [\texttt{gmake <\var{config}>-configinfo}] displays the options of the configuration (\texttt{cat configs/<\var{config}>/config-info}). -\item[\texttt{gmake <\var{config}>-cvsupdate}] updates the Flesh and this +\item[\texttt{gmake <\var{config}>-cvsupdate}] updates the flesh and this configuration's thorns from the CVS repositories. \item [\texttt{gmake <\var{config}>-delete}] deletes a configuration (\texttt{rm -r configs/<\var{config}>}). -\item [\texttt{gmake <config>-editthorns}] edits the ThornList. +\item [\texttt{gmake <config>-editthorns}] edits the \texttt{ThornList}. \item [\texttt{gmake <\var{config}>-examples}] copies all the example parameter files relevant for this configuration to the directory \texttt{examples} in the Cactus home directory. If a file of the same name is already there, it will not overwrite it. @@ -1050,13 +1049,13 @@ configuration using its previous configuration options from the file \texttt{configs/<\var{config}>/config-info}. \item [\texttt{gmake <\var{config}>--testsuite}] runs the test programs -associated with each thorn in the configuration. See section +associated with each thorn in the configuration. See Section \ref{sec:testing} for information about the test suite mechanism. \item[\texttt{gmake <\var{config}>-ThornGuide}] builds documentation for the thorns in this configuration - (see section \ref{sec:OtherGmakeTargetsDoc} on page - \pageref{sec:OtherGmakeTargetsDoc} for other targets to build documentation + (see Section \ref{sec:OtherGmakeTargetsDoc}, page + \pageref{sec:OtherGmakeTargetsDoc}, for other targets to build documentation for thorns). \item [\texttt{gmake <\var{config}>-thornlist}] regenerates the @@ -1070,51 +1069,49 @@ utilities can be selected by giving their names in the \texttt{UTILS} variable. -\subsection{Compiling in thorns} +\subsection{Compiling in Thorns} \label{sec:cointh} Cactus will try to compile all thorns listed in \texttt{configs/<\var{config}>/ThornList}. The \texttt{ThornList} file is simply a list of the form \texttt{<\var{arrangement}>/<\var{thorn}>}. All text after a pound sign -`\texttt{\#}' or exclamation mark `\texttt{!}' -on a line is treated as a comment and ignored. -The first time that you compile a configuration, if you did -not specify a ThornList already during configuration, -you will be shown a list of all the thorns in your arrangement -directory, and asked if you with to edit them. You can regenerate +`\texttt{\#}', or exclamation mark `\texttt{!}' +on a line is treated as a comment and is ignored. If you did +not specify a ThornList already during configuration the first time that you compile a configuration, you will be shown a list of all the thorns in your arrangement +directory, and asked if you want to edit them. You can regenerate this list at anytime by typing \begin{alltt} gmake <\var{config}>-thornlist -\end{alltt} +\end{alltt}, or you can edit it using \begin{alltt} gmake <\var{config}>-editthorns -\end{alltt} +\end{alltt}. Instead of using the editor to specify the thorns you want to have compiled, you can \emph{edit} the \texttt{ThornList} outside the make process. It is located in \texttt{configs/<\var{config}>/ThornList}, where \texttt{<\var{config}>} refers to the name of your configuration. - The directory, \texttt{./configs}, exists \emph{ after} the very first + The directory \texttt{./configs} exists \emph{after} the very first make phase for the first configuration. \subsection{Notes and Caveats} \begin{itemize} \item{} If during the build you see the error ``\texttt{missing - separator}'' you are probably not using GNU make. + separator}'', you are probably not using GNU make. \item{} \textit{The EDITOR environment variable}. You may not be aware of - this, but this thing very often exists and may be set by default to - something scary like \texttt{vi}. If you don't know how to use \texttt{vi} + this, but this variable usually exists, and may be set by default to + something scary like \texttt{vi}. If you don't know how to use \texttt{vi}, or wish to use your favorite editor instead, reset this environment variable. (To exit \texttt{vi} type \texttt{<ESC> :q!}) \end{itemize} -\subsection{\texttt{gmake} options for building configurations} +\subsection{\texttt{gmake} Options for Building Configurations} \label{sec:gmopfobuco} An \textit{option} for \texttt{gmake} can be thought of as an argument which tells @@ -1124,13 +1121,13 @@ the same. \begin{Lentry} \item [\texttt{gmake <\var{target}> PROMPT=no}] turns off all prompts from the make system. -\item [\texttt{gmake <\var{target}> SILENT=no}] print the commands that gmake +\item [\texttt{gmake <\var{target}> SILENT=no}] prints the commands that gmake is executing. -\item [\texttt{gmake <\var{target}> WARN=yes}] show compiler warnings during +\item [\texttt{gmake <\var{target}> WARN=yes}] shows compiler warnings during compilation. -\item [\texttt{gmake <\var{target}> FJOBS=<\var{number}>}] compile in parallel, +\item [\texttt{gmake <\var{target}> FJOBS=<\var{number}>}] compiles in parallel, across files within each thorn. -\item [\texttt{gmake <\var{target}> TJOBS=<\var{number}>}] compile in parallel, +\item [\texttt{gmake <\var{target}> TJOBS=<\var{number}>}] compiles in parallel, across thorns. \end{Lentry} @@ -1144,29 +1141,29 @@ normal \texttt{-j <\var{number}>} flag to gmake to get parallel compilation. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -\section{Other gmake targets} +\section{Other gmake Targets} \begin{Lentry} \item [\texttt{gmake help}] lists all make options. \item [\texttt{gmake checkout}] allows you to easily checkout Cactus -arrangements and thorns. For example it can checkout all the thorns +arrangements and thorns. For example, it can checkout all the thorns in any thornlist file found in the \texttt{thornlists} subdirectory of the Cactus root directory. % (usually \texttt{Cactus}). -\item [\texttt{gmake cvsdiff}] shows differences between checked out version of Cactus and that in the CVS repositories. +\item [\texttt{gmake cvsdiff}] shows differences between the checked out version of Cactus and that in the CVS repositories. -\item [\texttt{gmake cvsstatus}] shows status of checked out version of Cactus, reporting which files have been modified or need updating. +\item [\texttt{gmake cvsstatus}] shows status of the checked out version of Cactus, reporting which files have been modified or need updating. -\item [\texttt{gmake cvsupdate}] updates Flesh and all thorns from CVS repositories. +\item [\texttt{gmake cvsupdate}] updates flesh and all thorns from CVS repositories. \item [\texttt{gmake configinfo}] prints configuration options for every configuration found in user's \texttt{configs} subdirectory. \item [\texttt{gmake default}] creates a new configuration with a default name. -\item [\texttt{gmake distclean}] deletes your \texttt{configs} directory and hence all your configurations. +\item [\texttt{gmake distclean}] deletes your \texttt{configs} directory, and hence all your configurations. \item [\texttt{gmake downsize}] removes non-essential files as documents and test suites to allow for minimal installation size. @@ -1174,10 +1171,10 @@ configuration found in user's \texttt{configs} subdirectory. \item [\texttt{gmake newthorn}] creates a new thorn, prompting for the necessary information and creating template files. -\item [\texttt{gmake TAGS}] creates an Emacs style TAGS file. See section +\item [\texttt{gmake TAGS}] creates an Emacs style TAGS file. See Section \ref{sec:Appendix.tags} for using tags within Cactus. -\item [\texttt{gmake tags}] creates a \texttt{vi} style tags file. See section +\item [\texttt{gmake tags}] creates a \texttt{vi} style tags file. See Section \ref{sec:Appendix.tags} for using tags within Cactus. \end{Lentry} @@ -1228,10 +1225,10 @@ These test suite serve the dual purpose of \begin{Lentry} \item [Regression testing] -i.e. making sure that changes to the thorn or the Flesh don't affect the +i.e. making sure that changes to the thorn or the flesh don't affect the output from a known parameter file. \item [Portability testing] -i.e. checking that the results are independent of the architecture --- this +i.e. checking that the results are independent of the architecture---this is also of use when trying to get Cactus to work on a new architecture. \end{Lentry} @@ -1240,7 +1237,7 @@ is also of use when trying to get Cactus to work on a new architecture. \chapter{Running Cactus} -Cactus executables always run from a parameter file (which may be a +Cactus executables always run from a parameter file (that may be a specified as a command line argument taken from standard input), which specifies which thorns to use and sets the values of any parameters which are different @@ -1254,17 +1251,16 @@ a parameter file is then \texttt{./cactus\_<\var{config}> <\var{parameter file}> [\var{command line options}]} -or if the parameter file should be taken from standard input by appending -a dash (``\texttt{-}'') on the command line like this: +or if the parameter file should be taken from standard input, the syntax includes a dash (``\texttt{-}'') on the command line like this: \texttt{./cactus\_<\var{config}> [\var{command line options}] -} The remainder of this chapter covers all aspects for running your -Cactus executable. These include: command line options, parameter +Cactus executable. These include: command-line options, parameter file syntax, understanding screen output, environment variables, and creating thorn documentation. -\section{Command Line Options} +\section{Command-Line Options} \label{sec:command_line_options} Cactus uses the standard GNU style of long-named command-line options; @@ -1273,21 +1269,20 @@ The options follow the usual GNU rules: \begin{itemize} \item A long-named option \verb|--foo| which takes an argument \verb|bar| may be written as either \verb|--foo bar| or as \verb|--foo=bar|. -\item A long-named option may be abbreviated so long as the abbreviation +\item A long-named option may be abbreviated, so long as the abbreviation is unambiguous. \item The preferred way of spelling a long-named option is \verb|--foo|, - but \verb|-foo| also accepted, though this is deprecated - (we plan to remove it after the next beta release). -\item A short option \verb|-X| which takes an argument \verb|bar| + but \verb|-foo| also accepted, though this is deprecated. +\item A short option, \verb|-X|, which takes an argument \verb|bar|, may be written as either \verb|-Xbar| or as \verb|-X=bar|. \item An option which can be interpreted as either a short option, or as an abbreviated \verb|-foo|-style long option, is interpreted - as the former. In particularl, \verb|-re| is interpreted as + as the former. In particular, \verb|-re| is interpreted as an abbreviation for \verb|-redirect|, rather than as \verb|-r=e|. \end{itemize} -The Cactus command line options are specified in -table~\ref{tab:command-line-options}, and are as follows: +The Cactus command-line options are specified in +Table~\ref{tab:command-line-options}, and are as follows: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \begin{table} @@ -1346,7 +1341,7 @@ parameter \texttt{v} (i.e. \texttt{-Ov} to give verbose information about all parameters). \item [\texttt{-o<\var{param}>} or \texttt{--describe-parameter=<\var{param}>}] -Prints the description and allowed values for a given parameter --- takes one +Prints the description and allowed values for a given parameter---takes one argument. \item [\texttt{-S} or \texttt{--print-schedule}] @@ -1356,7 +1351,7 @@ Print only the schedule tree. Prints a list of all the thorns which were compiled in. \item [\texttt{-t<\var{arrangement or thorn}>} or \texttt{--test-thorn-compiled=<\var{arrangement or thorn>}} ] -Checks if a given thorn was compiled in --- takes one argument. +Checks if a given thorn was compiled in---takes one argument. \item [\texttt{-h}, \texttt{-?} or \texttt{--help}] Prints a help message. @@ -1371,7 +1366,7 @@ Prints version information of the code. \item [\texttt{-L<\var{level}>} or \texttt{--logging-level=<\var{level}>}] Sets the logging level of the code. All warning messages are given a -level --- the lower the level the greater the severity. This +level---the lower the level, the greater the severity. This parameter \texttt{-L} controls the level of messages to be seen, with all warnings of level $\le$ \texttt{<\var{level}>} printed to standard output. The default is a logging level of~0, meaning that only level~0 messages @@ -1389,8 +1384,8 @@ Similar to \texttt{-W}, but for fatal errors: Cactus treats all warnings with level $\le$ \texttt{<\var{level}>} as fatal errors, and aborts the Cactus run immediately (after printing the warning message%%% \footnote{%%% - Cactus imposes the constraints that - $\hbox{the \texttt{-W} level} \ge \hbox{the \texttt{-E} level} \ge 0$, + Cactus imposes the constraint, + $\hbox{ \texttt{-W} level} \ge \hbox{ \texttt{-E} level} \ge 0$, so any fatal-error message will always be printed (first). }%%% ). The default value is zero, \ie{} only level~0 warnings @@ -1398,8 +1393,8 @@ will abort the Cactus run. \item [\texttt{-r[o|e|oe|eo]} or \texttt{--redirect=[o|e|oe|eo]}] Redirects the standard output (`\texttt{o}') and/or standard error -(`\texttt{e}') of each processor to a file. By default -the standard outputs from processors other than processor 0 are discarded. +(`\texttt{e}') of each processor to a file. By default, +the standard outputs from processors, other than processor 0, are discarded. \item [\texttt{--logdir=<\var{directory}>}] Sets the output directory for logfiles created by the \texttt{-r} option. @@ -1412,7 +1407,7 @@ If the directory doesn't exist yet, it will be created by Cactus. and is then written in large chunks. This improves performance considerably. Line buffering means that output is written whenever a newline character is encountered; full buffering means that output - is written once e.g.\ 1000 characters have accmulated. The default + is written, say, once 1000 characters have accumulated. The default setting is line buffering for I/O that goes to a terminal, and full buffering for I/O that goes to a file. For debugging purposes, it is sometimes useful to reduce the amount of buffering. Error @@ -1435,7 +1430,7 @@ Causes the next argument on the command line to be ignored. \section{Parameter File Syntax} \label{sec:Parameter_File} -A \textit{parameter file} ( or \textit{par file}) is used to control the +A \textit{parameter file} (or \textit{par file}) is used to control the behaviour of a Cactus executable. It specifies initial values for parameters as defined in the various thorns' \texttt{param.ccl} files (see Chapter~\ref{chap:Cactus_parameters}). @@ -1455,27 +1450,27 @@ The first parameter statement in any parameter file should set \texttt{ActiveTho which is a special parameter that tells the program which \textit{thorns} are to be activated. Only parameters from active thorns can be set (and only those routines \textit{scheduled} by active thorns -are run). By default all thorns are inactive. For example, the first +are run). By default, all thorns are inactive. For example, the first entry in a parameter file which is using just the two thorns -\texttt{CactusPUGH/PUGH} and \texttt{CactusBase/CartGrid3D} should be +\texttt{CactusPUGH/PUGH} and \texttt{CactusBase/CartGrid3D}, should be \texttt{ActiveThorns = "PUGH CartGrid3D"} -Parameters following the \texttt{ActiveThorns} parameter all have names +All parameters following the \texttt{ActiveThorns} parameter have names whose syntax depends on the scope (see Section~\ref{sec:Cactus_parameters.scope}) of the parameter: \begin{Lentry} \item [\texttt{Global parameters}] Just the name of the parameter itself. Global parameters are to be avoided; -there are none in the Flesh and Cactus Toolkits. +there are none in the flesh and Cactus Toolkits. \item [\texttt{Restricted parameters}] The name of the \textit{implementation} which defined the parameter, followed by two colons, -then the name of the parameter --- e.g. \texttt{driver::global\_nx}. +then the name of the parameter---e.g. \texttt{driver::global\_nx}. \item [\texttt{Private parameters}] The name of the \textit{thorn} which defined the parameter, two colons, -and the name of the parameter --- e.g. \texttt{wavetoyF77::amplitude}. +and the name of the parameter---e.g. \texttt{wavetoyF77::amplitude}. \end{Lentry} This notation is not currently strictly enforced in the code. It is @@ -1483,11 +1478,11 @@ sufficient to specify the first part of the parameter name using either the implementation name, or the thorn name. However, we recommend that the above convention be followed. -The Cactus Flesh performs checks for consistency and range of parameters. +The Cactus flesh performs checks for consistency and range of parameters. The severity of these checks is controlled by the command line argument -\texttt{--parameter-level} which can take the following values +\texttt{--parameter-level}, which can take the following values \begin{Lentry} -\item[\texttt{relaxed}] Cactus will issue a level 0 warning (that is the +\item[\texttt{relaxed}] Cactus will issue a level 0 warning (that is, the default behaviour will be to terminate) if \begin{itemize} \item{} The specified parameter value is outside of the allowed range. @@ -1495,7 +1490,7 @@ default behaviour will be to terminate) if \item [\texttt{normal}] This is the default, and provides the same warnings as the -\texttt{relaxed} level, with in addition a level 0 warning issued for +\texttt{relaxed} level, with the addition of a level 0 warning issued for \begin{itemize} \item{} An implementation and/or thorn \texttt{foo} is active, but the parameter \texttt{foo::bar} was not defined. @@ -1505,8 +1500,8 @@ This is the default, and provides the same warnings as the \end{itemize} \item [\texttt{strict}] -This provides the same warnings as the \texttt{normal} level, with in -addition a level 0 warning issued for +This provides the same warnings as the \texttt{normal} level, with the +addition of a level 0 warning issued for \begin{itemize} \item{} The parameter \texttt{foo::bar} is specified in the parameter file, but no implementation or thorn with the name \texttt{bar} is active. @@ -1518,7 +1513,7 @@ Notes: \begin{itemize} \item{} You can obtain lists of the parameters associated with -each thorn using the command line options \texttt{-o} and \texttt{-O} +each thorn using the command-line options \texttt{-o} and \texttt{-O} (Section~\ref{sec:command_line_options}). \item{} The parameter file is read \emph{sequentially} from top to bottom, @@ -1527,16 +1522,16 @@ each thorn using the command line options \texttt{-o} and \texttt{-O} why the \texttt{ActiveThorns} parameter is always first in the file). \item{} String parameter values can be specified either as unquoted tokens (not - containing any whitespace) or as quoted values. If a quoted string - parameter value spans multiple lines all whitespaces, including newline + containing any whitespace), or as quoted values. If a quoted string + parameter value spans multiple lines, all whitespaces, including newline characters, are preserved. -\item{} Some parameters are \textit{steerable} and can be changed during - the execution of a Cactus program using parameter steering interfaces - For example, thorn \texttt{CactusConnect/HTTPD}, or using a +\item{} Some parameters are \textit{steerable}, and can be changed during + the execution of a Cactus program by using parameter steering interfaces, + for example, thorn \texttt{CactusConnect/HTTPD}, or by using a parameter file when recovering from a checkpoint file. -\item{} For examples of parameter files, look in the \texttt{par} directory +\item{} For examples of parameter files, look in the \texttt{par} directory, which can be found in most thorns. \end{itemize} @@ -1547,17 +1542,17 @@ each thorn using the command line options \texttt{-o} and \texttt{-O} The Cactus make system provides a mechanism for generating a \textit{Thorn Guide} containing separate chapters for each thorn and arrangement in your configuration. The documentation provided for an -individual thorn obviously depends on what the thorn authors added, +individual thorn, obviously depends on what the thorn authors added, but the Thorn Guide is a good place to first look for special -instructions on how to run and interpret the output from a Thorn. +instructions on how to run and interpret the output from a thorn. Details about parameters, grid variables and scheduling are -automatically included in from a thorns CCL files into the Thorn +automatically included from the thorns' CCL files into the Thorn Guide. To construct a Thorn Guide for the configuration \texttt{$<$\var{config}$>$} use \texttt{gmake $<$\var{config}$>$-ThornGuide} -or to make a Thorn Guide for all the thorns in the \texttt{arrangements} directory +or to make a Thorn Guide for all the thorns in the \texttt{arrangements} directory, \texttt{gmake $<$\var{config}$>$}. @@ -1569,15 +1564,15 @@ documentation to your own thorns. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -\chapter{Getting and looking at output} +\chapter{Getting and Looking at Output} -\section{Screen output} +\section{Screen Output} As your Cactus executable runs, standard output and standard error are usually written to the screen. Standard output provides you with information about the run, and standard error reports warnings -and errors from the Flesh and thorns. +and errors from the flesh and thorns. As the program runs, the normal output provides the following information: @@ -1587,8 +1582,8 @@ As the program runs, the normal output provides the following information: A report is made as each of the thorns in the \texttt{ActiveThorns} parameters from the parameter file (see Section~\ref{sec:Parameter_File}) is attempted to be activated. This report -shows whether the thorn activation was successful, and if successful gives the -thorn's implementation. For example +shows whether the thorn activation was successful, and if successful, gives the +thorn's implementation. For example, \begin{verbatim} Activating thorn idscalarwave...Success -> active implementation idscalarwave @@ -1598,20 +1593,20 @@ Activating thorn idscalarwave...Success -> active implementation idscalarwave If any of the parameters in the parameter file does not belong to any of the active thorns, or if the parameter value is not in the allowed range (see Section~\ref{sec:Parameters.Types_and_Ranges}), -an error is registered. For example, if the parameter is not recognised +an error is registered. For example, if the parameter is not recognised, \begin{verbatim} Unknown parameter time::ddtfac \end{verbatim} -or if the parameter value is not in the allowed range +or if the parameter value is not in the allowed range, \begin{verbatim} Unable to set keyword CartGrid3D::type - ByMouth not in any active range \end{verbatim} \item [Scheduling information] - The scheduled routines (see Section~\ref{chap:scheduling}), -are listed, in the order that they will be executed. For example + The scheduled routines (see Section~\ref{chap:scheduling}) +are listed in the order that they will be executed. For example, \begin{verbatim} ---------------------------------------------------------------------- @@ -1647,7 +1642,7 @@ are listed, in the order that they will be executed. For example \item [Thorn banners] Usually a thorn registers a short piece of text as a \emph{banner}. - This banner of each thorn is displayed in the standard output when + The banner of each thorn is displayed in the standard output when the thorn is initialised. \end{Lentry} @@ -1669,7 +1664,7 @@ iterations, this output should be performed. There is also usually a parameter to define the directory in which the output should be placed, defaulting to the directory from which the executable is run. -See Chapter~\ref{chap:io_methods} for details on creating your own IO method. +See Chapter~\ref{chap:io_methods} for details on creating your own I/O method. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -1684,7 +1679,7 @@ all the data from the checkpoint file. Cactus checkpointing and recovery methods are provided by thorns. In general, these thorns decide how often to generate a checkpoint. -They also register their recovery routines with the Flesh; these recovery +They also register their recovery routines with the flesh; these recovery routines may then be called during initialisation of a subsequent run to perform the recovery of the state of the run. Such a recovery is requested by setting a parameter in the parameter file. diff --git a/doc/UsersGuide/ThornWriters.tex b/doc/UsersGuide/ThornWriters.tex index 2ab8b67a..3a0d5c38 100644 --- a/doc/UsersGuide/ThornWriters.tex +++ b/doc/UsersGuide/ThornWriters.tex @@ -16,7 +16,7 @@ % @endhistory % @@*/ -\begin{cactuspart}{Application thorn writing}{$RCSfile$}{$Revision$} +\begin{cactuspart}{Application Thorn Writing}{$RCSfile$}{$Revision$} \label{part:ThornWriters} \renewcommand{\thepage}{\Alph{part}\arabic{page}} @@ -37,7 +37,7 @@ you may use to gain extra functionality. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -\chapter{Thorn concepts} +\chapter{Thorn Concepts} \label{chap:thorn_concepts} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -46,10 +46,10 @@ you may use to gain extra functionality. A thorn is the basic working module within Cactus. All user supplied code goes into thorns, which are, by and large, independent of each other. -Thorns communicate with each other via calls to the Flesh API, plus, more +Thorns communicate with each other via calls to the flesh API, plus, more rarely, custom APIs of other thorns. -The connection from a thorn to the Flesh or to other thorns is specified in +The connection from a thorn to the flesh, or to other thorns, is specified in configuration files which are parsed at compile time and used to generate glue code which encapsulates the external appearance of a thorn. @@ -73,7 +73,7 @@ and all your evolution thorns in another arrangement, or you may want to have separate arrangements for your developments, private and shared thorns. -The arrangements live in the \texttt{arrangements} directory off the main +The arrangements live in the \texttt{arrangements} directory of the main Cactus directory. Arrangement names must be (case independently) unique, must start with a letter, and can only contain @@ -93,12 +93,12 @@ belonging to the arrangement. One of the key concepts for thorns is the concept of the \textit{implementation}. Relationships among thorns are all based upon relationships among the implementations they provide. -In principle it should be possible to swap one thorn providing an +In principle, it should be possible to swap one thorn providing an implementation with another thorn providing that implementation, without affecting any other thorn. An implementation defines a group of variables and parameters which -are used to implement some functionality. For example the thorn +are used to implement some functionality. For example, the thorn \texttt{CactusPUGH/PUGH} provides the implementation \texttt{driver}. This implementation is responsible for providing memory for grid variables and for communication. Another thorn can also implement \texttt{driver}, @@ -116,7 +116,7 @@ the other thorns. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -\chapter{Anatomy of a thorn} +\chapter{Anatomy of a Thorn} \label{chap:thorn_anatomy} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -128,14 +128,14 @@ administrative files: \begin{Lentry} \item[\texttt{interface.ccl}] the Cactus interface, which defines the grid -functions, variables, etc. See \ref{sec:Appendix.interface}. +functions, variables, etc. See Appendix \ref{sec:Appendix.interface}. \item[\texttt{param.ccl}] the parameters introduced by this thorn, and the -parameters needed from other thorns. See +parameters needed from other thorns. See Appendix \ref{sec:Appendix.param}. \item[\texttt{schedule.ccl}] scheduling information for routines called by -the Flesh. See \ref{sec:Appendix.schedule}. +the flesh. See Appendix \ref{sec:Appendix.schedule}. \item[\texttt{configuration.ccl}] configuration options for the thorn. See -\ref{sec:Appendix.configuration.ccl}. + Appendix \ref{sec:Appendix.configuration.ccl}. \end{Lentry} Thorns can also contain @@ -147,12 +147,12 @@ Thorns can also contain \item a \texttt{doc} directory for documentation \item a \texttt{par} directory for example parameter files \item a \texttt{test} subdirectory may also be added, to hold the thorn's - test suite. See \ref{sec:adding_test_suite} for details. + test suite. See Section \ref{sec:adding_test_suite} for details. \end{itemize} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -\section{Creating a thorn} +\section{Creating a Thorn} To simplify the creation of a thorn, a \texttt{make} target {\tt @@ -167,9 +167,9 @@ pick one from the list of available arrangements that are shown. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -\section{Configuring your thorn} +\section{Configuring your Thorn} -The interaction of a thorn with the Flesh and other thorns is controlled +The interaction of a thorn with the flesh and other thorns is controlled by certain configuration files. These are: @@ -190,12 +190,12 @@ This defines which functions are called from the thorn and when they are called. It also handles memory and communication assignment for grid variables. \item [\texttt{configuration.ccl}] -This file is optional for a thorn. If it exists it contains extra +This file is optional for a thorn. If it exists, it contains extra configuration options of this thorn. \end{Lentry} -\subsection{General syntax of CCL files} +\subsection{General Syntax of CCL Files} \textit{Cactus Configuration Language} (CCL) files are simple text files used to define configuration information for a thorn. CCL files are @@ -205,13 +205,13 @@ non-blank character of a line in a CCL file is a backslash `\texttt{$\backslash$}', the following line is treated as a continuation of the current line. -\subsection{The \texttt{interface.ccl} file} +\subsection{The \texttt{interface.ccl} File} \label{subsec:interface_ccl} The \texttt{interface.ccl} file is used to declare \begin{itemize} -\item the implementation provided by the thorn, +\item the implementation provided by the thorn \item the variables provided by the thorn \item the include files provided by the thorn \item the functions provided by the thorn (in development) @@ -293,21 +293,21 @@ where \texttt{xsize}, \texttt{ysize}, \texttt{gxsize}, \texttt{gysize} are all parameters defined in the thorn's \texttt{param.ccl}. -By default all groups are \texttt{private}, to change this an access +By default, all groups are \texttt{private}, to change this, an access specification of the form \texttt{public:} or \texttt{protected:} (or \texttt{private:} to change it back) may be placed on a line by itself. This changes the access level for any group defined in the file from that point on. All variables seen by any one thorn must have distinct names. -\subsection{The \texttt{param.ccl} file} +\subsection{The \texttt{param.ccl} File} \label{subsec:param_ccl} Users control the operation of thorns via parameters given in a file at runtime. The \texttt{param.ccl} file is used to specify the parameters used to control an individual thorn, and to specify the values these parameters are allowed to take. When the code -is run it reads a parameter file and sets the parameters if they fall +is run, it reads a parameter file and sets the parameters if they fall within the allowed values. If a parameter is not assigned in a parameter file, it is given its default value. @@ -325,7 +325,7 @@ These are only seen by this thorn. A parameter specification consists of: \begin{itemize} -\item the parameter type (each may have an optional {\t CCTK\_} in front) +\item The parameter type (each may have an optional {\t CCTK\_} in front) \begin{Lentry} \item [\texttt{REAL}] \item [\texttt{INT}] @@ -338,27 +338,26 @@ A boolean type which can take values {\t 1}, {\t t}, {\t true}, {\t yes} or {\t 0}, {\t f}, {\t false}, {\t no}. \end{Lentry} -\item the parameter name +\item The parameter name -\item an optional size (in square brackets) -- if this is present the - parameter is a ``parameter array'', i.e.\ will actually be an array - of parameters each of which has the same properties but a different +\item An optional size (in square brackets)--if this is present, the + parameter is a ``parameter array'', i.e.\ it will actually be an array + of parameters, each of which has the same properties, but a different value. Such arrays appear as normal arrays in C or Fortran, 0-based - in C and 1-based in Fortran. In the parameter file the value of + in C, and 1-based in Fortran. In the parameter file the value of each element is specified with square brackets and is 0-based. The size must be an integer. -\item a description of the parameter +\item A description of the parameter -\item an allowed value block -- This consists of a brace delimited +\item An allowed value block--this consists of a brace-delimited block of lines\footnote{The beginning brace (\texttt{\{}) must sit on a line by itself; the ending brace (\texttt{\}}) must be preceded by a carriage return.} describing the allowed values of the parameter. Each range may have a -description associated with it by placing a \texttt{::} on the line and putting +description associated with it by placing a \texttt{::} on the line, and putting the description afterwards. -\item the default value -- -This must be one of the allowed values. +\item The default value--this must be one of the allowed values. \end{itemize} @@ -367,7 +366,7 @@ of a string of the form lower-bound:upper-bound:step, where a missing number or an asterisk `\texttt{*}' denotes anything (i.e.\ infinite bounds or an infinitesimal step). -For example +For example, \begin{verbatim} REAL Coeff "Important coefficient" { @@ -375,13 +374,13 @@ REAL Coeff "Important coefficient" } 0.0 #No need to define a range for BOOLEAN -BOOLEAN nice "Nice weather ?" +BOOLEAN nice "Nice weather?" { }"yes" # A example for a set of keywords and its default (which has to be # defined in the body) -KEYWORD confused "Are we getting confused ?" +KEYWORD confused "Are we getting confused?" { "yes" :: "absolutely positively" "perhaps" :: "we are not sure" @@ -394,17 +393,17 @@ REAL Length[2] "Length in each direction" } 1.0 \end{verbatim} -defines a \texttt{REAL} parameter, a \texttt{BOOLEAN} parameter, a \texttt{KEYWORD} +defines a \texttt{REAL} parameter, a \texttt{BOOLEAN} parameter, a \texttt{KEYWORD}, and an array of \texttt{REAL} parameters. -By default all parameters are \texttt{private}; to change this, an access +By default, all parameters are \texttt{private}; to change this, an access specification of the form \texttt{global:} or \texttt{restricted:} (or \texttt{private:} to change it back) may be placed on a line by itself. This changes the access level for any parameter defined in the file from that point on. To access \texttt{restricted} parameters from another implementation, a line containing \texttt{shares: <\var{name}>} declares that all parameters mentioned in -the file from now until the next access specification originate in +the file, from now until the next access specification, originate in implementation \texttt{<\var{name}>}. (Note that only one implementation can be specified on each \texttt{shares:} line.) Each of these parameters must be qualified by the initial token \texttt{USES} or \texttt{EXTENDS}, where \begin{Lentry} @@ -413,7 +412,7 @@ specified on each \texttt{shares:} line.) Each of these parameters must be qual \end{Lentry} In contrast to parameter declarations in other access blocks, the default -value must be omitted --- it is impossible to set the default value of any +value must be omitted---it is impossible to set the default value of any parameter not originating in this thorn. For example, the following block adds possible values to the keyword \texttt{initial\_data} originally defined in the implementation \texttt{einstein}, @@ -439,10 +438,10 @@ Note that you must compile at least one thorn which implements \texttt{einstein} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -\subsection{The \texttt{schedule.ccl} file} +\subsection{The \texttt{schedule.ccl} File} \label{subsec:schedule_ccl} -By default no routine of a thorn will be run. The \texttt{schedule.ccl} -file defines those that should be run and when and under which +By default, no routine of a thorn will be run. The \texttt{schedule.ccl} +file defines those that should be run, and when and under which conditions they should be run. The specification of routine scheduling is via a schedule block which @@ -458,7 +457,7 @@ schedule <\var{name}> at <\var{time bin}> [\var{other options}] \} "\var{A description}" \end{alltt} where \texttt{<\var{name}>} is the name of the routine, and -\texttt{<\var{time bin}>} is the name of a schedule bins (the \texttt{CCTK\_} +\texttt{<\var{time bin}>} is the name of a schedule bin (the \texttt{CCTK\_} prefix is optional). A list of the most useful schedule bins for application thorns is given here, a complete and more descriptive list is provided in Appendix~\ref{sec:Appendix.schedule_bins}: @@ -466,15 +465,15 @@ Appendix~\ref{sec:Appendix.schedule_bins}: \begin{Lentry} \item [\texttt{CCTK\_STARTUP}] -For routines run before the grid hierarchy is set up, for example function +For routines, run before the grid hierarchy is set up, for example, function registration. \item [\texttt{CCTK\_PARAMCHECK}] -For routines which check parameter combinations, routines registered here +For routines that check parameter combinations, routines registered here only have access to the grid size and the parameters. \item [\texttt{CCTK\_BASEGRID}] -Responsible for setting up coordinates etc. +Responsible for setting up coordinates, etc. \item [\texttt{CCTK\_INITIAL}] For generating initial data. @@ -496,7 +495,7 @@ For analysing data. \end{Lentry} -The \var{other options} allow finer grained control of the scheduling. It is +The \var{other options} allow finer-grained control of the scheduling. It is possible to state that the routine must run \texttt{BEFORE} or \texttt{AFTER} another routine or set of routines. It is also possible to schedule the routine under an @@ -519,7 +518,7 @@ the number of timelevels of each group for which storage should be activated. \texttt{STORAGE: <\var{group1}>[\var{timelevels1}], <\var{group2}>[\var{timelevels2}]} - This number can range from one to the maximum number of timelevels for the group as specified in the group definition in its \texttt{interface.ccl} file. If this maximum number is one, the timelevel specification can be omitted from the + This number can range from one to the maximum number of timelevels for the group, as specified in the group definition in its \texttt{interface.ccl} file. If this maximum number is one, the timelevel specification can be omitted from the \texttt{STORAGE} statement. \item[\texttt{TRIGGERS}] \texttt{TRIGGERS} is used when the routine is @@ -536,11 +535,11 @@ grid variables in \texttt{schedule.ccl} is an alternative to calling the functions \texttt{CCTK\_SyncGroup()} or \texttt{CCTK\_SyncGroupsI()} (see the Reference Manual) from inside a routine. Using the \texttt{SYNC} keyword in the \texttt{schedule.ccl} is the preferred method, since it -provides the Flesh with more information about the behaviour of your code. +provides the flesh with more information about the behaviour of your code. \end{Lentry} -Besides schedule blocks it's possible to embed C style \texttt{if/else} +Besides schedule blocks, it's possible to embed C style \texttt{if/else} statements in the \texttt{schedule.ccl} file. These can be used to schedule things based upon the value of a parameter. @@ -551,13 +550,13 @@ If the parameter \texttt{evolve\_hydro} is positively set, the Fortran routine \texttt{hydro\_predictor} is scheduled to run in the \textit{evolution} loop, after the routine \texttt{metric\_predictor} and before \texttt{metric\_corrector}. The routine names \texttt{metric\_predictor} and -\texttt{metric\_corrector} may either be real routine names from the same +\texttt{metric\_corrector}, may either be real routine names from the same or a different thorn, or they may be \texttt{aliased} routine names (see the next example). -Before entry to \texttt{hydro\_predictor} +Before entry to \texttt{hydro\_predictor}, storage will be allocated for one timelevel for the group of grid -variables \texttt{hydro\_variables}, on exit from the routine this storage +variables \texttt{hydro\_variables} on exit from the routine this storage will be deallocated and the contents of the variables will be lost. \begin{verbatim} if(CCTK_Equals(evolve_hydro,"yes")) @@ -572,14 +571,14 @@ if(CCTK_Equals(evolve_hydro,"yes")) If the parameter \texttt{evolve\_hydro} is set negatively, the {\tt hydro\_predictor} routine will not be called by the scheduler. Note -that if the \texttt{evolve\_hydro} parameter is \texttt{STEERABLE} it can be +that if the \texttt{evolve\_hydro} parameter is \texttt{STEERABLE}, it can be dynamically scheduled and de-scheduled during a run if a steering interface is available. \vskip .5cm {\bf Example II:} -The thorns \texttt{WaveToy77} and \texttt{WaveToyC} each provide a +The thorns \texttt{WaveToy77} and \texttt{WaveToyC}, each provide a routine to evolve the 3D wave equation: \texttt{WaveToyF77\_Evolution} and \texttt{WaveToyC\_Evolution}. The routine names have to be different, so that both thorns can be compiled at the same time, their functionality @@ -592,7 +591,7 @@ WaveToy\_Evolution} to allow relative scheduling ({\tt BEFORE/AFTER}) independent of the actual routine name (which may change depending on the activation in the parameter file). -In both cases the group of variables \texttt{scalarfield} are synchronised +In both cases, the group of variables \texttt{scalarfield} are synchronised across processes when the routine is exited. \begin{verbatim} schedule WaveToyF77_Evolution AS WaveToy_Evolution AT evol @@ -624,7 +623,7 @@ schedule WaveBinary AT evol AFTER WaveToy_Evolution The keyword \texttt{STORAGE} can also be used outside of the schedule blocks to indicate that storage for these groups should be switched on at the start of the run. Note that the storage is only allocated in -this way at the start, a thorn could explicitly switch the storage off +this way at the start; a thorn could explicitly switch the storage off (although this is not recommended practise). As for the \texttt{STORAGE} statement in schedule blocks, each group must also specify how many timelevels to activate storage for. @@ -637,7 +636,7 @@ timelevels to activate storage for. [{\bf NOTE:} the \texttt{configuration.ccl} is a new feature, and not all the features described in this section have been fully implemented yet. -PROVIDES and REQUIRES should work, but script support and OPTIONAL are still +PROVIDES, REQUIRES and SCRIPT should work, but OPTIONAL is still being developed.] The \texttt{configuration.ccl} file is optional. It can be used for two @@ -654,13 +653,13 @@ the CST of features to: add to the header files included by thorns using this capability; add to the make files used to build thorns using this capability; or add to the main Cactus link line. The script may also indicate that this capability is not present by returning a -non-zero exit code --- e.g., if the thorn is providing access to an +non-zero exit code---e.g. if the thorn is providing access to an external library, it should return an error if the library is not installed on the system. A thorn may either require a capability to be present, in which case it is an error if there is no thorn providing that capability in the -configuration's ThornList, or it may optionally use a capability, in +configuration's \texttt{ThornList}, or it may optionally use a capability, in which case a macro is defined in the thorn's header file if a thorn providing the capability is present. @@ -685,15 +684,15 @@ OPTIONAL <\var{Yet_Another_Capability}> which states that this thorn provides the capability \verb|My_Capability|, and a script \verb|MyConfigScript| should be run to detect features of this capability; the script is in language -\verb|My_Language| --- the CST will use the appropriate environment or +\verb|My_Language|---the CST will use the appropriate environment or interpreter to invoke the script. The example requires a thorn providing \verb|Another_Capability| to be -in the ThornList, and, if a thorn providing -\verb|Yet_Another_Capability| is in the ThornList, the preprocessor +in the \texttt{ThornList}, and, if a thorn providing +\verb|Yet_Another_Capability| is in the \texttt{ThornList}, the preprocessor macro \verb|macro| will be defined, and set to 1. -As an example a thorn A may be able to use PVM for parallelism if it is +As an example, a thorn A may be able to use PVM for parallelism if it is present, but can still work in the absence of it. There could be a thorn providing PVM, and thorn A would then have @@ -757,7 +756,7 @@ preprocessed. A complete description of Fortran fixed and free format can be found in any textbook on Fortran 90. The most obvious differences are that in fixed format, code must begin after the 5th column and line continuations are -indicated by a character in column 5, while in free format lines can begin +indicated by a character in column 5, while in free format, lines can begin anywhere, and line continuations are indicated by an ampersand at the end of the line to be continued. Also note that statement labels are handled very differently. @@ -769,7 +768,7 @@ The following restrictions apply to file names: system being case sensitive (e.g.\ having \texttt{MyFile.c} and \texttt{MYFILE.f77} is allright, but \texttt{MyFile.c} and \texttt{MYFILE.c} could cause problems). -\item Currently all source files within a thorn must have distinct names, +\item Currently, all source files within a thorn must have distinct names, regardless of whether they are placed in different subdirectories. We hope to relax this in future. Different thorns may have files with the same names, however. @@ -777,9 +776,9 @@ however. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -\section{Adding source files} +\section{Adding Source Files} -By default the CCTK looks in the \texttt{src} directory of the thorn for source +By default, the CCTK looks in the \texttt{src} directory of the thorn for source files. There are two ways in which to specify the sources. The easiest is to use the @@ -862,7 +861,7 @@ and has a working directory of \item \texttt{F77FLAGS}: Flags which are passed to \texttt{F77} \item \texttt{F90FLAGS}: Flags which are passed to \texttt{F90} \item \texttt{LD}: The binder. This should not be directly - \texttt{ld}, but should be a compiler driver such as \texttt{c++}. + \texttt{ld}, but should be a compiler driver such as C++. Often, \texttt{LD} is the same as \texttt{CXX} \item \texttt{LDFLAGS}: Flags which are passed to \texttt{LD} \end{itemize} @@ -873,12 +872,12 @@ and has a working directory of \chapter{Cactus Variables} \label{chap:cactus_variables} -A \textit{grid variable} is Cactus program variable passed among thorns, +A \textit{grid variable} is a Cactus program variable passed among thorns, (or routines belonging to the same thorn interface), -by way of calls to the Flesh. +by way of calls to the flesh. They are the only variables known to Cactus. Such variables represent values of functions on the computational grid, -and are therefore often called \textit{grid functions}. +and are, therefore, often called \textit{grid functions}. In the Cactus context, grid variables are often referred to simply as \textit{variables}. @@ -886,13 +885,13 @@ to simply as \textit{variables}. Cactus variables are used instead of local variables for a number of reasons: \begin{itemize} \item Cactus variables can be made visible to other thorns, allowing - thorns to communicate and share data, + thorns to communicate and share data. \item Cactus variables can be distributed and communicated - among processors, allowing parallel computation, + among processors, allowing parallel computation. \item A database of Cactus variables, and their attributes, is held by - the Flesh, and this information is used by thorns, for example for - obtaining a list of variables for checkpointing, -\item Many Cactus APIs and tools can only be used with Cactus variables, + the flesh, and this information is used by thorns, for example, for + obtaining a list of variables for checkpointing. +\item Many Cactus APIs and tools can only be used with Cactus variables. \item Cactus provides features for error checking based on Cactus variables and their attributes. \end{itemize} @@ -938,9 +937,8 @@ bytes occupied by the a variable of the type). This is a 1 byte data type. \end{Lentry} -Normally a thorn should use the default types --- -\texttt{CCTK\_INT}, \texttt{CCTK\_REAL}, \texttt{CCTK\_COMPLEX} --- rather than explicitly setting the size, as this gives maximum -portability. Also, the defaults can be changed at configuration time (see +Normally a thorn should use the default types---\texttt{CCTK\_INT}, \texttt{CCTK\_REAL}, \texttt{CCTK\_COMPLEX}---rather than explicitly setting the size, as this gives maximum +portability. Also, the defaults can be changed at configuration time (see Section \ref{subsec:Compilation-Available_Options}), and this allows people to compile the code with different precisions to test for roundoff effects, or to run more quickly with a lower accuracy. @@ -958,7 +956,7 @@ types. \begin{Lentry} \item[\texttt{SCALAR}] This is just a single number, e.g.\ the total energy of some field. These -variables aren't communicated between processors --- what would be the +variables aren't communicated between processors---what would be the result of such communication? \item[\texttt{GF}] This is the most common group type. A GF is an array with a @@ -984,7 +982,7 @@ Consider the 1-D wave equation \begin{equation} \frac{\partial^2 \phi}{\partial t^2} = \frac{\partial^2 \phi}{\partial x^2} \end{equation} -To solve this by partial differences, one discretises the derivatives, to get +To solve this by partial differences, one discretises the derivatives to get an equation relating the solution at different times. There are many ways to do this, one of which produces the following difference equation \begin{equation} @@ -993,7 +991,7 @@ to do this, one of which produces the following difference equation \end{equation} which relates the three timelevels $t+\Delta t$, $t$, and $t-\Delta t$. -Obviously the code could just use three variables, one for each timelevel. +Obviously, the code could just use three variables, one for each timelevel. This turns out, however, to be inefficient, because as soon as the time is incremented to $t+\Delta t$, it would be necessary to copy data from the $t$ variable to the $t-\Delta t$ variable and from the $t+\Delta t$ variable @@ -1002,7 +1000,7 @@ to the $t$ variable. To remove this extraneous copy, Cactus allows you to specify the number of timelevels used by your numerical scheme. It then generates variables with the base name (e.g.\ \texttt{phi}) suffixed by a qualifier for -which timelevel is being referred to --- no suffix for the +which timelevel is being referred to---no suffix for the next timelevel, and \texttt{\_p} for each previous timelevel. The timelevels are rotated (by the driver thorn) at the start @@ -1025,10 +1023,10 @@ loop: Timelevel rotation means that, for example, \texttt{phi\_p} now holds the values of the former \texttt{phi}, and \texttt{phi\_p\_p} the values of the former \texttt{phi\_p}, etc. -Note that after rotation \texttt{phi} is undefined, and its values should +Note that after rotation, \texttt{phi} is undefined, and its values should not be used until they have been updated. -All timelevels, except the current level, should be considered \emph{read-only} during evolution, that is their values should not be changed by thorns. +All timelevels, except the current level, should be considered \emph{read-only} during evolution, that is, their values should not be changed by thorns. The exception to this rule is for function initialisation, when the values at the previous timelevels do need to be explicitly filled out. @@ -1039,14 +1037,14 @@ values at the previous timelevels do need to be explicitly filled out. A Cactus grid function or array has a size set at runtime by parameters. This size can either be the global size of the array across all processors (\texttt{DISTRIB=DEFAULT}), -or, if \texttt{DISTRIB=CONSTANT} the specified +or, if \texttt{DISTRIB=CONSTANT}, the specified size on \emph{each} processor. If the size is split across processors, the driver thorn is responsible for assigning the size on each processor. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -\section{Ghost Size} +\section{Ghost Zones} \label{sec:ghost_size} Cactus is based upon a \textit{distributed computing} paradigm. That is, @@ -1056,9 +1054,9 @@ need to communicate at the edges. To illustrate this, take the example of the wave equation again. Equation \ref{equation:difference} describes a possible time-evolution -scheme. On examination you can see that to generate the data at the +scheme. On examination, you can see that to generate the data at the point ($t+\Delta t$, $x$) we need data from the four points ($t$, -$x$), ($t-\Delta t$, $x$), ($t$, $x+\Delta x$) and ($t$, $x-\Delta x$) +$x$), ($t-\Delta t$, $x$), ($t$, $x+\Delta x$), and ($t$, $x-\Delta x$) only. Ignoring the points at $x$, which are obviously always available on a given processor, you can see that the algorithm requires a point on either side of the point $x$, i.e.\ this algorithm @@ -1067,7 +1065,7 @@ on either side have stencil size 2, etc. Now, if you evolve the above scheme, it becomes apparent that at each iteration the number of grid points you can evolve decreases by one at -each edge (see figure \ref{fig:noghost}). +each edge (see Figure \ref{fig:noghost}). \begin{figure}[ht] \begin{center} @@ -1077,19 +1075,19 @@ each edge (see figure \ref{fig:noghost}). \label{fig:noghost} \end{figure} -At the outer boundary of the physical domain the data for the boundary -point can be generated by the boundary conditions, however at internal -boundaries the data has to be copied from the adjacent processor. It +At the outer boundary of the physical domain, the data for the boundary +point can be generated by the boundary conditions, however, at internal +boundaries, the data has to be copied from the adjacent processor. It would be inefficient to copy each point individually, so instead, a number of \textit{ghostzones} are created at the internal boundaries. A -ghostzone consists of a copy of the whole plane (in 3d, line in 2d, -point in 1d) of the data from the adjacent processor. That is, the array +ghostzone consists of a copy of the whole plane (in 3D, line in 2D, +point in 1D) of the data from the adjacent processor. That is, the array on each processor is augmented with copies of points from the adjacent processors, thus allowing the algorithm to proceed \textit{on the points owned by this processor} without having to worry about copying data. Once the data has been evolved one step, the data in the ghostzones can be exchanged (or \textit{synchronised}) between processors in one -fell swoop before the next evolution step. (See figure +fell swoop before the next evolution step. (See Figure \ref{fig:withghost}.) Note that you should have at least as many ghostzones as your stencil size requires. @@ -1125,7 +1123,7 @@ used. {\bf Specifying the staggertype} The type of staggering applied to a grid function can be specified in -the \texttt{interface.ccl} file by the attribute \texttt{stagger} (see +the \texttt{interface.ccl} file by the attribute \texttt{stagger} (see Appendix \ref{sec:Appendix.interface}). Cactus supports three kinds of staggering per dimension. The physical location of a grid function is shifted relative to the default position by adding the following values to the @@ -1138,13 +1136,13 @@ half of the grid spacing in the positive direction (or to the right). \item[\texttt{P}] full staggered. \texttt{P} refers to plus. The physical location is offset by a full grid spacing in the positive direction (or the right). \end{Lentry} -For multi dimensional grid functions you concatenate the code -characters in $xyz$ order. In Figure \ref{fig:stagger1} we show four different +For multi-dimensional grid functions you concatenate the code +characters in $xyz$ order. In Figure \ref{fig:stagger1}, we show four different staggerings of a two dimensional grid function. The solid black grid circles show the default location of the grid function at the intersections of the grid lines. In (A) we show an additional grid function of type \texttt{stagger=MC}: no staggering in $x$ direction, -center staggered in $y$ direction. In (B) we have \texttt{stagger=CM} and +center staggered in $y$ direction. In (B) we have \texttt{stagger=CM}, and staggering each direction (\texttt{stagger=CC}) is shown in (C). The full staggering in (D) (\texttt{stagger=PP}) obeys the same rules, but is rather unusual; it is included here for completeness. @@ -1164,50 +1162,50 @@ points (arrows) are accessed by the same indices.} \section{Information about Grid Variables} -The Flesh holds a database with information related to grid variables, and +The flesh holds a database with information related to grid variables, and provides a set of querying APIs. \subsection{Group Information} Fundamental information about grid functions (e.g.\ local grid size and location, number of ghostzones) is passed through the argument list of -scheduled routines (see~\ref{sec:cactus_variables_c}). To obtain similar information +scheduled routines (see Section~\ref{sec:cactus_variables_c}). To obtain similar information from non-scheduled routines, or for general grid variables, a set of functions are provided, the last two letters of which specify whether the information is requested using a group name (\texttt{GN}) or index (\texttt{GI}), or a variable name (\texttt{VN}) or index (\texttt{VI}). -\begin{itemize} +\begin{Lentry} -\item \texttt{CCTK\_Grouplsh[GN|GI|VN|VI]} An array of integers +\item [\texttt{CCTK\_Grouplsh[GN|GI|VN|VI]}] An array of integers with the local grid size on this processor. -\item \texttt{CCTK\_Groupgsh[GN|GI|VN|VI]} An array of integers +\item [\texttt{CCTK\_Groupgsh[GN|GI|VN|VI]}] An array of integers with the global grid size. -\item \texttt{CCTK\_Groupbbox[GN|GI|VN|VI]} An array of integers +\item [\texttt{CCTK\_Groupbbox[GN|GI|VN|VI]}] An array of integers which indicate whether the boundaries are internal boundaries (e.g.\ between processors), or physical boundaries. A value of \texttt{1} indicates a physical (outer) boundary at the edge of the computational grid, and \texttt{0} indicates an internal boundary. -\item \texttt{CCTK\_Groupnghostzones[GN|GI|VN|VI]} An array of integers with +\item [\texttt{CCTK\_Groupnghostzones[GN|GI|VN|VI]}] An array of integers with the number of ghostzones used in each direction. -\item \texttt{CCTK\_Grouplbnd[GN|GI|VN|VI]} An array of integers +\item [\texttt{CCTK\_Grouplbnd[GN|GI|VN|VI]}] An array of integers containing the lowest index (in each direction) of the local grid, as seen on the global grid. Note that these indices start from zero, so you need to add one when using them in Fortran thorns. -\item \texttt{CCTK\_Groupubnd[GN|GI|VN|VI]} An array of integers +\item [\texttt{CCTK\_Groupubnd[GN|GI|VN|VI]}] An array of integers containing the largest index (in each direction) of the local grid, as seen on the global grid. Note that these indices start from zero, so you need to add one when using them in Fortran thorns. -\end{itemize} +\end{Lentry} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -1221,17 +1219,17 @@ range of allowed values and a default value. These are declared in the thorn's \texttt{param.ccl} file. The thorn determines which parameters can be used in other thorns by -specifying a \textit{scope} for the thorn, as explained in +specifying a \textit{scope} for the thorn, as explained in Section \ref{sec:Cactus_parameters.scope}. The user may specify initial values for parameters in the parameter file -(see Section~\ref{sec:Parameter_File}); the Flesh validates these values +(see Section~\ref{sec:Parameter_File}); the flesh validates these values against the parameters' allowed ranges. Otherwise, the initial value of the parameter is taken to be its default. Once validated, parameter values are fixed, and cannot be changed, unless the parameter is specified to be \textit{steerable} (see \ref{sec:Cactus_parameters.steerable}). -For a detailed discussion of the \texttt{param.ccl} syntax see +For a detailed discussion of the \texttt{param.ccl} syntax, see Appendix~\ref{sec:Appendix.param}. The full specification for a parameter declaration is @@ -1259,7 +1257,7 @@ Parameters can be of these types: \item[Real] Can take any floating point value \item[Keyword] Can have a value consisting of one of a choice of strings \item[Boolean] Can be true or false ({\t 1}, {\t t}, {\t true}, or -{\t 0}, {\t f}, {\t false}). +{\t 0}, {\t f}, {\t false}) \item[String] Can have any string value \end{Lentry} @@ -1277,7 +1275,7 @@ The range specification is of the form \end{alltt} where \var{lower} and \var{upper} specify the lower and upper allowed -range, and \var{stride} allows numbers to be be missed out. E.g.\ +range, and \var{stride} allows numbers to be be missed out, e.g.\ \begin{verbatim} 1:21:2 \end{verbatim} @@ -1300,7 +1298,7 @@ infinity. The above is inclusive of the endpoints. A `\texttt{(}' endpoint. The numbers written in a \texttt{param.ccl} file are interpreted as C code. -To express a number in `scientific notation', use +To express a number in `scientific notation', use, e.g.\ `\texttt{1e-10}', which is a double precision constant in C. (If the floating precision of the variable to which it is assigned is not double, then C will typecast appropriately. If you \emph{really} want to @@ -1338,7 +1336,7 @@ it. A parameter can be changed dynamically if it is specified to be \textit{steerable} (see Section~\ref{subsec:Appendix.param.specification_items}). -It can then be changed by a call to the Flesh function +It can then be changed by a call to the flesh function \texttt{CCTK\_ParameterSet} (see the Reference Guide for a description of this function). @@ -1349,7 +1347,7 @@ of this function). \label{chap:scheduling} Cactus contains a rule-based scheduling system, which determines which -routines from which thorns are run in which order. The scheduler +routines, from which thorns are run in which order. The scheduler determines if the specifications are inconsistent, but does allow the user to schedule a routine with respect to another routine which may not exist. @@ -1391,15 +1389,15 @@ Each schedule item is scheduled either \texttt{AT} a particular \label{scheduling:schedule_bins} These are the main times at which scheduled functions are run, from -fixed points in the Flesh and driver thorn (schedule bins can easily +fixed points in the flesh and driver thorn (schedule bins can easily be traversed from any thorn, although this is not usual). When a schedule bin is \textit{traversed}, all the functions scheduled in that -particular are called, in the manner described in +particular are called, in the manner described in Section \ref{scheduling:calling_scheduled_functions} and respecting the -requested ordering of functions~\ref{scheduling:schedule_options}. In the absence of any ordering, functions in a particular schedule bin will be called in +requested ordering of functions(Section~\ref{scheduling:schedule_options}). In the absence of any ordering, functions in a particular schedule bin will be called in an undetermined order. -The schedule bins are described in \ref{subsec:schedule_ccl}. Note that +The schedule bins are described in Section \ref{subsec:schedule_ccl}. Note that the preceding \texttt{CCTK\_} is optional for the use of the bin names in the \texttt{schedule.ccl} file. @@ -1411,7 +1409,7 @@ in the \texttt{schedule.ccl} file. If the optional \texttt{GROUP} specifier is used, the item is a schedule group rather than a normal function. Schedule groups are effectively new, user-defined, schedule bins. Functions or groups may be -scheduled \texttt{IN} these in the same way as they are scheduled {\tt +scheduled \texttt{IN} these, in the same way as they are scheduled {\tt AT} the main schedule bins. (That is, groups may be nested.) %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -1431,13 +1429,13 @@ allowing other thorns to operate independently of which one is active. \item[\texttt{WHILE}] This specifies a \texttt{CCTK\_INT} grid scalar which is used to control the execution of this item. As long as the grid scalar has -a nonzero value the schedule item will be executed repeatedly. This +a nonzero value, the schedule item will be executed repeatedly. This allows dynamic behaviour with scheduling. \item[\texttt{IF}] This specifies a \texttt{CCTK\_INT} grid scalar which is used to control the execution of this item. If the grid scalar has a -nonzero value the schedule item will be executed, otherwise the item +nonzero value, the schedule item will be executed, otherwise the item will be ignored. This allows dynamic behaviour with scheduling. If both an \texttt{IF} and a \texttt{WHILE} clause are present, then @@ -1472,7 +1470,7 @@ schedule FOO BEFORE (A B C) ... Note that the set of all \texttt{BEFORE}/\texttt{AFTER} options in all active schedule blocks of all active thorns, \emph{must} specify -a (directed) graph with no cycles; if there are any cycles then the +a (directed) graph with no cycles; if there are any cycles, then the scheduler will abort with a fatal error. \end{Lentry} @@ -1499,11 +1497,11 @@ The item will only be executed if output is due for at least one variable in one of the listed groups. (The item will also be called if there is no group listed.) \item[\texttt{SYNC}] -On exit from this item the ghost zones of the listed groups will be +On exit from this item, the ghost zones of the listed groups will be exchanged. \item[\texttt{OPTIONS}] This is for miscellaneous options. The list of accepted options is -given in appendix~\ref{app:allopts}. +given in Appendix~\ref{app:allopts}. \end{Lentry} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -1511,22 +1509,21 @@ given in appendix~\ref{app:allopts}. \section{How Cactus Calls Scheduled Functions} \label{scheduling:calling_scheduled_functions} -For each scheduled function called, the Flesh performs a variety of jobs +For each scheduled function called, the flesh performs a variety of jobs at entry and exit. On entry to a scheduled routine, if the routine is being called at the -\texttt{CCTK\_ANALYSIS} timebin first a check is made to see if the routine should +\texttt{CCTK\_ANALYSIS} timebin first, a check is made to see if the routine should actually be called on this timestep. For this, all grid variables in the trigger groups for the routine are checked with all registered output methods to determine if it is time to output any triggers. The routine will only be called if at least one is due to be output. Note that once -a grid variable has been analyzed it gets marked as such and will not +a grid variable has been analyzed, it gets marked as such, and will not be analyzed again during this iteration. (Note that a routine without any trigger groups will also be called.) -Thus if more than one analysis -routine should be triggered on the same trigger variable(s) they must -be scheduled in a single group.\\ -Routines from all timebins other than \texttt{ANALYSIS} are always called. +Thus, if more than one analysis +routine should be triggered on the same trigger variable(s), they must +be scheduled in a single group. Routines from all timebins, other than \texttt{ANALYSIS}, are always called. Next, storage is assigned for any required variables, remembering the original state of storage. @@ -1546,14 +1543,14 @@ returned to the original state. \section{Thorn Programming Languages} When you start writing a new thorn, the first decision to make is -which programming language to use? The source code in Cactus thorns +which programming language to use. The source code in Cactus thorns can be written in any mixture of Fortran 77, Fortran 90, C or C++. The following points should be considered when choosing a language to work in \begin{itemize} \item All functions designed for application thorn writers are available - in all languages, however some interfaces for infrastructure + in all languages, however, some interfaces for infrastructure thorn writing are only available from C or C++. % This is no longer relevant? @@ -1572,9 +1569,9 @@ use machine dependent extensions. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -\section{What the Flesh provides} +\section{What the Flesh Provides} -The Flesh provides for thorns: +The flesh provides for thorns: \begin{Lentry} \item [\texttt{Variables}] \item [\texttt{Parameters}] @@ -1582,7 +1579,7 @@ The Flesh provides for thorns: \begin{itemize} \item{} Driver (parallelisation) utilities - \item{} IO utilities + \item{} I/O utilities \item{} Coordinates utilities \item{} Reduction utilities \item{} Interpolation utilities @@ -1599,12 +1596,12 @@ the header file \texttt{cctk.h} using the line #include "cctk.h" \end{verbatim} (Fortran programmers should not be put off by this being a C style -header file --- most Cactus files are run through a C preprocessor -before compilation). +header file---most Cactus files are run through a C preprocessor +before compilation.) \subsubsection{Variables} -Any routine using Cactus argument lists (for example all routines called from +Any routine using Cactus argument lists (for example, all routines called from the scheduler at time bins between {\t CCTK\_STARTUP} and {\t CCTK\_SHUTDOWN}) should include at the top of the file the header \begin{verbatim} @@ -1614,7 +1611,7 @@ should include at the top of the file the header A Cactus macro \texttt{CCTK\_ARGUMENTS} is defined for each thorn to contain: \begin{itemize} -\item General information about the grid hierarchy, for example the number +\item General information about the grid hierarchy, for example, the number of grid points used. See Section \ref{sec:cactus_variables_c} for a complete list. \item All the grid variables defined in the thorn's \texttt{interface.ccl} @@ -1629,19 +1626,19 @@ To pass the arguments to another routine in the same thorn use the macro \texttt{CCTK\_ARGUMENTS} in the receiving routine. Note that you cannot use Cactus argument lists in routines scheduled at the -{\t CCTK\_STARTUP} and {\t CCTK\_SHUTDOWN} time bins because at this time +{\t CCTK\_STARTUP} and {\t CCTK\_SHUTDOWN} time bins, because at this time no grid hierarchy exists. \subsubsection{Parameters} All parameters defined in a thorn's \texttt{param.ccl} and all \texttt{global} -parameters appear as local variables of the corresponding \texttt{CCTK} data type -in Fortran source code, i.e.. Booleans and Integers appear as \texttt{CCTK\_INT} +parameters, appear as local variables of the corresponding \texttt{CCTK} data type +in Fortran source code, i.e. Booleans and Integers appear as \texttt{CCTK\_INT} types (with nonzero/zero values for boolean {\t yes/no}), Reals as \texttt{CCTK\_REAL}, and Keywords and String parameters as \texttt{CCTK\_STRING} (see -also note below). These variables are \emph{read only} and \emph{changes should +also note below). These variables are \emph{read only}, and \emph{changes should not be made to them}. The effect of changing a parameter is undefined (at best). Any routine using Cactus parameters should include at @@ -1657,13 +1654,13 @@ In Fortran, special care should be taken with string valued parameters. These parameters are passed as C pointers, and can not be treated as normal Fortran strings. To compare a string valued parameter and Fortran -string use the macro \texttt{CCTK\_EQUALS()} or the function \texttt{CCTK\_Equals()} +string, use the macro \texttt{CCTK\_EQUALS()} or the function \texttt{CCTK\_Equals()} (see the reference manual for a description of the \texttt{CCTK\_} functions). To print the value of a string valued parameter to screen, use the subroutine \texttt{CCTK\_PrintString()}. A further function \texttt{CCTK\_FortranString} provides a mechanism for converting a string parameter to a Fortran string. For example, if \texttt{operator} is a Cactus string parameter holding the name of a reduction operator whose handle you need to find, you cannot pass it -directly into the subroutine \texttt{CCTK\_LocalArrayReductionHandle} which is expecting +directly into the subroutine \texttt{CCTK\_LocalArrayReductionHandle}, which is expecting a Fortran string. Instead, the following is needed: % \begin{verbatim} @@ -1679,9 +1676,9 @@ a Fortran string. Instead, the following is needed: \subsubsection{Fortran Example} -The Fortran routine \verb|MyFRoutine| is scheduled in the {\tt +The Fortran routine, \verb|MyFRoutine|, is scheduled in the {\tt schedule.ccl} file, doesn't use Cactus parameters, and calls another -routine, in the same thorn, \verb|MyNewRoutine| which does use +routine, in the same thorn, \verb|MyNewRoutine|, which does use parameters. This routine needs to be passed an integer flag as well as the standard Cactus variables. The source file should look like % @@ -1722,7 +1719,7 @@ c Main code goes here \subsubsection{Cactus Fortran Functions} -Cactus Fortran functions, for example \texttt{CCTK\_MyProc} and {\tt +Cactus Fortran functions, for example, \texttt{CCTK\_MyProc} and {\tt CCTK\_Equals}, can all be declared by adding the statement % \begin{verbatim} @@ -1742,13 +1739,13 @@ statement, but before any executable code. Fortran modules should be placed into source files that have the same name as the module, followed by the corresponding file name suffix. A -module \texttt{metric} should thus be placed e.g.\ into a file +module \texttt{metric} should thus be placed, e.g.\ into a file \texttt{metric.F90}. This convention allows the Cactus build system to automatically deduce the compile time dependencies. If you do not follow this convention, then you have to include the modules into the thorn's \texttt{make.code.deps} file -(\ref{sec:mabathbu}) to ensure they are compiled before the routines +(Section \ref{sec:mabathbu}) to ensure they are compiled before the routines which use them. This is especially important for parallel building. For example, if a routine in \texttt{MyRoutine.F90} uses a module in {\tt MyModule.F90}, then add the line: @@ -1761,7 +1758,7 @@ MyRoutine.F90.o: MyModule.F90.o The intrinsic function \texttt{MOD} in Fortran takes two integer arguments, which should both be of the same type. This means -that it may be necessary to cast the arguments to \textit{e.g.} +that it may be necessary to cast the arguments to, e.g. \texttt{INT} for some architectures. This can occur in particular when a \texttt{CCTK\_INT} parameter and the Cactus variable \texttt{cctk\_iteration} (which is declared to be \texttt{INTEGER}) are used, @@ -1781,8 +1778,8 @@ the header file \texttt{cctk.h} using the line \subsubsection{Variables} -Any routine using Cactus argument lists (for example all routines called from -the scheduler at time bins between {\t CCTK\_STARTUP} and {\t CCTK\_SHUTDOWN}) +Any routine using Cactus argument lists (for example, all routines called from +the scheduler at time bins between {\t CCTK\_STARTUP} and {\t CCTK\_SHUTDOWN}), should include at the top of the file the header \begin{verbatim} #include "cctk_Arguments.h" @@ -1791,7 +1788,7 @@ should include at the top of the file the header A Cactus macro \texttt{CCTK\_ARGUMENTS} is defined for each thorn to contain \begin{itemize} -\item General information about the grid hierarchy, for example the +\item General information about the grid hierarchy, for example, the number of grid points on the processor. See Section \ref{sec:cactus_variables_c} for a complete list. \item All the grid variables defined in the thorn's \texttt{interface.ccl} @@ -1802,23 +1799,23 @@ These variables must be declared at the start of the routine using the macro \texttt{DECLARE\_CCTK\_ARGUMENTS}. This macro should always be the first line of the routine. -To pass the arguments to another routine in the same thorn use the macro +To pass the arguments to another routine in the same thorn, use the macro \texttt{CCTK\_PASS\_CTOC} in the calling routine, and again the macro \texttt{CCTK\_ARGUMENTS} in the receiving routine. Note that you cannot use Cactus argument lists in routines scheduled at the -{\t CCTK\_STARTUP} and {\t CCTK\_SHUTDOWN} time bins because at this time +{\t CCTK\_STARTUP} and {\t CCTK\_SHUTDOWN} time bins, because at this time no grid hierarchy exists. \subsubsection{Parameters} All parameters defined in a thorn's \texttt{param.ccl} and all \texttt{global} -parameters appear as local variables of the corresponding \texttt{CCTK} data type +parameters, appear as local variables of the corresponding \texttt{CCTK} data type in C source code, i.e. Integers and Booleans appear as \texttt{CCTK\_INT} types (with nonzero/zero values for boolean {\t yes/no}), Reals as \texttt{CCTK\_REAL}, and Keywords and String parameters as \texttt{CCTK\_STRING}. -These variables are \emph{read only} and \emph{changes should not be made to +These variables are \emph{read only}, and \emph{changes should not be made to them}. The effect of changing a parameter is undefined (at best). Any routine using Cactus parameters should include at @@ -1925,20 +1922,20 @@ to access its elements. The following variables describe the location of the local grid (e.g.\ the grid treated on a given processor) within the global grid. -\begin{itemize} -\item \texttt{cctk\_lbnd} +\begin{Lentry} +\item [\texttt{cctk\_lbnd}] An array of \texttt{cctk\_dim} integers containing the lowest index (in each direction) of the local grid, as seen on the global grid. Note that these indices start from zero, so you need to add one when using them in Fortran thorns. -\item \texttt{cctk\_ubnd} +\item [\texttt{cctk\_ubnd}] An array of \texttt{cctk\_dim} integers containing the largest index (in each direction) of the local grid, as seen on the global grid. Note that these indices start from zero, so you need to add one when using them in Fortran thorns. -\item \texttt{cctk\_bbox} +\item [\texttt{cctk\_bbox}] An array of 2$*$\texttt{cctk\_dim} integers (in the order $[\mbox{dim}_0^{\mbox{min}}, \mbox{dim}_0^{\mbox{max}}, \mbox{dim}_1^{\mbox{min}}, \mbox{dim}_1^{\mbox{max}}, \ldots]$), @@ -1947,38 +1944,36 @@ the global grid. A value of 1 indicates a physical (outer) boundary at the edge of the computational grid, and 0 indicates an internal boundary. -\end{itemize} +\end{Lentry} The following variable is needed for grid refinement methods -\begin{itemize} -\item \texttt{cctk\_levfac} An array of \texttt{cctk\_dim} integer factors +\begin{Lentry} +\item [\texttt{cctk\_levfac}] An array of \texttt{cctk\_dim} integer factors by which the local grid is refined in the corresponding direction with respect to the base grid. -\item \texttt{cctk\_levoff} and \texttt{cctk\_levoffdenom} Two arrays of +\item [\texttt{cctk\_levoff}] and \texttt{cctk\_levoffdenom} Two arrays of \texttt{cctk\_dim} integers describing the distance by which the local grid is offset with respect to the base grid, measured in local grid spacings. The distance in direction \texttt{dir} is given by \texttt{1.0 * cctk\_levoff[dir] / cctk\_levoffdenom[dir]}. -\item \texttt{cctk\_timefac} The integer factor +\item [\texttt{cctk\_timefac}] The integer factor by which the time step size is reduced with respect to the base grid. -\end{itemize} +\end{Lentry} The following variables are used for identifying convergence levels. -\emph{NOTE:} -Convergence is not currently implemented by Cactus, so that -\texttt{cctk\_convlevel} is currently always $0$. -\begin{itemize} -\item \texttt{cctk\_convlevel} The convergence level of this grid hierarchy. + +\begin{Lentry} +\item [\texttt{cctk\_convlevel}] The convergence level of this grid hierarchy. The base level is $0$, and every level above that is coarsened by a factor of \texttt{cctk\_convfac}. -\item \texttt{cctk\_convfac} The factor between convergence levels. +\item [\texttt{cctk\_convfac}] The factor between convergence levels. The relation between the resolutions of different convergence levels is $\Delta x_L = \Delta x_0 \cdot F^L$, where $L$ is the convergence level and $F$ is the convergence factor. The convergence factor defaults to $2$. -\end{itemize} +\end{Lentry} The variables \texttt{cctk\_delta\_space}, \texttt{cctk\_delta\_time}, and \texttt{cctk\_origin\_space} denote the grid spacings, time step size, @@ -1993,7 +1988,7 @@ incorporate the effects of \texttt{cctk\_levfac}, \texttt{cctk\_levoff}, \texttt{cctk\_levoffdenom}, and \texttt{cctk\_timefac}, so that you do not explicitly have to take them into account. -Putting the above information together, figure~\ref{fig-global-xyz-coords} +Putting the above information together, Figure~\ref{fig-global-xyz-coords} shows two different ways to compute the global Cactus $xyz$ coordinates of the current grid point. Because the ``alternate calculation'' (the one using \verb|Grid::x|, \verb|Grid::y|, and \verb|Grid::z|) gives the @@ -2046,9 +2041,8 @@ int i,j,k; \subsection{Cactus Data Types} -The Cactus grid variables and parameters are defined and -declared using Cactus data types, to provide portability -across platforms. The most important of +To provide portability across platforms, the Cactus grid variables and parameters are defined and +declared using Cactus data types. The most important of these data types are described below, for a full description see Section~\ref{sec:datyansi}. These data types should be used to declare local variables where needed, and to @@ -2065,7 +2059,7 @@ declarations. \subsubsection{Example} -In the following example \verb|MyScalar| is a grid scalar which +In the following example, \verb|MyScalar| is a grid scalar which is declared in the \texttt{interface.ccl} as \texttt{CCTK\_REAL}. % \begin{verbatim} @@ -2084,7 +2078,7 @@ is declared in the \texttt{interface.ccl} as \texttt{CCTK\_REAL}. % Declaring \texttt{local\_var} to have a non-Cactus data type, e.g.\ \texttt{REAL*4}, or using one of the other Cactus real data types -described in Section~\ref{sec:datyansi} could give problems for +described in Section~\ref{sec:datyansi}, could give problems for different architectures or configurations. \subsection{Staggering} @@ -2096,14 +2090,14 @@ grid function: the black solid circles in diagram \ref{fig:stagger2} and their associated staggered grid functions (connected by arrows) have the same index! Since the grid function does not ``know'' anything about the physical -location (it's only addressed by indices) why add staggering if the +location (it's only addressed by indices), why add staggering if the indexing is the same? Indeed, you could roll your own, but there compelling reasons: Readability and the fact that you are able to query the staggertype of a -grid function. More important: in the way the grid is laid out, there is one grid +grid function. More important: In the way the grid is laid out, there is one grid point \emph{less} for \texttt{M} and \texttt{P} staggered grid functions. This is -illustrated in picture \ref{fig:stagger2}, which shows 15 gridpoints distributed +illustrated in Figure \ref{fig:stagger2}, which shows 15 gridpoints distributed across 3 processors. The solid black circles show the default location of the grid functions, the grey circles depict the ghostzones. Note that the number of center staggered gridpoints (fat crosses) @@ -2115,7 +2109,7 @@ The string specifying the staggering is encoded in a number called the \textit{staggerindex}. With the 3 supported staggerings, the string is converted into a base 3 number. Several routines exist to extract the staggering in a specific direction, called \textit{directional -staggerindex}. E.g.\ \texttt{stagger = MCM}: \textit{staggerindex} = 3, in the +staggerindex}. For example, \texttt{stagger = MCM}: \textit{staggerindex} = 3, in the $x$-direction: \textit{directional staggerindex} = \texttt{CCTK\_STAGGER\_M} (value 0), in the $y$-direction: \textit{directional staggerindex} = \texttt{CCTK\_STAGGER\_C} (value 1). @@ -2140,24 +2134,24 @@ $y$-direction: \textit{directional staggerindex} = \texttt{CCTK\_STAGGER\_C} (va ghostzone size of one (above) and two (below). The points are distributed across three processors. Note that the number of center staggered gridpoints (fat crosses) is one less on the outermost grid. How to -treat this case in a easy way is explained below. } +treat this case in an easy way is explained in the text. } \label{fig:stagger2} \end{figure} When a thorn programmer uses staggered gridpoints, he has to be aware of this gridpoint anomaly. This can be done most easily by using the \texttt{CCTK\_LSSH(<\var{dir\_staggertype}>,<\var{direction}>)} macro. -For a given staggertype and direction, this 2d array returns the local +For a given staggertype and direction, this 2D array returns the local number of gridpoints, including ghostzones and the necessary change for the staggering on the outermost processor. \begin{Lentry} -\item[\texttt{CCTL\_LSSH(<\var{dir\_staggertype}>,<\var{direction}>)}] -for a given staggertype and a direction this macro returns the number +\item[\texttt{CCTK\_LSSH(<\var{dir\_staggertype}>,<\var{direction}>)}] +for a given staggertype and a direction, this macro returns the number of processor local gridpoints, including ghostzones. \begin{itemize} -\item{macro has to be in capital letters}\item{This macro is C/Fortran indexing aware: +\item{For Fortran users, the macro has to be in capital letters}\item{This macro is C/Fortran indexing aware: can specify the dimension in C ranging from $0 \ldots$ and in Fortran ranging from $1 \ldots$.} \end{itemize} @@ -2234,19 +2228,19 @@ staggerindex. \section{Parallelisation} \label{sec:parallelisation} -The Flesh itself does not actually set up grid variables. This +The flesh itself does not actually set up grid variables. This is done by a \textit{driver} thorn. To allow the distribution of a grid over a number of processors, the driver thorn must also provide the grid decomposition, and routines to enable parallelisation. The method used to provide this parallelisation -(e.g.\ MPI, PVM) is not usually important for the thorn writer since +(e.g.\ MPI, PVM) is not usually important for the thorn writer, since the driver thorn provides routines which are called by standard interfaces -from the Flesh. Here we describe briefly the most important of these routines +from the flesh. Here, we describe briefly the most important of these routines for the application thorn writer. A more detailed description of these interfaces with their arguments, is given in the Reference Manual. A complete description of the -routines a driver thorn must provide will be provided in the -Infrastructure Thorn Writers guide (Part \ref{part:Infrastructure}). The standard driver thorn is +routines that a driver thorn must provide, will be provided in the +Infrastructure Thorn Writers Guide (Part \ref{part:Infrastructure}). The standard driver thorn is currently \texttt{PUGH} in the \texttt{CactusPUGH} package, which is a parallel unigrid driver. @@ -2256,7 +2250,7 @@ is a parallel unigrid driver. processor number zero) \item[\texttt{CCTK\_SyncGroup, CCTK\_SyncGroupsI}] Synchronises either a single group or a set of groups of grid arrays by - exchanging the values held in each processor ghostzones with the + exchanging the values held in each processor ghostzones, with the physical values of their neighbours (see the Reference Manual) \item[\texttt{CCTK\_Barrier}] Waits for all processors to reach this point before proceeding @@ -2272,20 +2266,16 @@ is a parallel unigrid driver. \section{Coordinates} \label{sec:CactusAPI.coordinates} -The Flesh provides utility routines for registering and querying -coordinate information. The Flesh does not provide any coordinates +The flesh provides utility routines for registering and querying +coordinate information. The flesh does not provide any coordinates itself, these must be supplied by a thorn. Thorns are not required to -register coordinates to the Flesh, but registering coordinates +register coordinates to the flesh, but registering coordinates provides a means for infrastructure thorns to make use of coordinate information. -Coordinate support is still being developed in the Cactus Flesh. At -the moment, it is assumed that coordinates will usually be grid -functions. - Coordinates are grouped into \textit{coordinate systems}, which have a specified dimension. Any number of coordinate systems can be -registered with the Flesh, and a coordinate system must be registered +registered with the flesh, and a coordinate system must be registered before any coordinates can be registered, since they must be associated with their corresponding system. Coordinates can be registered, with any chosen name, with an existing coordinate system, @@ -2294,7 +2284,7 @@ Optionally, the coordinate can also be associated with a given grid variable. A separate call can register the global range for a coordinate on a given grid hierarchy. -Following conventions for coordinate system and coordinate names +Following conventions for coordinate system and coordinate names, provides a means for other thorns to use the physical properties of coordinate systems, without being tied to a particular thorn. @@ -2310,7 +2300,7 @@ probably be phased out fairly soon. New code should use the APIs provided by the \texttt{CoordBase} thorn instead (this lives in the \texttt{CactusBase} arrangement).} -Coordinate systems and their properties can be registered at any time with the Flesh. +Coordinate systems and their properties can be registered at any time with the flesh. The registration utilities for thorns providing coordinates are: \begin{Lentry} @@ -2369,7 +2359,7 @@ ierr = CCTK_CoordRegisterRange(cctkGH, lower, upper, 2, NULL, "cart3d"); Implementing such things as symmetry properties for a grid leads to the need to know the details of the \emph{physical} section of a grid. -Such information is typically needed by IO thorns. The following call +Such information is typically needed by I/O thorns. The following call illustrates how to register the indices 3 and 25 as supplying the physical range of the \texttt{y} coordinate in the \texttt{cart3d} system @@ -2398,7 +2388,7 @@ The utilities for thorns using coordinates are: \item[\texttt{CCTK\_NumCoordSystems}] -Returns the number of coordinate systems registered with the Flesh. For example, +Returns the number of coordinate systems registered with the flesh. For example, % \begin{verbatim} int num; @@ -2408,7 +2398,7 @@ num = CCTK_NumCoordSystems(); \item[\texttt{CCTK\_CoordSystemName}] Provides the name of a registered coordinate system, given the integer -handle (or index) for the system in the Flesh's coordinate data base. +handle (or index) for the system in the flesh's coordinate data base. Note that the handle ranges between zero and the number of coordinate systems minus one: $0 \le \mbox{handle} \le \mbox{\texttt{CCTK\_NumCoordSystems()}}-1$. It is important to remember that the handle given to a coordinate system depends on the order in which systems are registered, and can be different @@ -2435,7 +2425,7 @@ dim = CCTK_CoordSystemDim("cart3d"); \item[\texttt{CCTK\_CoordSystemHandle}] Provides the integer handle for a given coordinate system name. The handle describes -the index for the coordinate system in the Flesh coordinate database, and its value +the index for the coordinate system in the flesh coordinate database, and its value will range between zero and the number of registered systems minus one. For example, the handle for the \texttt{cart3d} coordinate system can be found using % @@ -2446,8 +2436,8 @@ handle = CCTK_CoordSystemHandle("cart3d"); \item[\texttt{CCTK\_CoordSystemName}] -The inverse to the previous function call, this provides the name for a given coordinate system handle. -For example to find the first coordinate system in the Flesh database +The inverse to the previous function call. This provides the name for a given coordinate system handle. +For example, to find the first coordinate system in the flesh database % \begin{verbatim} int handle = 0; @@ -2458,7 +2448,7 @@ const char *name = CCTK_CoordSystemName(handle); Provides the grid variable index for a given coordinate. Note that it is not necessary for a registered coordinate to have an associated grid variable, -and if no such grid variable is found a negative integer will be returned. +and if no such grid variable is found, a negative integer will be returned. For example, to find the grid variable index associated with the \texttt{y} coordinate of the \texttt{cart3d} system, either of the two following calls could be made @@ -2492,8 +2482,7 @@ could not be found. \item[\texttt{CCTK\_CoordRange}] Provides the global range (that is, the minimum and maximum values across -the complete grid) of a coordinate on a given grid hierarchy. Currently -the minimum and maximum values must be of type \texttt{CCTK\_REAL}. The +the complete grid) of a coordinate on a given grid hierarchy. The minimum and maximum values must be of type \texttt{CCTK\_REAL}. The coordinate can be specified either by name or by its direction. Note that this call takes the \emph{addresses} of the minimum and maximum values. For example, the range of the \texttt{y} coordinate of the \texttt{cart3d} @@ -2516,7 +2505,7 @@ ierr = CCTK_CoordRange(cctkGH, &lower, &upper, 2, NULL, "cart3d"); \item[\texttt{CCTK\_CoordLocalRange}] Provides the local range of a coordinate on a processor for a given -grid hierarchy. WARNING: This utility only currently works for regular +grid hierarchy. WARNING: This utility only works for regular cartesian grids. For example, the local processor range of the \texttt{y} coordinate of the \texttt{cart3d} coordinate system can be found using % @@ -2559,7 +2548,7 @@ ierr = CCTK_CoordRangePhysIndex(cctkGH,&ilower,&iupper, 2, NULL, "cart3d"); \item[\texttt{CCTK\_CoordSystemImplementation}] This call returns the name of the implementation which registered a coordinate system. -Note that there is no guarantee that a thorn which registered a coordinate system is +Note that there is no guarantee that a thorn, which registered a coordinate system, is the same thorn which registers each of the coordinates in the system, although this should usually be the case. @@ -2567,50 +2556,50 @@ should usually be the case. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -\section{IO} +\section{I/O} \label{sec:io} -To allow flexible IO, the Flesh itself does not provide any output +To allow flexible I/O, the flesh itself does not provide any output routines, however it provides a mechanism for thorns to register -different routines as IO methods (see chapter \ref{chap:io_methods}). -Application thorns can interact with the different IO methods through +different routines as I/O methods (see Chapter \ref{chap:io_methods}). +Application thorns can interact with the different I/O methods through the following function calls: \begin{description} \item \texttt{CCTK\_OutputGH (const cGH *\var{GH})} -This call loops over all registered IO methods, calling the routine +This call loops over all registered I/O methods, calling the routine that each method has registered for {\t OutputGH}. The expected behaviour of any {\t OutputGH} routine is to loop over all GH -variables outputting them if the IO method contains appropriate +variables, outputting them if the I/O method contains appropriate routines (that is, not all methods will supply routines to output all -different types of variables) and if the method decides it is an +different types of variables), and if the method decides it is an appropriate time to output. \item \texttt{CCTK\_OutputVar (const cGH *\var{GH}, const char *\var{varname})} -Output a variable \var{varname} looping over all registered IO methods. -\var{varname} may have an optional I/O options string appended. +Outputs a variable \var{varname} looping over all registered I/O methods. +\var{varname} may have an optional I/O option string appended. The output should take place if at all possible. If output goes into -a file and the appropriate file exists the data is appended, otherwise +a file and the appropriate file exists, the data is appended, otherwise a new file is created. \item \texttt{CCTK\_OutputVarAs (const cGH *\var{GH}, const char *\var{varname}, const char *\var{alias})} -Output a variable \var{varname} looping over all registered IO methods. -\var{varname} may have an optional I/O options string appended. +Outputs a variable \var{varname} looping over all registered I/O methods. +\var{varname} may have an optional I/O option string appended. The output should take place if at all possible. If output goes into -a file and the appropriate file exists the data is appended, otherwise +a file and the appropriate file exists, the data is appended, otherwise a new file is created. Uses \var{alias} as the name of the variable for the purpose of constructing a filename. \item \texttt{CCTK\_OutputVarByMethod (const cGH *\var{GH}, const char *\var{varname}, const char *\var{methodname})} -Output a variable \var{varname} using the IO method \var{methodname} if it is -registered. \var{varname} may have an optional I/O options string appended. +Outputs a variable \var{varname} using the I/O method \var{methodname} if it is +registered. \var{varname} may have an optional I/O option string appended. The output should take place if at all possible. If -output goes into a file and the appropriate file exists the data is +output goes into a file and the appropriate file exists, the data is appended, otherwise a new file is created. \item \texttt{CCTK\_OutputVarAsByMethod (const cGH *\var{GH}, @@ -2618,10 +2607,10 @@ appended, otherwise a new file is created. const char *\var{methodname},\\ const char *\var{alias})} -Output a variable \var{varname} using the IO method \var{methodname} if it is -registered. \var{varname} may have an optional I/O options string appended. +Outputs a variable \var{varname} using the I/O method \var{methodname} if it is +registered. \var{varname} may have an optional I/O option string appended. The output should take place if at all possible. -If output goes into a file and the appropriate file exists the data is +If output goes into a file and the appropriate file exists, the data is appended, otherwise a new file is created. Uses \var{alias} as the name of the variable for the purpose of constructing a filename. @@ -2632,15 +2621,15 @@ name of the variable for the purpose of constructing a filename. \section{Interpolation Operators} \label{sec:inop} -The Flesh does not provide interpolation routines by itself. Instead -it offers a general function API to thorns for the registration and +The flesh does not provide interpolation routines by itself. Instead, +it offers a general function API to thorns, for the registration and invocation of interpolation operators. -There are two different Flesh APIs for interpolation, depending +There are two different flesh APIs for interpolation, depending on whether the data arrays are Cactus grid arrays or processor-local, programming language built-in arrays, and on what assumptions are made about the topology and spacing of the grid (these descriptions -are for 3-D, but the generalizations to other numbers of dimensions +are for 3D, but the generalisations to other numbers of dimensions should be obvious): \begin{Lentry} \item[\texttt{CCTK\_InterpGridArrays()}] @@ -2690,9 +2679,9 @@ should be obvious): %notyet by providing 3-D arrays giving their values at the grid points. \end{Lentry} -%notyet There are separate Flesh routines to register interpolation operators for +%notyet There are separate flesh routines to register interpolation operators for %notyet the APIs: -The Flesh provides an API to register local interpolation operators: +The flesh provides an API to register local interpolation operators: \begin{Lentry} \item[\texttt{CCTK\_InterpRegisterOpLocalUniform()}] Register a \verb|CCTK_InterpLocalUniform()| interpolation operator @@ -2705,7 +2694,7 @@ The Flesh provides an API to register local interpolation operators: This is described in detail in the Reference Manual. Each local interpolation operator is registered under a character string name; -at registration the name is mapped to a unique integer handle which +at registration, the name is mapped to a unique integer handle, which may be used to refer to the operator. \verb|CCTK_InterpHandle()| is used to get the handle corresponding to a given character string name. @@ -2728,15 +2717,15 @@ name. A reduction operation can be defined as an operation on variables distributed across multiple processor resulting in a single number. -Typical reduction operations are: sum, minimum/maximum value, boolean +Typical reduction operations are: sum, minimum/maximum value, and boolean operations. A typical application is, for example, finding the maximum reduction from processor local error estimates, -therefor making the previous processor local error known to all processors. +therefore, making the previous processor local error known to all processors. The exchange of information across processors needs the functionality of a -communication layer e.g. \texttt{CactusPUGH/PUGH}. For this reason, the -reduction operation itself is not part of the flesh, instead Cactus (again) +communication layer, e.g. \texttt{CactusPUGH/PUGH}. For this reason, the +reduction operation itself is not part of the flesh, instead, Cactus (again) provides a registration mechanism for thorns to register routines they provide as reduction operators. The different operators are identified by their name and/or a unique number, called a {\em handle}. @@ -2752,17 +2741,17 @@ the objects they can be applied to. There is a fundamental difference between the reduction operation on grid functions and quantities as arrays. -Currently the Flesh supports the new and old reduction specification. +Currently the flesh supports the new and old reduction specification. The old APIs will be deprecated in the next beta cycle in favour of the new specification. {\bf New Reduction Specification API documentation} \vskip .25cm -In the new reduction specification, there are two different Flesh APIs for reduction, depending +In the new reduction specification, there are two different flesh APIs for reduction, depending on whether the data arrays are Cactus grid arrays or processor-local, programming language built-in arrays, and on what assumptions are made about the topology and spacing of the grid (these descriptions -are for 3-D, but the generalisations to other numbers of dimensions +are for 3D, but the generalisations to other numbers of dimensions should be obvious): \begin{Lentry} \item[\texttt{CCTK\_ReduceGridArrays()}] @@ -2779,7 +2768,7 @@ should be obvious): \emph{offsets}, \emph{strides} and \emph{masks}. \end{Lentry} -The Flesh provides an API to register local reduction operators: +The flesh provides an API to register local reduction operators: \begin{Lentry} \item[\texttt{CCTK\_RegisterLocalArrayReductionOperator()}] Register a \verb|CCTK_ReduceLocalArrays()| interpolation operator @@ -2787,18 +2776,18 @@ The Flesh provides an API to register local reduction operators: This is described in detail in the Reference Manual. Each local reduction operator is registered under a character string name; -at registration the name is mapped to a unique integer handle which +at registration, the name is mapped to a unique integer handle, which may be used to refer to the operator. \verb|CCTK_LocalArrayReductionHandle()| is used to get the handle corresponding to a given character string name. -{\bf Old Reduction Specification API documentation} +{\bf Old Reduction Specification API Documentation} \vskip .25cm {\bf Obtaining the reduction handle} Before calling the routine which performs the reduction operation, -the handle, which identifies the operation must be derived from its +the handle, which identifies the operation, must be derived from its registered name. \begin{verbatim} @@ -2823,15 +2812,15 @@ call CCTK_ReductionArrayHandle(reduction_handle, reduction_name) \begin{Lentry} \item[\texttt{reduction\_handle}] -In Fortran the name of the variable will -contain the handle value after the call. In C this value is the +in Fortran, the name of the variable will +contain the handle value after the call. In C, this value is the function value. \item[\texttt{reduction\_name}] is the name under which the operator has been registered by the providing thorn. The only thorn in the standard -Computational Toolkit release which provides reduction operators is +Computational Toolkit release, which provides reduction operators, is \texttt{CactusPUGH/PUGHReduce}. -\item[{\bf error checking}] +\item[error checking] negative handle value indicates failure to identify the correct operator. \end{Lentry} @@ -2841,22 +2830,22 @@ The operator is identified by the name it was registered with. (Note that although it would appear to be far more convenient to pass the name of the reduction operator directly to the following function call to {\t CCTK\_Reduce} this causes problems with the -translation of strings from {\t FORTRAN} to {\t C} with variable +translation of strings from Fortran to C with variable argument lists). \vskip 0.25cm -{\bf The general reduction interface} +{\bf The general reduction interface.} The main interfaces for reduction operations are quite powerful (and -hence rather complicated) To ease the use of these main interfaces, wrappers +hence rather complicated). To ease the use of these main interfaces, wrappers designed for specific and more restricted use are described below. If uncertain, you should use these. {\t \begin{verbatim} -int CCTK_Reduce( cGH *GH, +int CCTK_Reduce( const cGH *GH, int proc, int operation_handle, int num_out_vals, @@ -2876,7 +2865,7 @@ call CCTK_Reduce( int returnvalue, int num_in_fields, ... ) -int CCTK_ReduceArray( cGH *GH, +int CCTK_ReduceArray( const cGH *GH, int proc, int operation_handle, int num_out_vals, @@ -2903,53 +2892,48 @@ call CCTK_ReduceArray(int returnvalue, \begin{Lentry} \item[\texttt{int returnvalue}] -the return value of the operation. negative +the return value of the operation. Negative value indicates failure to perform reduction. -Zero indicates successful operation. +Zero indicates a successful operation. \item[\texttt{cctkGH}] -in Fortran the pointer to the grid hierarchy -structure. Cannot be used within Fortran but +in Fortran, the pointer to the grid hierarchy +structure. Can not be used within Fortran, but can be used from within -C. Since this name is fixed write it out shown -above. +C. Since this name is fixed, write it out as shown. \item[\texttt{cGH *GH}] - in the C the pointer to the grid hierarchy. + in C, it is the pointer to the grid hierarchy. \item[\texttt{int processor}] the processor which collects the -information, negative value ($-1$) will distribute the data to all +information, a negative value ($-1$) will distribute the data to all processors. \item[\texttt{int operation\_handle}] the number of the reduction operation handle, needs to be found by calling \texttt{CCTK\_ReductionHandle} or \texttt{CCTK\_ReductionArrayHandle}. -\item[\texttt{int num\_out\_vals}] integer defining the number of ?? +\item[\texttt{int num\_out\_vals}] integer defining the number of output values. \item[\texttt{int type\_out\_arrays}, \texttt{type\_in\_arrays}] specifies the type of the gridfunction -you are communicating. Use the values as specified in -\ref{sec:datyansi}. Note: do not {\em mix} datatypes: e.g. in -Fortran do not declare a variable as \texttt{integer} and then specify +you are communicating. Use the values as specified in Section +\ref{sec:datyansi}. Note: Do not mix data types, e.g. in +Fortran, do not declare a variable as \texttt{integer} and then specify the type \texttt{CCTK\_VARIABLE\_INT} in the reduction command. These types need not be the same on some architectures and will conflict. -\item[\texttt{out\_vals}] -\item[\texttt{int num\_in\_fields}] specifies the number of to follow -\item[\texttt{...}] indicates a variable argument list: specify the size -of the array in each dimension, comma separated numbers or integer -variables. Specify the arrays which will be reduced, the number of +\item[\texttt{out\_vals}] an array that will contain the output values. +\item[\texttt{int num\_in\_fields}] specifies the number of input fields. +\item[\texttt{...}] indicates a variable argument list: specify the arrays which will be reduced, the number of specified arrays must be the same as the value of the {\tt num\_in\_fields} variable. -\item[{\bf error checking}] a return value other than zero indicates +\item[{\bf error checking}] a return value, other than zero, indicates failure to perform the operation. \end{Lentry} \vskip 0.25cm -{\bf Special reduction interfaces} - -The routines are designed for the purpose of reducing scalars, arrays +{\bf Special reduction interfaces.} The routines are designed for the purpose of reducing scalars, arrays and grid functions. They hide many of the options of the generic -interface described above.\\ +interface described above. -{\bf Reduction of local scalars} across multiple processors, the result of +{\bf Reduction of local scalars across multiple processors.} The result of the reduction operation will be on the specified processor or on all processors. {\t @@ -2973,17 +2957,17 @@ call CCTK_ReduceLocScalar(int returnvalue, \begin{Lentry} \item[\texttt{in\_scalar}] the processor local variable with local value to be reduced \item[\texttt{out\_scalar}] the reduction result: a processor local variable -with the global value (same on all procs) if \texttt{processor} has been -set to $-1$. Otherwise \texttt{processor} will hold the reduction result. +with the global value (same on all processors), if \texttt{processor} has been +set to $-1$. Otherwise, \texttt{processor} will hold the reduction result. \item[\texttt{data\_type}] specifies the type of the gridfunction -you are communicating. Use the values as specified in +you are communicating. Use the values as specified in Section \ref{sec:datyansi}. \end{Lentry} \vskip 0.25cm -{\bf Reduction of local 1d arrays} to a local arrays, this reduction is carried +{\bf Reduction of local 1d arrays to a local arrays.} This reduction is carried out element by element. The arrays need to have the same size on all processors. {\t @@ -3008,7 +2992,7 @@ call CCTK_ReduceLocArrayToArray1D(int returnvalue } \begin{Lentry} -\item[\texttt{in\_array1d}] one dimensional local arrays, to be reduced across a +\item[\texttt{in\_array1d}] one dimensional local arrays to be reduced across a processors, element by element. \item[\texttt{out\_array1d}] array holding the reduction result. out\_array1d[1] = Reduction(in\_array[1]). @@ -3017,7 +3001,7 @@ processors, element by element. \vskip 0.25cm -{\bf Reduction of local 2d arrays} to a local 2d array, this reduction is carried +{\bf Reduction of local 2d arrays to a local 2d array.} This reduction is carried out element by element. The arrays need to have the same size on all processors. {\t @@ -3056,7 +3040,7 @@ result. out\_array2d[i,j]= Reduction(in\_array2d[i,j]). \vskip 0.25cm -{\bf Reduction of local 3d arrays} to a local 3d array, this reduction is carried +{\bf Reduction of local 3D arrays to a local 3D array.} This reduction is carried out element by element. The arrays need to have the same size on all processors. {\t @@ -3122,8 +3106,8 @@ processors. This quantity can then be reassigned to \texttt{normerr}. \end{verbatim} -{\bf Reduction of a local 2d array:} a two dimensional array $(2x3)$ is -reduced, reduction results (array of same size: \texttt{bla\_tmp}) is seen +{\bf Reduction of a local 2D array:} a two dimensional array $(2\times3)$ is +reduced, reduction results (array of same size: \texttt{bla\_tmp}) are seen on all processors ($-1$ entry as the third argument); also demonstrates some simple error checking with the \texttt{CCTKi\_EXPECTOK} macro. \begin{verbatim} @@ -3157,11 +3141,11 @@ the reduction call is made. Note that since most source files (see Section~\ref{nacofosofi} for exceptions) pass through a C preprocessor, C style comments can be -used in Fortran code. (Note that C++ comments (that is ones starting +used in Fortran code. Note that C++ comments (those ones starting with ``\texttt{//}''), -should only be used in C++ source code). +should only be used in C++ source code. -The Flesh and the Cactus thorns use the \texttt{grdoc} Code Documenting +The flesh and the Cactus thorns use the \texttt{grdoc} Code Documenting System\\(\url{http://jean-luc.aei.mpg.de/Codes/grdoc/}) to document source code. @@ -3170,7 +3154,7 @@ source code. \section{Providing Runtime Information} \label{sec:prrutiin} -To write from thorns to standard output (\textit{i.e.} the screen) +To write from thorns to standard output (i.e. the screen) at runtime, use the macro \texttt{CCTK\_INFO} or the function \texttt{CCTK\_VInfo()}. For example, from the Fortran thorn \texttt{MyThorn}, @@ -3188,8 +3172,8 @@ will be printed to screen by default. The standard output of other processors will usually be discarded unless the ``\texttt{-r}'' command line option is used (Section~\ref{sec:command_line_options}). -Note that the routine \texttt{CCTK\_VInfo()} can only be called from C because -Fortran doesn't know about variable argument lists. So including variables in +Note that the routine \texttt{CCTK\_VInfo()} can only be called from C, because +Fortran doesn't know about variable argument lists. So, including variables in the info message using \texttt{CCTK\_INFO} is currently more tricky, since you need to build the string to be output. @@ -3200,7 +3184,7 @@ For example, in C you would just write CCTK_VInfo(CCTK_THORNSTRING, "The integer is %d", myint); \end{verbatim} -But in Fortran you have to do the following: +But in Fortran you have to do the following \begin{verbatim} integer myint character*200 message @@ -3209,7 +3193,7 @@ But in Fortran you have to do the following: call CCTK_INFO (message) \end{verbatim} -In Fortran 90, you can also do: +In Fortran 90, you can also do \begin{verbatim} integer myint character(200) message @@ -3218,7 +3202,7 @@ In Fortran 90, you can also do: call CCTK_INFO (message) \end{verbatim} -Note that: +Note that \begin{itemize} \item{} \texttt{CCTK\_INFO} is just a macro which expands to a call to the internal function \texttt{CCTK\_Info()} and automatically includes @@ -3233,22 +3217,22 @@ Note that: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -\section{Error handling, warnings and code termination} +\section{Error Handling, Warnings and Code Termination} \sectionmark{Error handling, ...} \label{sec:erhawancote} -The Cactus function \texttt{CCTK\_VWarn()} and its accompanying -\texttt{CCTK\_WARN} macro should be used to issue warning messages +The Cactus function \texttt{CCTK\_VWarn()}, and its accompanying +\texttt{CCTK\_WARN} macro, should be used to issue warning messages during code execution. Along with the warning message, an integer is given to indicate the severity of the warning. Only warnings with severity levels less -than or equal to the global Cactus warning level threshold%%% +than, or equal to, the global Cactus warning level threshold%%% \footnote{%%% - As discussed in section~\ref{sec:command_line_options} + As discussed in Section~\ref{sec:command_line_options} of this manual, the Cactus warning level threshold is set with the \texttt{-W} or \texttt{-warning-level} command-line option when running Cactus; see - section~\ref{sec:command_line_options}. + Section~\ref{sec:command_line_options}. }%%% {} will be printed. A level~0 warning indicates the highest severity (and is guaranteed to abort the Cactus run), while larger numbers @@ -3291,30 +3275,29 @@ and from C \end{verbatim} Note that \texttt{CCTK\_WARN} is just a macro which expands to a call -to an internal function. The macro automatically includes the thorn name -and the source code file name and line number in the message.%%% +to an internal function. The macro automatically includes the thorn name, the source code file name and line number in the message.%%% \footnote{%%% - In calling \texttt{CCTK\_VWarn()} you need to + In calling \texttt{CCTK\_VWarn()}, you need to provide this information yourself. Cactus - provides the macro \texttt{CCTK\_THORNSTRING} + provides the macro \texttt{CCTK\_THORNSTRING}, which is the character-string name of the current thorn. In C, you can get the source file name and line number from the predefined preprocessor macros \texttt{\_\_FILE\_\_} and - \texttt{\_\_LINE\_\_} respectively. In Fortran + \texttt{\_\_LINE\_\_}, respectively. In Fortran you're out of luck. :( }%%% {} (For this reason it is important for Fortran code that capital letters are always used in order to expand the macro.) -If the Flesh parameter \texttt{cctk\_full\_warnings} is set to true, then the +If the flesh parameter \texttt{cctk\_full\_warnings} is set to true, then the source file name and line number will be printed to standard error along with the originating processor number, the thorn name and the warning message. The default is to omit the source file name and line number. -Note that the routine \texttt{CCTK\_VWarn()} can only be called from C because +Note that the routine \texttt{CCTK\_VWarn()} can only be called from C, because Fortran doesn't know about variable argument lists. So including variables in -the warning message using \texttt{CCTK\_WARN} is currently more tricky, since +the warning message using \texttt{CCTK\_WARN}, is currently more tricky, since you need to build the string to be output. For example, in C you would just write @@ -3327,7 +3310,7 @@ For example, in C you would just write myreal, myint); \end{verbatim} -But in Fortran you have to do the following: +But in Fortran you have to do the following \begin{verbatim} integer myint real myreal @@ -3337,7 +3320,7 @@ But in Fortran you have to do the following: call CCTK_WARN (CCTK_WARN_ALERT, message) \end{verbatim} -In Fortran 90, you can also do: +In Fortran 90, you can also do \begin{verbatim} integer myint real myreal @@ -3348,9 +3331,9 @@ In Fortran 90, you can also do: \end{verbatim} Besides the default methods to handle warning and information -messages, the Flesh also implements a callback scheme to let thorn -writers get information and warning messages as they are produced. \footnote{These -functions can, for the moment, only be used from C.} +messages, the flesh also implements a callback scheme to let thorn +writers get information and warning messages as they are produced.\footnote{For the moment, these +functions can only be used from C.} For warning messages, a function with the following prototype \begin{verbatim} @@ -3361,7 +3344,7 @@ void my_warnfunc(int level, const char *message, void *data); \end{verbatim} -should be implemented and then registered with +should be implemented, and then registered with \begin{verbatim} CCTK_WarnCallbackRegister(int minlevel, int maxlevel, @@ -3374,7 +3357,7 @@ registered function, e.g. a file descriptor or a format string. Multiple functions can be registered as above; when \texttt{CCTK\_VWarn()} is called, all the registered functions will be -called if the warning is within the minimum and maximum levels +called, if the warning is within the minimum and maximum levels indicated. The basic procedure is exactly the same for information messages. @@ -3392,15 +3375,15 @@ CCTK_InfoCallbackRegister(void *data, cctk_infofunc my_infofunc); %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -\section{Adding documentation} +\section{Adding Documentation} \label{sec:Adding_documentation} Documentation is a vital part of your thorn, helping to ensure its -ease of use and longevity, not only for others but also for the thorn +ease of use and longevity, not only for others, but also for the thorn authors. Although any kind of independent documentation can be added to a thorn (ideally in the \texttt{doc} directory), there are two standard places for adding thorn documentation, a \texttt{README} and a -file \texttt{doc/documentation.tex} for including in Thorn Guides. +\texttt{doc/documentation.tex} file for including in Thorn Guides. \subsection{\texttt{README}} @@ -3410,7 +3393,7 @@ copyright details, and a synopsis of what the thorn does. \subsection{Contribution to Thorn Guide} -The LaTeX file \texttt{doc/documentation.tex} is included in Thorn Guides +The LaTeX file, \texttt{doc/documentation.tex}, is included in Thorn Guides built by the Cactus make system. (e.g.\ by \texttt{gmake $<$\var{config}$>$-ThornGuide}). Ideally this file should contain complete (and \textit{up-to-date}) details about the thorn, exactly what is @@ -3421,29 +3404,29 @@ sections include: \begin{itemize} - \item{\bf model} A description of the system which the thorn is modelling, - including the equations etc.\ which are being solved or implemented. + \item{\bf Model.} A description of the system which the thorn is modelling, + including the equations, etc., which are being solved or implemented. - \item{\bf numerical implementation} Details about how the model is + \item{\bf Numerical implementation.} Details about how the model is numerically implemented in the thorn. - \item{\bf using the thorn} Any special details needed for using the + \item{\bf Using the thorn.} Any special details needed for using the thorn, tricky parameters, particular operating systems or additional required software, interactions with other thorns and examples of use. - \item{\bf history} Here is where you should describe why the thorn + \item{\bf History.} Here is where you should describe why the thorn was written, any previous software or experience which was made use of, the authors of the thorn source code and documentation, how to get - hold of the thorn etc. + hold of the thorn, etc. - \item{\bf references} A bibliography can be included, referencing papers + \item{\bf References.} A bibliography can be included, referencing papers published using or about this thorn, or additional information about the model or numerics used. \end{itemize} A LaTeX template for the Thorn Guide documentation can be found in the -Flesh distribution at +flesh distribution at \texttt{doc/ThornGuide/template.tex}, @@ -3457,7 +3440,7 @@ documentation: \begin{itemize} - \item Use the Cactus Thorn Guide style file, located in the Flesh + \item Use the Cactus Thorn Guide style file, located in the flesh distribution at \texttt{doc/latex/cactus.sty}. This should be included using a relative link, so that updates to the style file are applied to all thorns. @@ -3484,7 +3467,7 @@ and \item Do not use the {\t $\backslash$appendix} command, instead include any appendices you have as standard sections. - \item Currently we only support PostScript (\texttt{.eps or .ps}) figures. + \item We only support PDF (\texttt{.pdf}) figures. Graphics figures should be included using the \texttt{includegraphics} command (not \texttt{epsffig}), with no file extension specified. For example, % @@ -3521,7 +3504,7 @@ and %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -\section{Adding a test suite} +\section{Adding a Test Suite} \label{sec:adding_test_suite} To add a test suite to your thorn, devise a series of parameter @@ -3562,22 +3545,22 @@ TEST <\var{test_example}> \} \end{alltt} -which states that when comparing files of test \verb|test_example|, -\verb|absolute_tol| and \verb|relative_tol| {\space} should be used as +which states that when comparing files of test \verb|test_example|, both +\verb|absolute_tol| and \verb|relative_tol| should be used as the absolute and relative tolerances. For all other tests in the thorn, the default value of absolute and relative tolerances are set to \verb|thorn_absolute_tolerance| and \verb|thorn_relative_tolerance|. The \texttt{NPROCS} option specifies the number of processors required to run a given testsuite \var{test\_example} or all testsuites of a thorn successfully. If no \texttt{NPROCS} option is present, the testsuite(s) -are assumed to run with any number of processors. +is (are) assumed to run with any number of processors. The \texttt{EXTENSIONS} option adds \verb|extension_1|, \verb|extension_2| and \verb|extension_3| to the list of file extensions that are compared. This list is global over all tests in a configuration. Test specific tolerances have precedence over all tolerances, next -come thorn wide tolerances and then cactus default tolerances. +come thorn wide tolerances, and then cactus default tolerances. Absolute and relative tolerances are independent: you can choose to use test specific absolute tolerance and thorn specific relative tolerance when running a test. For example, @@ -3598,15 +3581,15 @@ and a relative tolerance of $10^{-12}$ when running all other tests. For details on running the test suites, see Section~\ref{sec:testing}. -\subsection{Best practices for test suites} +\subsection{Best Practices for Test Suites} When writing a test suite, there are a few things you should keep in mind: \begin{itemize} \item The test suite will be run together with many other test - suites. It should therefore finish quickly (say, in under two - minutes) and not use too much memory (so that it can run on a + suites. It should, therefore, finish quickly (say, in under two + minutes), and not use too much memory (so that it can run on a ``normal'' workstation). \item The test suite will be run automatically, often in situations where no one checks the screen output. All important output should @@ -3618,18 +3601,18 @@ mind: normally the output files should be small, so that there are not more than a few Megabytes of output files per test suite. \item The test suite output files should always be the same. That - means that they should not contain no time stamps etc. It is - therfore best to use the option \verb|IO::out_fileinfo="none"|. + means that they should not contain time stamps, etc. It is, + therefore, best to use the option \verb|IO::out_fileinfo="none"|. \item Norms are unfortunately quite insensitive to changes to a few grid points only, even if the changes are significant. It is necessary to output grid point values directly, not only norms. -\item Try to use as few thorns as possible in a test case. E.g., do +\item Try to use as few thorns as possible in a test case. For example, do not active 3D output thorns (unless you use it). The fewer thorns you use, the easier it is to run the test suite. \item It is not necessary that a test suite result is ``physically correct'', or that it uses parameters that ensure a stable time evolution. A test suite will usually take only a few time steps, so - that a grid size of e.g.\ $20^3$ grid points \emph{without} + that a grid size of, e.g.\ $20^3$ grid points \emph{without} dissipation can be sufficient. Test suites cannot test whether the result of a simulation is physically feasible; they only test whether anything changed at all. Ensuring that the physics is still @@ -3646,14 +3629,14 @@ mind: \section{Using Cactus Timers} \label{sec:timers} -\subsection{What are timers?} +\subsection{What are Timers?} %The standard timing information available during a simulation was %described in Section~\ref{????}. %[FIXME: This isn't filled in yet because it is being reworked] Cactus provides a flexible mechanism for timing different sections of your -thorns using various \textit{clocks} which have been registered with the Flesh. -By default, the Flesh provides two clocks that measure time in different +thorns using various \textit{clocks} which have been registered with the flesh. +By default, the flesh provides two clocks that measure time in different ways (provided the underlying functionality is available on your system): \begin{Lentry} @@ -3665,7 +3648,7 @@ ways (provided the underlying functionality is available on your system): \end{Lentry} Additional clocks can be implemented by thorns and registered with the -Flesh (see Chapter \ref{chap:clocks}). +flesh (see Chapter \ref{chap:clocks}). To use Cactus timing, you create a \textit{timer}, which provides time information for all the registered clocks. @@ -3674,14 +3657,14 @@ You can add any number of timers to your thorn source code, providing each with a name of your choice, and then use Cactus timing functions to switch on the timers, stop or reset them, and recover timing information. -Setting the Flesh parameter \texttt{cactus::cctk\_timer\_output = "full"} +Setting the flesh parameter \texttt{cactus::cctk\_timer\_output = "full"} will cause some summary timing information to be printed at the end of a run. Some other thorns have their own timer printing parameters as well. -\subsection{Timing calls} +\subsection{Timing Calls} Many of the timing calls come in two versions, one whose name ends with -the letter \texttt{I} and one without. The calls whose names end with the +the letter \texttt{I}, and one without. The calls whose names end with the letter \texttt{I} refer to the timer or clock by index, which is a non-negative \texttt{int} value; the other calls refer to a timer by name. If a timer is created without a name, it can be referred to only by its index, @@ -3763,11 +3746,11 @@ from the {\t cTimerVal} pointer argument. \end{Lentry} -\subsection{How to insert timers in your code} +\subsection{How to Insert Timers in your Code} The function prototypes and structure definitions are contained in the include file \texttt{cctk\_Timers.h}, which is included in the standard -thorn header file \texttt{cctk.h}. At the moment the timer calls are only +thorn header file \texttt{cctk.h}. At the moment, the timer calls are only available from C. The following example, which uses a timer to @@ -4392,7 +4375,7 @@ function reference section. \section{General Naming Conventions} -The following naming conventions are followed by the Flesh and the +The following naming conventions are followed by the flesh and the supported Cactus arrangements. They are not compulsory, but if followed allow for a homogeneous code. |