\documentstyle{report} \parskip = 2 pt \oddsidemargin = 0 cm \textwidth = 16 cm \topmargin = -1 cm \textheight = 24 cm \def\q{\bf QUERY: } \def\t{\tt \obeylines } \begin{document} \date{\verb$Date$} \title{Cactus 4.0 Documentation\\ \small{\verb$Revision$} } \maketitle \tableofcontents %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \part{Introduction} \chapter{Overview of documentation} This documentation splits naturally into these parts: \begin{itemize} \item {\bf The history of Cactus and the Concepts behind the CCTK. Getting you up and running.} We give an overview on the required hardware and software and we willll talk you through the installation of a working CCTK. You will be able to verify the correct installation by the CCTK testsuite. We show how to perform some basic simulation and introduce the concepts behind data output. We provide brief information on how to vizualize the 1D, 2D and 3D data. \item {\bf Part2: How to write a physics thorn.} We introduce a sample thorn that illustrates the implementation of simple initial data and the subsequent evolution. You will learn how to use the programming interface to take advantage of parallelism and modularity. \item {\bf Part3: How to writes an infrastructure thorn.} In this more advanced part, we talk about user supplied infrastructure routines as {\em additional output routines, boundary conditions, etc.} \item {\bf The standard toolkit.} \end{itemize} Other topics will be discussed in separate documents: \begin{itemize} \item The numerical relativity toolkit. \item A description of the flesh, for the maintainers. \end{itemize} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \chapter{A Cactus history} To begin with a one-phrase summary: Cactus is a numerical infrastructure for solving partial differential equations (typically in 3-dimensional space and time), a modular code for solving the Einstein equations, and an international collaboration in numerical general relativity pooling ideas in this code. Cactus 1.0 was written in 1996 by Joan Mass\'o and Paul Walker out of frustration with the numerical relativity codes they had been working on. It has come a long way. The name is meant to suggest a modular structure consisting of a ``flesh'' (or backbone, or core) and ``thorns'' (or modules, or collections of subroutines). Some essential features of the code have been present from the beginning: \begin{itemize} \item Parallelism (the work is split between multiple processors, or multiple computers) that is largely automatic and hidden from the user. \item A modular structure with a fixed interface between the flesh and the thorns, which allows users to add new parts. \item Giving the user a choice of C and Fortran for writing thorns, while automating the interface between the two languages. \item A spirit of open-source collaboration similar to the gnu project. \end{itemize} Other aspects have changed completely since Cactus 1.0. The flesh started out as a complete numerical relativity code, using one particular formulation of the Einstein equations, with the thorns providing only initial data, analysis, and other peripheral functions. In 4.0 not only the Einstein equations and general relativity basics, but even input/output and parallelism are provided by thorns. The only function of the flesh is now to define the interface between thorns, to hold varying collections of thorns, and to provide an interface between C and Fortran routines. The aim of this radicalism is to allow for future growth. We now distinguish informally between physics thorns and infrastructure thorns. The main infrastructure project for now is adding different flavors of adaptive mesh refinement -- while the user writing a physics thorn can still large she is working a single-processor, unigrid code. Similarly, an infrastructure thorn could be replaced by one that does the same job, but simply better. Even the flesh could be rewritten in the future, but the interface between the flesh and thorns is to remain unchanged. Cactus thorns can be written in F90, F77 and C (or a mixture). The decision to support Fortran was made for the pragmatic reason that many people like it, or don't write C. This decision does make the flesh more complex. The flesh is written in C, and in perl that generates C. The change from Cactus 3.0 to Cactus 4.0 is the largest one yet. The new interface forces modifications on all existing thorns, but it is intended not to change again. {\q Can we realistically say any more?} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \chapter{Cactus concepts} Cactus is designed to solve time evolution problems in three space dimensions by finite differencing on a Cartesian grid. This typically includes solving elliptic equations to obtain initial data, evolving them by hyperbolic or parabolic equations, and perhaps solving more elliptic equations at each time step. One basic concept is therefore a series of initialization steps followed by a do-loop that implements the time evolution, along with periodic analysis and output. Problems that do not naturally fit into this concept are methods which are global in space or time (such as pseudo-spectral methods) or methods which do not have a grid (such as finite elements). Thorns interact with the flesh by, among other things, announcing when they want to be called in this basic schedule. Another basic concept is parallelism by distributing spatial domains among processors. Finite differencing is an almost local procedure, which means that each processor can work on its domain of space, needing to communicate only with neighboring processors in order to work on points of the boundary of its domain. The domain decomposition is automatic and largely hidden. A physics thorn can pretend to work on a single grid, which in reality is only a chunk of the complete numerical domain. This chunk consists of an inner region of points the processor ``owns'', and needs to update, surrounded by a ``ghost zone'' of points which it can use but doesn't need to update -- they are updated by neighboring processors. Clearly this concept is geared towards explicit finite differencing schemes for the time-evolution part. {\q Say something about elliptic equations here, and implicit schemes} The notion that we are dealing with a collection of fields that live on the same (typically three-dimensional) space, and that evolve together in time defines the third basic concept, that of ``grid functions'' grouped together in a ``grid hierarchy''. Multiple grid hierarchies arise, for example, when one wants to cover the surface of a sphere with two Cartesian coordinate patches, or in adaptive mesh refinement. Grid functions are defined on Cartesian grids that are equally spaced in each of the three coordinates $x$, $y$ and $z$. (These could be spherical type coordinates, although that was not the original intention.) Grid functions can be two or one-dimensional as well as three dimensional. Routines are grouped in thorns, and thorns informally in toolkits, such as the standard toolkit, and the general relativity toolkit. The routines inside a thorn can interact in any way, but thorns interact with the flesh and with each other only through a well-defined interface. Confirming to this interface guarantees that thorns will be compatible with, and useful for, all future extensions of the code. The interface also helps users and developers to understand what a given thorn does. The flesh and many thorns are public and can be used by anyone, assuming ethical behavior. Public thorns and the flesh are distributed by CVS (see {\q where?} below). Public thorns are checked out in toolkits. The flesh may change in the future but this should never affect users, as Cactus 4.0 is making every effort to codify and freeze the interface. Public thorns should of course not lose important functionality they once had. (Nevertheless, CVS gives the user access to all earlier versions.) %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \chapter{Installation} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{Required software} In general, Cactus {\em requires} the following set of software to function in a single processor mode. Please refer to the architecture section \ref{sec:architecture} for architecture specific items. \begin{itemize} \item{\tt Perl5.0} Perl is used extensively during the CCTK configuration phase. {\tt Perl} is available for nearly all operating systems known to man and can be obtained at {\tt http://www.perl.org}. \item{\tt gmake} The GNU make utility needss to be installed. Errors {\q wanna name them here ?} during the make process are often caused by not using {\tt gmake}. {\tt gmake} can be obtained from any friendly GNU site. \item{\tt C/C++} A C/C++ compiler. These can be the GNU compilers as well. \item{\tt Fortran 90} \item{\tt CVS} The {\em ``Concurrent Versioning System''} is not needed to run/compile the CCTK, but you are strongly encourage to install this software to take advantage of the update procedures. \end{itemize} \noindent The following software is {\em optional} and not needed to get the basic CCTK installation on track. Its listed in the order of importance. \begin{itemize} \item{\tt Fortran 90} CCTK requires a F90 compiler to run modules written in F77 or F90. There is no GNU F90 and just a very limited set of free F90 compilers for various architectures. \item{\tt MPI} Currently the communication layer of the CCTK uses the {\em Message Passing Interface (MPI)} provide inter process communication. Supercomputers very often supply a native MPI implementation at their site. CCTK is very likely to work with them. For local area networks we suggest installing the {\tt MPICH} version, which can be obtained for various architectures and operating systems at {\tt http://www-unix.mcs.anl.gov/mpi/}. \item{\tt TAGS}: Tags enables you browse through the calling structure of a program by help of function call database. Navigating the CCTK and Toolkits becomes very easy. Emacs and vi both support this database. See \ref{sec:tags} how to install {\tt TAGS}. \end{itemize} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{Supported architectures} Cactus runs on many machines, under a large number of Unix operating systems. Here we list all the machines we have compiled and verfied Cactus on, including some architecture specific notes. \begin{itemize} \item{\bf SGI Origin 2000} \item{\bf SGI} \item{\bf Cray T3E} \item{\bf Dec Alpha}: Dec operating system and Linux. Single processor mode and MPICH. supported. The Decs need to have the GNU {\tt C/C++} compilers installed. \item{\bf Intel Linux}: There is a free Linux F90 compiler available from {\tt http://www.sierra.com} -- the only free we know of. Single processor mode and MPICH supported. \item{\bf Windows32 using Cygwin32 extensions} {\q still true?} \item{\bf Windows NT}: succesful compile with the following software: \begin{itemize} \item{\tt DIGITAL Visual Fortran Optimizing Compiler Version: V5.0C} \item{\tt Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 11.00.7022 for 80x86} \item{Cygnus Unix suite version 1.19} \end{itemize} \item{\bf Solaris } {\q What is the status on this ??} \end{itemize} If you did not find you architecture/operating system of choice, see section sec\ref{sec:how_to_port} on how to port to an unsupported architecture. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{Checkout procedure} Cactus is distributed, added to, and updated through the free CVS software. CVS ({\em ``Concurrent Versioning System''}) allows many people to work on a large software project together without getting into a tangle by keeping track of changes made by the different users. For the beginner, we summarize the basics in appendix \ref{sec:CVS}, please refer to this section for small description of the CVS syntax. The installation requires about {\q HOW MANY MEGS ? }. See section \ref{sec:downsize_CCTK} on how to downsize CCTK installtion if space is tight. {\q gmake small ??} \begin{itemize} \item{\bf Login}: Prior to any CVS operation, you need to loginto the central repository. For an anoymous checkout, type:\\ {\tt cvs -d :pserver:cvs\_anon@hod.aei-potsdam.mpg.de:/usr/users/cactus login } You will be prompted for a password which is {\tt anon}. \item{\bf Checkout}: To obtain a fresh CCTK checkout, type {\t cvs -d :pserver:cvs\_anon@hod.aei-potsdam.mpg.de:/usr/users/cactus checkout cactus } The CVS checkout procedure will create a directory called {\tt ./CCTK} and install the CCTK inside this directory. From now on we will reference all directory names relative to {\tt ./CCTK}. \item{\bf Update}: To update an existing CCTK checkout (to patch in possible changes, etc.), do the following {\em within} the {\tt ./CCTK} directory. {\t cvs -d :pserver:cvs\_anon@hod.aei-potsdam.mpg.de:/usr/users/cactus update } The update process will operate downwards relative to you current position within the CCTK tree. To update only certain directories, change into these directories and issue the update command. \item{\bf CVS status}: to obtain a status report on the ``age'' of your CCTK or Toolkit routines (from your current directory position downward), type {\t cvs -d :pserver:cvs\_anon@hod.aei-potsdam.mpg.de:/usr/users/cactus status } \item{\bf non-anonymous CVS}: if you have an account at the central repository and you would like to perform any of the operation above {\em non-anonymously}, replace {\tt cvs\_anon} by your login name and provide the appropriate password during the CVS login process. Depending on your permissions, you may then make commits to the CCTK or its Toolkits. \item{\bf Commits}: you need to perform a personalized login and have proper permissions to commit code to the repository. \end{itemize} For more CVS commands please refer to appendix \ref{sec:CVS}. {\q Change CCTK to cactus in the distribution} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{Directory structure} A fresh CCTK checkout creates a directory {\tt ./CCTK} with the following files substructure: \begin{itemize} \item{\tt ./CVS/} the CVS book-keeping directory, present in every subdirectory. \item{\tt ./doc/} CCTK documentation \item{\tt ./lib/} contains libraries. \item{\tt ./src/} contains the source code for the CCTK \item {\tt ./toolkits/} contains the Cactus toolkits. The toolkits (the actual ``physics'') is not supplied by the CCTK. If the toolkits you want to use are part of the central repository, they can be checked out in similar way the CCTK. Each directory inside {\tt toolkits/} contains thorns. {\q ??} \end{itemize} When Cactus is first compiled it creates a new directory {\tt ./CCTK/build/}. Disk space may be a problem on supercomputers where home directories are small. A workaround is to first create a build directory on scratch space, say {\tt scratch/cactus\_build/} (where {\tt scratch/} is your scratch directory), and soft link ({\tt ln -s scratch/cactus\_build cactus/build/}) it to the Cactus directory. When the code has been compiled for some configuration CONF, the directory {\tt build/CONF/} contains the following: \begin{itemize} \item {\tt ActiveThorns} \item {\tt datestamp.o} \item {\tt bindings/} contains all the files created by the perl scripts, sorted into subdirectories {\tt Parameters}, {\tt Variables}, {\tt Schedule}, {\tt thorn\_***} (one for each thorn). \item {\tt build/} contains the post-processed source code, the dependencies and the object files, sorted into the subdirectories {\tt Cactus}, {\tt CactusBindings} and {\tt thorn\_***}. \item {\tt config-data} : contains the files created by the configure script: \begin{itemize} \item {\tt config.cache} \item {\tt config.h} \item {\tt config.log} \item {\tt config.status} \item {\tt fortran\_name.pl} (used to determine how fortran routines are called from C) \item {\tt make.config.defn} (contains compilers etc. for a configuration) \item {\tt make.config.deps} \item {\tt make.config.rule} (rules to generate object files from source files) \end{itemize} \item {\tt lib} : contains the libraries built from the object files, \begin{itemize} \item {\tt libCactus.a} for the flesh \item{\tt llibCactusBinding.a} for the Bindings \item {\tt llib***.a} for each thorn \end{itemize} {\q It might be useful to give a couple of commands like nm for seeing what is in libraries.} \item {\tt scratch}: empty at the end. \end{itemize} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{Getting help} Cactus Maint and GNATS %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{Testing} simple make and test suite %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \chapter{Compilation} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{Compilation of the flesh, and configurations} Cactus can be built in different configurations from the same copy of the source files, and these different configurations coexist in the {\tt cactus/} directory. Different configurations can be for different machines. This is useful when your {\tt cactus/} directory is mounted on different machines in a cluster, and you want to run cactus on all these machines while only keeping and updating a single copy of the source files. It is also useful if you want to compile cactus with different compilers or optimization settings on the same machine, and compare one against the other. As a third example, you may want to compile with different sets of thorns. As a final example, you might want to build cactus using different multi-processor communication protocols, such as MPI (currently the cactus standard) or GLOBUS. Building the code should be easy. Just type {\t gmake } where {\tt <...>} denotes optional arguments. {\tt gmake} is the gnu make program. Note that {\tt make} will not work (if you try, you'll get funny error messages.) Without any arguments, the make process picks a default configuration (looking for standard compilers and so on) and gives this configuration a default name (typically the name of your operating system). It then runs configuration scripts (in {\tt lib/make/}) which figure out details of your computer. You'll see that the make process has created a {\tt build/} directory, and in it a directory {\tt build/CONF}, where {\tt CONF} is the name of the configuration, in this case the default name chosen for you. Inside {\tt CONF}, {\tt CONF/config-data/} now contains the results of the configuration process. To actually compile with this configuration, type {\t gmake CONF} where {\tt CONF} is the name of the configuration. The argument {\tt CONF} is now obligatory. gmake will ask if you want to edit the file {\tt CONF/ActiveThorns}. As the name says, it contains the names of all the thorns you want compiled in. By default it contains all the thorns you have checked out. If you say {\tt yes}, gmake opens the editor specified in your {\tt EDITOR} shell variable. gmake continues when you close the editor. Alternatively, you could have created {\tt CONF/ActiveThorns} beforehand. {\tt ActiveThorns} can contain the names of individual thorns or entire toolkits {\q Implement this}. {\q move setup file elsewhere.} {\q move executable elsewhere, and make one for each configuration.} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{Compiling in thorns} Cactus compiles in all thorns listed in {\tt build/CONF/ActiveThorns}. {\q If the thorn has not been checked out...} Cactus looks for source code in a thorn's {\tt src} directory. It first looks for a file {\tt make.code.defn} which should contain \begin{itemize} \item {\t SRCS = } \item {\t SUBDIRS = } \end{itemize} and each subdirectory mentioned should also contain a {\tt make.code.defn} file with a {\tt SRCS} line ({\tt SUBDIRS} in a subdirectory {\tt make.code.defn} file will be ignored). The {\tt make.code.defn} file in a directory is included at the top of the makefile {\q where does it sit?} used to build the sources. If a file {\tt make.code.deps} exists in the directory {\q where?} it is included at the bottom. The standard make system may be overridden by placing a makefile called {\tt Makefile} in the {\tt src/} directory. This can do whatever it likes, but must create a library called {\tt \$(NAME)} (which is actually {\tt \$(CCTK\_LIBDIR)/lib.a }). {\q We need more detail here. What does ``is actually'' mean?} {\q Watch for CCTK here and elsewhere. Don't remove this query before CCTK is gone.} A makefile is passed the following variables {\q How many makefiles are involved? Which are created at runtime etc.} \begin{itemize} \item {\tt \$(CCTK\_HOME)} - the main CCTK directory \item {\tt \$(TOP)} - the CONF directory \item {\tt \$(SRCDIR)} - the directory in which the source files can be found \item {\tt \$(CONFIG)} - the directory containing the configuration files \item {\tt \$(THORN)} - the thorn name \item {\tt \$(SCRATCH\_BUILD)} - the scratch directory where f90 module files should end up if they need to be seen by other thorns. \item {\tt \$(NAME)}. \end{itemize} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section {gmake targets} (A target for {\tt gmake} can be naively thought of as an argument that tells it which of several things listed in the {\tt Makefile} it is to do.) \begin{itemize} \item {\tt gmake config} will create the default (uname) configuration \item {\tt gmake -clean} cleans a configuration (rm -r *.o) \item {\tt gmake -delete} deletes a configuration (rm -r build/) \item {\tt gmake help} lists all make options \item {\tt gmake tags} creates a {\tt vi} style tags file \item {\tt gmake TAGS} creates an Emacs style TAGS file \item {\tt gmake distclean} nukes your build directory. \end{itemize} {\q Tom, this is for you} b) Make methodology (dependencies,libraries, thorn make files etc) d) Configuration options f) (New platforms??) %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \chapter{Running Cactus} a) Command line options b) Simple parameter file syntax (will be explained fully in thorn writers part) c) Description of standard parameters (although maybe should be an appendix) d) Checkpointing and outputting e) Reference to web pages with machine dependant stuff f) Understanding the screen output (RFR tree, errors and warnings etc) g) Convergence mode h) Environment variables %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \chapter{Looking at output} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \part{Application thorn writing} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \chapter{Thorn concepts} (This section has to contain enough explanation to make the rest of the writers guide readable the first time through) (The hardest bugs to find are those arising from plausible but incorrect assumptions about the behavior of someone else's thorn.) a) Again probably emphasize collaboration, what are thorns, toolkits, how to share them. b) Things to think about before you start programming: Language, read all the documentation, emphasize use of standard supported Cactus infrastructure c) Available data types i) Scalars ii) Arrays and GFs iii) Groups d) Ghost zones and parallelism e) Understanding the RFR concept f) Understanding the GH concept %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \chapter{Anatomy of a thorn} Each thorn must be in a toolkit. Toolkits are fairly informal. If you want, you can just make your own toolkit directory, and stick your new thorn into it. a) Creating a thorn (gmake new thorn) b) Thorn directory structure c) Thorn make files d) Thorn configuration (i) Parameters (param.ccl) (ii) Variables (interface.ccl) (iii) Scheduler (schedule.ccl) %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \chapter{Putting code into your thorn} a) What the flesh basically provides (header files, macros, ...) b) Argument lists and parameters c) A simple example (Baloney) d) Differences between F and C with the interface e) A more complex example (Wavetoy) f) Error handling and code stopping g) Calling C from F, F from C, (C++ from F? ....) h) Good Cactus Style Programming Guidelines i) Adding a test suite j) Sharing your thorn with the world. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \part*{Appendices} \appendix \chapter{Using GNATS} \chapter{Using CVS} \chapter{Using TAGS} \section{TAGS with emacs} \section{TAGS with vi} \chapter{Configuration file syntax} \section{General Concepts} Each thorn is configured by three compulsory files in the top level thorn directory: \begin{itemize} \item{} interface.ccl \item{} param.ccl \item{} schedule.ccl \end{itemize} These files are written in the {\it Cactus Configuration Language} which is case insensitive. \section{interface.ccl} The interface configuration file consists of a header block giving details of the thorns relationship with other thorns, and then a series of blocks listing the thorn's global variables. The header block has the form: {\t \begin{verbatim} implements: [inherits: , ] [friend: , ] \end{verbatim} } where \begin{itemize} \item{} The implentation name must be unique among all thorns, except between thorns which have the same public and protected variables and parameters. \item{} Inheriting from another implentation makes all that implentations public variables available to your thorn. At least one thorn providing any inherited implentation must be present at compile time. A thorn cannot inherit from itself. Inheritance is associative and recursive, but not commutative. \item{} Being a friend of another implementation makes all that implementations protected variables available to your thorn. At least one thorn providing an implementation for each friend must be present at compile time. A thorn cannot be its own friend. Friendship is associative and commutative. Note that your thorn is also a friend of all your thorns friend's friends. \end{itemize} The thorn's variables are defined by: {\t \begin{verbatim} [:] VTYPE [TYPE=] [DIM=] [{ [,] }] \end{verbatim}} \begin{itemize} \item{} {\t access} defines which thorns have can use the following groups, and can be either {\t public}, {\t protected} or {\t private}. \item{} {\t VTYPE} defines the data type of the variables in the group. Supported data types are {\t LOGICAL}, {\t INTEGER}, {\t CHAR} and {\t REAL}. \item{} {\t group name} must be a alpha-numeric name which is unique within the scope of the thorn. A group name is compulsory. \item{} {\t TYPE} designates the kind of variables held by the group. The choices are {\t GF}, {\t ARRAY} or {\t SCALAR}. This field is optional, with the default variable type being {\t SCALAR}. \item{} {\t DIM} defines the spatial dimension if the group is of type {\t ARRAY} or {\t GF}, and can take the value {\t 1}, {\t 2}, or {\t 3}. The default value is {\t DIM=3}. \item{} The block following the group declaration line contains a list of variables contained in the group. All variables in a group have the same data type, variable type and dimension. The list can be separated by spaces, commas, or new lines. The variable names must be unique within the scope of the thorn. A variable can only be a member of one group. The block must be delimited by brackets on new lines. If no block is given after a group declaration line, a variable with the same name as the group is created. \end{itemize} \section{param.ccl} The parameter configuration file consists of a list of {\it parameter object specification items} (OSIs) giving the type and range of the parameter separated by optional {\it parameter data scoping items} (DSIs) which detail access to the parameter. \subsection{Parameter Data Scoping Items} {\t \begin{verbatim} : \end{verbatim} } The key word {\t access} designates that all parameter object specification items up to the next parameter data scoping item are in the same protection or scoping class. {\t access} can take the values: \begin{itemize} \item{} {\t public} --- all thorns have access to public parameters \item{} {\t protected} --- other thorns can have access to these parameters if they specifically request it in their own param.ccl \item{} {\t private} --- only your thorn has access to private parameters \item{} {\t friend} --- in this case an {\t implementation} name must follow the colon. It declare that all the parameters in the following scoping block are protected variables from the specified {\t implementation}. \end{itemize} \subsection{Parameter Object Specification Items} {\t \begin{verbatim} [EXTENDS] ``'' { } \end{verbatim} } \begin{itemize} \item{} Allowed {\t parameter types} are \begin{itemize} \item{} {\t INTEGER} The specification of parameter values takes the form or any number of comma-separated blocks of the form: {\t \begin{verbatim} [][::][][::] \end{verbatim} } Where an empty field, or a {\t *} in the place of {\t low-range} or {\t high-range} indicates $-\infty$ and $\infty$ respectively. The default value for {\t step} is 1. A number by itself denotes that this number is the only acceptable value. \item{} {\t REAL} The range specification is the same as integers, except that here, no {\t step} implies a continuum of values. \item{} {\t KEYWORD} Each entry in the list of acceptable values for a keyword has the form {\t \begin{verbatim} ``'', ``'' :: ``'' \end{verbatim} } \item{} {\t STRING} No allowed values should be specified for a logical parameter. \item{} {\t LOGICAL} No allowed values should be specified for a logical parameter. \end{itemize} \item{} The {\t parameter name} must be unique within the scope of the thorn. \item{} A thorn can declare that it {\t EXTENDS} a parameter that it is a friend of. This allows it to declare additional acceptable values. By default it is acceptable for two thorns to declare the same value as acceptable. \end{itemize} \section{schedule.ccl} The schedule configuration files consists of \begin{itemize} \item{} assignments to switch on storage and communications for array variables at the start of program execution. \item{} {\it schedule blocks} which schedule a subroutine from the thorn to be called at a specified time during program executation. Statements within the schedule block can be used to switch on storage and communication for groups of variables during the duration that the subroutine is called. \item{} Conditional statements \end{itemize} {\it assignments statements} have the form: {\t \begin{verbatim} [STORAGE: , ] [COMMUNICATION: , ] \end{verbatim} } Each {\it schedule block} in the file {\t schedule.ccl} must have the syntax: {\t \begin{verbatim} schedule at