\documentstyle{report} \parskip = 2 pt \oddsidemargin = 0 cm \textwidth = 16 cm \topmargin = -1 cm \textheight = 24 cm \def\q{\bf QUERY: } \def\t{\tt \obeylines } \begin{document} \date{\verb$Date$} \title{Cactus 4.0 Documentation\\ \small{\verb$Revision$} } \maketitle \tableofcontents %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \part{Introduction} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \chapter{A Cactus history} To begin with a one-phrase summary: Cactus is a numerical infrastructure for solving partial differential equations (typically in 3-dimensional space and time), a modular code for solving the Einstein equations, and an international collaboration in numerical general relativity pooling ideas in this code. Cactus 1.0 was written in 1996 by Joan Mass\'o and Paul Walker out of frustration with the numerical relativity codes they had been working on. It has come a long way. The name is meant to suggest a modular structure consisting of a ``flesh'' (or backbone, or core) and ``thorns'' (or modules, or collections of subroutines). Some essential features of the code have been present from the beginning: \begin{itemize} \item Parallelism (the work is split between multiple processors, or multiple computers) that is largely automatic and hidden from the user. \item A modular structure with a fixed interface between the flesh and the thorns, which allows users to add new parts. \item Giving the user a choice of C and Fortran for writing thorns, while automating the interface between the two languages. \item A spirit of open-source collaboration similar to the gnu project. \end{itemize} Other aspects have changed completely since Cactus 1.0. The flesh started out as a complete numerical relativity code, using one particular formulation of the Einstein equations, with the thorns providing only initial data, analysis, and other peripheral functions. In 4.0 not only the Einstein equations and general relativity basics, but even input/output and parallelism are provided by thorns. The only function of the flesh is now to define the interface between thorns, to hold varying collections of thorns, and to provide an interface between C and Fortran routines. The aim of this radicalism is to allow for future growth. We now distinguish informally between physics thorns and infrastructure thorns. The main infrastructure project for now is adding different flavors of adaptive mesh refinement -- while the user writing a physics thorn can still largely pretend she is working a single-processor, unigrid code. Similarly, an infrastructure thorn could be replaced by one that does the same job, but simply better. Even the flesh could be rewritten in the future, but the interface between the flesh and thorns is to remain unchanged. Cactus thorns can be written in F90, F77, C, C++ (or a mixture). The decision to support Fortran was made for the pragmatic reason that many people like it, or don't write C. This decision does make the flesh more complex. The flesh is written in C, and in perl that generates C. The change from Cactus 3.0 to Cactus 4.0 is the largest one yet. The new interface forces modifications on all existing thorns, but it is intended not to change again. {\q Can we realistically say any more?} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \chapter{Cactus concepts} Cactus is designed to solve time evolution problems in three space dimensions by finite differencing on a Cartesian grid. This typically includes solving elliptic equations to obtain initial data, evolving them by hyperbolic or parabolic equations, and perhaps solving more elliptic equations at each time step. One basic concept is therefore a series of initialization steps followed by a do-loop that implements the time evolution, along with periodic analysis and output. Problems that do not naturally fit into this concept are methods which are global in space or time (such as pseudo-spectral methods) or methods which do not have a grid (such as finite elements). Thorns interact with the flesh by, among other things, announcing when they want to be called in this basic schedule. Another basic concept is parallelism by distributing spatial domains among processors. Finite differencing is an almost local procedure, which means that each processor can work on its domain of space, needing to communicate only with neighboring processors in order to work on points of the boundary of its domain. The domain decomposition is automatic and largely hidden. A physics thorn can pretend to work on a single grid, which in reality is only a chunk of the complete numerical domain. This chunk consists of an inner region of points the processor ``owns'', and needs to update, surrounded by a ``ghost zone'' of points which it can use but doesn't need to update -- they are updated by neighboring processors. Clearly this concept is geared towards explicit finite differencing schemes for the time-evolution part. {\q Say something about elliptic equations here, and implicit schemes} The notion that we are dealing with a collection of fields that live on the same (typically three-dimensional) space, and that evolve together in time defines the third basic concept, that of ``grid functions'' grouped together in a ``grid hierarchy''. Multiple grid hierarchies arise, for example, when one wants to cover the surface of a sphere with two Cartesian coordinate patches, or in adaptive mesh refinement. Grid functions are defined on Cartesian grids that are equally spaced in each of the three coordinates $x$, $y$ and $z$. (These could be spherical type coordinates, although that was not the original intention.) Grid functions can be two or one-dimensional as well as three dimensional. Routines are grouped in thorns, and thorns informally in toolkits, such as the standard toolkit, and the general relativity toolkit. The routines inside a thorn can interact in any way, but thorns interact with the flesh and with each other only through a well-defined interface. Confirming to this interface guarantees that thorns will be compatible with, and useful for, all future extensions of the code. The interface also helps users and developers to understand what a given thorn does. The flesh and many thorns are public and can be used by anyone, assuming ethical behavior. Public thorns and the flesh are distributed by CVS (see {\q where?} below). Public thorns are checked out in toolkits. The flesh may change in the future but this should never affect users, as Cactus 4.0 is making every effort to codify and freeze the interface. Public thorns should of course not lose important functionality they once had. (Nevertheless, CVS gives the user access to all earlier versions.) %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \chapter{Overview of documentation} This documentation splits naturally into these parts: \begin{itemize} \item What one needs to know to obtain, compile and run the code, and look at output. \item How one writes a physics thorn. \item How one writes an infrastructure thorn. \item The standard toolkit. \end{itemize} Other topics will be discussed in separate documents: \begin{itemize} \item The numerical relativity toolkit. \item A description of the flesh, for the maintainers. \end{itemize} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \part{Using Cactus} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \chapter{Installation} (Contains details of everything needed to get the code, compile it, and check that it works) %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{Required software} To check out cactus you will require CVS, and to build it you will require Gnu Make, and perl. All these are freely available and may be downloaded from...{\q fill these in}. You will also need a C compiler to build the flesh, and most thorns require FORTRAN77 or Fortran 90 compilers. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{Supported architectures} Cactus runs on many machines, under a large number of Unix operating systems, and also under Windows NT. The flesh is written in ANSI standard C, and so should compile on any machine with a compiler supporting this. {\q Elsewhere} we discuss how to port cactus to a new operating system or hardware platform. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{Checkout procedure} Cactus is distributed, added to, and updated through the free CVS software. Basically, CVS (Concurrent Versioning System) allows many people to work on a large software project together without getting into a tangle. For the beginner, we summarize the basics in an appendix to this document. To check out the Cactus Computational Toolkit from the central repository, type in {\t cvs -d :pserver:cvs\_anon@hod.aei-potsdam.mpg.de:/usr/users/cactus login cvs -d :pserver:cvs\_anon@hod.aei-potsdam.mpg.de:/usr/users/cactus checkout cactus } Note our typesetting conventions: these are two separate unix commands (two lines, each followed by the Enter key). Note also that the cvs commands are basically {\tt cvs login} and {\tt cvs checkout cactus}. The middle bit tells CVS where to look. The first line will prompt you for a password, which is {\tt anon}. {\q Change CCTK to cactus in the distribution} (If you contribute to cactus, you will have your own account, as we'll discuss in the section on writing thorns. Then {\tt cvs\_anon} will be replaced by your user name, and the password will be different.) In order to update your checkout later (because someone else has changed the repository version), go to the directory you want to update (eg {\tt cactus}) and do {\tt cvs update}. To find out if you are up to date, do {\tt cvs status}. Note the {\tt CVS/} directories at each directory level: they tell CVS what you have checked out, and when you last updated it. See the appendix on CVS for more information. (Note our typesetting convention: {\tt CVS/} means that {\tt CVS} is a directory. Your unix {\tt ls} may or may not show the slash.) %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{Directory structure} In the {\tt cactus/} directory you see a file called {\tt Makefile}. It tells the make program what to do in order to build Cactus. {\tt cactus/} also contains these directories: \begin{itemize} \item {\tt doc/} contains documentation. \item {\tt lib/} contains libraries and support tools for building the code. \item {\tt src/} contains the source code for the flesh. (If you are familiar with unix, all this will be obvious to you.) \item {\tt toolkits/} contains the Cactus toolkits. It is empty now. Each directory inside {\tt toolkits/} contains thorns. \end{itemize} When Cactus is first compiled it creates a new directory {\tt cactus/build/}. Disk space may be a problem on supercomputers where home directories are small. A workaround is to first create a build directory on scratch space, say {\tt scratch/cactus\_build/} (where {\tt scratch/} is your scratch directory), and soft link ({\tt ln -s scratch/cactus\_build cactus/build/}) it to the Cactus directory. When the code has been compiled for some configuration CONF, the directory {\tt build/CONF/} contains the following: \begin{itemize} \item {\tt ActiveThorns} \item {\tt datestamp.o} \item {\tt bindings/} contains all the files created by the perl scripts, sorted into subdirectories {\tt Parameters}, {\tt Variables}, {\tt Schedule}, {\tt thorn\_***} (one for each thorn). \item {\tt build/} contains the post-processed source code, the dependencies and the object files, sorted into the subdirectories {\tt Cactus}, {\tt CactusBindings} and {\tt thorn\_***}. \item {\tt config-data} : contains the files created by the configure script: \begin{itemize} \item {\tt config.cache} \item {\tt config.h} \item {\tt config.log} \item {\tt config.status} \item {\tt fortran\_name.pl} (used to determine how fortran routines are called from C) \item {\tt make.config.defn} (contains compilers etc. for a configuration) \item {\tt make.config.deps} \item {\tt make.config.rule} (rules to generate object files and their dependencies from source files) \end{itemize} \item {\tt lib} : contains the libraries built from the object files, \begin{itemize} \item {\tt libCactus.a} for the flesh \item{\tt libCactusBindings.a} for the bindings between the flesh and the thorns \item {\tt lib***.a} for each thorn \end{itemize} {\q It might be useful to give a couple of commands like nm for seeing what is in libraries.} \item {\tt scratch}: contains any F90 module files produced by the compiler. \end{itemize} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{Getting help} Cactus Maint and GNATS %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{Testing} simple make and test suite %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \chapter{Compilation} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{Compilation of the flesh, and configurations} Cactus can be built in different configurations from the same copy of the source files, and these different configurations coexist in the {\tt cactus/} directory. Different configurations can be for different machines. This is useful when your {\tt cactus/} directory is mounted on different machines in a cluster, and you want to run cactus on all these machines while only keeping and updating a single copy of the source files. It is also useful if you want to compile cactus with different compilers or optimization settings on the same machine, and compare one against the other. As a third example, you may want to compile with different sets of thorns. As a final example, you might want to build cactus using different multi-processor communication protocols, such as different implementations of MPI (currently the cactus standard), PVM, or shared memory. Building the code should be easy. Just type {\t gmake } where {\tt <...>} denotes optional arguments. {\tt gmake} is the gnu make program. Note that {\tt make} will not work (if you try, you'll get funny error messages.) Without any arguments, the make process picks a default configuration (looking for standard compilers and so on) and gives this configuration a default name (typically the name of your operating system). It then runs configuration scripts (in {\tt lib/make/}) which figure out details of your computer. You'll see that the make process has created a {\tt build/} directory, and in it a directory {\tt build/CONF}, where {\tt CONF} is the name of the configuration, in this case the default name chosen for you. Inside {\tt CONF}, {\tt CONF/config-data/} now contains the results of the configuration process. To actually compile with this configuration, type {\t gmake CONF} where {\tt CONF} is the name of the configuration. The argument {\tt CONF} is now obligatory. gmake will ask if you want to edit the file {\tt CONF/ActiveThorns}. As the name says, it contains the names of all the thorns you want compiled in. By default it contains all the thorns you have checked out. If you say {\tt yes}, gmake opens the editor specified in your {\tt EDITOR} shell variable. gmake continues when you close the editor. Alternatively, you could have created {\tt CONF/ActiveThorns} beforehand. {\tt ActiveThorns} can contain the names of individual thorns or entire toolkits {\q Implement this}. {\q move setup file elsewhere.} {\q move executable elsewhere, and make one for each configuration.} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{Compiling in thorns} Cactus compiles in all thorns listed in {\tt build/CONF/ActiveThorns}. {\q If the thorn has not been checked out...} Cactus looks for source code in a thorn's {\tt src} directory. It first looks for a file {\tt make.code.defn} which should contain \begin{itemize} \item {\t SRCS = } \item {\t SUBDIRS = } \end{itemize} and each subdirectory mentioned should also contain a {\tt make.code.defn} file with a {\tt SRCS} line ({\tt SUBDIRS} in a subdirectory {\tt make.code.defn} file will be ignored). The {\tt make.code.defn} file in a source directory is included at the top of the makefile used to build the sources, and thus can be used to change compiler flags on a per-directory basis. If a file {\tt make.code.deps} exists in a source directory it is similarly included at the bottom; this can be used to add dependencies or customised build rules for particular files. The standard make system may be overridden by placing a makefile called {\tt Makefile} in the {\tt src/} directory. This can do whatever it likes, but must create a library called {\tt \$(NAME)} (which is actually {\tt CONF/lib/lib.a }). {\q Watch for CCTK here and elsewhere. Don't remove this query before CCTK is gone.} A makefile is passed the following variables {\q How many makefiles are involved? Which are created at runtime etc.} \begin{itemize} \item {\tt \$(CCTK\_HOME)} - the main CCTK directory \item {\tt \$(TOP)} - the CONF directory \item {\tt \$(SRCDIR)} - the directory in which the source files can be found \item {\tt \$(CONFIG)} - the directory containing the configuration files \item {\tt \$(THORN)} - the thorn name \item {\tt \$(SCRATCH\_BUILD)} - the scratch directory where f90 module files should end up if they need to be seen by other thorns. \item {\tt \$(NAME)}. \end{itemize} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section {gmake targets} (A target for {\tt gmake} can be naively thought of as an argument that tells it which of several things listed in the {\tt Makefile} it is to do.) \begin{itemize} \item {\tt gmake config} will create the default (uname) configuration \item {\tt gmake -clean} cleans a configuration (rm -r *.o) \item {\tt gmake -delete} deletes a configuration (rm -r build/) \item {\tt gmake help} lists all make options \item {\tt gmake tags} creates a {\tt vi} style tags file \item {\tt gmake TAGS} creates an {\tt Emacs} style TAGS file \item {\tt gmake distclean} deletes your whole build directory. \end{itemize} {\q Tom, this is for you} b) Make methodology (dependencies,libraries, thorn make files etc) d) Configuration options f) (New platforms??) %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \chapter{Running Cactus} a) Command line options b) Simple parameter file syntax (will be explained fully in thorn writers part) c) Description of standard parameters (although maybe should be an appendix) d) Checkpointing and outputting e) Reference to web pages with machine dependant stuff f) Understanding the screen output (RFR tree, errors and warnings etc) g) Convergence mode h) Environment variables %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \chapter{Looking at output} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \part{Application thorn writing} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \chapter{Thorn concepts} (This section has to contain enough explanation to make the rest of the writers guide readable the first time through) (The hardest bugs to find are those arising from plausible but incorrect assumptions about the behavior of someone else's thorn.) a) Again probably emphasize collaboration, what are thorns, toolkits, how to share them. b) Things to think about before you start programming: Language, read all the documentation, emphasize use of standard supported Cactus infrastructure c) Available data types i) Scalars ii) Arrays and GFs iii) Groups d) Ghost zones and parallelism e) Understanding the RFR concept f) Understanding the GH concept %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \chapter{Anatomy of a thorn} Each thorn must be in a toolkit. Toolkits are fairly informal. If you want, you can just make your own toolkit directory, and stick your new thorn into it. a) Creating a thorn (gmake new thorn) b) Thorn directory structure c) Thorn make files d) Thorn configuration (i) Parameters (param.ccl) (ii) Variables (interface.ccl) (iii) Scheduler (schedule.ccl) %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \chapter{Putting code into your thorn} a) What the flesh basically provides (header files, macros, ...) b) Argument lists and parameters c) A simple example (Baloney) d) Differences between F and C with the interface e) A more complex example (Wavetoy) f) Error handling and code stopping g) Calling C from F, F from C, (C++ from F? ....) h) Good Cactus Style Programming Guidelines i) Adding a test suite j) Sharing your thorn with the world. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \part*{Appendices} \appendix \chapter{Using GNATS} \chapter{Using CVS} \chapter{Using TAGS} \section{TAGS with emacs} \section{TAGS with vi} \chapter{Configuration file syntax} \section{General Concepts} Each thorn is configured by three compulsory files in the top level thorn directory: \begin{itemize} \item{} interface.ccl \item{} param.ccl \item{} schedule.ccl \end{itemize} These files are written in the {\it Cactus Configuration Language} which is case insensitive. \section{interface.ccl} The interface configuration file consists of a header block giving details of the thorns relationship with other thorns, and then a series of blocks listing the thorn's global variables. The header block has the form: {\t \begin{verbatim} implements: [inherits: , ] [friend: , ] \end{verbatim} } where \begin{itemize} \item{} The implentation name must be unique among all thorns, except between thorns which have the same public and protected variables and parameters. \item{} Inheriting from another implentation makes all that implentations public variables available to your thorn. At least one thorn providing any inherited implentation must be present at compile time. A thorn cannot inherit from itself. Inheritance is associative and recursive, but not commutative. \item{} Being a friend of another implementation makes all that implementations protected variables available to your thorn. At least one thorn providing an implementation for each friend must be present at compile time. A thorn cannot be its own friend. Friendship is associative and commutative. Note that your thorn is also a friend of all your thorns friend's friends. \end{itemize} The thorn's variables are defined by: {\t \begin{verbatim} [:] VTYPE [TYPE=] [DIM=] [{ [,] }] \end{verbatim}} \begin{itemize} \item{} {\t access} defines which thorns have can use the following groups, and can be either {\t public}, {\t protected} or {\t private}. \item{} {\t VTYPE} defines the data type of the variables in the group. Supported data types are {\t LOGICAL}, {\t INTEGER}, {\t CHAR} and {\t REAL}. \item{} {\t group name} must be a alpha-numeric name which is unique within the scope of the thorn. A group name is compulsory. \item{} {\t TYPE} designates the kind of variables held by the group. The choices are {\t GF}, {\t ARRAY} or {\t SCALAR}. This field is optional, with the default variable type being {\t SCALAR}. \item{} {\t DIM} defines the spatial dimension if the group is of type {\t ARRAY} or {\t GF}, and can take the value {\t 1}, {\t 2}, or {\t 3}. The default value is {\t DIM=3}. \item{} The block following the group declaration line contains a list of variables contained in the group. All variables in a group have the same data type, variable type and dimension. The list can be separated by spaces, commas, or new lines. The variable names must be unique within the scope of the thorn. A variable can only be a member of one group. The block must be delimited by brackets on new lines. If no block is given after a group declaration line, a variable with the same name as the group is created. \end{itemize} \section{param.ccl} The parameter configuration file consists of a list of {\it parameter object specification items} (OSIs) giving the type and range of the parameter separated by optional {\it parameter data scoping items} (DSIs) which detail access to the parameter. \subsection{Parameter Data Scoping Items} {\t \begin{verbatim} : \end{verbatim} } The key word {\t access} designates that all parameter object specification items up to the next parameter data scoping item are in the same protection or scoping class. {\t access} can take the values: \begin{itemize} \item{} {\t public} --- all thorns have access to public parameters \item{} {\t protected} --- other thorns can have access to these parameters if they specifically request it in their own param.ccl \item{} {\t private} --- only your thorn has access to private parameters \item{} {\t friend} --- in this case an {\t implementation} name must follow the colon. It declare that all the parameters in the following scoping block are protected variables from the specified {\t implementation}. \end{itemize} \subsection{Parameter Object Specification Items} {\t \begin{verbatim} [EXTENDS] ``'' { } \end{verbatim} } \begin{itemize} \item{} Allowed {\t parameter types} are \begin{itemize} \item{} {\t INTEGER} The specification of parameter values takes the form or any number of comma-separated blocks of the form: {\t \begin{verbatim} [][::][][::] \end{verbatim} } Where an empty field, or a {\t *} in the place of {\t low-range} or {\t high-range} indicates $-\infty$ and $\infty$ respectively. The default value for {\t step} is 1. A number by itself denotes that this number is the only acceptable value. \item{} {\t REAL} The range specification is the same as integers, except that here, no {\t step} implies a continuum of values. \item{} {\t KEYWORD} Each entry in the list of acceptable values for a keyword has the form {\t \begin{verbatim} ``'', ``'' :: ``'' \end{verbatim} } \item{} {\t STRING} No allowed values should be specified for a logical parameter. \item{} {\t LOGICAL} No allowed values should be specified for a logical parameter. \end{itemize} \item{} The {\t parameter name} must be unique within the scope of the thorn. \item{} A thorn can declare that it {\t EXTENDS} a parameter that it is a friend of. This allows it to declare additional acceptable values. By default it is acceptable for two thorns to declare the same value as acceptable. \end{itemize} \section{schedule.ccl} The schedule configuration files consists of \begin{itemize} \item{} assignments to switch on storage and communications for array variables at the start of program execution. \item{} {\it schedule blocks} which schedule a subroutine from the thorn to be called at a specified time during program executation. Statements within the schedule block can be used to switch on storage and communication for groups of variables during the duration that the subroutine is called. \item{} Conditional statements \end{itemize} {\it assignments statements} have the form: {\t \begin{verbatim} [STORAGE: , ] [COMMUNICATION: , ] \end{verbatim} } Each {\it schedule block} in the file {\t schedule.ccl} must have the syntax: {\t \begin{verbatim} schedule at