% /*@@
%   @file      Appendices.tex
%   @date      27 Jan 1999
%   @author    Tom Goodale, Gabrielle Allen, Gerd Lanferman, Thomas Radke
%   @desc
%              Appendices for the Cactus User's Guide
%   @enddesc
%   @version   $Header$
% @@*/

\begin{cactuspart}{Appendices}{$RCSfile$}{$Revision$}
\label{part:Appendices}
\renewcommand{\thepage}{\Alph{part}\arabic{page}}

Note that these appendices appear (identically) in both the
Cactus Users' Guide and the Cactus Reference Manual.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\chapter{Glossary}
\label{sec:glossary}

\begin{Lentry}

\item[alias function]
  See \textit{function aliasing}.
\item[AMR]
  \textit{Automatic Mesh Refinement}
\item[analysis]
\item[API]
  \textit{Applications Programming Interface}, the interface provided by
  some software component to programmers who use the component. 
  An API usually consists of subroutine/function calls, but may also include
  structure definitions and definition of constant values.
  The Cactus Reference Manual documents most of the Cactus Flesh APIs.
\item[arrangement]
  A collection of thorns, stored in a subdirectory of the Cactus
  \verb|arrangements| directory.  See Section~\ref{sec:arrangements}.
\item[autoconf]
  A GNU program which builds a configuration script which can be used
  to make a Makefile.
\item[boundary zone]
  A boundary zone is a set of points at the edge of a grid, interpreted as
  the boundary of the physical problem, and which contains boundary data,
  e.g.\ Dirichlet conditions or von Neumann conditions.
  (See also \textit{symmetry zone}, \textit{ghost zone}.)
\item[Cactus]
  A green fleshy plant with lots of thorns, usually inflicting pain if
  touched.
\item[CCTK]
  \textit{Cactus Computational Tool Kit} (The Cactus Flesh and computational
  thorns).
\item[CCL]
  The \textit{Cactus Configuration Language}, this is the language that
  the thorn configuration files are written in.
  See Section~\ref{sec:Appendix.ccl}.
\item[configuration]
  The combination of a set of thorns, and all the Cactus configure
  options which affect what binary will be produced when compiling
  Cactus.  For example, the choice of compilers (Cactus \verb|CC|,
  \verb|CXX|, \verb|F77|, and \verb|F90| configure options) and the
  compiler optimization settings (\verb|OPTIMISE|/\verb|OPTIMIZE| and
  \verb|*_OPTIMISE_FLAGS| configure options) are part of a
  configuration (these flags change what binary is produced), but the
  Cactus \verb|SILENT| and \verb|WARN| configure options aren't part
  of a configuration (they don't change what binary will be produced).
  See Section~\ref{sec:configurations}.
\item[checkout]
  Get a copy of source code from CVS.  See Section~\ref{sec:checkout}.
\item[checkpoint]
  Save the entire state of a Cactus run to a file so that the run can be
  restarted at a later time.
  See Sections~\ref{sec:checkpointing}, \ref{chap:cp_recovery_methods}.
\item[computational grid]
  A discrete finite set of spatial points in $\Re^n$
  (typically $1 \le n \le 3$).
  Historically Cactus has required these points to be uniformly spaced
  (we say the grid is uniformly spaced), but we are now starting to add
  support for non-uniform spacings (we say the grid is non-uniformly spaced).

  The grid consists of the physical domain and the boundary and symmetry
  points.

  See \textit{grid functions} for the typical use of grid points.
\item[convergence]
  Important, but often neglected.
\item[CST]
  The \textit{Cactus Specification Tool}, which is the set of Perl
  scripts which parse the thorns' \texttt{.ccl} files and generates the
  code that binds the thorn source files with the Flesh.
\item[CVS]
  The \textit{Concurrent Versions System} is the favoured code
  distribution system for Cactus.  See Sections~\ref{sec:checkout},
  \ref{sec:Appendix.cvs}.
\item[domain decomposition]
  The technique of breaking up a large computational problem into parts
  that are easier to solve.  In Cactus, refers especially to a decomposition
  wherein the parts are solved in parallel on separate computer processors.
\item[driver]
  A special kind of thorn which creates and handles grid hierarchies 
  and grid variables.
%Drivers are responsible for memory management for grid variables, and for all parallel operations, in response to requests from the scheduler.
  See Section~\ref{sec:parallelisation}.
\item[evolution]
  An iteration interpreted as a step through time.  Also, a particular Cactus
  schedule bin for executing routines when evolution occurs.
\item[Flesh]
  The Cactus routines which hold the thorns together, allowing them to 
  communicate and scheduling things to happen with them. This is what you
  get if you check out Cactus from our CVS repository.
\item[friend] Interfaces that are \textit{friends} share their collective
  set of protected grid variables.
  See Section~\ref{subsec:interface_ccl}. \ref{sec:Appendix.interface}.
\item[function aliasing]
  The process of referring to a function to be provided by an interface
  independently of which thorn actually contains the function, or what
  language the function is written in.  The function is called an
  \textit{alias function}.  See Section~\ref{sec:function_aliasing},
  \ref{subsec:Appendix.interface.function_aliasing}.
\item[GA]
  Shorthand for a \textit{grid array}.
\item[GF]
  Shorthand for a \textit{grid function}.
\item[gmake]
  GNU version of the {\tt make} utility.
\item[ghost zone] 
  A set of points added for parallelisation purposes to a block of a
  grid lying on one processor, corresponding to points on the boundary
  of an adjoining block of the grid lying on another processor.
  Points from the boundary of the one block are copied (via an
  inter-processor communication mechanism) during synchronization (see)
  to the corresponding ghost zone of the other block, and vice-versa.
  In single processor runs there are no ghost zones.
  Contrast with symmetry or boundary zones.
  See Section~\ref{sec:ghost_size}.
\item[grid]
  Short for \textit{computational grid}.
\item[grid array]
  A \textit{grid variable} whose global size need not be that of the
  computational grid; instead, the size is declared explicitly in an
  \verb|interface.ccl| file.  
\item[grid function]
  A \textit{grid variable} whose global size is the size of the
  computational grid.  (See also \textit{local array}.)
  From another perspective,
  \textit{grid functions} are functions (of any of the Cactus
  data types (see section~\ref{sect-ThornWriting/DataTypes})
  defined on the domain of grid points.
  Typically grid functions are used to discretely approximate functions
  defined on the domain $\Re^n$, with \textit{finite differencing}
  used to approximate partial derivatives.
\item[grid hierarchy]
  A \textit{computational grid}, and the \textit{grid variables} associated
  with it.
\item[grid point]
  A spatial point in the \textit{computational grid}.
\item[grid scalar]
  A \textit{grid variable} with index zero,
  \textit{i.e.} just a number on each processor.
\item[grid variable]
  A variable which is passed through the Flesh interface, either between 
  thorns or between routines of the same thorn.
  This implies the variable is related to the computational grid as opposed
  to being an internal variable of the thorn or one of its routines. 
  \textit{grid scalar}, \textit{grid function}, and \textit{grid array}
  are all examples of \textit{grid variables}. 
  See Sections~\ref{sec:cactus_variables-groups},
  \ref{subsec:Appendix.interface-variables}
\item[GNATS]
  The GNU program we use for reporting and tracking bugs, comments and
  suggestions.
\item[GNU]
  \textit{GNU's Not Unix}: a freely-distributable code project.  
  See \url{http://www.gnu.org/}.
\item[GV]
  Shorthand for \textit{grid variable}.
\item[handle]
  A signed integer value $>= 0$ passed by many Cactus routines and
  used to represent a dynamic data or code object.
\item[HDF5]
  \textit{Hierarchical Data Format} version~5, an API, subroutine library, and
  file format for storing structured data.  An HDF5 file can
  store both data (for example Cactus grid variables), and meta data
  (data describing the other data, for example Cactus coordinate
  systems).
  See Section~\ref{subsec:hdf5}, also \url{http://hdf.ncsa.uiuc.edu/HDF5/}.
\item[implementation]
  Defines the interface that a thorn presents to the rest of a Cactus program.
  See Section~\ref{sec:implementations}.
\item[inherit] A thorn that \textit{inherits} from another implementation
  can access all the other implementation's public variables.
  See Section~\ref{subsec:interface_ccl}. \ref{sec:Appendix.interface}.
\item[interface]
\item[interpolation]
  Given a set of grid variables and interpolation points (points in the
  grid coordinate space which are typically distinct from the grid points)
  interpolation is the act of producing values for the grid variables 
  at each interpolation point over the entire grid hierarchy.  
  (Contrast with \textit{local interpolation}.)
\item[local array]
  An array that is declared in thorn code but not declared in the thorn's
  \verb|interface.ccl|, as opposed to a \textit{grid array}.
\item[local interpolation]
  Given a set of grid variables and interpolation points (points in the
  grid coordinate space which are typically distinct from the grid points)
  interpolation is the act of producing values for the grid variables
  at each interpolation point on a particular grid.  
  (Contrast with \textit{interpolation}.)
\item[Makefile]
  The default input file for \texttt{make} (or \texttt{gmake}).  Includes
  rules for building targets.
\item[make] A system for building software.  It uses rules involving
  dependencies of one part of software on another, and information of what
  has changed since the last build, to determine what parts need to be
  built.
\item[MPI]
  \textit{Message Passing Interface}, an API and software library for sending
  messages between processors in a multiprocessor system.
  See Sections~\ref{subsec:Compilation-Available_Options},
  \ref{subsubsec:Compiling-MPI}.
\item[multi-patch]
\item[mutual recursion]
  See \textit{recursion, mutual}.
\item[NUL character]
  The C programming language uses a ``NUL character'' to terminate
  character strings.  A NUL character has the integer value zero, but
  it's useful to write it as \verb|'\0'| to emphasize to human readers
  that this has type \verb|char| rather than \verb|int|.
\item[null pointer, NULL pointer]
  C defines a ``null pointer'' (often (slightly incorrectly) called
  a ``NULL pointer'') which is guaranteed not to point to any object.
  You get a null pointer by converting the integer constant 0 to a
  pointer type, e.g.\ \verb|int* ptr = 0;|.%%%
\footnote{%%%
       Note that if you have an expression which
       has the value zero, but which isn't an
       integer constant, converting this to a
       pointer type is \emph{not} guaranteed to
       give a NULL pointer, e.g.:\\
       {\tt int i = 0;}\\
       {\tt int* ptr = i;     /* ptr is NOT guaranteed to be a NULL pointer! */}\\
       }%%%

  Many programmers prefer to use the predefined macro \verb|NULL|
  (defined in \verb|<stdlib.h>|, \verb|<stdio.h>|, and possibly other
  system header files) to create null pointers,
  e.g.\ \verb|int* ptr = NULL;|, to emphasize to human readers that
  this is a null \emph{pointer} rather than ``just'' the integer zero.

  Note that it is explicitly \emph{not} defined whether a null pointer
  is represented by a bit pattern of all zero bits --- this varies from
  system to system, and there are real-world systems where null pointers
  are in fact \emph{not} represented this way.

  For further information, see the section ``Null pointers''
  in the (excellent) {\tt comp.lang.c FAQ}, available online at
  \url{http://www.eskimo.com/~scs/C-faq/top.html}.
\item[parallelisation]
  The process of utilising multiple computer processors to work on different
  parts of a computational problem at the same time, in order to obtain a
  solution of the problem more quickly.  Cactus achieves parallelisation
  by means of \textit{domain decomposition}.
\item[parameter]
  A variable that controls the run time behaviour of the Cactus executable.
  Parameters have default values which can be set in a
  \textit{parameter file} (See Chapter~\ref{chap:Cactus_parameters}).
  The Flesh has parameters; thorn parameters are made available to the rest
  of Cactus by describing them in the thorn's 
  {\tt param.ccl} file (See Appendix~\ref{sec:Appendix.param}).
\item[parameter file]
  (Also called \textit{par file}.) A text file used as the input of a
  Cactus program, specifying initial values of thorn parameters.
  See Section~\ref{sec:Parameter_File}.
\item[processor topology]
\item[PUGH]
  The default driver thorn for Cactus which uses MPI.
  See Section~\ref{sec:required_software}.
\item[PVM]
  \textit{Parallel Virtual Machine}, provides interprocessor communication.
  See Section~\ref{sec:required_software}.
\item[recursion, mutual]
  See \textit{mutual recursion}.
\item[reduction]
  Given a set of grid variables on a computational grid, \textit{reduction} 
  is the process of producing values for the variables on a proper subset of
  points from the grid.
\item[scheduler]
  The part of the Cactus Flesh that determines the order and circumstances
  in which to execute Cactus routines.  Thorn functions and schedule groups 
  are registered with the Flesh via the thorn's {\tt schedule.ccl} file to
  be executed in a certain schedule bin, before or after another function 
  or group executes, and so forth.  See Chapter~\ref{chap:scheduling},
  and Section \ref{sec:Appendix.schedule}.
\item[schedule bin]
  One of a set of special timebins pre-defined by Cactus.
  See Section \ref{sec:Appendix.schedule_bins} for a list.
\item[schedule group]
  A timebin defined by a thorn, in its {\tt schedule.ccl} file (see
  Appendix \ref{sec:Appendix.schedule}).
  Each schedule group must be defined to occur in a Cactus schedule bin or
  another schedule group.  
  See Chapter~\ref{chap:scheduling}, \ref{scheduling:schedule_bins}.
\item[shares] An implementation may \textit{share} restricted parameters
  with another implementation, which means the other implementation can
  get the parameter values, and if the parameters are steerable, it can
  change them.
  See Section~\ref{subsec:param_ccl}. \ref{sec:Appendix.param}.
\item[staggering]
  See Section~\ref{sec:staggering}.
\item[steerable parameter]
  A parameter which can be changed at any time after the program has been
  initialised.  See Section~\ref{sec:Cactus_parameters.steerable}.
\item[symmetry operation]
  A grid operation that is a manifestation of a geometrical symmetry,
  especially rotation or reflection.
\item[symmetry zone]
  A set of points laying at the edge of the computational grid and
  containing data obtained by some symmetry operation from
  another part of the same grid.
  (Contrast with \textit{boundary zone}, \textit{ghost zone}.)
\item[synchronisation]
  The process of copying information from the outer part of a 
  computational interior on one processor to the corresponding ghost zone 
  (see) on another processor.  Also refers to a special Cactus timebin
  corresponding to the occurrence of this process. 
  See Section~\ref{sec:ghost_size}.
\item[TAGS]
  See Section~\ref{sec:Appendix.tags}.
\item[target]
  A \textit{make target} is the name of a set of rules for 
  \texttt{make} (or \texttt{gmake}).  When the target is included in the
  command line for \texttt{make} the rules are executed, usually to
  build some software.
\item[test suite]
  See Sections~\ref{sec:testing}, \ref{sec:adding_test_suite}.
\item[thorn]
  A collection of subroutines defining a Cactus interface.  
  See Chapters~\ref{chap:thorn_concepts}, \ref{chap:thorn_anatomy}.
\item[ThornList]
  A file used by the Cactus CST to determine which thorns to compile 
  into a Cactus executable
  (see Section~\ref{sec:gmtafobuanadco}, \ref{sec:cointh}). 
  Can also be used to determine which thorns
  to check out from CVS (see Section~\ref{sec:checkout}).
  A ThornList for each Cactus configuration lies in the configuration
  subdirectory of the Cactus {\tt configs} directory.
\item[time bin]
  A time interval in the duration of a Cactus run wherein the Flesh 
  runs specified routines.  See \textit{scheduler}, \textit{schedule bin}.
\item[time level]
\item[timer]
  A Cactus API for reporting time.  See Section~\ref{sec:timers}.
\item[trigger]
\item[unigrid]
\item[WMPI]
  \textit{Win32 Message Passing Interface}.
  See Sections~\ref{subsec:Compilation-Available_Options},
  \ref{subsubsec:Compiling-MPI}.
\item[wrapper]

\end{Lentry}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\chapter{Configuration file syntax}
\label{sec:Appendix.ccl}

\section{General Concepts}

Each thorn is configured by three compulsory and one optional files in the
top level thorn directory:
\begin{itemize}
\item{} {\tt interface.ccl}
\item{} {\tt param.ccl}
\item{} {\tt schedule.ccl}
\item{} {\tt configuration.ccl} (optional)
\end{itemize}
These files are written in the \textit{Cactus Configuration Language} which is
case insensitive.

% A note on optional arguments and square brackets:  If e.g.\ a variable or
% include file is provided by this thorn, then the corresponding
% section of the (in this case) interface.ccl is not optional.  Thus
% these sections are not enclosed in [] as a whole (though parts of them can of
% course be enclosed in []).

\section{interface.ccl}
\label{sec:Appendix.interface}

The interface configuration file consists of
\begin{itemize}
\item a header block giving details of the thorn's relationship with
other thorns.
\item a block detailing which include files are used from other
thorns, and which include files are provided by this thorn
\item blocks detailing aliased functions provided or used by this thorn
\item a series of blocks listing the thorn's global variables.
\end{itemize}
(For a more extensive discussion of Cactus variables see Chapter
\ref{chap:cactus_variables}.)

\subsection{Header block}
The header block has the form:
\begin{alltt}
implements: <\var{implementation}>
inherits: <\var{implementation}>, <\var{implementation}>
friend: <\var{implementation}>, <\var{implementation}>
\end{alltt}
where
\begin{itemize}
\item{} The implementation name must be unique among all thorns, except
        between thorns which have the same public and protected variables and
        global and restricted parameters.
\item{} Inheriting from another implementation makes all that implementation's
        public variables available to your thorn. At least one thorn
        providing any inherited implementation must be present at compile time.
        A thorn cannot inherit from itself. Inheritance is transitive
	(if $A$ inherits from $B$, and $B$ inherits from $C$, then
	$A$ also implicitly inherits from $C$),
	but not commutative.
% FIXME: Jonathan and Thomas aren't sure if this is true!
%	(if $A$ inherits from $B$, then $B$ can't inherit from $A$)
\item{} Being a friend of another implementation makes all that
        implementation's protected variables available to your thorn.
	At least one thorn providing an implementation for each friend
	must be present at compile time.
	A thorn cannot be its own friend.
% FIXME: what does "associative" mean here???
	Friendship is associative,
	commutative
% FIXME: Jonathan and Thomas aren't sure whether B *must* be declared this way
%        or is *implicitly* declared this way
%	(if $A$ is a friend of $B$, then $B$ is also a friend of $A$),
	and transitive (i.e.~if $A$ is a friend of $B$
	and $B$ is a friend of $C$, then $A$ is implicitly a friend of $C$).
\end{itemize}

\subsection{Include files}
The include file section has the form:
\begin{alltt}
USES INCLUDE [SOURCE|HEADER]: <\var{file_name}>
INCLUDE[S] [SOURCE|HEADER]: <\var{file_to_include}> in <\var{file_name}>
\end{alltt}
The former is used when a thorn wishes to use an include file from
another thorn.  The latter indicates that this thorn adds the code in
\texttt{<\var{file\_to\_include}>} to the include file \texttt{<\var{file\_name}>}.  If
the include file is described as \verb|SOURCE|, the included code is
only executed if the providing thorn is active.
Both default to \verb|HEADER|.

\subsection{Function aliasing}
\label{subsec:Appendix.interface.function_aliasing}
If any aliased function is to be used or provided by the thorn then
the prototype must be declared with the form:
\begin{alltt}
<\var{return_type}> FUNCTION <\var{alias}>(<\var{arg1_type}> <\var{intent1}> [ARRAY] <\var{arg1}>, ...)
\end{alltt}
The \texttt{<\var{return\_type}>} must be either \verb|void|, \verb|CCTK_INT|
or \verb|CCTK_REAL|. The keyword \verb|SUBROUTINE| is equivalent to
\verb|void FUNCTION|. The name of the aliased function \texttt{<\var{alias}>}
must contain at least one uppercase and one lowercase letter and
follow the C standard for function names.  The type of each argument,
\texttt{<\var{arg*\_type}>} must be a \verb|CCTK| type in
\verb|INT,REAL,STRING,POINTER|. The intent of each argument,
\texttt{<\var{intent*}>} must be either \verb|IN|, \verb|OUT|, or
\verb|INOUT|. An argument should only be modified if it is declared
to have intent \verb|OUT| or \verb|INOUT|. If the argument is an array
then the prefix \verb|ARRAY| should also be given.

If the argument \texttt{<\var{arg*}>} is a function pointer then the argument
itself (which will preceded by the return type) should be
\begin{alltt}
CCTK_FPOINTER <\var{function_arg1}>(<\var{arg1_type}> <\var{intent1}> <\var{arg1}>, ...)
\end{alltt}
Function pointers may not be nested. 

If an aliased function is to be required then the block
\begin{alltt}
REQUIRES FUNCTION <\var{alias}>
\end{alltt}
is required.

If an aliased function is to be (optionally) used then the block
\begin{alltt}
USES FUNCTION <\var{alias}>
\end{alltt}
is required.

If a function is provided then the block
\begin{alltt}
PROVIDES FUNCTION <\var{alias}> WITH <\var{provider}> LANGUAGE <\var{providing_language}>
\end{alltt}
is required. As with the alias name, \texttt{<\var{provider}>} must contain at
least one uppercase and one lowercase letter and follow the C standard
for function names. Currently the only supported values of
\texttt{<\var{providing\_language}>} are \verb|C| and \verb|Fortran|.


\subsection{Variable blocks}
\label{subsec:Appendix.interface-variables}
The thorn's variables are collected into groups. This is not only
for convenience, but for collecting like variables together.
Storage assignment, communication assignment, and ghostzone synchronization
take place for groups only.

The thorn's variables are defined by:

\begin{alltt}
[<\var{access}>:]

<\var{data_type}> <\var{group_name}>[[<\var{number}>]] [TYPE=<\var{group_type}>] [DIM=<\var{dim}>]
[TIMELEVELS=<\var{num}>]
[SIZE=<\var{size in each direction}>] [DISTRIB=<\var{distribution_type}>]
[GHOSTSIZE=<\var{ghostsize}>] [STAGGER=<\var{stagger-specification}>]
[TAGS=<\var{string}>]  ["<\var{group_description}>"]
[\{
 [ <\var{variable_name}>[,]<\var{variable_name}>
   <\var{variable_name}> ]
\} ["<\var{group_description}>"] ]
\end{alltt}
%
(The options {\t TYPE}, {\t DIM}, etc.\ following {\t <\var{group\_name}>}
must all appear on one line.)  Note that the beginning brace (\{) must
sit on a line by itself; the ending brace (\}) must be preceded by a
carriage return.
%
\begin{itemize}
\item{} {\t \var{access}} defines which thorns can use the following
        groups of variables. {\t \var{access}} can be either
        {\t public}, {\t protected} or {\t private}.
\item{} {\t \var{data\_type}} defines the data type of the variables in the
group.  Supported data types are {\t CHAR}, {\t BYTE}, {\t INT}, {\t REAL} and
{\t COMPLEX}. 
\item{} {\t \var{group\_name}} must be an alphanumeric name (which may also
contain underscores) which is unique across group and variable names
within the scope of the thorn. A group name is compulsory.
\item{} {\t [\var{number}]}, if present, indicates that this is a
  \emph{vector} group.  The number can be any valid arithmetical
  expression consisting of integers or integer-valued parameters.
  Each variable in that group appears as a one-dimensional array of
  grid variables.  When the variable is accessed in the code, then the
  last index is the member-index, and any other indices are the normal
  spatial indices for a group of this type and dimension.
\item{} {\t TYPE} designates the kind of variables held by the group.
The choices are {\t GF}, {\t ARRAY} or {\t SCALAR}. This field is
optional, with the default variable type being {\t SCALAR}.
\item{} {\t DIM} defines the spatial dimension of the {\t ARRAY} or
{\t GF}.  The default value is {\t DIM=3}.
\item{} {\t TIMELEVELS} defines the number of timelevels a group has if
        the group is of type {\t ARRAY} or {\t GF}, and can take any positive
        value. The default is one timelevel.
\item{} {\t SIZE} defines the number grid-points an {\tt ARRAY} has in each direction.
        This should be a comma-separated list of valid arithmetical
        expressions consisting of integers or integer-valued parameters.
\item{} {\t DISTRIB} defines the processor decomposition of an {\tt ARRAY}.
        {\tt DISTRIB=DEFAULT} distributes {\tt SIZE} grid-points
        across all processors. {\tt DISTRIB=CONSTANT} implies that
        {\tt SIZE} grid-points should be allocated on each
        processor. The default value is {\tt DISTRIB=DEFAULT}.
\item{} {\t GHOSTSIZE} defines number of ghost-zones in each dimension
of an {\tt ARRAY}.  
%Does GHOSTSIZE default to one for a GF and zero for a GA?
\item{} {\t STAGGER} defines position of grid-points of a {\tt GF} with respect to
        the underlying grid.  It consists of a string made up of a combination {\tt DIM}
        of the letters {\tt M}, {\tt C}, {\tt P} depending on whether the layout in
        that direction is on the {\tt M}inus face, {\tt C}entre, or {\tt P}lus face
        of the cell in that dimension.
\item{} {\t TAGS} defines an optional string which is used to create a
	set of key-value pairs associated with the group. The keys are case
	independent.  The string (which must be deliminated by single or
	double quotes) is interpreted by the function
	{\t Util\_TableSetFromString()}, which is described in the
        ReferenceManual.\\
        Currently the CST parser and the flesh do not evaluate any information
        passed in an optional {\t TAGS} string. Thorns may do so by
        querying the key/value table information for a group by using
        {\t CCTK\_GroupTagsTable()} and the
        appropriate {\t Util\_TableGet*()} utility functions
        (see the ReferenceManual for detailed descriptions).\\
        For a list of currently supported {\t TAGS} key-value table information
        please refer to the corresponding chapter in the documentation of the
        \verb|CactusDoc| arrangement (section \ref{sec:OtherGmakeTargetsDoc} on
        page \pageref{sec:OtherGmakeTargetsDoc} explains how to build this
        documentation).
\item{} The (optional) block following the group declaration line
contains a list of variables contained in the group. All variables in
a group have the same data type, variable type, dimension and
distribution. The list
can be separated by spaces, commas, or new lines. The variable names
must be unique within the scope of the thorn.  A variable can only be
a member of one group. The block must be delimited by brackets on new
lines.  If no block is given after a group declaration line, a
variable with the same name as the group is created. Apart from this case, 
a group name cannot be the same as the name of any variable seen by this thorn.
\item{} An optional description of the group can be given on the last
line.  If the variable block is omitted, this description can be given
at the end of the declaration line.
\end{itemize}

The process of sharing code among thorns using include files is
discussed in Section~\ref{sec:includefiles}.

\section{param.ccl}
\label{sec:Appendix.param}

The parameter configuration file consists of a list of
\textit{parameter object specification items} (OSIs) giving the type and
range of the parameter separated by optional
\textit{parameter data scoping items} (DSIs) which detail access to the
parameter.  (For a more extensive discussion of Cactus parameters see Chapter
\ref{chap:Cactus_parameters}.)

\subsection{Parameter data scoping items}

\begin{alltt}
<\var{access}>:
\end{alltt}

The key word {\t \var{access}} designates that all parameter object specification
items up to the next parameter data scoping item are in the same
protection or scoping class. {\tt \var{access}} can take the values:
\begin{Lentry}
\item[{\tt global}] all thorns have access to global parameters
\item[{\tt restricted}] other thorns can have access to these
                           parameters if they specifically request
                           it in their own param.ccl
\item[{\tt private}] only your thorn has access to private parameters
\item[{\tt shares}] in this case an {\t implementation} name must
follow the colon. It declares that all the parameters in the following
scoping block are restricted variables from the specified {\tt
implementation}.  (Note: only one implementation can be specified
on this line.)
\end{Lentry}


\subsection{Parameter object specification items}
\label{subsec:Appendix.param.specification_items}

\begin{alltt}
[EXTENDS|USES] <\var{parameter type}> <\var{parameter name}>[[<\var{integer}>]] "<\var{parameter description}>" 
[AS <\var{alias}>] [STEERABLE=<NEVER|ALWAYS|RECOVER>] 
[ACCUMULATOR=<\var{expression}>] [ACCUMULATOR-BASE=<\var{parameter name}>]
\{
  <\var{PARAMETER VALUES}>
\} <\var{default value}>
\end{alltt}
where the options {\t AS}, {\t STEERABLE}, etc. following {\t <\var{parameter description}>}
must all appear on one line.
  Note that the beginning brace ({\t\{}) must
sit on a line by itself; the ending brace ({\t\}}) must be at the
beginning of a line followed by \var{<default value>} on that same line.
\begin{itemize}
\item
  Allowed {\t \var{parameter type}}s are
  \begin{Lentry}
  \item[{\t INT}]
    The specification of \var{parameter value}s takes the form of one
    or more lines, each of the form
    \begin{alltt}
      \var{<range description>} [::"\var{<comment describing this range>}"]
    \end{alltt}
    Here a \var{<range description>} specifies a set of integers,
    and has one of the following forms:
    \begin{alltt}
      *                           \# means any integer
      \var{<integer>}                   \# means only \var{<integer>}
      \var{<lower bound>}:\var{<upper bound>} \# means all integers in the range
                                  \# from \var{<lower bound>} to \var{<upper bound>}
      \var{<lower bound>}:\var{<upper bound>}:\var{<positive integer step>}
                                  \# means all integers in the range
                                  \# from \var{<lower bound>} to \var{<upper bound>}
                                  \# in steps of \var{<positive integer step>}
    \end{alltt}
    where \var{<lower bound>} has one of the forms
    \begin{alltt}
      \var{<empty field>}   \# means no lower limit
      *               \# means no lower limit
      \var{<integer>}       \# means a closed interval starting at \var{<integer>}
      (\var{<integer>}      \# means an open interval starting at \var{<integer>}
    \end{alltt}
    and \var{<upper bound>} has one of the forms
    \begin{alltt}
      \var{<empty field>}   \# means no upper limit
      *               \# means no upper limit
      \var{<integer>}       \# means a closed interval ending at \var{<integer>}
      \var{<integer>})      \# means an open interval ending at \var{<integer>}
    \end{alltt}

  \item[{\t REAL}]
    The range specification is the same as integers, except that here,
    no \var{step} implies a continuum of values.  Note that numeric
    constants should be expressed as in C (e.g.\ {\t 1e-10}).  Note
    also that one cannot use the Cactus types such as {\t CCTK\_REAL4}
    to specify the precision of the parameter; parameters always have
    the default precision.

  \item[{\t KEYWORD}]
    Each entry in the list of acceptable values for a keyword has the
    form
    \begin{alltt}
      <\var{keyword value}>, <\var{keyword value}> :: "<\var{description}>"
    \end{alltt}
    Keyword values should be enclosed in double quotes.  The double
    quotes are mandatory if the keyword contains spaces.

  \item[{\t STRING}]
    Allowed values for strings should be specified using regular
    expressions. To allow any string, the regular expression {\tt ""}
    should be used.  (An empty regular expression matches anything.)
    Regular expressions and string values should be enclosed in double
    quotes.  The double quotes are mandatory if the regular expression
    or the string value is empty or contains spaces.

  \item[{\t BOOLEAN}]
    No \var{PARAMETER VALUES} should be specified for a boolean
    parameter. The default value for a boolean can be
    \begin{itemize}
    \item True: {\t 1}, {\t yes}, {\t y}, {\t t}, {\t true}
    \item False: {\t 0}, {\t no}, {\t n}, {\t f}, {\t false}
    \end{itemize}
    Boolean values may optionally be enclosed in double quotes.

  \end{Lentry}

\item
  The \var{parameter name} must be unique within the scope of the
  thorn.

\item
  A thorn can declare that it {\t EXTENDS} a parameter of another
  thorn. This allows it to declare additional acceptable values.  By
  default it is acceptable for two thorns to declare the same value as
  acceptable.

\item
  If the thorn wants to simply use a parameter from another thorn,
  without declaring additional values, use {\t USES} instead.

\item
  {\tt [\var{integer}]}, if present, specifies that this is an {\tt
  array} parameter. (Note that the notation used above for the
  parameter specification breaks down here, there must be square
  brackets around the given integer).  The parameter is then a
  one-dimensional array of values of the specified type.

\item
  \var{alias} allows a parameter to appear as a different name in this
  thorn than its original name in another thorn.  The name as seen in
  the parameter file is unchanged.

\item
  {\t STEERABLE} specifies when a parameter value may be changed.  By
  default parameters may not be changed after the parameter file has
  been read, or on restarting from checkpoint.  This option relaxes
  this restricting, specifying that the parameter may be changed at
  recovery time from a parameter file or at any time using the Flesh
  routine {\tt CCTK\_ParameterSet} --- see the Reference Guide.

\item
  {\t ACCUMULATOR} specifies that this is an \textit{accumulator}
  parameter.  Such parameters cannot be set directly, but are set by
  other parameters who specify this one as an {\tt ACCUMULATOR-BASE}.
  The expression is a two parameter arithmetical expression of $x$ and
  $y$.  Setting the parameter consists of evaluating this expression
  successively, with $x$ being the current value of the parameter (at
  the first iteration this is the default value) and $y$ the value of
  the setting parameter.  This procedure is repeated, starting from
  the default value of the parameter, each time one of the setting
  parameters changes.

\item
  {\t ACCUMULATOR-BASE} specifies the name of an {\tt ACCUMULATOR}
  parameter which this parameter sets.

\end{itemize}

\section{schedule.ccl}
\label{sec:Appendix.schedule}

(A more extensive discussion of Cactus scheduling is provided in Chapter
\ref{chap:scheduling}.) A schedule configuration file consists of
\begin{itemize}

\item{} \textit{assignment statements} to switch on storage for
  grid variables for the entire duration of program execution

\item{} \textit{schedule blocks} to schedule a subroutine from a thorn
  to be called at specific times during program execution in a given manner

\item {} \textit{conditional statements} for both assignment statements and
  schedule blocks to allow them to be processed depending on parameter values

\end{itemize}

\subsection{Assignment Statements}

\textit{Assignment statements}, currently only assign storage.

These lines have the form:
\begin{alltt}
[STORAGE: <\var{group}>[\var{timelevels}], <\var{group}>[\var{timelevels}]]
\end{alltt}

If the thorn is active, storage will be allocated for the given groups
for the duration of program execution (unless storage is explicitly
switched off by some call to {\tt CCTK\_DisableGroupStorage} within a
thorn). 

The storage line includes the number of timelevels to activate storage
for, this number can be from 1 up to the maximum number or timelevels
for the group as specified in the defining {\tt interface.ccl}
file. If the maximum number of timelevels is 1 (the default), this
number may be omitted.

The behaviour of an assignment statement is independent of its
position in the schedule file (so long as it is outside a schedule
block). 

\subsection{Schedule Blocks}

Each \textit{schedule block} in the file {\t schedule.ccl} must have the syntax

\begin{alltt}
schedule [GROUP] <\var{function name}|\var{group name}> AT|IN <\var{time}> \verb|\|
     [BEFORE|AFTER <\var{function name}>|(<\var{function name}> <\var{function name}> ...)] \verb|\|
     [WHILE <\var{variable}>] [AS <\var{alias}>]
\{
  [LANG: <\var{language}>]
  [STORAGE:       <\var{group}>[\var{timelevels}],<\var{group}>[\var{timelevels}]...]
  [TRIGGER:       <\var{group}>,<\var{group}>...]
  [SYNCHRONISE:   <\var{group}>,<\var{group}>...]
  [OPTIONS:       <\var{option}>,<\var{option}>...]
\} "\var{Description of function}"
\end{alltt}

\begin{Lentry}
  \item[{\tt GROUP}] Schedule a schedule group with the same options
  as a schedule function.  The schedule group will be created if it doesn't exist.

  \item[{\tt <\var{function name}|\var{group name}>}] The name of a function or a
  schedule group to be scheduled.  Function and schedule group names
  are case sensitive

  \item[{\tt <\var{group}>}] A group of grid variables. Variable groups
  inherited from other thorns may be used, but they must then be fully
  qualified with the implementation name.

  \item[{\tt AT}] Functions can be scheduled to run at the Cactus
  schedule bins, for example {\tt CCTK\_EVOL}, {\tt CCTK\_STARTUP}. A
  complete list and description of these is provided in
  Appendix~\ref{sec:Appendix.schedule_bins}.  The initial letters
  {\tt CCTK\_} are optional. Grid variables cannot be used in the
  {\tt CCTK\_STARTUP} and {\tt CCTK\_SHUTDOWN} timebins.

  \item[{\tt IN}] Schedules a function or schedule group to run in a
  schedule group rather than in a Cactus timebin.

  \item[{\tt BEFORE/AFTER}] Takes a function name, a function alias,
  a schedule group name, or a parentheses-enclosed whitespace-separated
  list of these.  (Any names that are not provided by an active thorn
  are ignored.)  Note that a single schedule block may have multiple
  {\tt BEFORE/AFTER} clauses.  See section~\ref{chap:scheduling}
  (``Scheduling'') in the Cactus Users' Guide for more information
  about {\tt BEFORE/AFTER} clauses.

  \item[{\tt WHILE}] Executes a function or schedule group until the given
    variable (which must be a fully qualified integer grid scalar) has the value zero.

  \item[{\tt AS}] Provides an alias for a function or schedule group
  which should be used for scheduling before, after or in.  This can
  be used to provide thorn independence for other thorns scheduling
  functions or schedule groups relative to this one.

  \item[{\tt LANG}] The code language for the function (either {\tt C} or {\tt
    FORTRAN}). No language should be specified for a schedule group.

  \item[{\tt STORAGE}] List of variable groups which should have storage
  switched on for the duration of the function or schedule group.
  Each group must specify how many timelevels to activate storage for,
  from 1 up to the maximum number for the group as specified in the
  defining {\tt interface.ccl} file. If the maximum is 1 (the default)
  this number may be omitted.

  \item[{\tt TRIGGER}] List of grid variables or groups to be used as
  triggers for causing an {\tt ANALYSIS} function or group to be
  executed.  Any schedule block for an analysis function or analysis
  group may contain a {\tt TRIGGER} line.

  \item[{\tt SYNCHRONISE}] List of groups to be synchronised as soon
    as the function or schedule group is exited.

  \item[{\tt OPTIONS}] List of additional options (see
    Section~\ref{app:allopts}) for the scheduled function or group of functions

\end{Lentry}

\subsubsection{Allowed Options}

\label{app:allopts}

Cactus understands the following options.  These options are
interpreted by the driver, not by Cactus.  The current set of options
is useful for Berger-Oliger mesh refinement which has subcycling in
time, and for multi-patch simulations in which the domain is split
into several distinct patches.  Given this, the meanings of the
options below is only tentative, and their exact meaning needs to be
obtained from the driver documentation.  The standard driver PUGH
ignores all options.

Option names are case-insensitive.  There can be several options given
at the same time.

\begin{Lentry}
  
\item[{\tt META}] This routine will only be called once, even if
  several simulations are performed at the same time.  This can be
  used, for example, to initialise external libraries, or to set up
  data structures that live in global variables.
  
\item[{\tt GLOBAL}] This routine will only be called once on a grid
  hierarchy, not for all subgrids making up the hierarchy.  This can
  be used, for example, for analysis routines which use global
  reduction or interpolation routines rather than the local subgrid
  passed to them, and hence should only be called once.
  
\item[{\tt LEVEL}] This routine will only be called once on any
  ``level'' of the grid hierarchy.  That is, it will only be called
  once for any set of sub-grids which have the same
  \texttt{cctk\_levfac} numbers.
  
\item[{\tt SINGLEMAP}] This routine will only be called once on any of
  the ``patches'' that form a ``level'' of the grid hierarchy.
  
\item[{\tt LOCAL} (this is the default)] This routine will be called
  on every ``component''.

\end{Lentry}

When the above options are used, it is often the case that a certain
routine should e.g.\ be called at the time for a \texttt{GLOBAL}
routine, but should actually loop over all ``components''.  The
following set of options allows this:

\begin{Lentry}
  
\item[{\tt LOOP-META}] Loop once.
  
\item[{\tt LOOP-GLOBAL}] Loop over all simulations.
  
\item[{\tt LOOP-LEVEL}] Loop over all ``levels''.
  
\item[{\tt LOOP-SINGLEMAP}] Loop over all ``patches''.
  
\item[{\tt LOOP-LOCAL}] Loop over all ``components''.

\end{Lentry}

For example, the specification
\begin{alltt}
  OPTIONS: global loop-local
\end{alltt}
schedules a routine at those time when a \texttt{GLOBAL} routine is
scheduled, and then calls the routine in a loop over all
``components''.


\subsection{Conditional Statements}

Any schedule block or assignment statements can be optionally
surrounded by conditional {\t if-elseif-else}
constructs using the parameter data base. These can be nested,
and have the general form:

\begin{alltt}
if (CCTK_Equals(<\var{parameter}>,<\var{string}>))
\{
  [<\var{assignments}>]
  [<\var{schedule blocks}>]
\}\end{alltt}

Such conditionals are evaluated only at program startup and are used
to pick between different static schedule options.  For dynamic
scheduling the {\tt SCHEDULE WHILE} construction should be used.

Conditional constructs cannot be used inside a schedule block.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{configuration.ccl}
\label{sec:Appendix.configuration.ccl}

[{\bf NOTE:} The configuration.ccl is still relatively new, and not
all features listed below may be fully implemented or functional.]

A configuration.ccl file defines {\bf capabilities} which a thorn
either provides or requires, or may use if available.  Unlike {\bf
implementations}, only one thorn providing a particular capability may
be compiled into a configuration at one time.  Thus this mechanism may
be used to, for example: provide access to external libraries; provide
access to functions which other thorns must call, but are too complex
for function aliasing; or to split a thorn into several thorns, all of
which require some common (not aliased) functions.

A configuration options file can contain any number of the following
sections:

\begin{itemize}

\item

\begin{alltt}
PROVIDES <\var{Capability}>
\{
  SCRIPT <\var{Configuration script}>
  LANG <\var{Language}>
  [OPTIONS [<option>[,<option>]...]] 
\}
\end{alltt}

Informs the CST that this thorn provides a given capability, and that
this capability has a given detection script which may be used to
configure it (e.g. running an autoconf script or detecting an external
library's location).  The script should output configuration
information on its standard output --- the syntax is described below
in sectio \ref{sec:Appendix.configuration.ccl.configscript}.  The
script may also indicate the failure to detect a capability by
returning a non-zero exit code;  this will stop the build after the
CST stage.

Scripts can be in any language.  If an interpreter is needed to run
the script, for example \verb|Perl|, this should be indicated by the
\verb|LANG| option.

The specified options are checked for in the original configuration
and any options passed on the command line (including an `options'
file) at compile time when the thorn is added, or if the CST is
rerun.  These options need be set only once, and will be remembered
between builds.

\item

\begin{alltt}
REQUIRES  <\var{Capability}>
\end{alltt}

Informs the CST that this thorn requires a certain capability to be
present.  If no thorn providing the capability is in the ThornList,
the build will stop after the CST stage.

\begin{alltt}
OPTIONAL <\var{Capability}>
\{
  DEFINE <\var{macro}>
\}
\end{alltt}

Informs the CST that this thorn may use a certain capability if a
thorn providing it is in the ThornList.  If present, the preprocessor
macro \verb|macro| will be defined and given the value ``1''.

\end{itemize}

\subsection{Configuration Scripts}
\label{sec:Appendix.configuration.ccl.configscript}

The configuration script may tell the CST to add certain features to
the Cactus environment --- either to the make system or to header
files included by thorns.  It does this by outputting lines to its
standard output:

\begin{itemize}

\item

\begin{alltt}
BEGIN DEFINE 
<text>
END DEFINE
\end{alltt}

Places a set of definitions in a header file which will be included by
all thorns using this capability (either through an OPTIONAL or
REQUIRES entry in their configuration.ccl files).

\item

\begin{alltt}
INCLUDE_DIRECTORY  <directory>
\end{alltt}

Adds a directory to the include path used for compiling files in
thorns using this capability.

\item

\begin{alltt}
BEGIN MAKE_DEFINITION 
<text>
END MAKE_DEFINITION
\end{alltt}

Adds a makefile definition into the compilation of all thorns using
this capability.

\item

\begin{alltt}
BEGIN MAKE_DEPENDENCY 
<text>
END MAKE_DEPENDENCY 
\end{alltt}

Adds makefile dependency information into the compilation of all
thorns using this capability.

\item

\begin{alltt}
LIBRARY <library>
\end{alltt}

Adds a library to the final cactus link.

\item

\begin{alltt}
LIBRARY_DIRECTORY <library>
\end{alltt}

Adds a directory to the list of directories searched for libraries at
link time.

\end{itemize}

No other lines should be output by the script.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\chapter{Schedule bins}
\label{sec:Appendix.schedule_bins}

Using the {\tt schedule.ccl} files, thorn functions can be scheduled to run 
in the different timebins which are executed by the Cactus Flesh. This chapter
describes these standard timebins, and shows the flow of program execution
through them.

Scheduled functions must be declared as
\begin{Lentry}

\item[In C:]
\begin{verbatim}
#include "cctk_Arguments.h"
void MyFunction (CCTK_ARGUMENTS);
\end{verbatim}

\item[In C++:]
\begin{verbatim}
#include "cctk_Arguments.h"
extern "C" void MyFunction (CCTK_ARGUMENTS);
\end{verbatim}

\item[In Fortran:]
\begin{verbatim}
#include "cctk_Arguments.h"
subroutine MyFunction (CCTK_ARGUMENTS)
   DECLARE_CCTK_ARGUMENTS
end
\end{verbatim}
\end{Lentry}

Exceptions are the functions that are scheduled in the bins {\tt
CCTK\_STARTUP}, {\tt CCTK\_RECOVER\_PARAMETERS}, and {\tt
CCTK\_SHUTDOWN}.  They do not take arguments,
and they return an integer.  They must be declared as
\begin{Lentry}

\item[In C:]
\begin{verbatim}
int MyFunction (void);
\end{verbatim}

\item[In C++]
\begin{verbatim}
extern "C" int MyFunction ();
\end{verbatim}

\item[In Fortran:]
\begin{verbatim}
integer function MyFunction ()
end
\end{verbatim}
\end{Lentry}

The return value in {\tt CCTK\_STARTUP} and {\tt CCTK\_SHUTDOWN} is
unused, and might in the future be used to indicate whether an error
occurred.  You should return 0.

The return value in {\tt CCTK\_RECOVER\_PARAMETERS} should be zero,
positive, or negative, indicating that no parameters were recovered,
that parameters were recovered successfully, or that an error
occurred, respectively.  Only the first routine that returns a
positive value in the {\tt CCTK\_RECOVER\_PARAMETERS} bin will be
executed; all later routines will be ignored.  The execution order in
this bin is unspecified.



\begin{Lentry}

\item[{\tt CCTK\_RECOVER\_PARAMETERS}]
        Used by thorns with relevant IO methods as the point 
        to read parameters when recovering from checkpoint files.
        Grid variables are not available in this timebin.  This bin is
        special (see above); do not schedule any routines here except
        if they recover parameters.

\item[{\tt CCTK\_STARTUP}] 
        Run before any grids are constructed, this is 
        the timebin for example where grid independent information 
        (e.g.\ output methods, reduction operators) is registered. 
        Note that since no grids are setup at this point, grid 
        variables cannot be used in routines scheduled here.

\item[{\tt CCTK\_WRAGH}]
        This timebin is executed when all parameters are known, but
        before the driver thorn constructs the grid.  It should only
        be used to set up information that is needed by the driver.

\item[{\tt CCTK\_PARAMCHECK}] 
        This timebin is for thorns to check the validity of
        parameter combinations. This bin is also executed before the
        grid hierarchy is made, so that routines scheduled here only
        have access to the global grid size and the parameters.

\item[{\tt CCTK\_BASEGRID}]
        This timebin is executed very early after a driver thorn
        constructs grid; this bin should only be used to set up
        coordinate systems on the newly created grids.

\item[{\tt CCTK\_INITIAL}] 
        This is the place to set up any required initial data. This timebin
        is not run when recovering from a checkpoint file.

\item[{\tt CCTK\_POSTINITIAL}]
        This is the place to modify initial data, or to calculate data
        that depend on the initial data.  This timebin is also not run
        when recovering from a checkpoint file.

\item[{\tt CCTK\_RECOVER\_VARIABLES}]
        Used by thorns with relevant IO methods as the point 
        to read in all the grid variables when recovering from 
        checkpoint files.
        
\item[{\tt CCTK\_POST\_RECOVER\_VARIABLES}]
        This timebin exists for scheduling any functions which need 
        to modify grid variables after recovery.

\item[{\tt CCTK\_CPINITIAL}]
        Used by thorns with relevant IO methods as the point to checkpoint
        initial data if required.

\item[{\tt CCTK\_CHECKPOINT}]
        Used by thorns with relevant IO methods as the point to checkpoint
        data during the iterative loop when required.
                
\item[{\tt CCTK\_PRESTEP}]
        The timebin for scheduling any routines which need to be 
        executed before any routines in the main evolution step. This 
        timebin exists for thorn writers convenience, the {\tt BEFORE}, 
        {\tt AFTER} etc. functionality of the {\tt schedule.ccl} file
        should allow all functions to be scheduled in the main {\tt CCTK\_EVOL}
        timebin.
        
\item[{\tt CCTK\_EVOL}]
        The timebin for the main evolution step.

\item[{\tt CCTK\_POSTSTEP}]
        The timebin for scheduling any routines which need to be 
        executed after all the routines in the main evolution step. This 
        timebin exists for thorn writers convenience, the {\tt BEFORE}, 
        {\tt AFTER} etc. functionality of the {\tt schedule.ccl} file
        should allow all functions to be scheduled in the main {\tt CCTK\_EVOL}
        timebin.
        
\item[{\tt CCTK\_POSTRESTRICT}]
        This timebin is used only in mesh refinement settings.  It is
        ignored for unigrid runs.  This bin is executed after each
        restriction operation.  It is, e.g., necessary to re-apply the
        boundary conditions after every restriction operation.

\item[{\tt CCTK\_POSTREGRID}]
        This timebin is used in mesh refinement settings, or when there are
        multiple grid patches.  It is ignored for unigrid runs.  This bin is
        executed whenever the grid hierarchy or patch setup has changed.  It
        is, e.g., necessary to re-apply the boundary conditions or
        recalculate the grid points' coordinates after every changing the
        grid hierarchy.

\item[{\tt CCTK\_ANALYSIS}]
        The {\tt ANALYSIS} timebin is special, in that it is closely coupled 
        with output, and routines which are scheduled here are typically
        only executed if output of analysis variables is required. 
        Routines which perform analysis should be independent of the main 
        evolution loop (that is, it should not matter for the results
        of a simulation whether routines in this timebin are executed or 
        not). 

\item[{\tt CCTK\_TERMINATE}]
        Called after the main iteration loop when Cactus terminates. 
        Note that sometime in this timebin a driver thorn should be 
        destroying the grid hierarchy and removing grid variables. 

\item[{\tt CCTK\_SHUTDOWN}]
        Cactus final shutdown routines, after the grid hierarchy has been 
        destroyed. Grid variables are no longer available.

\end{Lentry}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\chapter{Flesh parameters}

The Flesh parameters are defined in the file {\tt src/param.ccl}.

\section{Private parameters}

\begin{Lentry}

\item[{\tt allow\_mixeddim\_gfs}]
Allow use of GFs from different dimensions [{\tt no}]

\item [{\tt cctk\_brief\_output}]
Give only brief output [{\tt no}]

\item[{\tt cctk\_full\_warnings}]
Give detailed information for each warning statement [{\tt yes}]

\item [{\tt cctk\_run\_title}]
Description of this simulation [{\tt ""}]

\item [{\tt cctk\_show\_banners}]
Show any registered banners for the different thorns [{\tt yes}]

\item [{\tt cctk\_show\_schedule}]
Print the scheduling tree to standard output [{\tt yes}]

\item[{\tt cctk\_strong\_param\_check}]
Die on parameter errors in {\tt CCTK\_PARAMCHECK} [{\tt yes}]

\item[{\tt cctk\_timer\_output}]
Give timing information [{\tt off}] \{{\tt off, full}\}

\item[{\tt manual\_cache\_setup}]
Set the cache size manually [{\tt no}]

\item[{\tt manual\_cache\_size}]
The size to set the cache to if not done automatically (bytes) [{\tt 0}]

\item[{\tt manual\_cacheline\_bytes}]
The size of a cacheline if not set automatically (bytes) [{\tt 0}]

\item[{\tt recovery\_mode}]
How to behave when recovering from a checkpoint [{\tt strict}] \{{\tt strict, relaxed}\}

\item[{\tt highlight\_warning\_messages}]
Highlight CCTK warning messages [{\tt yes}]

\item[{\tt info\_format}]
Specifies the content and format of \code{CCTK\_INFO()}/\code{CCTK\_VInfo()}
messages. [{\tt basic}]
% don't want lines broken in the middle of one of the keywords!!
\{%%%
\hbox{{\tt "basic"}},
\hbox{{\tt "numeric time stamp"}},
\hbox{{\tt "human-readable time stamp"}},\\
\hbox{{\tt "full time stamp"}}%%%
\}

\end{Lentry}

\section{Restricted parameters}

\begin{Lentry}

\item [{\tt cctk\_final\_time}] Final time for evolution, overridden by
{\tt cctk\_itlast} unless it is positive [{\tt -1.0}]

\item[{\tt cctk\_initial\_time}]
Initial time for evolution [{\tt 0.0}]

\item [{\tt cctk\_itlast}]
Final iteration number [{\tt 10}]

\item [{\tt max\_runtime}]
Terminate evolution loop after a certain elapsed runtime (in minutes); set to zero to disable this termination condition [{\tt 0}]

\item [{\tt terminate}]
Condition on which to terminate evolution loop [{\tt iteration}] \{{\tt never, iteration, time, runtime, any, all}\}

\item [{\tt terminate\_next}]
Terminate on next iteration ? [{\tt no}]

\end{Lentry}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\chapter{Using GNATS}
\label{sec:Appendix.gnats}
GNATS is a freely redistributable set of tools for tracking bug
reports. It allows users to categorize their problem report and submit
them to the GNATS. The bug tracker will assign appropriate maintainers
to analyze and solve the problem.
We are currently supporting a web based interface at
\url{http://www.cactuscode.org} which lets you
interactively file a bug report. Here, we briefly
describe the main categories when creating a Cactus
problem report.
\begin{Lentry}
\item[{\bf Reporters email}] Your email address so we can get in
contact with you.
\item[{\bf Category}] there is currently a category for each of the
Cactus thorns and arrangements, also a category for the old Cactus3.x and
some general subjects like Web,etc. Select whatever category fits
best.
\item[{\bf Synopsis}] A brief and informative subject line.

\item[{\bf Confidential}] Unused, all PRs are public.

\item[{\bf Severity}] Pick one three levels.

\item[{\bf Class}] In the selected category, specify what kind of
problem you are dealing with.

\item[{\bf submitter ID}] Unused

\item[{\bf Originator}] Your name. Anonymous is OK, but you real name
   would be best.

\item[{\bf Release}] The Cactus release you are using. You can find this
   out, for example, from an executable by typing {\tt cactus\_<config> -v}.

\item[{\bf Environment}] Very important: specify the environment,
e.g.\ {\tt uname -a}, MPI/non-MPI, etc.

\item[{\bf Description}] Describe your problem precisely, if you get a
core dump, include the stack trace, give the minimal number of thorns,
this problems occurs with.

\item[{\bf How-To-Repeat}] Tell us how to repeat the problem if it is
software related.

\item[{\bf Fix}] If you can provide a fix, let us know.
\end{Lentry}

We also provide the customized {\tt send-pr} and {\tt send-pr.el} programs at
our web site. These commands are compiled to submit Cactus problem
reports in your shell and from within Emacs, respectively.



%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\chapter{Using CVS}
\label{sec:Appendix.cvs}
CVS is a version control system, which allows you to  keep
old versions of files (usually source code), log of
when, and why changes occurred, and who made them,  etc.
Unlike the simpler systems, CVS does not just operate on one file at a
time or one directory at a time,  but
operates  on  hierarchical collections of directories consisting of
version controlled files.  CVS helps to  manage
releases  and  to control the concurrent editing of source
files among multiple  authors. CVS can be obtained from
\url{http://www.cyclic.com}.

A CVS \textit{repository} located on a \textit{server} may consist of an arbitrary
number of \textit{modules}, which can be checked out (that is downloaded)
independently. The Cactus Flesh and the Cactus
arrangements are organized as modules, their CVS \textit{server} is {\tt cvs.cactuscode.org}.

\section{Essential CVS commands}

\begin{Lentry}
\item[{\bf cvs login}]
Logs into the repository. You will be prompted for a \textit{password}.
This CVS command leaves a file {\tt .cvspass} in your
home directory. There is no need to login every time you issue a CVS
command, as long as this file exists. For a Cactus checkout, you have
to log into the CVS server, using the CVS option {\tt -d} to specify {\tt CVSROOT}:\\
{\tt cvs -d :pserver:cvs\_anon@cvs.cactuscode.org:/cactus login}

\emph{Note}: that there is no ``logout'' command: if you log in with
administrative rights from an arbitrary account, you should be aware
that the password file enables subsequent administrative logins from
that account. \emph{Delete the file if necessary}.

\item[{\bf cvs checkout} \textit{modules} \ldots]
This command  creates
your private copy of the source for \textit{modules}. You can work
with this copy  without  interfering  with  others'
work.   At  least  one subdirectory level is always created: it does
not write all files into your current directory but creates a
directory. For Cactus, you need to either include the {\tt -d} options to
specify the {\tt CVSROOT} directory and the CVS server, or specify them
with an environment variable (see below). Once you
have checked out the repository there is no need to include the {\tt
-d} option and its rather lengthy argument: the necessary information
is contained in the local {\tt CVS/} directories.

\item[{\bf cvs update}]
Execute  this  command  from  \emph{within}  your  private
source  directory  when  you  wish  to  update your
copies of source  files  from  changes  that  other
developers have made to the source in the repository.
Merges are performed automatically when possible, a warning is issued
if manual  resolution  is  required  for conflicting  changes. If your
local copy is several versions behind the actual repository copy, CVS
will \emph{refetch} the whole file instead of applying multiple
patches.

\item[{\bf cvs add} {\tt file}]
Use this command to enroll new files in CVS records
of your working directory.  The files will be added
to the  repository  the  next  time  you  run  `{\tt cvs
commit}'.

\item[{\bf cvs commit} {\tt file}]
Use this command to add your local changes to the source to
the repository and thereby making it publically available to
checkouts and updates by other users. You cannot commit a
newly created file unless you have \emph{added} it.

\item[{\bf cvs diff} {\tt file}]
Show differences between a file in your working directory
and  a file in the source repository, or between two revisions in
source repository.  (Does not change either repository or working
directory.) For example, to see the difference between versions
{\tt 1.8} and {\tt 1.9} of a file {\tt foobar.c}:

{\tt
\begin{verbatim}
cvs diff -r 1.8 1.9 foobar.c
\end{verbatim}
}

\item[{\bf cvs remove} {\tt file}]
Remove files from the source repository, pending  a {\tt cvs commit} on
the same files.

\item[{\bf cvs status} {[}file{]}]
This command returns the current status of your local copy relative to
the repository: e.g.\ it indicates local modifications and possible
updates.

\item[{\bf cvs import} {\tt repository tag1 tag2}]
Import adds an entire source distribution (starting from the
directory you issue the command in) to the repository directory.
Use this command to add new arrangements to the Cactus4.0 repository. The
{\tt repository} argument is a directory name (or a path to a
directory) and the CVS root directory for repositories; to obtain this
directory on the CVS server, send a request to {\tt
cactus@cactuscode.org}. {\tt tag1} and {\tt tag2} are two tags (vendor
 and release tags) that have to be supplied. For example, to add {\tt MyThorn}
to the {\tt MyArrangement} arrangement, which may or may not already exist on
the CVS repository

{\tt
\begin{verbatim}
cvs -d :pserver:<name>@cvs.cactuscode.org:/cactus import MyArrangement/MyThorn
start v1
\end{verbatim}
}

After you import a thorn, you should check it out from the repository straight
away, and only edit this version.

\end{Lentry}

\section{CVS Options}
The CVS command line can include the following:
\begin{Lentry}
\item[{\bf cvs options}] which apply to the overall CVS program
\item[{\bf a cvs command}]  which defines a particular task carried out by
CVS
\item[{\bf command options}] to specify certain working modes for the CVS
command.
\item[{\bf command arguments}] to specify which file to act on.
\end{Lentry}

The options must be put \emph{relative} to the CVS \emph{command} as the
same option name can mean different things: CVS \emph{options} go to the
\emph{left} of the CVS command, \emph{command options} go to the \emph{right}
of the CVS command. Here is a list of essential CVS options:

\begin{Lentry}

\item[{\bf -d} \textit{cvs\_root\_directory}]
Use \textit{cvs\_root\_directory} as the root directory  pathname  of
the  master source repository.  Overrides
the setting of the  {\tt CVSROOT}  environment  variable.
This value should be specified as an absolute pathname.
In the Cactus checkout procedure, you specify the Cactus CVS server:\\
{\tt -d :pserver:cvs\_anon@cvs.cactuscode.org:/cactus/}

\item[{\bf -z} \textit{compression-level}]
When transferring files across the network use {\tt gzip}
with compression level  \textit{compression-level}  to  compress  and
decompress  data as it is transferred.
Requires the presence of the GNU {\tt gzip} program  in
the current search path at both ends of the link.

\item[{\bf -n}]
Do not change any file. Attempt to execute the CVS \textit{command} but
only to issue reports. Does not remove, update, etc. any files. Very
effective for testing.

\item[{\bf -v}]
Displays version information of the installed CVS.

\item[{\bf -H} \textit{cvs-command}]
Displays usage information about the specified CVS command. Without
\textit{cvs-command}, a list of all available commands is returned.
\end{Lentry}

Here is a list of essential command options with the
commands they are used with. They go after the CVS command. For a more
complete list of all options, please refer to the manual page.

\begin{Lentry}

\item[{\bf -P}]
Prune  (remove)  directories  that  are empty after being updated, on
{\tt checkout}, or  {\tt update}.   Normally, an  empty  directory
(one that is void of revision controlled files) is  left  alone.
Specifying  {\tt -P} will cause these directories to be silently
removed from the sources you have checked out.  This does not remove
the directory from the repository, only from your checked out copy.

\item[{\bf -m} \textit{"Text"}]
Specify a logging message explaining changes, etc. on {\tt commit},
{\tt import}. If you do not specify a message, your default editor
is invoked to allow you to enter one.

\item[\bf -d]
Use this option with the {\tt update} command to create any
directories if they are missing from your local copy. This is normally
the case if another user has added files or directories to the
repository. By default the {\tt update} command only acts on files in
your local copy. Note that omitting this option is a frequent cause of
files missing during compilation.  (You can change this
default behavior of CVS by putting a {\tt .cvsrc} in your home directory
with the contents ``{\tt update -d}''.)

\end{Lentry}

\section{CVS Examples}
We list some sample CVS commands to treat the most typical Cactus 4.0
CVS situations.
\begin{description}
\item\textbf{Logging into the server}\newline
{\tt cvs -d :pserver:cvs\_anon@cvs.cactuscode.org:/cactus
login} \\ You will be asked for the password for user \textit{cvs\_anon}, which is {\tt anon}.

\item\textbf{Checking out the code}\newline
{\tt cvs -d :pserver:cvs\_anon@cvs.cactuscode.org:/cactus
checkout Cactus}\\
check out a CVS module named ``Cactus'', in this case it checks out the
Cactus Computational Toolkit. A directory {\tt ./Cactus} is created if
it doesn't already exist. If you perform a checkout on an already
existing and locally modified copy of the module, CVS will try to merge the files
with your local copy.

\item\textbf{Updating a file or directory}\newline
Assuming that you have a file {\tt ./foobar} in your checked out
copy, you may perform a \\
{\tt cvs status ./foobar}\\
to inform yourself about the necessary updates, etc. To update the
file issue \\
{\tt cvs update ./foobar}\\
If that was file was locally modified, CVS will try to merge the
changes. Manual merging might be necessary and will be indicated by a
CVS warning.

\item\textbf{Updating a directory}\newline
To recursively update the current directory and all subdirectories,
type\\
{\tt cvs update .}\\
To update a directory {\tt ./mysources}, type\\
{\tt cvs update ./path/to/mysources}

\item\textbf{Committing a changed file}\newline
To commit changes you have applied to your local copy, your file must be in
sync with the repository: your changes must be done to the
latest version, otherwise CVS will instruct you to perform an {\tt
update} first. To commit changes made to a file {\tt ./foobar}, type\\
{\tt cvs commit -m "Reason for the change" ./foobar}\\
You may specify several files to commit.

\item\textbf{Adding and committing a new file}\newline
Adding a new file to the repository is a two fold procedure you first
schedule the file for addition, then you commit it:\\
{\tt cvs add ./newfoo}\\
{\tt cvs commit -m "new few message" ./newfoo}

\item\textbf{Creating a new thorn}\newline

To add a new \textit{module} (e.g.\ an arrangement) to a Cactus repository we
first have to create a directory for you with the right permissions.
Please contact {\tt cactus@cactuscode.org} providing the name of the
requested module, and who should be able to commit changes to the module.

To add the new module, change directory so that you are in the first directory
that you want to commit to the repository. (e.g.\ if you want to commit
a new arrangement called {\tt MyArrange} then change directory to
{\tt MyArrange}). Then type\\
{\tt cvs -d :pserver:}\textit{your\_login}{\tt
@cvs.cactuscode.org:<repository name> } import \textit{module\_name} {\tt start V1}\\
(where {\tt start} and {\tt V1} are the vendor and release tags, which you could change to something different).

To add a new \textit{directory} {\tt <new dir>} to an existing module (that you have write permissions for), either add the directory using\\
{\tt cvs add <new dir>}\\
and then recursing down adding all the new files and directories contained
inside, or import the directory by changing directory to sit inside it, and then using\\
{\tt cvs -d :pserver:}\textit{your\_login}{\tt
@cvs.cactuscode.org:<repository\_name> } import {\tt <relative name> start V1}\\
Where {\tt <relative name>} means the position of the directory within the module. (For example, if you have a module called \textit{AMod} which contains a
directory \textit{BMod}, and you want to add \textit{CMod} inside \textit{BMod}, then change directory to \textit{BMod}, and use \textit{AMod/BMod} for the \textit{relative name}).


\end{description}

\section{Checking out Flesh and thorns with CVS}

\begin{Lentry}
\item[{\bf Login}] Prior to any CVS operation, you need to log into the Cactus
  repository. For an anonymous checkout, type:\\
  {\tt
    cvs -d :pserver:cvs\_anon@cvs.cactuscode.org:/cactus login
    }
  You will be prompted for a password which is {\tt anon}.
\item[{\bf Checkout}] To obtain a fresh copy of Cactus, move to a directory
  which does not contain a previously checked out version, and type
  {\t
    cvs -d :pserver:cvs\_anon@cvs.cactuscode.org:/cactus checkout Cactus
    }
  The CVS checkout procedure will create a directory called {\tt
  Cactus} and install the code inside this directory.  From now on we
  will reference all directory names relative to {\tt Cactus}.

\noindent
  If you want to compile Cactus with thorns, you now need to checkout
  separately the required arrangement (or \textit{arrangements})
   into the {\tt arrangements} directory. To see the
  available Cactus arrangements and thorns type
  {\t
    cvs -d :pserver:cvs\_anon@cvs.cactuscode.org:/cactus checkout -s
  }
  To check out an arrangement or thorn go to the arrangements directory,  {\t cd arrangements},
  and  for an arrangement type
{\t
  cvs checkout <arrangement\_name>
  }
        or for just one thorn
{\t
cvs checkout <arrangement\_name/thorn\_name>
}

To simplify this procedure you may use {\t gmake checkout} in the Cactus
home directory which provides menus to pick arrangements and thorns from.


\item[{\bf Update}] To update an existing Cactus checkout (to patch in
  possible changes, etc.), do the following \emph{within} the {\tt Cactus} directory.
  {\t
    cvs update
    }
  The update process will operate recursively downwards from your current position
  within the Cactus tree. To update only on certain directories, change
  into these directories and issue the update command.
\item[{\bf CVS status}] To obtain a status report on the ``age'' of your
  Cactus or arrangement routines (from your current directory position
  downward), type
  {\t
    cvs status
    }
\item[{\bf non-anonymous CVS}] If you have an account at the central
  repository ({\tt cvs.cactuscode.org}) and would like to perform
  any of the operation above
  \emph{non-anonymously}, replace {\tt cvs\_anon} by your login name
  and provide the appropriate password during the CVS login
  process. Depending on your permissions, you may then make commits to Cactus
  or its arrangements.
\item[{\bf Commits}] You need to perform a personalized login and have
  proper permissions to commit code to the repository.
\end{Lentry}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\chapter{Using Tags}
\label{sec:Appendix.tags}
Finding your way around in the Cactus structure can be pretty
difficult to handle. To make life easier there is support for \textit{tags},
which lets you browse the code easily from within Emacs/XEmacs or {\tt vi}.
A tags database can be generated with {\tt gmake}:

\section{Tags with Emacs}

The command {\tt gmake TAGS} will create a database for a routine reference
table to be used within Emacs. This database can be accessed within
Emacs if you add either of the following lines to your {\tt .emacs} file:\\
{\tt (setq tags-file-name "CACTUS\_HOME/TAGS")} XOR \\
{\tt (setq tag-table-alist '(("CACTUS\_HOME" . "CACTUS\_HOME/TAGS")))}\\
where {\tt CACTUS\_HOME} is your Cactus directory.\\

You can now easily navigate your Cactus Flesh and Toolkits by searching for
functions or ``tags'':
\begin{enumerate}
\item \textbf{ Alt.} will find a tag
\item \textbf{ Alt,} will find the next matching tag
\item \textbf{ Alt*} will go back to the last matched tag
\end{enumerate}
If you add the following lines to your {\tt .emacs} file, the
files found with tags will opened in \emph{read-only} mode:
\begin{verbatim}
(defun find-tag-readonly (&rest a)
  (interactive)
  (call-interactively `find-tag a)
  (if (eq nil buffer-read-only) (setq buffer-read-only t))  )

(defun find-tag-readonly-next (&rest a)
  (interactive)
  (call-interactively `tags-loop-continue a)
  (if (eq nil buffer-read-only) (setq buffer-read-only t))  )

(global-set-key [(control meta \.)] 'find-tag-readonly)
(global-set-key [(control meta \,)] 'find-tag-readonly-next)
\end{verbatim}
The key strokes to use when you want to browse in read-only mode are:
\begin{enumerate}
\item{CTRL Alt.} will find a tag and open the file in read-only mode
\item{CTRL Alt,} will find the next matching tag in read-only mode
\end{enumerate}

\section{Tags with \code{vi}}

The commands available are highly dependent upon the version of \code{vi}, but
the following is a selection of commands which may work.


\begin{enumerate}

\item \textbf{vi -t tag}
Start {\tt vi} and position the cursor at the file and line where `tag' is defined.

\item \textbf{Control-{]}}
Find the tag under the cursor.

\item \textbf{:ta tag}
Find a tag.

\item \textbf{:tnext}
Find the next matching tag.

\item \textbf{:pop}
Return to previous location before jump to tag.

\item \textbf{Control-T}
Return to previous location before jump to tag (not widely implemented).

\end{enumerate}

\vspace{1.1cm}

\emph{Note: Currently some of the \texttt{CCTK\_FILEVERSION()} macros
at the top of every source file don't have a trailing semicolon, which
confuses the \texttt{etags} and \texttt{ctags} programs, so tags does not
find the first subroutine in any file with this problem.}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\chapter{ThornLists}

\label{chap:th}

This section still needs to be written.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\end{cactuspart}

%%% Local Variables: 
%%% mode: latex
%%% TeX-master: "UsersGuide"
%%% End: 
