1 files changed, 332 insertions, 0 deletions
diff --git a/doc/mclachlan.tex b/doc/mclachlan.tex
new file mode 100644
index 0000000..1f2db47
--- /dev/null
+++ b/doc/mclachlan.tex
@@ -0,0 +1,332 @@
+\documentclass[11pt, tightenlines]{revtex4}
+
+\usepackage{amsmath}
+\usepackage{amssymb}
+\usepackage{color}
+\usepackage{float}
+\usepackage{graphicx}
+\usepackage[latin9]{inputenc}
+\usepackage{url}
+
+% Put this package last
+\usepackage[bookmarks, bookmarksopen, bookmarksnumbered]{hyperref}
+% Put this package after hyperref
+\usepackage[all]{hypcap}
+% Don't use tt font for urls
+\urlstyle{rm}
+
+% Make a comment stand out visually
+\newcommand{\todo}[1]{{\color{blue}$\blacksquare$~\textsf{[TODO: #1]}}}
+% Name of a code
+\newcommand{\code}[1]{\texttt{#1}}
+
+\hyphenation{Cac-tus-Ein-stein Schwarz-schild South-amp-ton}
+\sloppypar
+
+\begin{document}
+
+
+
+\title{McLachlan}
+\date{August 18, 2009}
+
+\author{Erik Schnetter}
+\address{Center for Computation \& Technology, Louisiana State
+  University, USA}
+\homepage{http://www.cct.lsu.edu/~eschnett/McLachlan/}
+\email{schnetter@cct.lsu.edu}
+
+\begin{abstract}
+  McLachlan is a free Einstein solver that uses the Cactus framework
+  and the Einstein toolkit.  This document describes the basic
+  features of the code, and also how to obtain, build, and use the
+  code.
+\end{abstract}
+
+\maketitle
+
+
+
+\section{McLachlan}
+
+McLachlan is a free Einstein solver that uses the Cactus framework and
+the Einstein toolkit.  McLachlan was developed by Erik Schnetter and
+Peter Diener with the help of Jian Tao and Ian Hinder.  It was first
+described in \cite{ES-Brown2007b}, where (to our knowledge) the first
+fully fourth order accurate black hole evolution with adaptive mesh
+refinement is presented.  The McLachlan web pages are located at
+\cite{ES-mclachlanweb}.
+
+
+
+% \section{Formulation}
+% 
+% \todo{To be written.  BSSN equations, time evolution, constraints.}
+
+
+
+% \section{Automated Code Generation}
+% 
+% \todo{To be written.  Kranc.  Modifying the equations.}
+
+
+
+\section{Obtaining McLachlan}
+
+McLachlan uses the Cactus Software Framework and the Einstein Toolkit.
+Cactus organises applications into \emph{thorns} (modules) that can be
+maintained independently of each other.  In order to use McLachlan, it
+is necessary to obtain Cactus as well as a set of supporting thorns.
+
+The subsections below describe how to obtain Cactus and other
+necessary thorns.  McLachlan contains also a shell script
+\code{checkout.sh} that attempts to automate this.  However, this
+script is very basic and does not handle errors well.
+
+\subsection{Tools}
+
+Cactus thorns are ususall stored in \emph{repositories} that are
+managed by version control systems such as CVS \cite{cvsweb}, SVN
+(subversion) \cite{svnweb}, or git \cite{gitweb}.
+
+Before getting the code, you will need to install the following
+software on your local system:
+\begin{enumerate}
+\item wget (or curl)
+\item CVS
+\item SVN
+\item git
+\item Perl
+\end{enumerate}
+These are standard packages, and they should be easily available for
+all Linux systems.
+
+\subsection{Cactus}
+\label{sec:cactus}
+
+Cactus \cite{Goodale02a, ES-cactusweb} is a software framework that
+makes it possible to maintain parts of applications independent of
+each other, and combine them into an efficient code when building the
+application.  Cactus is described at \url{http://www.cactuscode.org/},
+and a new version of the of Cactus web site is currently being
+prepared at \url{http://preview.cactuscode.org/download/}.
+
+To obtain Cactus itself as well as a set of basic thorns, follow the
+instructions at \url{http://preview.cactuscode.org/download/}.
+(Additional thorns specific to numerical relativity are also located
+elsewhere.)  Don't use a particular thorn list for this; instead,
+download all the basic thorn that Cactus offers.
+
+For reference, here is a brief overview over the commands to do this:
+\begin{enumerate}
+\item\verb+wget http://preview.cactuscode.org/download/GetCactus+
+\item\verb-chmod a+x GetCactus-
+\item\verb+./GetCactus+
+  
+  This checks out Cactus itself (the \emph{flesh}) into a new
+  subdirectory \code{Cactus}.  Use the default answer for all
+  questions.
+\item\verb+cd Cactus+
+\item\verb+gmake checkout+
+  
+  This checks out some arrangements with basic thorns for Cactus,
+  including the important CactusEinstein arrangement.  Again, use the
+  default answer for all questions, except when you are asked for the
+  second time whetyer you want to quit.  In this case, quit.
+\end{enumerate}
+
+\subsection{Carpet}
+
+Carpet \cite{ES-Schnetter2003b, ES-Schnetter2006a, ES-carpetweb} is a
+\emph{driver} for Cactus.  A driver manages memory, handles
+parallelism, and performs I/O on behalf of the application.  Carpet
+supports adaptive mesh refinement (AMR) and multi-block methods.
+Carpet is described at \url{http://www.carpetcode.org/}.
+
+To obtain Carpet, follow the instructions at
+\url{http://www.carpetcode.org/get-carpet.html}.  Please check out the
+\emph{Development Version}, which is currently quite stable.  (We are
+planning to release a new stable version soon.)
+
+In particular, the commands to obtain the development version are:
+\begin{enumerate}
+\item\verb+cd Cactus+
+\item\verb+git clone -o carpet git://carpetcode.dyndns.org/carpet.git+
+\item\verb+cd arrangements+
+\item\verb+ln -s ../carpet/Carpet* .+
+\end{enumerate}
+(Don't miss the dot after the \verb+Carpet*+ in the last line.)  Note
+that Carpet should be checked out into the main Cactus directory, and
+the \code{arrangements} subdirectory needs to contain symbolic links
+pointing into the \code{carpet} directory.
+
+\subsection{McLachlan}
+
+McLachlan \cite{ES-Brown2007b, ES-mclachlanweb} in an Einstein solver.
+It uses one of the BSSN formulations of the Einstein equations.
+McLachlan is described at
+\url{http://www.cct.lsu.edu/~eschnett/McLachlan/}, which is where you
+may have obtained this documentation.
+
+To obtain McLachlan, issue the following commands:
+\begin{enumerate}
+\item\verb+cd Cactus+
+\item\verb+cd arrangements+
+\item\verb+git clone git://carpetcode.dyndns.org/McLachlan.git+
+\end{enumerate}
+Note that McLachlan needs to be checked out directly into the
+\code{arrangements} subdirectory.
+
+McLachlan uses the Kranc code generation package \cite{kranc04,
+  Husa:2004ip, krancweb, ES-krancweb}.  Kranc also contains some
+thorns that McLachlan needs.  (However, it is not necessary to run
+Kranc in order to use McLachlan.  It is only necessary to run Kranc if
+McLachlan is modified.)
+
+To obtain Kranc, issue the following commands:
+\begin{enumerate}
+\item\verb+cd Cactus+
+\item\verb+git clone http://www.aei.mpg.de/~ianhin/kranc.git+
+\item\verb+cd arrangements+
+\item\verb+ln -s ../kranc/Auxiliary/Cactus/KrancNumericalTools .+
+\end{enumerate}
+(Don't miss the dot at the end of the last line.)  Note that Kranc
+needs to be check out into the main Cactus directory, and the
+\code{arrangements} subdirectory needs to contain symbolic links
+pointing into the \code{kranc} directory.
+
+\subsection{Other Thorns}
+
+All other thorns, including the public Whisky thorns, can be obtained
+via the GetCactus script that was downloaded above (see section
+\ref{sec:cactus}):
+
+\begin{enumerate}
+\item\verb+cd Cactus+
+\item\verb+cd ..+
+\item\verb+./GetCactus qc0-mclachlan-public.th+
+  
+  Use the default answer for all questions.
+\end{enumerate}
+
+\subsection{Consistency Check}
+
+The \emph{thorn list} \code{qc0-mclachlan-public.th} lists the thorns
+that are necessary for a simple spacetime evolution.  The thorns are
+grouped into arrangements.  All thorns listed in this file must now be
+present in the \code{arrangements} subdirectory of the main Cactus
+directory.
+
+
+
+\section{Building McLachlan}
+
+Building McLachlan and the other thorns requires C, C++, and Fortran
+90 compilers, MPI, as well as the BLAS, GSL, HDF5, and LAPACK
+libraries.
+
+\subsection{Documentation}
+
+It is best to begin building Cactus with building documentation:
+\begin{enumerate}
+\item\verb+cd Cactus+
+\item\verb+make UsersGuide+
+\end{enumerate}
+This creates the users' guide as \code{doc/UsersGuid.pdf}.
+
+All Cactus commands are listed with
+\begin{enumerate}
+\item\verb+make help+
+\end{enumerate}
+
+\subsection{Option List}
+
+To build a Cactus application one needs to create an \emph{options
+  list}.  This is a text file containing the configuration options
+that tell Cactus what compilers and compiler options to use, and where
+MPI and the auxiliary libraries are installed.  This process is very
+specific to each machine and may require some trial and error.
+
+We distribute options lists for a range of machines that we are using
+on the Cactus web site at
+\url{http://preview.cactuscode.org/download/configfiles/}.  Option
+lists are also available together with the Simulation Factory
+\cite{ES-simfactoryweb} at
+\url{https://svn.cct.lsu.edu/repos/numrel/simfactory/optionlists/}.
+
+\subsection{Building}
+
+To build a Cactus application one starts with an option list and a
+thorn list.  The text below assumes that you have an option list
+called \code{einstein-redshift-gcc.cfg}.
+
+To configure an application
+\begin{enumerate}
+\item\verb+cd Cactus+
+\item\verb+make sim-config options=einstein-redshift-gcc.cfg \+\\
+  \verb+THORNLIST=qc0-mclachlan-public.th+
+\item\verb+make sim+
+\end{enumerate}
+This is necessary only once, or when the configuration options change.
+This will create an application called \code{sim}; of course, the name
+could also be different.  Different applications, e.g.\ with different
+options or different thorn lists, can exist side by side.
+
+To build the application:
+\begin{enumerate}
+\item\verb+cd Cactus+
+\item\verb+make sim -j4+
+\end{enumerate}
+The make option \code{-j4} builds 4 files at the same time.  Use this
+option if you have several processors available; this will speed up
+building the application.  The executable is called \code{cactus\_sim}
+and is placed in the \code{exe} subdirectory.
+
+\subsection{Cleaning}
+
+You can also clean the application, removing all object files but
+keeping the configuration options and thorn list:
+\begin{enumerate}
+\item\verb+cd Cactus+
+\item\verb+make sim-realclean+
+\end{enumerate}
+
+
+
+\section{Running McLachlan}
+
+To run the McLachlan code, one needs a \emph{parameter file}.
+Parameter files select which thorns are activated at run time, and
+what values the thorns' run-time parameters have.  They typically have
+a \code{.par} suffix.  Below, we use the parameter file
+\code{qc0-mclachlan-public.par}.
+
+Cactus applications are started like a regular MPI application.  The
+exact mechanism depends on the particular MPI implementation.  On
+Redshift, the command is
+\begin{enumerate}
+\item\verb+cd Cactus+
+\item\verb+mkdir simulations+
+\item\verb+cd simulations+
+\item\verb+env OMP_NUM_THREADS=1 mpirun ../exe/cactus_sim qc0-mclachlan-public.par+
+\end{enumerate}
+This parameter file simulates a single, stationary, spinning black
+hole in Kerr-Schild coordinates.  It requires about 4~GByte of RAM to
+run.
+
+\emph{Note:} If the options list enables OpenMP, then the Cactus
+application will be multi-threaded.  Multi-threading can improve
+performance and reduce memory consumption, especially when many
+($>100$) cores are used.  However, it is usually a bad idea to
+over-subscribe cores by having too many threads per node.  It is
+usually best to choose both the number of MPI processes per node and
+the number of OpenMP threads per process such that their product
+equals the number of cores on a node.  The way in which these numbers
+are chosen depend on the MPI implementation.
+
+
+
+\bibliographystyle{apsrev-titles-manyauthors}
+\bibliography{references,publications-schnetter}
+
+\end{document}