* reorder sections so "how to use it" info comes before "how it works"

* expand discussion of I/O parameters
* add acknowledgments
* lots of smaller cleanups


git-svn-id: http://svn.einsteintoolkit.org/cactus/EinsteinAnalysis/AHFinderDirect/trunk@896 f88db872-0e4f-0410-b76b-b9085cfa78c5

1 files changed, 343 insertions, 240 deletions

diff --git a/doc/documentation.tex b/doc/documentation.tex
index a72f8fd..9475b08 100644
--- a/doc/documentation.tex
+++ b/doc/documentation.tex
@@ -172,198 +172,12 @@ the irreducable mass associated with the area.
 
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
-\section{How \thorn{AHFinderDirect} Works}
-\label{AEIDevelopment_AHFinderDirect_sect-how-ahfinderdirect-works}
-
-The apparent horizon (henceforth \defn{horizon}) finding algorithm used
-in this thorn is basically that of
-\cite{AEIDevelopment_AHFinderDirect_Thornburg-1996-apparent-horizon-finding},
-but modified to work with $g_\ij$ and $K_\ij$ on a Cartesian ($xyz$) grid:
-
-I assume that a local coordinate origin has been chosen such that
-relative to that origin, the horizon is a \defn{Strahlk\"{o}rper}
-(literally ``ray body'', or more commonly ``star-shaped region''),
-defined by Minkowski
-(\cite[p.~108]{AEIDevelopment_AHFinderDirect_Schroeder-1986-number-theory})
-as
-   \begin{quote}
-   a region in $n$-dimensional Euclidean space containing the
-   origin and whose surface, as seen from the origin, exhibits
-   only one point in any direction.
-   \end{quote}
-I can thus parameterize the horizon's shape by $r = h(\text{angle})$
-for some single-valued \defn{horizon shape function} $h$ defined on the
-2-sphere $S^2$.
-
-I then write~\eqref{AEIDevelopment_AHFinderDirect_eqn-horizon} as a
-2-D elliptic PDE on $S^2$ for the function $h$.  I finite difference
-this in angle to obtain a system of simultaneous nonlinear algebraic
-equations for $h$ at the angular grid points, and solve this system
-of equations by a global Newton's method (or a variant with improved
-convergence).
-
-Computationally, this algorithm has 3 main parts:
-\begin{itemize}
-\item	Computation of the ``horizon function'' $H(h)$ given a trial
-	surface defined by a trial horizon shape function $h$.  This is
-	done by interpolating the Cactus geometry fields $g_\ij$ and
-	$K_\ij$ (and optionally $\psi$) from the 3-D $xyz$ grid
-	to the (2-D set of) trial-horizon-surface grid points (also
-	computing $\partial_k g_\ij$ in the interpolation process),
-	then doing all further computations with angular grid functions
-	defined solely on $S^2$ (\ie{} at the horizon-surface grid points).
-\item	Computation of the Jacobian matrix $\Jac[H(h)]$ of $H(h)$.  This
-	thorn incorporates the \defn{symbolic differentiation} technique
-	described in
-\cite{AEIDevelopment_AHFinderDirect_Thornburg-1996-apparent-horizon-finding},
-	so this computation is quite fast.  The Jacobian is a highly
-	sparse matrix, but at the moment we neglect this and store it
-	as a full (dense) matrix.  (This should change soon.)
-\item	Solving the nonlinear equations $H(h) = 0$ by a global Newton's
-	method or a variant.  At the moment this thorn uses a simple
-	Newton's-method solver, and solves the linear system using
-	$\sf LU$ decomposition routines from LAPACK
-	(this should change to a sparse $\sf LU$ decomposition/solution
-	soon).
-\end{itemize}
-
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
-
-\section{The Multipatch System}
-
-Perhaps the most unusual feature of \thorn{AHFinderDirect} is the
-``multipatch'' system used to cover $S^2$ without coordinate singularities.
-In general there are 6~patches, one each covering a neighborhood of
-the $\pm z$, $\pm x$, and $\pm y$ axes, but this may be reduced in
-the presence of suitable symmetries.  For example,
-figure~\ref{AEIDevelopment_AHFinderDirect_fig-3patch}
-on page~\pageref{AEIDevelopment_AHFinderDirect_fig-3patch} shows a system
-of 3~patches covering the $+xyz$~octant of $S^2$.  This would be
-suitable for finding an apparent horizon with mirror symmetry about
-the (local) $z=0$~plane, and either 90~degree periodic rotation symmetry
-about the (local) $z$~axis, or mirror symmetry about each of the (local)
-$x$~and $y$~axes.
-
-\begin{figure}[htbp]
-\begin{center}
-\includegraphics{AEIDevelopment_AHFinderDirect_3patch.eps}
-\end{center}
-\caption[Illustration of the Multipatch System]
-	{
-	This figure shows a multipatch system covering the
-	$(+,+,+)$~octant of the unit sphere~$S^2$ with 3~patches.
-	The angular resolution is 5~degrees.  Notice that the
-	patches overlap by several ``ghost zone'' grid points.
-	}
-\label{AEIDevelopment_AHFinderDirect_fig-3patch}
-\end{figure}
-
-To allow easy angular finite differencing within the patch system,
-each patch is extended beyond its nominal extent by a ``ghost zone''%%%
-\footnote{%%%
-	 Note that this terminology differs somewhat
-	 from that used by Cactus in general; Cactus
-	 would call these ``patch zones'' or ``symmetry
-	 zones''.
-	 }%%%
-{} (2~grid points wide in
-figure~\ref{AEIDevelopment_AHFinderDirect_fig-3patch}).  Angular grid
-function values in the ghost zone can be obtained by interpatch
-interpolation%%%
-\footnote{%%%
-	 Due to the way the patch coordinates are defined,
-	 adjacent patches always share a common ``perpendicular''
-	 angular coordinate, so only 1-D interpolation
-	 is needed here.
-	 }%%%
-{} or by applying symmetry operations.  Once this is done, then angular
-finite differencing within the nominal extent of each patch can proceed
-normally, ignoring the patch boundaries.  \thorn{AHFinderDirect} can
-be configured at compile time to use either 2nd~order or 4th~order
-angular finite differencing (3~point or 5~point angular molecules);
-the default is 4th~order (5~point).  See \verb|src/include/config.h|
-if you want to change this.
-
-By default \thorn{AHFinderDirect} will automagically choose a patch
-system type for each apparent horizon searched for, based on the local
-coordinate origin and the symmetries implicit in the Cactus grid type.
-This generally works well, but if desired you can instead manually
-specify the patch system type, the angular resolution, the width of
-the ghost zones, etc.  See the \verb|param.ccl| file for details.
-
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
-
-\section{Using \thorn{AHFinderDirect}}
-
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
-
-\subsection{Obtaining and Compiling \thorn{AHFinderDirect}}
-
-You should be able to obtain the source code for this thorn via the
-usual procedures (\eg{}~cvs checkout); at present it lives in the
-\arrangement{AEIDevelopment} arrangement.
-
-This thorn is written primarily in \Cplusplus{}, calling C and
-Fortran~77 numerical libraries.%%%
-\footnote{%%%
-	 The (C) code for the relativity computations
-	 (\ie{}~the apparent horizon equation itself)
-	 is mainly machine-generated from Maple code,
-	 which in turn uses a Perl preprocessor.
-	 But the machine-generated C code is already
-	 in CVS along with the rest of this thorn's
-	 source code, so you don't need to have Maple
-	 installed unless you want to modify the
-	 relativity computations.%%%
-	 }%%%
-{}  In theory the code should be quite portable to modern \Cplusplus{}
-compilers, but in practice I've had a number of portability problems
-with various compilers.  See the ``Code Notes'' and ``Compiler Notes''
-sections in the top-level \code{README} file for details and lists of
-compilers currently known to be ok or not.
-
-To compile (a Cactus configuration which includes) this thorn, you
-must set the environment variable \code{LAPACK\_DIR}, and possibly
-also the environment variables \code{LAPACK\_EXTRA\_LIBS} and
-\code{LAPACK\_EXTRA\_LIBDIRS}.  See the ``Library Configuration''
-section in the top-level \code{README} file for details.
-
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
-
-\subsection{What \thorn{AHFinderDirect} Needs}
-
-Your Cactus configuration and run must satisfy a number of requirements
-in order to use \thorn{AHFinderDirect}:
-\begin{itemize}
-\item	I recommend using a current-CVS checkout of the Cactus flesh
-	and of all relevant thorns.  I haven't tested \thorn{AHFinderDirect}
-	at all with older versions of the flesh or other thorns.
-\item	Obviously, your Cactus configuration must include
-	\thorn{AHFinderDirect}, and your \code{ActiveThorns} parameter(s)
-	must activate it.  \thorn{AHFinderDirect} inherits from \thorn{Grid},
-	\thorn{ADMBase}, and \thorn{StaticConformal}, so you'll need
-	those (or more precisely some thorns providing them), too.
-\item	\verb|Grid::domain = "full"|, \verb|"bitant"|,
-	\verb|"quadrant"|, and \verb|"octant"| are supported.
-	Alas, at present rotating (or more precisely nonlocal)
-	symmetry boundary conditions aren't supported.
-\item	The \verb|ADMBase::metric_type| values \verb|"physical"| and
-	\verb|"static conformal"| are supported; for the latter you must
-	have storage turned on for at least the conformal factor
-	\verb|StaticConformal::psi|.  (The Cactus 3-D grid functions
-	for 1st and 2nd derivatives of \verb|psi| aren't used.)
-\item	\thorn{AHFinderDirect} uses the new \verb|CCTK_InterpLocalUniform()|
-	interpolator API, which at present is only supported by
-	\thorn{LocalInterp}, so you need that thorn compiled in to
-	your configuration and activated.
-\item	At present \thorn{AHFinderDirect} only works for single-processor
-	Cactus runs.  This isn't checked, so if you try to activate
-	it in a multi-procssor run you'll probably get garbage results.
-	I hope to properly support multi-processor operation soon.
-\end{itemize}
+\section{What \thorn{AHFinderDirect} Needs}
+\label{AEIDeveopment_AHFinderDirect_sect-what-AHFinderDirect-needs}
 
-There are also some restrictions on the spacetime, or more precisely
-on each slice where you want to find apparent horizons:
+There are some restrictions on the spacetime, or more precisely
+on each slice where you want to find apparent horizons, which are
+necessary in order for \thorn{AHFinderDirect} to work:
 \begin{itemize}
 \item	\thorn{AHFinderDirect} requires that the Cactus geometry
 	($g_{ij}$, $K_{ij}$, and optionally $\psi$) be nonsingular
@@ -409,11 +223,91 @@ on each slice where you want to find apparent horizons:
 	properly support excision (at least in the sense of checking
 	for the trial horizon surface being too close to an excised
 	region) soon.
+\item	\thorn{AHFinderDirect} requires that any apparent horizon
+	it's going to (try to) find must be a \defn{Strahlk\"{o}rper}
+	(literally ``ray body'', or more commonly ``star-shaped region'')
+	relative to some local coordinate origin (which you must specify).
+	A Strahlk\"{o}rper is defined by Minkowski
+(\cite[p.~108]{AEIDevelopment_AHFinderDirect_Schroeder-1986-number-theory})
+	as
+	   \begin{quote}
+	   a region in $n$-dimensional Euclidean space containing the
+	   origin and whose surface, as seen from the origin, exhibits
+	   only one point in any direction.
+	   \end{quote}
+	In other words, using polar spherical coordinates relative
+	to the local coordinate origin, the apparent horizon's shape
+	must be parameterizable as $r = h(\text{angle})$ for some
+	single-value function $h: S^2 \to \Re^+$.  (\thorn{AHFinderDirect}
+	uses precisely this parameterization.)
 \end{itemize}
 
+There are also some restrictions on your Cactus configuration and run:
+\begin{itemize}
+\item	I recommend using a current-CVS checkout of the Cactus flesh
+	and of all relevant thorns.  I haven't tested \thorn{AHFinderDirect}
+	at all with older versions of the flesh or other thorns.
+\item	Obviously, your Cactus configuration must include
+	\thorn{AHFinderDirect}, and your \code{ActiveThorns} parameter(s)
+	must activate it.  \thorn{AHFinderDirect} inherits from \thorn{Grid},
+	\thorn{ADMBase}, and \thorn{StaticConformal}, so you'll need
+	those (or more precisely some thorns providing them), too.
+\item	\verb|Grid::domain = "full"|, \verb|"bitant"|,
+	\verb|"quadrant"|, and \verb|"octant"| are supported.
+	Alas, at present rotating (or more precisely nonlocal)
+	symmetry boundary conditions aren't supported.
+\item	The \verb|ADMBase::metric_type| values \verb|"physical"| and
+	\verb|"static conformal"| are supported; for the latter you must
+	have storage turned on for at least the conformal factor
+	\verb|StaticConformal::psi|.  (The Cactus 3-D grid functions
+	for 1st and 2nd derivatives of \verb|psi| aren't used.)
+\item	\thorn{AHFinderDirect} uses the new \verb|CCTK_InterpLocalUniform()|
+	interpolator API, which at present is only supported by
+	\thorn{LocalInterp}, so you need that thorn compiled in to
+	your configuration and activated.
+\item	At present \thorn{AHFinderDirect} only works for single-processor
+	Cactus runs.  This isn't checked, so if you try to activate
+	it in a multi-procssor run you'll probably get garbage results.
+	I hope to properly support multi-processor operation soon.
+\end{itemize}
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\section{Obtaining and Compiling \thorn{AHFinderDirect}}
+
+You should be able to obtain the source code for this thorn via the
+usual procedures (\eg{}~cvs checkout); at present it lives in the
+\arrangement{AEIDevelopment} arrangement.
+
+This thorn is written primarily in \Cplusplus{}, calling C and
+Fortran~77 numerical libraries.%%%
+\footnote{%%%
+	 The (C) code for the relativity computations
+	 (\ie{}~the apparent horizon equation itself)
+	 is mainly machine-generated from Maple code,
+	 which in turn uses a Perl preprocessor.
+	 But the machine-generated C code is already
+	 in CVS along with the rest of this thorn's
+	 source code, so you don't need to have Maple
+	 installed unless you want to modify the
+	 relativity computations.%%%
+	 }%%%
+{}  In theory the code should be quite portable to modern \Cplusplus{}
+compilers, but in practice I've had a number of portability problems
+with various compilers.  See the ``Code Notes'' and ``Compiler Notes''
+sections in the top-level \code{README} file for details and lists of
+compilers currently known to be ok or not.
+
+To compile (a Cactus configuration which includes) this thorn, at present
+you must set the environment variable \code{LAPACK\_DIR}, and possibly
+also the environment variables \code{LAPACK\_EXTRA\_LIBS} and
+\code{LAPACK\_EXTRA\_LIBDIRS}.  See the ``Library Configuration''
+section in the top-level \code{README} file for details.
+(I plan to replace this by an autoconfiguration scheme at some point.)
+
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
-\subsection{Parameters}
+\section{\thorn{AHFinderDirect} Parameters}
 
 This thorn has lots of parameters, but most of them have reasonable
 default values which you probably won't need to change.  Here I describe
@@ -422,12 +316,13 @@ possibly set explicitly.
 
 Note that all of the ``\code{[}$n$\code{]}'' parameters are Cactus
 array parameters, which you need to specify separately in your parameter
-file for each apparent horizon.  If this isn't clear, see the the
-examples in section~\ref{AEIDevelopment_AHFinderDirect_sect-examples}.
+file for each apparent horizon.  The example in
+section~\ref{AEIDevelopment_AHFinderDirect_sect-examples} should make
+this clear.
 
 %%%%%%%%%%%%%%%%%%%%
 
-\subsubsection{Overall Parameters}
+\subsection{Overall Parameters}
 
 \begin{description}
 \item[%%%
@@ -437,11 +332,13 @@ examples in section~\ref{AEIDevelopment_AHFinderDirect_sect-examples}.
       \end{tabular}
      ]
 \mbox{}\\
-	Do you want to find apparent horizons in the initial data
-	(at the \code{POSTINITIAL} Cactus schedule bin),
-	at each time step of a time evolution
-	(at the \code{POSTSTEP} Cactus schedule bin),
-	or both (the default)?
+	These are Boolean parameters specifying whether
+	\thorn{AHFinderDirect} should try to find apparent horizons
+	in the initial data (at the \code{CCTK\_POSTINITIAL} Cactus
+	schedule bin), at each time step of a time evolution
+	(at the \code{CCTK\_POSTSTEP} Cactus schedule bin),
+	or both (the default).
+
 \item[\code{N\_horizons}]
 \mbox{}\\
 	How many apparent horizons do you want to find in each slice?
@@ -467,6 +364,7 @@ examples in section~\ref{AEIDevelopment_AHFinderDirect_sect-examples}.
 	apparent horizon surrounding both black holes, then you should
 	set \verb|N_horizons = 2|, since you're finding at most 2
 	apparent horizons within the numerical grid.
+
 \item[\code{verbose\_level}]
 \mbox{}\\
 	This controls how verbose this thorn is in printing
@@ -499,12 +397,12 @@ examples in section~\ref{AEIDevelopment_AHFinderDirect_sect-examples}.
 
 %%%%%%%%%%%%%%%%%%%%
 
-\subsubsection{Choosing the Local Coordinate Origins (Expansion Centers)}
+\subsection{Choosing the Local Coordinate Origin for each Apparent Horizon}
 
 For each apparent horizon you want to find, you need to specify
 the Cactus $(x,y,z)$ coordinates of a local coordinate system origin.
 As described in
-section~\ref{AEIDevelopment_AHFinderDirect_sect-how-ahfinderdirect-works},
+section~\ref{AEIDeveopment_AHFinderDirect_sect-what-AHFinderDirect-needs},
 each apparent horizon must be a Strahlk\"{o}rper with respect to its
 local coordinate system origin.
 
@@ -542,12 +440,12 @@ I hope to add such a provision soon.%%%
 
 %%%%%%%%%%%%%%%%%%%%
 
-\subsubsection{Specifying the Initial Guess}
+\subsection{Specifying the Initial Guess}
 
 \thorn{AHFinderDirect} requires an initial guess
 for the apparent horizon's coordinate position and shape
 (that is, for the $h(\text{angle})$ function defined in
-section~\ref{AEIDevelopment_AHFinderDirect_sect-how-ahfinderdirect-works})
+section~\ref{AEIDeveopment_AHFinderDirect_sect-what-AHFinderDirect-needs}),
 for each apparent horizon you want to find.  Unlike some other apparent
 horizon finders (\eg{}~the curvature flow method in \thorn{AHFinder}),
 for \thorn{AHFinderDirect} there's no restriction on whether the initial
@@ -581,9 +479,9 @@ There are a number of parameters for specifying the initial guess:
 \footnote{%%%
 	 At the moment \code{initial\_guess\_method} is
 	 a single parameter which applies to all apparent
-	 horizons.  This might change (to it being a Cactus
-	 array parameter to be set separately for each
-	 apparent horizon) if there's popular demand\dots
+	 horizons.  This will probably change soon to it
+	 being a Cactus array parameter, which must be set
+	 separately for each apparent horizon.
 	 }%%%
 {}	There are several possibilities, most with their own sets
 	of subparameters:%%%
@@ -695,14 +593,25 @@ $^,$%%%
 
 %%%%%%%%%%%%%%%%%%%%
 
-\subsubsection{I/O Parameters}
+\subsection{I/O Parameters for the Apparent Horizon Shape(s)}
 
 The main output of this thorn is the computed horizon shape function
 $h({\rm angle})$, and correspondingly the $(x,y,z)$ coordinate positions
 of the apparent-horizon-surface (angular) grid points.  There are several
-parameters controlling if and how these should be written to data files:
+parameters controlling if, how often, and how these should be written
+to data files:
 
 \begin{description}
+\item[\code{how\_often\_to\_output\_h}]
+\mbox{}\\
+	If \verb|find_AHs_at_poststep| is set to true,
+	\thorn{AHFinderDirect} will find the apparent horizon(s)
+	at every time step.  However, you can control how often
+	(if at all) the apparent horizon shape(s) are written to
+	data files: this is only done if \verb|how_often_to_output_h|
+	is nonzero, and the Cactus time step number is an
+	integral multiple of this parameter.
+
 \item[\code{file\_format}]
 \mbox{}\\
 	This specifies the file format for $h$ (and other
@@ -714,9 +623,13 @@ parameters controlling if and how these should be written to data files:
 		This is a simple ASCII format designed for easy
 		plotting with gnuplot:
 		\begin{itemize}
-		\item	At each Cactus time step, each apparent horizon
-			is written to a separate data file.  The time
-			step number and the apparent horizon number
+		\item	\thorn{AHFinderDirect} writes a separate data file
+			for each Cactus time step and for each apparent
+			horizon found.  By default these are all written
+			in the starting directory of the Cactus run.
+			(I plan to change this at some point to use the 
+			\verb|IOUtil::out_dir| directory.)
+		\item	The time step number and the apparent horizon number
 			are both encoded in the file name; the actual
 			file name is given by a \verb|printf()| format
 			\verb|"%s.t%d.ah%d.%s"|, where
@@ -729,9 +642,9 @@ parameters controlling if and how these should be written to data files:
 			\item	the second \verb|%d| is the
 				apparent horizon number
 			\item	the second \verb|%s| is the
-				file name extension, which defaults to
-				\verb|"gp"| (see the \verb|param.ccl|
-				file if you want to change this)
+				file name extension, set by the
+				\verb|ASCII_gnuplot_file_name_extension|
+				parameter; this defaults to \verb|"gp"|
 			\end{itemize}
 		\item	Comment lines begin with \verb|#|.
 		\item	Patches are separated by 2 blank lines;
@@ -764,42 +677,110 @@ parameters controlling if and how these should be written to data files:
 		\end{verbatim}
 		will plot the apparent horizon surface in 3-D.
 	\end{description}
-\item[\code{how\_often\_to\_output\_h}]
-\mbox{}\\
-	This parameter controls how often \thorn{AHFinderDirect}
-	writes $h$ and other ``large'' \verb|file_format|-format
-	data files: it only writes them if this parameter
-	is nonzero, and the Cactus time step number is an
-	integral multiple of this parameter.
+
 \item[\code{h\_base\_file\_name}]
 \mbox{}\\
 	This specifies the base file name for $h$ data files, as
-	described above.
+	described above.  This defaults to "h".
+
+\item[\code{ASCII\_gnuplot\_file\_name\_extension}]
+\mbox{}\\
+	This specifies the file name extension for $h$ data files, as
+	described above.  This defaults to "gp".
+\end{description}
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\subsection{Other I/O Parameters}
+
+As well as the apparent horizon shape files, this thorn can also
+write files giving time series of various diagnostics.  These are
+controlled by the following parameters:
+
+\begin{description}
+\item[\code{output\_BH\_diagnostics}]
+\mbox{}\\
+	If this Boolean parameter is set to true,
+	\thorn{AHFinderDirect} will write a ``black hole diagnostics''
+	file for each distinct apparent horizon found
+	(up to \verb|N_horizons| files in all).  Each such file
+	contains a time series of various diagnostics for all time
+	steps when the corresponding apparent horizon was found.
+	The file format is again a simple ASCII format designed
+	for easy plotting with gnuplot:
+	\begin{itemize}
+	\item	The apparent horizon number is encoded in the file name;
+		the actual file name is given by a \verb|printf()| format
+		\verb|"%s.ah%d.%s"|, where
+		\begin{itemize}
+		\item	the first \verb|%s| is the base file name
+			set by the \verb|BH_diagnostics_base_file_name|
+			parameter (see below); this defaults to
+			\verb|"BH_diagnostics"|
+		\item	the \verb|%d| is the apparent horizon number
+		\item	the second \verb|%s| is the file name extension,
+			set by the \verb|BH_diagnostics_file_name_extension|
+			parameter; this defaults to \verb|"gp"|
+		\end{itemize}
+	\item	The file begins with a block of header comments
+		(lines begining with \verb|#|) describing the data
+		fields.
+	\item	Each time this apparent horizon is found, a single
+		line is appended to the data file, containing various
+		whitespace-separated fields.  The the precise list of
+		fields, see the header comments, or see the function
+		\verb|output_BH_diagnostics_fn()| in \verb|src/driver/io.cc|.
+		As of this writing the fields are:
+		\begin{itemize}
+		\item	the Cactus iteration number \verb|cctk_iteration|
+		\item	the Cactus time coordinate \verb|cctk_time|
+		\item	the Cactus $(x,y,z)$ coordinates of the apparent
+			horizon centroid
+		\item	the mean radius of the apparent horizon
+		\item	the $xy$, $xz$, and $yz$-plane circumferences
+			of the apparent horizon%%%
+\footnote{%%%
+	 Alas, these are computed using the
+	 $xy$, $xz$, and $yz$-planes passing through
+	 the local coordinate origin, so they may be
+	 considerably different (smaller) than the
+	 actual circumferences if the apparent horizon
+	 is displaced significantly from the local
+	 coordinate origin.
+	 }%%%
+		\item	the area of the apparent horizon, $A$
+		\item	the irreducible mass of the apparent horizon,
+			$\sqrt{A/16\pi}$
+		\end{itemize}
+	\end{itemize}
+
+\item[\code{BH\_diagnostics\_base\_file\_name}]
+	This specifies the base file name for black hole diagnostics
+	data files, as described above.  This defaults to
+	\verb|"BH_diagnostics"|.
+
+\item[\code{BH\_diagnostics\_file\_name\_extension}]
+	This specifies the file name extension for black hole diagnostics
+	data files, as described above.  This defaults to \verb|"gp"|.
 \end{description}
 
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
-\section{Accuracy and Diagnostics}
+\section{Accuracy}
 
 The apparent horizon positions are typically computed very accurately;
 tests on Kerr spacetimes give typical errors of $10^{-4}m$ to $10^{-5}m$.
 
-This thorn also prints various messages to standard output, as
-controlled by the \code{verbose\_level} parameter.  In particular,
-for each apparent horizon that it finds, it prints the horizon area,
-mass, and centroid position.  The area is computed very accurately
-(tests on Kerr spacetimes give errors below 1~part per million).
-The centroid position
-$\bar{\bf x}^i \equiv \int\! {\bf x}^i \, dS \big/ \int\! dS$
-is computed about as accurately, but is quite coordinate-dependent
-and gives only an approximate estimate of the black hole position.
-
-At present the only mass estimate available for an apparent horizon
-is the irreducible mass $m_\text{irreducible} \equiv \sqrt{A/16\pi}$.
-Note that even for Kerr spacetime, this formula deviates significantly
-from the ADM black hole mass if the spin is large.  (For example,
-for spin~$0.6$, $0.8$, and $0.999$, gives
-$m_\text{irreducible}/m_\text{ADM} = 0.949$, $0.894$, and $0.723$.)
+The various diagnostics printed to standard output and written to the
+black hole diagnostics file(s), are typically computed to accuracies
+on the order of a part per million or so.
+
+Note, however, that the irreducible mass $m_\text{irreducible}$
+may differ considerably from the black hole's local mass or its
+contribution to the slice's ADM mass.  For example, for Kerr spacetime
+in Kerr-Schild coordinates,
+$m_\text{irreducible}/m_\text{ADM} = 0.949$, $0.894$, and $0.723$
+for spin parameters $a \equiv J/m^2 = 0.6$, $0.8$, and $0.999$, respectively.
 It would be better to (also) use the ``isolated horizons'' formalism of
 \cite{AEIDevelopment_AHFinderDirect_Dreyer-etal-2002-isolated-horizons};
 at some point this thorn may be enhanced to do this.
@@ -876,6 +857,119 @@ AHFinderDirect::initial_guess__coord_sphere__radius[1] = 2.0
 
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
+\section{How \thorn{AHFinderDirect} Works}
+\label{AEIDevelopment_AHFinderDirect_sect-how-ahfinderdirect-works}
+
+\thorn{AHFinderDirect} uses the apparent horizon (henceforth \defn{horizon})
+finding algorithm of
+\cite{AEIDevelopment_AHFinderDirect_Thornburg-1996-apparent-horizon-finding},
+modified slightly to work with $g_\ij$ and $K_\ij$ on a Cartesian ($xyz$)
+grid:
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\subsection{General Description of the Algorithm}
+
+As described above, I parameterizes the horizon shape by
+$r = h(\text{angle})$ for some single-value function $h: S^2 \to \Re^+$.
+The apparent horizon equation~\eqref{AEIDevelopment_AHFinderDirect_eqn-horizon}
+then becomes a 2-D elliptic PDE on $S^2$ for the function $h$.  I finite
+difference this in angle to obtain a system of simultaneous nonlinear
+algebraic equations for $h$ at the angular grid points, and solve this
+system of equations by a global Newton's method (or a variant with
+improved convergence).
+
+Computationally, this algorithm has 3 main parts:
+\begin{itemize}
+\item	Computation of the ``horizon function'' $H(h)$ given a trial
+	surface defined by a trial horizon shape function $h$.  This is
+	done by interpolating the Cactus geometry fields $g_\ij$ and
+	$K_\ij$ (and optionally $\psi$) from the 3-D $xyz$ grid
+	to the (2-D set of) trial-horizon-surface grid points (also
+	computing $\partial_k g_\ij$ in the interpolation process),
+	then doing all further computations with angular grid functions
+	defined solely on $S^2$ (\ie{} at the horizon-surface grid points).
+\item	Computation of the Jacobian matrix $\Jac[H(h)]$ of $H(h)$.  This
+	thorn incorporates the \defn{symbolic differentiation} technique
+	described in
+\cite{AEIDevelopment_AHFinderDirect_Thornburg-1996-apparent-horizon-finding},
+	so this computation is quite fast.  The Jacobian is a highly
+	sparse matrix, but at the moment we neglect this and store it
+	as a full (dense) matrix.  (This should change soon.)
+\item	Solving the nonlinear equations $H(h) = 0$ by a global Newton's
+	method or a variant.  At the moment this thorn uses a simple
+	Newton's-method solver, and solves the linear system using
+	$\sf LU$ decomposition routines from LAPACK
+	(this should change to a sparse $\sf LU$ decomposition/solution
+	soon).
+\end{itemize}
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\subsection{The Multipatch System}
+
+Perhaps the most unusual feature of \thorn{AHFinderDirect} is the
+``multipatch'' system used to cover $S^2$ without coordinate singularities.
+In general there are 6~patches, one each covering a neighborhood of
+the $\pm z$, $\pm x$, and $\pm y$ axes, but this may be reduced in
+the presence of suitable symmetries.  For example,
+figure~\ref{AEIDevelopment_AHFinderDirect_fig-3patch}
+on page~\pageref{AEIDevelopment_AHFinderDirect_fig-3patch} shows a system
+of 3~patches covering the $+xyz$~octant of $S^2$.  This would be
+suitable for finding an apparent horizon with mirror symmetry about
+the (local) $z=0$~plane, and either 90~degree periodic rotation symmetry
+about the (local) $z$~axis, or mirror symmetry about each of the (local)
+$x$~and $y$~axes.
+
+\begin{figure}[htbp]
+\begin{center}
+\includegraphics{AEIDevelopment_AHFinderDirect_3patch.eps}
+\end{center}
+\caption[Illustration of the Multipatch System]
+	{
+	This figure shows a multipatch system covering the
+	$(+,+,+)$~octant of the unit sphere~$S^2$ with 3~patches.
+	The angular resolution is 5~degrees.  Notice that the
+	patches overlap by several ``ghost zone'' grid points.
+	}
+\label{AEIDevelopment_AHFinderDirect_fig-3patch}
+\end{figure}
+
+To allow easy angular finite differencing within the patch system,
+each patch is extended beyond its nominal extent by a ``ghost zone''%%%
+\footnote{%%%
+	 Note that this terminology differs somewhat
+	 from that used by Cactus in general; Cactus
+	 would call these ``patch zones'' or ``symmetry
+	 zones''.
+	 }%%%
+{} (2~grid points wide in
+figure~\ref{AEIDevelopment_AHFinderDirect_fig-3patch}).  Angular grid
+function values in the ghost zone can be obtained by interpatch
+interpolation%%%
+\footnote{%%%
+	 Due to the way the patch coordinates are defined,
+	 adjacent patches always share a common ``perpendicular''
+	 angular coordinate, so only 1-D interpolation
+	 is needed here.
+	 }%%%
+{} or by applying symmetry operations.  Once this is done, then angular
+finite differencing within the nominal extent of each patch can proceed
+normally, ignoring the patch boundaries.  \thorn{AHFinderDirect} can
+be configured at compile time to use either 2nd~order or 4th~order
+angular finite differencing (3~point or 5~point angular molecules);
+the default is 4th~order (5~point).  See \verb|src/include/config.h|
+if you want to change this.
+
+By default \thorn{AHFinderDirect} will automagically choose a patch
+system type for each apparent horizon searched for, based on the local
+coordinate origin and the symmetries implicit in the Cactus grid type.
+This generally works well, but if desired you can instead manually
+specify the patch system type, the angular resolution, the width of
+the ghost zones, etc.  See the \verb|param.ccl| file for details.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 \section{Other Related Thorns}
 
 If you're interested in \thorn{AHFinderDirect}, you might also be
@@ -899,6 +993,15 @@ interested in some other related thorns:
 
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
+\section{Acknowledgments}
+
+I thank Ian Hawke, Peter Diener, and Erik Schnetter for many valuable
+conversations, and the whole Cactus crew for a great infrastructure!
+I thank the Alexander von Humboldt foundation and the AEI visitors'
+and postdoctoral fellowships programs for financial support.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
 % make LaTeX read in ahfinderdirect.bbl produced by bibtex
 % run 'make bib' in this directory to update this
 \bibliography{ahfinderdirect}