More documentation added

git-svn-id: http://svn.cactuscode.org/arrangements/CactusBase/IOUtil/trunk@76 b32723a9-ab3a-4a60-88e2-2e5d99d7c17a

1 files changed, 183 insertions, 18 deletions

diff --git a/doc/documentation.tex b/doc/documentation.tex
index 16ccf37..02f9fb6 100644
--- a/doc/documentation.tex
+++ b/doc/documentation.tex
@@ -2,7 +2,7 @@
 \begin{document}
 
 \title{IOUtil}
-\author{Thomas Radke}
+\author{Thomas Radke, Gabrielle Allen}
 \date{June 2000}
 \maketitle
 
@@ -23,6 +23,9 @@ decide which variables to output), and a general set of parameters which
 are inherited by the different IO methods (such as the output directory). 
 Thorn {\tt IOUtil} by itself provides no IO methods.
 
+More information about IO and Visualisation of Cactus data can be found in 
+the individual IO thorns, and in the {\tt Visualization-HOWTO} available 
+on the Cactus web pages.
 
 \section{IO methods in Cactus}
 
@@ -33,7 +36,7 @@ and all methods are registered with the flesh, satisfying the
 Cactus API, allowing them to be called directly from application thorns. 
 An IO method registers itself with the flesh along with it's name, and 
 these registered names are the labels we now use to describe the various
-methods.
+methods. 
 
 
 \begin{table}
@@ -95,6 +98,24 @@ For example, you may want scalar and 1D output at every iteration
 but computationally expensive 3D output only every 10th iteration, 
 with these files going into another directory on a scratch partition.
 
+\section{External Packages}
+
+NOTE: This section will not be filled out properly until the new treatment of external 
+packages is implemented.
+
+Some of the IO methods used by Cactus require that external packages are installed and available 
+on your machine. 
+
+\begin{itemize}
+
+\item{\tt HDF5} {\bf Hierachical Data Format}\\
+{\tt http://hdf.ncsa.uiuc.edu/}
+
+\item{\tt IEEEIO} \\
+{\tt http://zeus.ncsa.uiuc.edu/}
+
+\end{itemize}
+
 \section{Standard Parameters}
 
 Here we describe a few of the standard parameters used by {\tt IOUtil}
@@ -221,14 +242,68 @@ checkpoint files, the same IO modes are used (see Section~\ref{iomodes}).
 Note that the recovery routines in Cactus can process both chunked and 
 unchunked checkpoint files if you restart on the same number of processors.
 That's why you should always use one of the parallel IO modes for 
-checkpointing.
-
-<CAN RESTART ON DIFFERENT NUMBER OF PROCESSORS OR DIFFERENT MACHINES>
-
-\section{Data Filename Extensions}
-
+checkpointing. If you want to restart on a different number of processors, you first need to 
+{\it recombine} the data in the checkpoint file(s) to create a single file with
+unchunked data.
+Note that you can restart from your checkpoint file on a different machine.
+
+\vskip .5cm
+\noindent{\bf WARNING:}
+
+\noindent
+Checkpointing and recovery should {\bf always} be tested for a new thorn set. This is because
+only Cactus grid variables and parameters are saved in a checkpoint file. If a thorn has made 
+use of saved local
+variables, the state of a recovered simulation may differ from the original run. To test 
+checkpointing and recovery, simply perform one run of say 10 timesteps, and compare output data
+with a checkpointed and recovered run at say the 5th timestep. The output data should match 
+exactly if recovery was successful.
+
+\section{Data Filenames and Filename Extensions}
+
+The standard IO thorns in Cactus make use of a consistent set of filenames and extensions, which 
+identify the variables and data format used in the file. The filenames are listed in Table~\ref{two},
+and the extensions in Table~\ref{three}
+
+\begin{center}
 \begin{table}
 \label{two}
+\begin{tabular}{|l|l|}
+  \hline
+  {\bf IO method} & {\bf Filename for output of variable {\tt phi}}\\
+  \hline
+  {\tt Scalar} & 
+phi\_nm1.tl, phi\_nm2.tl phi\_max.tl phi\_min.tl
+\\
+  {\tt Info}      & 
+Provides no output file (the same data is available with {\tt Scalar} output)
+\\
+  {\tt IOASCII\_1D} & 
+phi.xl, phi.yl, phi.zl, phi.tl
+\\
+  {\tt IOASCII\_2D} & 
+??
+\\
+  {\tt IOHDF5\_2D} & 
+phi2d.h5
+\\
+  {\tt IOHDF5\_3D} & 
+phi3d.h5
+\\
+  {\tt IOFlexIO\_2D} & 
+phi2d.ieee
+\\
+  {\tt IOFlexIO\_3D} & 
+phi3d.ieee
+\\
+  \hline
+\end{tabular}
+\caption{Filenames used by standard IO thorns}
+\end{table}
+\end{center}
+\begin{center}
+\begin{table}
+\label{three}
 \begin{tabular}{|c|p{4.5cm}|l|}
 \hline
 {\bf Extension} & {\bf Description} & {\bf Thorn} \\
@@ -260,38 +335,38 @@ checkpointing.
 \end{tabular}
 \caption{File extensions used by the standard IO thorns in the Cactus distribution}
 \end{table}
-
+\end{center}
  
 \section{Example Parameter Files}
 
 Here we give examples of the parameters for the different IO methods.
 
 \begin{itemize}
-\item{Screen Information {\tt info}}
+\item{\bf Screen Information {\tt info}}
 \begin{verbatim}
-ActiveThorns = ``iobasic ioutil ...''
+ActiveThorns = "iobasic ioutil ..."
 
 # Output using all methods on iteration 0, 10, 20, ...
 io::out_every = 10            
 
 # Group of variables to output to screen
-iobasic::outinfo_vars = ``evolve::vars''
+iobasic::outinfo_vars = "evolve::vars"
 \end{verbatim}
 
-\item{Scalar Output {\tt scalar}}
+\item{\bf Scalar Output {\tt scalar}}
 \begin{verbatim}
-ActiveThorns = ``iobasic ioutil ...''
+ActiveThorns = "iobasic ioutil ..."
 
 # Output vars using scalar method on iteration 0, 10, 20, ...
 iobasic::outscalar_every = 10            
 
 # Group of variables to output to file
-iobasic::outscalar_vars = ``evolve::vars''
+iobasic::outscalar_vars = "evolve::vars"
 \end{verbatim}
 
-\item{ASCII 1D and 2D Output {\tt ioascii\_1d}, {\tt ioascii\_2d}}
+\item{\bf ASCII 1D and 2D Output {\tt ioascii\_1d}, {\tt ioascii\_2d}}
 \begin{verbatim}
-ActiveThorns = ``ioascii ioutil hyperslab ...''
+ActiveThorns = "ioascii ioutil hyperslab ..."
 
 # Output vars using ascii 1d method on iteration 0, 10, 20, ...
 ioascii::out1d_every = 10            
@@ -300,7 +375,97 @@ ioascii::out1d_every = 10
 ioascii::out1d_every = 100
 
 # Group of variables to output to file for each method
-iobasic::outscalar_vars = ``evolve::vars''
+iobasic::outscalar_vars = "evolve::vars"
+\end{verbatim}
+
+\item{\bf HDF5 2D and 3D Output {\tt iohdf5\_2d}, {\tt iohdf5\_3d}}
+\begin{verbatim}
+ActiveThorns = "iohdf5 ioutil hyperslab ..."
+
+# Output vars using ascii 2d method on iteration 0, 100, 200, ...
+iohdf5::out2d_every = 100            
+
+# Output vars using ascii 3d method on iteration 0, 5, 10, ...
+iohdf5::out3d_every = 5
+
+# Standard IO directory for 2D output
+io::outdir = "test"
+
+# Special IO directory for 3D output
+iohdf5::outdir3D = "/scratch/tmp"
+
+# 3D Output unchunked to one file 
+# (Only using a small number of processors)
+io::out3D_mode = "onefile"
+io::out3D_unchunked = "yes"
+
+# Downsample 3D data by a factor of 3 in each direction
+io::out3D_downsample_x = 3
+io::out3D_downsample_y = 3
+io::out3D_downsample_z = 3
+
+# Group of variables to output to file for each method
+iohdf5::out2D_vars = "evolve::vars"
+iohdf5::out3D_vars = "evolve::vars"
+\end{verbatim}
+
+
+\item{\bf IEEEIO 2D and 3D Output {\tt ioflexio\_2d}, {\tt ioflexio\_3d}}
+\begin{verbatim}
+ActiveThorns = "ioflexio ieeeio ioutil hyperslab ..."
+
+# Output vars using 2D FlexIO method on iteration 0, 100, 200, ...
+ioflexio::out2d_every = 100            
+
+# Output vars using 3D FlexIO method on iteration 0, 5, 10, ...
+ioflexio::out3d_every = 5
+
+# Standard IO directory for 2D output
+io::outdir = "test"
+
+# Special IO directory for 3D output
+ioflexio::outdir3D = "/scratch/tmp"
+
+# 3D Output chunked to one file for every eight processors
+# (Run on large number of processors)
+io::out3D_mode = "np"
+io::out3D_nproc = 8
+io::out3D_unchunked = "no"
+
+# Downsample 3D data by a factor of 3 in each direction
+io::out3D_downsample_x = 3
+io::out3D_downsample_y = 3
+io::out3D_downsample_z = 3
+
+# Group of variables to output to file for each method
+ioflexio::out2D_vars = "evolve::vars"
+ioflexio::out3D_vars = "evolve::vars"
+\end{verbatim}
+
+\item{\bf Checkpointing using IEEEIO}
+\begin{verbatim}
+ActiveThorns = "ioflexio ieeeio ioutil hyperslab ..."
+
+# Use IEEEIO data format 
+ioflexio::checkpoint = "yes"
+
+# Make a new checkpoint file every 300 iterations
+io::checkpoint_every = 300
+
+# Name and directory of checkpoint file
+io::checkpoint_file = "run5"
+io::checkpoint_dir  = "/scratch/tmp"
+\end{verbatim}
+
+\item{\bf Recovering from a checkpoint file}
+\begin{verbatim}
+ActiveThorns = "ioflexio ieeeio ioutil hyperslab ..."
+
+io::recover = "auto"
+
+# Name and directory of checkpoint file
+io::recover_file = "run5"
+io::recover_dir  = "/scratch/tmp"
 \end{verbatim}
 
 \end{itemize}