Filled in the ThornGuide chapter for thorn PUGHSlab.

It describes the CCTK Hyperslab API specification, and what parts of it are
currently implemented by PUGHSlab.


git-svn-id: http://svn.cactuscode.org/arrangements/CactusPUGH/PUGHSlab/trunk@95 10716dce-81a3-4424-a2c8-48026a0d3035

1 files changed, 442 insertions, 100 deletions

diff --git a/doc/documentation.tex b/doc/documentation.tex
index b097bbe..241fee3 100644
--- a/doc/documentation.tex
+++ b/doc/documentation.tex
@@ -1,137 +1,479 @@
-% *======================================================================*
-%  Cactus Thorn template for ThornGuide documentation
-%  Author: Ian Kelley
-%  Date: Sun Jun 02, 2002
-%  $Header$                                                             
-%
-%  Thorn documentation in the latex file doc/documentation.tex 
-%  will be included in ThornGuides built with the Cactus make system.
-%  The scripts employed by the make system automatically include 
-%  pages about variables, parameters and scheduling parsed from the 
-%  relevent thorn CCL files.
-%  
-%  This template contains guidelines which help to assure that your     
-%  documentation will be correctly added to ThornGuides. More 
-%  information is available in the Cactus UsersGuide.
-%                                                    
-%  Guidelines:
-%   - Do not change anything before the line
-%       % BEGIN CACTUS THORNGUIDE",
-%     except for filling in the title, author, date etc. fields.
-%   - You can define your own macros are OK, but they must appear after
-%     the BEGIN CACTUS THORNGUIDE line, and do not redefine standard 
-%     latex commands.
-%   - To avoid name clashes with other thorns, 'labels', 'citations', 
-%     'references', and 'image' names should conform to the following 
-%     convention:          
-%       ARRANGEMENT_THORN_LABEL
-%     For example, an image wave.eps in the arrangement CactusWave and 
-%     thorn WaveToyC should be renamed to CactusWave_WaveToyC_wave.eps
-%   - Graphics should only be included using the graphix package. 
-%     More specifically, with the "includegraphics" command. Do
-%     not specify any graphic file extensions in your .tex file. This 
-%     will allow us (later) to create a PDF version of the ThornGuide
-%     via pdflatex. |
-%   - References should be included with the latex "bibitem" command.  
-%   - For the benefit of our Perl scripts, and for future extensions, 
-%     please use simple latex.     
-%
-% *======================================================================* 
-% 
-% Example of including a graphic image:
-%    \begin{figure}[ht]
-% 	\begin{center}
-%    	   \includegraphics[width=6cm]{MyArrangement_MyThorn_MyFigure}
-% 	\end{center}
-% 	\caption{Illustration of this and that}
-% 	\label{MyArrangement_MyThorn_MyLabel}
-%    \end{figure}
-%
-% Example of using a label:
-%   \label{MyArrangement_MyThorn_MyLabel}
-%
-% Example of a citation:
-%    \cite{MyArrangement_MyThorn_Author99}
-%
-% Example of including a reference
-%   \bibitem{MyArrangement_MyThorn_Author99}
-%   {J. Author, {\em The Title of the Book, Journal, or periodical}, 1 (1999), 
-%   1--16. {\tt http://www.nowhere.com/}}
-%
-% *======================================================================* 
-
-% If you are using CVS use this line to give version information
-% $Header$
-
 \documentclass{article}
 
 % Use the Cactus ThornGuide style file
-% (Automatically used from Cactus distribution, if you have a 
+% (Automatically used from Cactus distribution, if you have a
 %  thorn without the Cactus Flesh download this from the Cactus
 %  homepage at www.cactuscode.org)
 \usepackage{../../../../doc/ThornGuide/cactus}
 
 \begin{document}
 
-% The author of the documentation
-\author{} 
-
-% The title of the document (not necessarily the name of the Thorn)
-\title{}
-
-% the date your document was last changed, if your document is in CVS, 
-% please us:
-%    \date{$ $Date$ $}
-\date{}
+\title{PUGHSlab}
+\author{Gabrielle Allen, Tom Goodale, Thomas Radke,\\
+with many comments and suggestions from Erik Schnetter and Jonathan Thornburg}
+\date{October 2001}
+%\date{$ $Date$ $}
 
 \maketitle
 
 % Do not delete next line
 % START CACTUS THORNGUIDE
 
-% Add all definitions used in this documentation here 
-%   \def\mydef etc
-
-% Add an abstract for this thorn's documentation
 \begin{abstract}
-
+Thorn PUGHSlab implements the generic hyperslab data extraction API for CCTK arrays.
 \end{abstract}
 
-% The following sections are suggestive only.
-% Remove them or add your own.
-
 \section{Introduction}
 
-\section{Physical System}
+Many I/O thorns output data from distributed CCTK array variables.
+If -- in a multiprocessor run -- output is done by only one processor, it needs to collect the
+data from the others. This ties the I/O thorn to the driver
+since it has to know about domain-decomposed data layout, interprocessor
+ghostzones, etc.
 
-\section{Numerical Implementation}
+A clean way of separating the I/O code from the driver is to use
+a thorn which provides a generic interface to get/put the distributed
+data on/from the I/O processor for writing/reading. This interface can also
+provide more features such as downsampling, datatype conversions,
+or hyperslab selections. A hyperslab is defined in this context a subset of a
+global CCTK array, with its own dimension, origin, direction, and extents.\\
 
-\section{Using This Thorn}
+Another possible use of hyperslabs is the implementation of certain boundary
+conditions (eg. reflection). By having the boundary condition code calling
+generic hyperslab get/put calls, it can be written without special knowledge
+about driver specifics.\\
 
-\subsection{Obtaining This Thorn}
+This thorn documentation describes the complete generic hyperslab API.
+All routines use CCTK data types in their argument lists and as return codes
+exclusively. This allows actual implementations of this API to be realized as
+CCTK function aliases. Different hyperslab thorns can then implementing the API
+using the same function names, and other thorns using the API can be independent
+of the actual hyperslab thorns which are compiled in.\\
 
-\subsection{Basic Usage}
+The current version of thorn {\tt PUGHSlab} implements only parts of the CCTK
+hyperslab API.
+Please refer to section \ref{implementation} for implementation details.
 
-\subsection{Special Behaviour}
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
-\subsection{Interaction With Other Thorns}
+\section{CCTK Hyperslab API Specification}
+\label{specification}
 
-\subsection{Support and Feedback}
+In general, a hyperslab get/put operation is done in a three-level scheme:
 
-\section{History}
+\begin{enumerate}
+  \item In a first step, a hyperslab mapping is defined by calling one of the
+    following routines:\\
+    {\tt
+      \hspace*{3ex}Hyperslab\_DefineLocalMappingByIndex()\\
+      \hspace*{3ex}Hyperslab\_DefineLocalMappingByPhys()\\
+      \hspace*{3ex}Hyperslab\_DefineGlobalMappingByIndex()\\
+      \hspace*{3ex}Hyperslab\_DefineGlobalMappingByPhys()\\
+    }
+    There exists two complement sets of routines: one for the definition of
+    local hyperslabs (which apply to a processor's local patch of a distributed
+    grid variable only), and one for global hyperslabs (which spawn the entire
+    grid).
 
-\subsection{Thorn Source Code}
+    A mapping can be defined by either physical coordinates or by grid index points.
 
-\subsection{Thorn Documentation}
+    All hyperslab mapping routines return an integer handle which refers to an
+    internally allocated data structure describing the defined hyperslab.
 
-\subsection{Acknowledgements}
+  \item With such a mapping, hyperslabs can then be extracted/distributed by
+    one or more calls to
+    {\tt
+      \hspace*{3ex}Hyperslab\_Get()\\
+      \hspace*{3ex}Hyperslab\_GetList()\\
+      \hspace*{3ex}Hyperslab\_Put()\\
+      \hspace*{3ex}Hyperslab\_PutList()\\
+    }
+    There are routines for getting/putting a hyperslab from/to a single grid
+    variable or from/to a list of variables.
 
+  \item Once all hyperslabs are done, the hyperslab mapping should be freed by
+    a call to\\
+    {\tt\hspace*{3ex}Hyperslab\_FreeMapping()}.
+\end{enumerate}
 
-\begin{thebibliography}{9}
+If the {\tt Hyperslab\_Get*()/Hyperslab\_Put*()} get passed a mapping for a
+global hyperslab, a global, synchronuous operation will be performed
+(ie. it must be called in sync by every processor). All input arguments must be
+consistent between processors.
 
-\end{thebibliography}
 
-% Do not delete next line
-% END CACTUS THORNGUIDE
+\subsection{Defining a hyperslab mapping}
+
+An M-dimensional hyperslab subspace mapped into an N-dimensional space
+can be specified either by coordinates on the physical grid or by index
+points on the underlying computational grid.
+
+\begin{verbatim}
+CCTK_INT Hyperslab_DefineGlobalMappingByIndex (
+           CCTK_POINTER_TO_CONST GH,
+           CCTK_INT              vindex,
+           CCTK_INT              hdim,
+           const CCTK_INT       *direction         /* vdim*hdim */,
+           const CCTK_INT       *origin            /* vdim */,
+           const CCTK_INT       *extent            /* hdim */,
+           const CCTK_INT       *downsample        /* hdim */,
+           CCTK_INT              table_handle,
+           CCTK_FPOINTER         conversion_fn,
+           CCTK_INT             *hsize             /* hdim */);
+
+CCTK_INT Hyperslab_DefineGlobalMappingByPhys (
+           CCTK_POINTER_TO_CONST GH,
+           CCTK_INT              vindex,
+           CCTK_INT              hdim,
+           const CCTK_CHAR      *coord_system_name,
+           const CCTK_INT       *direction         /* vdim*hdim */,
+           const CCTK_REAL      *origin            /* vdim */,
+           const CCTK_REAL      *extent            /* hdim */,
+           const CCTK_INT       *downsample        /* hdim */,
+           CCTK_INT              table_handle,
+           CCTK_FPOINTER         conversion_fn,
+           CCTK_INT             *hsize             /* hdim */);
+
+
+CCTK_INT Hyperslab_DefineLocalMappingByIndex (
+           CCTK_POINTER_TO_CONST GH,
+           CCTK_INT              vindex,
+           CCTK_INT              hdim,
+           const CCTK_INT       *direction         /* vdim*hdim */,
+           const CCTK_INT       *origin            /* vdim */,
+           const CCTK_INT       *extent            /* hdim */,
+           const CCTK_INT       *downsample        /* hdim */,
+           CCTK_INT              table_handle,
+           CCTK_FPOINTER         conversion_fn,
+           CCTK_INT             *hsize_local,      /* hdim */
+           CCTK_INT             *hsize_global,     /* hdim */
+           CCTK_INT             *hoffset_global    /* hdim */);
+
+CTK_INT Hyperslab_DefineLocalMappingByPhys (
+           CCTK_POINTER_TO_CONST GH,
+           CCTK_INT              vindex,
+           CCTK_INT              hdim,
+           const CCTK_CHAR      *coord_system_name,
+           const CCTK_INT       *direction         /* vdim*hdim */,
+           const CCTK_REAL      *origin            /* vdim */,
+           const CCTK_REAL      *extent            /* hdim */,
+           const CCTK_INT       *downsample        /* hdim */,
+           CCTK_INT              table_handle,
+           CCTK_FPOINTER         conversion_fn,
+           CCTK_INT             *hsize_local,      /* hdim */
+           CCTK_INT             *hsize_global,     /* hdim */
+           CCTK_INT             *hoffset_global    /* hdim */);
+\end{verbatim}
+
+{\bf Function arguments:}
+\begin{itemize}
+  \item{\tt CCTK\_POINTER\_TO\_CONST GH}\\
+    The reference to the CCTK grid hierarchy.
+
+    In a C implementation, this should be a pointer of type {\tt const cGH *}.
+
+  \item{\tt CCTK\_INT vindex}\\
+    In order to compute a hyperslab mapping, a CCTK grid variable must be given
+    by this argument which will be used as a template in the following
+    hyperslab get/put operation to denote the input arrays' domain
+    decomposition (dimensionality and distribution over processors).
+    The domain decomposition of all input CCTK variables given by the
+    {\tt vindex, vindices} arguments in subsequent calls to
+    {\tt Hyperslab\_GetXXX()/Hyperslab\_PutXXX()} must match the one of the
+    template variable {\tt vindex}.
+
+  \item{\tt CCTK\_INT hdim}\\
+    The dimension of the hyperslab to get/put ($0 < {\tt hdim} <= {\tt vdim}$).
+
+  \item{\tt const CCTK\_INT *direction\\
+    const CCTK\_INT *origin\\
+    const CCTK\_INT *extent\\
+    const CCTK\_INT *downsample\\
+    --------------------------\\
+    const CCTK\_CHAR *coord\_system\_name\\
+    const CCTK\_INT  *direction\\
+    const CCTK\_REAL *origin\\
+    const CCTK\_REAL *extent\\
+    const CCTK\_INT  *downsample}\\
+    Arguments describing the actual mapping of the hyperslab to get/put.
+
+    The hyperslab location is identified by its origin (lower left corner),
+    the direction vectors starting from the origin and spanning the hyperslab
+    in the N-dimensional space, and its extents (size of the hyperslab in
+    each direction).
+
+    There are {\tt hdim} direction vectors (one for each hyperslab axis) with
+    {\tt vdim} elements. The direction vectors are given in grid index points and must
+    not be a linear combination of each other. The {\tt direction} argument must
+    be passed as an array {\tt direction[vdim\_index + hdim\_index*vdim]}
+    ({\tt vdim} is the fastest changing dimension).
+
+    The origin and extent can be given in either physical coordinates or
+    grid points -- for the first case a coordinate system needs to be given
+    to do the mapping onto the underlying grid.
+
+    The downsampling parameter denotes the downsampling factors for the
+    hyperslab to be extracted/distributed. They are given in terms of grid points in
+    each hyperslab direction. The downsampling parameter is optional --
+    if NULL is passed here, no downsampling will be applied.
+
+  \item{\tt CCTK\_INT table\_handle}\\
+    A key/value table can be passed in via its handle to provide additional information
+    to the hyperslab get/put routines about the hyperslab mapping.
+    For example, there could be a tolerance parameter for hyperslabs
+    which are not rectangular to the underlying grid. For grid points which offset
+    from the direction vectors, the tolerance would then specify a
+    (plus/minus) offset for the directions saying which points should
+    still be included in the hyperslab space.
+
+    Another example could be
+    whether to do interpolation between grid points or not.
+
+    Passing a table handle is optional, an invalid (negative) table handle
+    denotes no additional table information.
+
+  \item{\tt CCTK\_FPOINTER  conversion\_fn}\\
+    Reference to a user-defined datatype conversion function.
+
+    Users can request a type conversion between input and output data during
+    a hyperslab get/put operation. A hyperslab API implementation may provide
+    a set of predefined data type conversion routines for this purpose.
+    In addition to this feature, users can also provide their own data type
+    conversion function and pass a reference to it in the {conversion\_fn} argument.
+
+    For a C implementation, such a user-supplied conversion function should
+    be declared according to the following typedef:
+\begin{verbatim}
+typedef void (*t_hslabConversionFn) (CCTK_INT nelems,
+                                     CCTK_INT stride,
+                                     CCTK_INT from_type,
+                                     CCTK_INT to_type,
+                                     const void *from,
+                                     void *to);
+\end{verbatim}
+
+    A data type conversion function gets passed the number of elements to
+    convert ({\tt nelems}), and a stride between adjacent elements
+    ({\tt stride}), a pointer to the data to convert ({\tt from}), a pointer to
+    the conversion target buffer ({\tt to}), and the corresponding CCTK data
+    types for the data ({\tt from\_type, to\_type}).
+
+    If a user-supplied function is given ({\tt conversion\_fn} is not NULL),
+    subsequent hyperslab get/put calls will use for data type conversions.
+    Otherwise the hyperslab get/put calls should fall
+    back to using an appropriate predefined data conversion function (if any exists).
+
+  \item{\tt CCTK\_INT *hsize\\
+    CCTK\_INT *hsize\_local}\\
+    Reference to a size array with {\tt hdim} elements to be set by the
+    {\tt Hyperslab\_DefineXXXMappingByXXX()} routines.
+
+    The resulting size of the hyperslab to be extracted is set
+    according to the hyperslab extents and downsampling parameters chosen.
+    With this information, one can compute the overall size of the hyperslab,
+    allocate memory for it and pass it as a user-provided hyperslab data buffer
+    into subsequent calls to {\tt Hyperslab\_GetXXX()}.
+
+  \item{\tt CCTK\_INT *hsize\_global}\\
+    Reference to a size array with {\tt hdim} elements to be set by the
+    {\tt Hyperslab\_DefineLocalMappingBy*()} routine.
+
+    This array holds the sizes of the corresponding global hyperslab.
+    It is set according to the local hyperslab extents and downsampling
+    parameters chosen and locates the local hyperslab in the global grid.
+
+    A value of NULL can be passed for {\tt hsize\_global} if no information
+    about the global hyperslab size is needed.
+
+  \item{\tt CCTK\_INT *hoffset\_global}\\
+    Reference to an offset array with {\tt hdim} elements to be set by the
+    {\tt Hyperslab\_DefineLocalMappingBy*()} routine.
+
+    This array holds the offsets of the local hyperslab into the
+    corresponding global hyperslab. It is set according to the local
+    hyperslab extents and downsampling parameters chosen and locates the local
+    hyperslab in the global grid.
+
+    A value of NULL can be passed for {\tt hoffset\_global} if no information
+    about a hyperslab offsets is needed.
+\end{itemize}
+
+{\bf Return codes (according to the Cactus Coding Conventions):}
+\begin{itemize}
+  \item 0 for success
+  \item negative for some error condition (to be defined by an actual
+    implementation of the {\tt Hyperslab\_Define*MappingBy*()} routines)
+\end{itemize}
+
+
+\subsection{Extracting/distributing a hyperslab}
+
+Each set of hyperslab get/put routines has two functions: one
+which operates on a single hyperslab, and another which gets/puts hyperslabs
+for a list of variables.
+Depending on the actual hyperslab implementation it might be more efficient
+to operate on a list of grid variables using
+{\tt Hyperslab\_GetList()/Hyperslab\_PutList()} rather than doing sequential
+calls to {\tt Hyperslab\_Get()/Hyperslab\_Put()} with individual grid variables.
+
+\begin{verbatim}
+CCTK_INT Hyperslab_Get (CCTK_POINTER_TO_CONST GH,
+                        CCTK_INT       mapping_handle,
+                        CCTK_INT       proc,
+                        const CCTK_INT vindex,
+                        const CCTK_INT timelevel,
+                        const CCTK_INT hdatatype,
+                        void          *hdata);
+
+CCTK_INT Hyperslab_GetList (CCTK_POINTER_TO_CONST GH,
+                            CCTK_INT        mapping_handle,
+                            CCTK_INT        num_arrays,
+                            const CCTK_INT *procs      /* num_arrays */,
+                            const CCTK_INT *vindices   /* num_arrays */,
+                            const CCTK_INT *timelevels /* num_arrays */,
+                            const CCTK_INT *hdatatypes /* num_arrays */,
+                            void *const    *hdata      /* num_arrays */,
+                            CCTK_INT       *retvals    /* num_arrays */);
+
+CCTK_INT Hyperslab_Put (CCTK_POINTER_TO_CONST GH,
+                        CCTK_INT              mapping_handle,
+                        CCTK_INT              proc,
+                        CCTK_INT              vindex,
+                        CCTK_INT              timelevel,
+                        CCTK_INT              hdatatype,
+                        CCTK_POINTER_TO_CONST hdata);
+
+CCTK_INT Hyperslab_PutList (CCTK_POINTER_TO_CONST       GH,
+                            CCTK_INT                    mapping_handle,
+                            CCTK_INT                    num_arrays,
+                            const CCTK_INT             *procs      /* num_arrays */,
+                            const CCTK_INT             *vindices   /* num_arrays */,
+                            const CCTK_INT             *timelevels /* num_arrays */,
+                            const CCTK_INT             *hdatatypes /* num_arrays */,
+                            const CCTK_POINTER_TO_CONST hdata      /* num_arrays */,
+                            CCTK_INT                   *retvals    /* num_arrays */);
+\end{verbatim}
+
+{\bf Function arguments:}
+\begin{itemize}
+  \item{\tt CCTK\_POINTER\_TO\_CONST GH}\\
+    The reference to the CCTK grid hierarchy.
+
+    In a C implementation, this should be a pointer of type {\tt const cGH *}.
+
+  \item{\tt CCTK\_INT mapping\_handle}\\
+    The handle for the hyperslab mapping as returned by a previous call to
+    one of the hyperslab mapping routines.
+
+  \item{\tt CCTK\_INT num\_arrays}\\
+    The total number of input arrays to get/put a hyperslab from/to.
+
+    This must be a positive integer and match the number of array elements
+    in the arguments following.
+
+  \item{\tt CCTK\_INT proc\\
+    const CCTK\_INT *procs}\\
+    The (list of) processor(s) which will receive/provide the hyperslab data.
+
+    For {\tt Hyperslab\_GetXXX()}, there may be either exactly one processor
+    providing the hyperslab data (in this case, its processor ID must be given
+    in {\tt proc}), or all all processors will get the extracted hyperslab data
+    (an invalid (ie. negative) processor ID should be given as {\tt proc}, or
+    {\tt procs} is passed as a NULL pointer).
+    For {\tt Hyperslab\_PutXXX()}, there may only be one processor providing
+    the hyperslab data to be distributed to all others.
+
+  \item{\tt CCTK\_INT vindex\\
+    const CCTK\_INT *vindices}\\
+    The (list of) CCTK variable(s) to get/put a hyperslab from/to.
+
+    The grid variables are given by their CCTK indices; their domain
+    decomposition must match the template variable as given in a previous
+    hyperslab mapping routine call.
+
+  \item{\tt CCTK\_INT timelevel\\
+    const CCTK\_INT *timelevels}\\
+    The (list of) timelevel(s) for the grid variable(s) to get/put a hyperslab from/to.
+
+    Each element in the {\tt timelevels} array matches its entry in the
+    {\tt vindices} array argument.
+    If {\tt timelevels} is passed as a NULL pointer then all timelevels for
+    the list operation will default to 0 (denoting the current timelevel).
+
+  \item{\tt CCTK\_INT hdatatype\\
+    const CCTK\_INT *hdatatypes}\\
+    The (list of) CCTK data type(s) of the hyperslab data.
+
+    The hyperslab data to be extracted/distributed may be given in a data type
+    which is different to its corresponding grid variable. For this case,
+    the requested hyperslab data type must be specified explicitely.
+    The hyperslab routines will then do the neccessary data type conversions
+    either by using a user-supplied data type conversion function (as specified
+    in the {\tt conversion\_fn} argument to the hyperslab mapping routines),
+    or by choosing a built-in predefined data type conversion function.
+    convert the input array datatypes to some output array datatype.
+
+    A negative value for {\tt hdatatype} or type or passing a NULL pointer for
+    the {\tt hdatatypes} argument indicates that both the grid variable and
+    its corresponding hyperslab have the same CCTK data type so that no type
+    conversion is necessary.
+
+  \item{\tt (const) void *hdata\\
+    (const) void *const *hdata}\\
+    The (list of) user-supplied buffer(s) to store the extracted hyperslabs for
+    each input variable (for a get operation) or to read the hyperslab data
+    from (for a put operation).
+
+    This argument is only evaluated on processors which are part of the
+    hyperslab mapping.
+
+  \item{\tt CCTK\_INT *retvals}\\
+    User-provided array to store the status of each individual get/put
+    operation in a {\tt Hyperslab\_XXXList()} call.
+
+    Each element in the {\tt retvals} array will contain the status of the
+    corresponding hyperslab operation on grid variable {\it i}.
+    If {\tt retvals} is passed as a NULL pointer then no status codes for
+    individual hyperslab operations will be passed back to the caller.
+\end{itemize}
+
+{\bf Return Codes for these routines (according to the Cactus Code conventions:}
+\begin{itemize}
+  \item 0 for success
+  \item negative for some error condition (to be defined by an actual
+    implementation of the
+    {\tt Hyperslab\_GetXXX()/Hyperslab\_PutXXX()} routines)
+\end{itemize}
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+\section{Implementation Details}
+\label{implementation}
+
+The current version of thorn {\tt PUGHSlab} implements only parts of the
+CCTK hyperslab API as described in section \ref{specification}:
+\begin{enumerate}
+  \item the definition of local/global hyperslabs based on grid indicices\\
+    Currently, no additional hyperslab mapping information can be passed
+    through a key/value table; the {\tt table\_handle} argument is
+    completely ignored.
+
+    {\tt PUGHSlab} provides a set of predefined built-in functions
+    for the following classes of data type conversions:
+    \begin{itemize}
+      \item any {\tt CCTK\_INT} data type into any other {\tt CCTK\_INT}
+      \item any {\tt CCTK\_REAL} data type into any other {\tt CCTK\_REAL}
+      \item any {\tt CCTK\_COMPLEX} data type into any other {\tt CCTK\_COMPLEX}
+    \end{itemize}
+
+  \item local/global hyperslab extractions\\
+    Global hyperslab get requests will strip off all processor-boundary
+    ghostzones from the returned hyperslab data.
+
+    Local hyperslabs will always include processor-boundary ghostzones.
+    The {\tt hsize\_local, hoffset\_local} information returned by the
+    hyperslab mapping routines should be used to locate the locate hyperslab
+    within the global grid (eg. during a recombination of several local
+    hyperslabs into a single global one).
+\end{enumerate}
 
 \end{document}