diff options
Diffstat (limited to 'doc/documentation.tex')
| -rw-r--r-- | doc/documentation.tex | 542 |
1 files changed, 442 insertions, 100 deletions
diff --git a/doc/documentation.tex b/doc/documentation.tex index b097bbe..241fee3 100644 --- a/doc/documentation.tex +++ b/doc/documentation.tex @@ -1,137 +1,479 @@ -% *======================================================================* -% Cactus Thorn template for ThornGuide documentation -% Author: Ian Kelley -% Date: Sun Jun 02, 2002 -% $Header$ -% -% Thorn documentation in the latex file doc/documentation.tex -% will be included in ThornGuides built with the Cactus make system. -% The scripts employed by the make system automatically include -% pages about variables, parameters and scheduling parsed from the -% relevent thorn CCL files. -% -% This template contains guidelines which help to assure that your -% documentation will be correctly added to ThornGuides. More -% information is available in the Cactus UsersGuide. -% -% Guidelines: -% - Do not change anything before the line -% % BEGIN CACTUS THORNGUIDE", -% except for filling in the title, author, date etc. fields. -% - You can define your own macros are OK, but they must appear after -% the BEGIN CACTUS THORNGUIDE line, and do not redefine standard -% latex commands. -% - To avoid name clashes with other thorns, 'labels', 'citations', -% 'references', and 'image' names should conform to the following -% convention: -% ARRANGEMENT_THORN_LABEL -% For example, an image wave.eps in the arrangement CactusWave and -% thorn WaveToyC should be renamed to CactusWave_WaveToyC_wave.eps -% - Graphics should only be included using the graphix package. -% More specifically, with the "includegraphics" command. Do -% not specify any graphic file extensions in your .tex file. This -% will allow us (later) to create a PDF version of the ThornGuide -% via pdflatex. | -% - References should be included with the latex "bibitem" command. -% - For the benefit of our Perl scripts, and for future extensions, -% please use simple latex. -% -% *======================================================================* -% -% Example of including a graphic image: -% \begin{figure}[ht] -% \begin{center} -% \includegraphics[width=6cm]{MyArrangement_MyThorn_MyFigure} -% \end{center} -% \caption{Illustration of this and that} -% \label{MyArrangement_MyThorn_MyLabel} -% \end{figure} -% -% Example of using a label: -% \label{MyArrangement_MyThorn_MyLabel} -% -% Example of a citation: -% \cite{MyArrangement_MyThorn_Author99} -% -% Example of including a reference -% \bibitem{MyArrangement_MyThorn_Author99} -% {J. Author, {\em The Title of the Book, Journal, or periodical}, 1 (1999), -% 1--16. {\tt http://www.nowhere.com/}} -% -% *======================================================================* - -% If you are using CVS use this line to give version information -% $Header$ - \documentclass{article} % Use the Cactus ThornGuide style file -% (Automatically used from Cactus distribution, if you have a +% (Automatically used from Cactus distribution, if you have a % thorn without the Cactus Flesh download this from the Cactus % homepage at www.cactuscode.org) \usepackage{../../../../doc/ThornGuide/cactus} \begin{document} -% The author of the documentation -\author{} - -% The title of the document (not necessarily the name of the Thorn) -\title{} - -% the date your document was last changed, if your document is in CVS, -% please us: -% \date{$ $Date$ $} -\date{} +\title{PUGHSlab} +\author{Gabrielle Allen, Tom Goodale, Thomas Radke,\\ +with many comments and suggestions from Erik Schnetter and Jonathan Thornburg} +\date{October 2001} +%\date{$ $Date$ $} \maketitle % Do not delete next line % START CACTUS THORNGUIDE -% Add all definitions used in this documentation here -% \def\mydef etc - -% Add an abstract for this thorn's documentation \begin{abstract} - +Thorn PUGHSlab implements the generic hyperslab data extraction API for CCTK arrays. \end{abstract} -% The following sections are suggestive only. -% Remove them or add your own. - \section{Introduction} -\section{Physical System} +Many I/O thorns output data from distributed CCTK array variables. +If -- in a multiprocessor run -- output is done by only one processor, it needs to collect the +data from the others. This ties the I/O thorn to the driver +since it has to know about domain-decomposed data layout, interprocessor +ghostzones, etc. -\section{Numerical Implementation} +A clean way of separating the I/O code from the driver is to use +a thorn which provides a generic interface to get/put the distributed +data on/from the I/O processor for writing/reading. This interface can also +provide more features such as downsampling, datatype conversions, +or hyperslab selections. A hyperslab is defined in this context a subset of a +global CCTK array, with its own dimension, origin, direction, and extents.\\ -\section{Using This Thorn} +Another possible use of hyperslabs is the implementation of certain boundary +conditions (eg. reflection). By having the boundary condition code calling +generic hyperslab get/put calls, it can be written without special knowledge +about driver specifics.\\ -\subsection{Obtaining This Thorn} +This thorn documentation describes the complete generic hyperslab API. +All routines use CCTK data types in their argument lists and as return codes +exclusively. This allows actual implementations of this API to be realized as +CCTK function aliases. Different hyperslab thorns can then implementing the API +using the same function names, and other thorns using the API can be independent +of the actual hyperslab thorns which are compiled in.\\ -\subsection{Basic Usage} +The current version of thorn {\tt PUGHSlab} implements only parts of the CCTK +hyperslab API. +Please refer to section \ref{implementation} for implementation details. -\subsection{Special Behaviour} +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -\subsection{Interaction With Other Thorns} +\section{CCTK Hyperslab API Specification} +\label{specification} -\subsection{Support and Feedback} +In general, a hyperslab get/put operation is done in a three-level scheme: -\section{History} +\begin{enumerate} + \item In a first step, a hyperslab mapping is defined by calling one of the + following routines:\\ + {\tt + \hspace*{3ex}Hyperslab\_DefineLocalMappingByIndex()\\ + \hspace*{3ex}Hyperslab\_DefineLocalMappingByPhys()\\ + \hspace*{3ex}Hyperslab\_DefineGlobalMappingByIndex()\\ + \hspace*{3ex}Hyperslab\_DefineGlobalMappingByPhys()\\ + } + There exists two complement sets of routines: one for the definition of + local hyperslabs (which apply to a processor's local patch of a distributed + grid variable only), and one for global hyperslabs (which spawn the entire + grid). -\subsection{Thorn Source Code} + A mapping can be defined by either physical coordinates or by grid index points. -\subsection{Thorn Documentation} + All hyperslab mapping routines return an integer handle which refers to an + internally allocated data structure describing the defined hyperslab. -\subsection{Acknowledgements} + \item With such a mapping, hyperslabs can then be extracted/distributed by + one or more calls to + {\tt + \hspace*{3ex}Hyperslab\_Get()\\ + \hspace*{3ex}Hyperslab\_GetList()\\ + \hspace*{3ex}Hyperslab\_Put()\\ + \hspace*{3ex}Hyperslab\_PutList()\\ + } + There are routines for getting/putting a hyperslab from/to a single grid + variable or from/to a list of variables. + \item Once all hyperslabs are done, the hyperslab mapping should be freed by + a call to\\ + {\tt\hspace*{3ex}Hyperslab\_FreeMapping()}. +\end{enumerate} -\begin{thebibliography}{9} +If the {\tt Hyperslab\_Get*()/Hyperslab\_Put*()} get passed a mapping for a +global hyperslab, a global, synchronuous operation will be performed +(ie. it must be called in sync by every processor). All input arguments must be +consistent between processors. -\end{thebibliography} -% Do not delete next line -% END CACTUS THORNGUIDE +\subsection{Defining a hyperslab mapping} + +An M-dimensional hyperslab subspace mapped into an N-dimensional space +can be specified either by coordinates on the physical grid or by index +points on the underlying computational grid. + +\begin{verbatim} +CCTK_INT Hyperslab_DefineGlobalMappingByIndex ( + CCTK_POINTER_TO_CONST GH, + CCTK_INT vindex, + CCTK_INT hdim, + const CCTK_INT *direction /* vdim*hdim */, + const CCTK_INT *origin /* vdim */, + const CCTK_INT *extent /* hdim */, + const CCTK_INT *downsample /* hdim */, + CCTK_INT table_handle, + CCTK_FPOINTER conversion_fn, + CCTK_INT *hsize /* hdim */); + +CCTK_INT Hyperslab_DefineGlobalMappingByPhys ( + CCTK_POINTER_TO_CONST GH, + CCTK_INT vindex, + CCTK_INT hdim, + const CCTK_CHAR *coord_system_name, + const CCTK_INT *direction /* vdim*hdim */, + const CCTK_REAL *origin /* vdim */, + const CCTK_REAL *extent /* hdim */, + const CCTK_INT *downsample /* hdim */, + CCTK_INT table_handle, + CCTK_FPOINTER conversion_fn, + CCTK_INT *hsize /* hdim */); + + +CCTK_INT Hyperslab_DefineLocalMappingByIndex ( + CCTK_POINTER_TO_CONST GH, + CCTK_INT vindex, + CCTK_INT hdim, + const CCTK_INT *direction /* vdim*hdim */, + const CCTK_INT *origin /* vdim */, + const CCTK_INT *extent /* hdim */, + const CCTK_INT *downsample /* hdim */, + CCTK_INT table_handle, + CCTK_FPOINTER conversion_fn, + CCTK_INT *hsize_local, /* hdim */ + CCTK_INT *hsize_global, /* hdim */ + CCTK_INT *hoffset_global /* hdim */); + +CTK_INT Hyperslab_DefineLocalMappingByPhys ( + CCTK_POINTER_TO_CONST GH, + CCTK_INT vindex, + CCTK_INT hdim, + const CCTK_CHAR *coord_system_name, + const CCTK_INT *direction /* vdim*hdim */, + const CCTK_REAL *origin /* vdim */, + const CCTK_REAL *extent /* hdim */, + const CCTK_INT *downsample /* hdim */, + CCTK_INT table_handle, + CCTK_FPOINTER conversion_fn, + CCTK_INT *hsize_local, /* hdim */ + CCTK_INT *hsize_global, /* hdim */ + CCTK_INT *hoffset_global /* hdim */); +\end{verbatim} + +{\bf Function arguments:} +\begin{itemize} + \item{\tt CCTK\_POINTER\_TO\_CONST GH}\\ + The reference to the CCTK grid hierarchy. + + In a C implementation, this should be a pointer of type {\tt const cGH *}. + + \item{\tt CCTK\_INT vindex}\\ + In order to compute a hyperslab mapping, a CCTK grid variable must be given + by this argument which will be used as a template in the following + hyperslab get/put operation to denote the input arrays' domain + decomposition (dimensionality and distribution over processors). + The domain decomposition of all input CCTK variables given by the + {\tt vindex, vindices} arguments in subsequent calls to + {\tt Hyperslab\_GetXXX()/Hyperslab\_PutXXX()} must match the one of the + template variable {\tt vindex}. + + \item{\tt CCTK\_INT hdim}\\ + The dimension of the hyperslab to get/put ($0 < {\tt hdim} <= {\tt vdim}$). + + \item{\tt const CCTK\_INT *direction\\ + const CCTK\_INT *origin\\ + const CCTK\_INT *extent\\ + const CCTK\_INT *downsample\\ + --------------------------\\ + const CCTK\_CHAR *coord\_system\_name\\ + const CCTK\_INT *direction\\ + const CCTK\_REAL *origin\\ + const CCTK\_REAL *extent\\ + const CCTK\_INT *downsample}\\ + Arguments describing the actual mapping of the hyperslab to get/put. + + The hyperslab location is identified by its origin (lower left corner), + the direction vectors starting from the origin and spanning the hyperslab + in the N-dimensional space, and its extents (size of the hyperslab in + each direction). + + There are {\tt hdim} direction vectors (one for each hyperslab axis) with + {\tt vdim} elements. The direction vectors are given in grid index points and must + not be a linear combination of each other. The {\tt direction} argument must + be passed as an array {\tt direction[vdim\_index + hdim\_index*vdim]} + ({\tt vdim} is the fastest changing dimension). + + The origin and extent can be given in either physical coordinates or + grid points -- for the first case a coordinate system needs to be given + to do the mapping onto the underlying grid. + + The downsampling parameter denotes the downsampling factors for the + hyperslab to be extracted/distributed. They are given in terms of grid points in + each hyperslab direction. The downsampling parameter is optional -- + if NULL is passed here, no downsampling will be applied. + + \item{\tt CCTK\_INT table\_handle}\\ + A key/value table can be passed in via its handle to provide additional information + to the hyperslab get/put routines about the hyperslab mapping. + For example, there could be a tolerance parameter for hyperslabs + which are not rectangular to the underlying grid. For grid points which offset + from the direction vectors, the tolerance would then specify a + (plus/minus) offset for the directions saying which points should + still be included in the hyperslab space. + + Another example could be + whether to do interpolation between grid points or not. + + Passing a table handle is optional, an invalid (negative) table handle + denotes no additional table information. + + \item{\tt CCTK\_FPOINTER conversion\_fn}\\ + Reference to a user-defined datatype conversion function. + + Users can request a type conversion between input and output data during + a hyperslab get/put operation. A hyperslab API implementation may provide + a set of predefined data type conversion routines for this purpose. + In addition to this feature, users can also provide their own data type + conversion function and pass a reference to it in the {conversion\_fn} argument. + + For a C implementation, such a user-supplied conversion function should + be declared according to the following typedef: +\begin{verbatim} +typedef void (*t_hslabConversionFn) (CCTK_INT nelems, + CCTK_INT stride, + CCTK_INT from_type, + CCTK_INT to_type, + const void *from, + void *to); +\end{verbatim} + + A data type conversion function gets passed the number of elements to + convert ({\tt nelems}), and a stride between adjacent elements + ({\tt stride}), a pointer to the data to convert ({\tt from}), a pointer to + the conversion target buffer ({\tt to}), and the corresponding CCTK data + types for the data ({\tt from\_type, to\_type}). + + If a user-supplied function is given ({\tt conversion\_fn} is not NULL), + subsequent hyperslab get/put calls will use for data type conversions. + Otherwise the hyperslab get/put calls should fall + back to using an appropriate predefined data conversion function (if any exists). + + \item{\tt CCTK\_INT *hsize\\ + CCTK\_INT *hsize\_local}\\ + Reference to a size array with {\tt hdim} elements to be set by the + {\tt Hyperslab\_DefineXXXMappingByXXX()} routines. + + The resulting size of the hyperslab to be extracted is set + according to the hyperslab extents and downsampling parameters chosen. + With this information, one can compute the overall size of the hyperslab, + allocate memory for it and pass it as a user-provided hyperslab data buffer + into subsequent calls to {\tt Hyperslab\_GetXXX()}. + + \item{\tt CCTK\_INT *hsize\_global}\\ + Reference to a size array with {\tt hdim} elements to be set by the + {\tt Hyperslab\_DefineLocalMappingBy*()} routine. + + This array holds the sizes of the corresponding global hyperslab. + It is set according to the local hyperslab extents and downsampling + parameters chosen and locates the local hyperslab in the global grid. + + A value of NULL can be passed for {\tt hsize\_global} if no information + about the global hyperslab size is needed. + + \item{\tt CCTK\_INT *hoffset\_global}\\ + Reference to an offset array with {\tt hdim} elements to be set by the + {\tt Hyperslab\_DefineLocalMappingBy*()} routine. + + This array holds the offsets of the local hyperslab into the + corresponding global hyperslab. It is set according to the local + hyperslab extents and downsampling parameters chosen and locates the local + hyperslab in the global grid. + + A value of NULL can be passed for {\tt hoffset\_global} if no information + about a hyperslab offsets is needed. +\end{itemize} + +{\bf Return codes (according to the Cactus Coding Conventions):} +\begin{itemize} + \item 0 for success + \item negative for some error condition (to be defined by an actual + implementation of the {\tt Hyperslab\_Define*MappingBy*()} routines) +\end{itemize} + + +\subsection{Extracting/distributing a hyperslab} + +Each set of hyperslab get/put routines has two functions: one +which operates on a single hyperslab, and another which gets/puts hyperslabs +for a list of variables. +Depending on the actual hyperslab implementation it might be more efficient +to operate on a list of grid variables using +{\tt Hyperslab\_GetList()/Hyperslab\_PutList()} rather than doing sequential +calls to {\tt Hyperslab\_Get()/Hyperslab\_Put()} with individual grid variables. + +\begin{verbatim} +CCTK_INT Hyperslab_Get (CCTK_POINTER_TO_CONST GH, + CCTK_INT mapping_handle, + CCTK_INT proc, + const CCTK_INT vindex, + const CCTK_INT timelevel, + const CCTK_INT hdatatype, + void *hdata); + +CCTK_INT Hyperslab_GetList (CCTK_POINTER_TO_CONST GH, + CCTK_INT mapping_handle, + CCTK_INT num_arrays, + const CCTK_INT *procs /* num_arrays */, + const CCTK_INT *vindices /* num_arrays */, + const CCTK_INT *timelevels /* num_arrays */, + const CCTK_INT *hdatatypes /* num_arrays */, + void *const *hdata /* num_arrays */, + CCTK_INT *retvals /* num_arrays */); + +CCTK_INT Hyperslab_Put (CCTK_POINTER_TO_CONST GH, + CCTK_INT mapping_handle, + CCTK_INT proc, + CCTK_INT vindex, + CCTK_INT timelevel, + CCTK_INT hdatatype, + CCTK_POINTER_TO_CONST hdata); + +CCTK_INT Hyperslab_PutList (CCTK_POINTER_TO_CONST GH, + CCTK_INT mapping_handle, + CCTK_INT num_arrays, + const CCTK_INT *procs /* num_arrays */, + const CCTK_INT *vindices /* num_arrays */, + const CCTK_INT *timelevels /* num_arrays */, + const CCTK_INT *hdatatypes /* num_arrays */, + const CCTK_POINTER_TO_CONST hdata /* num_arrays */, + CCTK_INT *retvals /* num_arrays */); +\end{verbatim} + +{\bf Function arguments:} +\begin{itemize} + \item{\tt CCTK\_POINTER\_TO\_CONST GH}\\ + The reference to the CCTK grid hierarchy. + + In a C implementation, this should be a pointer of type {\tt const cGH *}. + + \item{\tt CCTK\_INT mapping\_handle}\\ + The handle for the hyperslab mapping as returned by a previous call to + one of the hyperslab mapping routines. + + \item{\tt CCTK\_INT num\_arrays}\\ + The total number of input arrays to get/put a hyperslab from/to. + + This must be a positive integer and match the number of array elements + in the arguments following. + + \item{\tt CCTK\_INT proc\\ + const CCTK\_INT *procs}\\ + The (list of) processor(s) which will receive/provide the hyperslab data. + + For {\tt Hyperslab\_GetXXX()}, there may be either exactly one processor + providing the hyperslab data (in this case, its processor ID must be given + in {\tt proc}), or all all processors will get the extracted hyperslab data + (an invalid (ie. negative) processor ID should be given as {\tt proc}, or + {\tt procs} is passed as a NULL pointer). + For {\tt Hyperslab\_PutXXX()}, there may only be one processor providing + the hyperslab data to be distributed to all others. + + \item{\tt CCTK\_INT vindex\\ + const CCTK\_INT *vindices}\\ + The (list of) CCTK variable(s) to get/put a hyperslab from/to. + + The grid variables are given by their CCTK indices; their domain + decomposition must match the template variable as given in a previous + hyperslab mapping routine call. + + \item{\tt CCTK\_INT timelevel\\ + const CCTK\_INT *timelevels}\\ + The (list of) timelevel(s) for the grid variable(s) to get/put a hyperslab from/to. + + Each element in the {\tt timelevels} array matches its entry in the + {\tt vindices} array argument. + If {\tt timelevels} is passed as a NULL pointer then all timelevels for + the list operation will default to 0 (denoting the current timelevel). + + \item{\tt CCTK\_INT hdatatype\\ + const CCTK\_INT *hdatatypes}\\ + The (list of) CCTK data type(s) of the hyperslab data. + + The hyperslab data to be extracted/distributed may be given in a data type + which is different to its corresponding grid variable. For this case, + the requested hyperslab data type must be specified explicitely. + The hyperslab routines will then do the neccessary data type conversions + either by using a user-supplied data type conversion function (as specified + in the {\tt conversion\_fn} argument to the hyperslab mapping routines), + or by choosing a built-in predefined data type conversion function. + convert the input array datatypes to some output array datatype. + + A negative value for {\tt hdatatype} or type or passing a NULL pointer for + the {\tt hdatatypes} argument indicates that both the grid variable and + its corresponding hyperslab have the same CCTK data type so that no type + conversion is necessary. + + \item{\tt (const) void *hdata\\ + (const) void *const *hdata}\\ + The (list of) user-supplied buffer(s) to store the extracted hyperslabs for + each input variable (for a get operation) or to read the hyperslab data + from (for a put operation). + + This argument is only evaluated on processors which are part of the + hyperslab mapping. + + \item{\tt CCTK\_INT *retvals}\\ + User-provided array to store the status of each individual get/put + operation in a {\tt Hyperslab\_XXXList()} call. + + Each element in the {\tt retvals} array will contain the status of the + corresponding hyperslab operation on grid variable {\it i}. + If {\tt retvals} is passed as a NULL pointer then no status codes for + individual hyperslab operations will be passed back to the caller. +\end{itemize} + +{\bf Return Codes for these routines (according to the Cactus Code conventions:} +\begin{itemize} + \item 0 for success + \item negative for some error condition (to be defined by an actual + implementation of the + {\tt Hyperslab\_GetXXX()/Hyperslab\_PutXXX()} routines) +\end{itemize} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +\section{Implementation Details} +\label{implementation} + +The current version of thorn {\tt PUGHSlab} implements only parts of the +CCTK hyperslab API as described in section \ref{specification}: +\begin{enumerate} + \item the definition of local/global hyperslabs based on grid indicices\\ + Currently, no additional hyperslab mapping information can be passed + through a key/value table; the {\tt table\_handle} argument is + completely ignored. + + {\tt PUGHSlab} provides a set of predefined built-in functions + for the following classes of data type conversions: + \begin{itemize} + \item any {\tt CCTK\_INT} data type into any other {\tt CCTK\_INT} + \item any {\tt CCTK\_REAL} data type into any other {\tt CCTK\_REAL} + \item any {\tt CCTK\_COMPLEX} data type into any other {\tt CCTK\_COMPLEX} + \end{itemize} + + \item local/global hyperslab extractions\\ + Global hyperslab get requests will strip off all processor-boundary + ghostzones from the returned hyperslab data. + + Local hyperslabs will always include processor-boundary ghostzones. + The {\tt hsize\_local, hoffset\_local} information returned by the + hyperslab mapping routines should be used to locate the locate hyperslab + within the global grid (eg. during a recombination of several local + hyperslabs into a single global one). +\end{enumerate} \end{document} |