Restore old reduction docs so people can understand current codes

and know how to migrate to new API.  Patch from Yaakoub.


git-svn-id: http://svn.cactuscode.org/flesh/trunk@3882 17b73243-c579-4c4c-a9d2-2d5706c11dac

1 files changed, 389 insertions, 7 deletions

diff --git a/doc/UsersGuide/ThornWriters.tex b/doc/UsersGuide/ThornWriters.tex
index 2f2458c5..b5dcf10f 100644
--- a/doc/UsersGuide/ThornWriters.tex
+++ b/doc/UsersGuide/ThornWriters.tex
@@ -2585,15 +2585,43 @@ name.
 \section{Reduction Operators}
 \label{sec:reop}
 
-The Flesh does not provide reduction routines by itself. Instead
-it offers a general function API to thorns for the registration and
-invocation of reduction operators.
-
-There are two different Flesh APIs for reduction, depending
+A reduction operation can be defined as an operation on variables
+distributed  across multiple processor resulting in a single number.
+Typical reduction operations are: sum, minimum/maximum value, boolean
+operations. A typical application is, for example, 
+finding the maximum reduction from processor local error estimates, 
+therefor making the previous processor local error known to all processors. 
+
+The exchange of
+information across processors needs the functionality of a
+communication layer e.g. {\tt CactusPUGH/PUGH}. For this reason, the
+reduction operation itself is not part of the flesh, instead Cactus (again)
+provides a registration mechanism for thorns to register routines they 
+provide as reduction operators. The different operators are
+identified by their name and/or a unique number, called a {\em handle}.
+
+The registration mechanism gives the advantage of a common
+interface while hiding the individual communication calls in the
+layer.
+
+In Cactus, reduction operators can be applied to 
+grid functions, arrays and scalars, as well as to local arrays. Note that 
+different implementations of reduction operators may be limited in 
+the objects they can be applied to.
+There is a fundamental difference between the reduction operation on
+grid functions and quantities as arrays.
+
+Currently the Flesh supports the new and old reduction specification.
+The old APIs will be deprecated in the next beta cycle in favour of the
+new specification.
+
+{\bf New Reduction Specification API documentation}
+\vskip .25cm
+In the new reduction specification, there are two different Flesh APIs for reduction, depending
 on whether the data arrays are Cactus grid arrays or processor-local,
 programming language built-in arrays, and on what assumptions
 are made about the topology and spacing of the grid (these descriptions
-are for 3-D, but the generalizations to other numbers of dimensions
+are for 3-D, but the generalisations to other numbers of dimensions
 should be obvious):
 \begin{Lentry}
 \item[{\tt CCTK\_ReduceGridArrays()}]
@@ -2623,6 +2651,360 @@ may be used to refer to the operator.  \verb|CCTK_LocalArrayReductionHandle()|
 is used to get the handle corresponding to a given character string
 name.
 
+{\bf Old Reduction Specification API documentation}
+\vskip .25cm
+
+{\bf Obtaining the reduction handle}
+
+Before calling the routine which performs the reduction operation,
+the handle, which identifies the operation must be derived from its
+registered name.
+
+\begin{verbatim}
+
+int CCTK_ReductionHandle(const char *reduction_name);
+
+integer       reduction_handle
+character*(*) reduction_name
+call CCTK_ReductionHandle(reduction_handle, reduction_name)
+
+
+int CCTK_ReductionArrayHandle(const char *reduction_name);
+
+integer       reduction_handle
+character*(*) reduction_name
+call CCTK_ReductionArrayHandle(reduction_handle, reduction_name)
+
+\end{verbatim}
+
+
+\vskip .25cm
+
+\begin{Lentry}
+\item[{\tt reduction\_handle}] 
+In Fortran the name of the variable will
+contain the handle value after the call. In C this value is the
+function value.
+\item[{\tt reduction\_name}] 
+is the name under which the operator has
+been registered by the providing thorn. The only thorn in the standard
+Computational Toolkit release which provides reduction operators is
+{\tt CactusPUGH/PUGHReduce}.
+\item[{\bf error checking}] 
+negative handle value indicates failure to
+identify the correct operator. 
+\end{Lentry}
+
+Get a integer handle corresponding to a given reduction operator.
+The operator is identified by the name it was registered with.
+(Note that although it would appear to be far more convenient to 
+pass the name of the reduction operator directly to the following
+function call to {\t CCTK\_Reduce} this causes problems with the
+translation of strings from {\t FORTRAN} to {\t C} with variable
+argument lists).
+
+\vskip 0.25cm
+
+
+{\bf The general reduction interface}
+The main interfaces for reduction operations are quite powerful (and
+hence rather complicated) To ease the use of these main interfaces, wrappers
+designed for specific and more restricted use are described below. If 
+uncertain, you should use these.
+
+{\t
+\begin{verbatim}
+
+int CCTK_Reduce(  cGH *GH, 
+                  int proc,
+                  int operation_handle,
+                  int num_out_vals,
+                  int type_out_vals,
+                  void *out_vals,
+                  int num_in_fields,
+                  ...);
+
+
+call CCTK_Reduce( int returnvalue, 
+                  cctkGH, 
+                  int processor,
+                  int operation_handle,
+                  int num_out_vals,
+                  int type_out_vals,
+                  out_vals,
+                  int num_in_fields,
+                  ... )
+
+int CCTK_ReduceArray(  cGH *GH,
+                       int proc,
+                       int operation_handle,
+                       int num_out_vals,
+                       int type_out_vals,
+                       void *out_vals,
+                       int num_dims,
+                       int num_in_arrays,
+                       int type_in_arrays,
+                       ... )
+
+call CCTK_ReduceArray(int returnvalue,
+                      cctkGH,
+                      int processor,
+                      int operation_handle,
+                      int num_out_vals,
+                      int type_out_arrays,
+                      void out_vals,
+                      int num_dims,
+                      int num_in_arrays,
+                      int type_in_arrays,
+                      ... )
+\end{verbatim}
+}
+
+\begin{Lentry}
+\item[{\tt int returnvalue}] 
+the return value of the operation. negative
+value indicates failure to perform reduction.
+Zero indicates successful operation.
+\item[{\tt cctkGH}]
+in Fortran the pointer to the grid hierarchy
+structure. Cannot be used within Fortran but
+can be used from within 
+C. Since this name is fixed write it out shown
+above.
+\item[{\tt cGH *GH}]
+ in the C the pointer to the grid hierarchy.
+\item[{\tt int processor}] 
+the processor which collects the
+information, negative value ($-1$) will distribute the data to all
+processors.
+\item[{\tt int operation\_handle}] the number of the reduction operation
+                handle, needs to be found by calling {\tt CCTK\_ReductionHandle} or
+                {\tt CCTK\_ReductionArrayHandle}.
+\item[{\tt int num\_out\_vals}] integer defining the number of ??
+\item[{\tt int type\_out\_arrays}, {\tt type\_in\_arrays}] 
+specifies the type of the gridfunction 
+you are communicating. Use the values as specified in
+\ref{sec:datyansi}. Note: do not {\em mix} datatypes: e.g. in
+Fortran do not declare a variable as {\tt integer} and then specify  
+the type {\tt CCTK\_VARIABLE\_INT} in the reduction command. These
+types need not be the same on some architectures and will conflict.
+\item[{\tt out\_vals}]
+\item[{\tt int num\_in\_fields}] specifies the number of  to follow
+\item[{\tt ...}] indicates a variable argument list: specify the size
+of the array in each dimension, comma separated numbers or integer
+variables. Specify the arrays which will be reduced, the number of
+specified arrays must be the same as the value of the {\tt
+num\_in\_fields} variable.
+\item[{\bf error checking}] a return value other than zero indicates
+failure to perform the operation.
+\end{Lentry}
+
+\vskip 0.25cm
+
+
+{\bf Special reduction interfaces}
+
+The routines are designed for the purpose of reducing scalars, arrays
+and grid functions. They hide many of the options of the generic
+interface described above.\\
+
+{\bf Reduction of local scalars} across multiple processors, the result of 
+the reduction operation will be on the specified processor or on all processors.
+
+{\t
+\begin{verbatim}
+int CCTK_ReduceLocScalar (cGH *GH, 
+                          int processor, 
+                          int operation_handle,
+                          void *in_scalar, 
+                          void *out_scalar, 
+                          int data_type)
+
+call CCTK_ReduceLocScalar(int returnvalue,
+                          cctkGH,
+                          int processor,
+                          int operation_handle,
+                          in_scalar,
+                          out_scalar,
+                          int data_type)
+\end{verbatim}
+}
+\begin{Lentry}
+\item[{\tt in\_scalar}] the processor local variable with local value to be reduced
+\item[{\tt out\_scalar}] the reduction result: a processor local variable 
+with the global value (same on all procs) if {\tt processor} has been
+set to $-1$. Otherwise {\tt processor} will hold the reduction result.
+\item[{\tt data\_type}]
+specifies the type of the gridfunction 
+you are communicating. Use the values as specified in
+\ref{sec:datyansi}.
+\end{Lentry}
+
+\vskip 0.25cm
+
+{\bf Reduction of local 1d arrays} to a local arrays, this reduction is carried
+out element by element. The arrays need to have the same size on all
+processors.
+{\t
+\begin{verbatim}
+int CCTK_ReduceLocArrayToArray1D( cGH *GH, 
+                                  int processor,
+                                  int operation_handle,
+                                  void *in_array1d, 
+                                  void *out_array1d, 
+                                  int xsize)
+                                  int data_type)
+
+call CCTK_ReduceLocArrayToArray1D( int returnvalue
+                                  cctkGH,
+                                  int processor,
+                                  int operation_handle,
+                                  in_array1d,
+                                  out_array1d,
+                                  int xsize,
+                                  int data_type)
+\end{verbatim}
+}
+
+\begin{Lentry}
+\item[{\tt in\_array1d}] one dimensional local arrays, to be reduced across a 
+processors, element by element. 
+\item[{\tt out\_array1d}] array holding the reduction result. out\_array1d[1]
+= Reduction(in\_array[1]).
+\item[{\tt xsize}] the size of the one dimensional array.
+\end{Lentry}
+
+\vskip 0.25cm
+
+{\bf Reduction of local 2d arrays} to a local 2d array, this reduction is carried
+out element by element. The arrays need to have the same size on all
+processors.
+{\t
+\begin{verbatim}
+int CCTK_ReduceLocArrayToArrayD( cGH *GH,
+                                 int processor,
+                                 int opertaion_handle,
+                                 in_array_2d,
+                                 out_array2d,
+                                 int xsize,
+                                 int ysize,
+                                 int data_type)
+                                 
+
+call CCTK_ReduceLocArrayToArray2D( int returnvalue
+                                   cctkGH,
+                                   int processor,
+                                   int operation_handle,
+                                   in_array2d,
+                                   out_array2d,
+                                   int xsize,
+                                   int ysize,
+                                   int data_type)
+                                 
+\end{verbatim}
+}
+
+\begin{Lentry}
+\item[{\tt in\_array1d}] two dimensional local arrays, to be reduced across a 
+processors, element by element. 
+\item[{\tt out\_array1d}] two dimensional array holding the reduction
+result. out\_array2d[i,j]= Reduction(in\_array2d[i,j]).
+\item[{\tt xsize}] the size of the one dimensional array in x direction.
+\item[{\tt ysize}] the size of the one dimensional array in y direction.
+\end{Lentry}
+
+\vskip 0.25cm
+
+{\bf Reduction of local 3d arrays} to a local 3d array, this reduction is carried
+out element by element. The arrays need to have the same size on all
+processors.
+{\t
+\begin{verbatim}
+int CCTK_ReduceLocArrayToArray3D(cGH *GH,
+                                 int processor,
+                                 int opertaion_handle,
+                                 in_array_3d,
+                                 out_array3d,
+                                 int xsize,
+                                 int ysize,
+                                 int zsize,
+                                 int data_type)
+
+call CCTK_ReduceLocArrayToArray3D(int returnvalue
+                                 cctkGH,
+                                 int processor,
+                                 int operation_handle,
+                                 in_array3d,
+                                 out_array3d,
+                                 int xsize,
+                                 int ysize,
+                                 int zsize,
+                                 int data_type)
+\end{verbatim}
+}
+
+\begin{Lentry}
+\item[{\tt in\_array3d}] two dimensional local arrays, to be reduced across a 
+processors, element by element. 
+\item[{\tt out\_array3d}] two dimensional array holding the reduction
+result. out\_array3d[i,j,k]= Reduction(in\_array3d[i,j,k]).
+\item[{\tt xsize}] the size of the one dimensional array in x direction.
+\item[{\tt ysize}] the size of the one dimensional array in y direction.
+\item[{\tt ysize}] the size of the one dimensional array in z direction.
+\end{Lentry}
+
+\vskip .25cm
+
+{\bf Some brief examples:}
+
+{\bf Reduction of a local scalars:} a local error is reduced across all
+processors with the maximum operation. The variable {\tt tmp} will
+hold the maximum of the error and  is the same on all
+processors. This quantity can then be reassigned to {\tt normerr}.
+\begin{verbatim}
+         CCTK_REAL normerr, tmp
+         integer   ierr, reduction_handle
+
+         call CCTK_ReductionArrayHandle(reduction_handle,"maximum")
+
+         if (reduction_handle.lt.0) then
+            call CCTK_WARN(1,"Cannot get reduction handle for maximum operation.")
+         endif
+
+         call CCTK_ReduceLocScalar(ierr, cctkGH, -1, 
+     .             reduction_handle,
+     .             normerr, tmp, CCTK_VARIABLE_REAL)
+         if (ierr.ne.0) then
+            call CCTK_WARN(1,"Reduction of norm failed!");
+         endif
+         normerr = tmp
+\end{verbatim}
+
+
+{\bf Reduction of a local 2d array:} a two dimensional array $(2x3)$ is 
+reduced, reduction results (array of same size: {\tt bla\_tmp}) is seen
+on all processors ($-1$ entry as the third argument); also demonstrates
+some simple error checking with the {\tt CCTKi\_EXPECTOK} macro.
+\begin{verbatim}
+      CCTK_REAL bla(2,3),bla_tmp(2,3);
+      integer   ierr, sum_handle
+
+      call CCTK_ReductionArrayHandle(sum_handle,"sum")
+      bla         =  1.0d0
+      write (*,*) "BLA ",bla
+
+      call CCTK_ReduceLocArrayToArray2D(ierr, cctkGH, -1, sum_handle,
+     .     bla, bla_tmp, 2, 3, CCTK_VARIABLE_REAL)
+      call CCTKi_EXPECTOK(ierr, 0, 1, "2D Reduction failed")
+
+      bla = bla_tmp
+      write (*,*) "BLA ",bla
+\end{verbatim}
+
+Note that the memory for the returned values must be allocated before
+the reduction call is made.
+
+
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
@@ -3019,7 +3401,7 @@ ways (provided the underlying functionality is available on your system):
 \item[{\tt GetTimeOfDay}]
         Provides ``wall clock time'' via the unix \texttt{gettimeofday} function.
 \item[{\tt GetrUsage}]
-	Provides CPU usage time via the unix \texttt{getrusage} function.
+        Provides CPU usage time via the unix \texttt{getrusage} function.
 \end{Lentry}
 
 Additional clocks can be implemented by thorns and registered with the