Initial revision

darcs-hash:20010301114010-f6438-12fb8a9ffcc80e86c0a97e37b5b0dae0dbc59b79.gz

49 files changed, 8107 insertions, 0 deletions

diff --git a/Carpet/CarpetIOHDF5/COPYING b/Carpet/CarpetIOHDF5/COPYING
new file mode 100644
index 000000000..1942c4334
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/COPYING
@@ -0,0 +1,341 @@
+		    GNU GENERAL PUBLIC LICENSE
+		       Version 2, June 1991
+
+ Copyright (C) 1989, 1991 Free Software Foundation, Inc.
+                          59 Temple Place - Suite 330
+                          Boston, MA 02111-1307, USA.
+ Everyone is permitted to copy and distribute verbatim copies
+ of this license document, but changing it is not allowed.
+
+			    Preamble
+
+  The licenses for most software are designed to take away your
+freedom to share and change it.  By contrast, the GNU General Public
+License is intended to guarantee your freedom to share and change free
+software--to make sure the software is free for all its users.  This
+General Public License applies to most of the Free Software
+Foundation's software and to any other program whose authors commit to
+using it.  (Some other Free Software Foundation software is covered by
+the GNU Library General Public License instead.)  You can apply it to
+your programs, too.
+
+  When we speak of free software, we are referring to freedom, not
+price.  Our General Public Licenses are designed to make sure that you
+have the freedom to distribute copies of free software (and charge for
+this service if you wish), that you receive source code or can get it
+if you want it, that you can change the software or use pieces of it
+in new free programs; and that you know you can do these things.
+
+  To protect your rights, we need to make restrictions that forbid
+anyone to deny you these rights or to ask you to surrender the rights.
+These restrictions translate to certain responsibilities for you if you
+distribute copies of the software, or if you modify it.
+
+  For example, if you distribute copies of such a program, whether
+gratis or for a fee, you must give the recipients all the rights that
+you have.  You must make sure that they, too, receive or can get the
+source code.  And you must show them these terms so they know their
+rights.
+
+  We protect your rights with two steps: (1) copyright the software, and
+(2) offer you this license which gives you legal permission to copy,
+distribute and/or modify the software.
+
+  Also, for each author's protection and ours, we want to make certain
+that everyone understands that there is no warranty for this free
+software.  If the software is modified by someone else and passed on, we
+want its recipients to know that what they have is not the original, so
+that any problems introduced by others will not reflect on the original
+authors' reputations.
+
+  Finally, any free program is threatened constantly by software
+patents.  We wish to avoid the danger that redistributors of a free
+program will individually obtain patent licenses, in effect making the
+program proprietary.  To prevent this, we have made it clear that any
+patent must be licensed for everyone's free use or not licensed at all.
+
+  The precise terms and conditions for copying, distribution and
+modification follow.
+
+		    GNU GENERAL PUBLIC LICENSE
+   TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
+
+  0. This License applies to any program or other work which contains
+a notice placed by the copyright holder saying it may be distributed
+under the terms of this General Public License.  The "Program", below,
+refers to any such program or work, and a "work based on the Program"
+means either the Program or any derivative work under copyright law:
+that is to say, a work containing the Program or a portion of it,
+either verbatim or with modifications and/or translated into another
+language.  (Hereinafter, translation is included without limitation in
+the term "modification".)  Each licensee is addressed as "you".
+
+Activities other than copying, distribution and modification are not
+covered by this License; they are outside its scope.  The act of
+running the Program is not restricted, and the output from the Program
+is covered only if its contents constitute a work based on the
+Program (independent of having been made by running the Program).
+Whether that is true depends on what the Program does.
+
+  1. You may copy and distribute verbatim copies of the Program's
+source code as you receive it, in any medium, provided that you
+conspicuously and appropriately publish on each copy an appropriate
+copyright notice and disclaimer of warranty; keep intact all the
+notices that refer to this License and to the absence of any warranty;
+and give any other recipients of the Program a copy of this License
+along with the Program.
+
+You may charge a fee for the physical act of transferring a copy, and
+you may at your option offer warranty protection in exchange for a fee.
+
+  2. You may modify your copy or copies of the Program or any portion
+of it, thus forming a work based on the Program, and copy and
+distribute such modifications or work under the terms of Section 1
+above, provided that you also meet all of these conditions:
+
+    a) You must cause the modified files to carry prominent notices
+    stating that you changed the files and the date of any change.
+
+    b) You must cause any work that you distribute or publish, that in
+    whole or in part contains or is derived from the Program or any
+    part thereof, to be licensed as a whole at no charge to all third
+    parties under the terms of this License.
+
+    c) If the modified program normally reads commands interactively
+    when run, you must cause it, when started running for such
+    interactive use in the most ordinary way, to print or display an
+    announcement including an appropriate copyright notice and a
+    notice that there is no warranty (or else, saying that you provide
+    a warranty) and that users may redistribute the program under
+    these conditions, and telling the user how to view a copy of this
+    License.  (Exception: if the Program itself is interactive but
+    does not normally print such an announcement, your work based on
+    the Program is not required to print an announcement.)
+
+These requirements apply to the modified work as a whole.  If
+identifiable sections of that work are not derived from the Program,
+and can be reasonably considered independent and separate works in
+themselves, then this License, and its terms, do not apply to those
+sections when you distribute them as separate works.  But when you
+distribute the same sections as part of a whole which is a work based
+on the Program, the distribution of the whole must be on the terms of
+this License, whose permissions for other licensees extend to the
+entire whole, and thus to each and every part regardless of who wrote it.
+
+Thus, it is not the intent of this section to claim rights or contest
+your rights to work written entirely by you; rather, the intent is to
+exercise the right to control the distribution of derivative or
+collective works based on the Program.
+
+In addition, mere aggregation of another work not based on the Program
+with the Program (or with a work based on the Program) on a volume of
+a storage or distribution medium does not bring the other work under
+the scope of this License.
+
+  3. You may copy and distribute the Program (or a work based on it,
+under Section 2) in object code or executable form under the terms of
+Sections 1 and 2 above provided that you also do one of the following:
+
+    a) Accompany it with the complete corresponding machine-readable
+    source code, which must be distributed under the terms of Sections
+    1 and 2 above on a medium customarily used for software interchange; or,
+
+    b) Accompany it with a written offer, valid for at least three
+    years, to give any third party, for a charge no more than your
+    cost of physically performing source distribution, a complete
+    machine-readable copy of the corresponding source code, to be
+    distributed under the terms of Sections 1 and 2 above on a medium
+    customarily used for software interchange; or,
+
+    c) Accompany it with the information you received as to the offer
+    to distribute corresponding source code.  (This alternative is
+    allowed only for noncommercial distribution and only if you
+    received the program in object code or executable form with such
+    an offer, in accord with Subsection b above.)
+
+The source code for a work means the preferred form of the work for
+making modifications to it.  For an executable work, complete source
+code means all the source code for all modules it contains, plus any
+associated interface definition files, plus the scripts used to
+control compilation and installation of the executable.  However, as a
+special exception, the source code distributed need not include
+anything that is normally distributed (in either source or binary
+form) with the major components (compiler, kernel, and so on) of the
+operating system on which the executable runs, unless that component
+itself accompanies the executable.
+
+If distribution of executable or object code is made by offering
+access to copy from a designated place, then offering equivalent
+access to copy the source code from the same place counts as
+distribution of the source code, even though third parties are not
+compelled to copy the source along with the object code.
+
+  4. You may not copy, modify, sublicense, or distribute the Program
+except as expressly provided under this License.  Any attempt
+otherwise to copy, modify, sublicense or distribute the Program is
+void, and will automatically terminate your rights under this License.
+However, parties who have received copies, or rights, from you under
+this License will not have their licenses terminated so long as such
+parties remain in full compliance.
+
+  5. You are not required to accept this License, since you have not
+signed it.  However, nothing else grants you permission to modify or
+distribute the Program or its derivative works.  These actions are
+prohibited by law if you do not accept this License.  Therefore, by
+modifying or distributing the Program (or any work based on the
+Program), you indicate your acceptance of this License to do so, and
+all its terms and conditions for copying, distributing or modifying
+the Program or works based on it.
+
+  6. Each time you redistribute the Program (or any work based on the
+Program), the recipient automatically receives a license from the
+original licensor to copy, distribute or modify the Program subject to
+these terms and conditions.  You may not impose any further
+restrictions on the recipients' exercise of the rights granted herein.
+You are not responsible for enforcing compliance by third parties to
+this License.
+
+  7. If, as a consequence of a court judgment or allegation of patent
+infringement or for any other reason (not limited to patent issues),
+conditions are imposed on you (whether by court order, agreement or
+otherwise) that contradict the conditions of this License, they do not
+excuse you from the conditions of this License.  If you cannot
+distribute so as to satisfy simultaneously your obligations under this
+License and any other pertinent obligations, then as a consequence you
+may not distribute the Program at all.  For example, if a patent
+license would not permit royalty-free redistribution of the Program by
+all those who receive copies directly or indirectly through you, then
+the only way you could satisfy both it and this License would be to
+refrain entirely from distribution of the Program.
+
+If any portion of this section is held invalid or unenforceable under
+any particular circumstance, the balance of the section is intended to
+apply and the section as a whole is intended to apply in other
+circumstances.
+
+It is not the purpose of this section to induce you to infringe any
+patents or other property right claims or to contest validity of any
+such claims; this section has the sole purpose of protecting the
+integrity of the free software distribution system, which is
+implemented by public license practices.  Many people have made
+generous contributions to the wide range of software distributed
+through that system in reliance on consistent application of that
+system; it is up to the author/donor to decide if he or she is willing
+to distribute software through any other system and a licensee cannot
+impose that choice.
+
+This section is intended to make thoroughly clear what is believed to
+be a consequence of the rest of this License.
+
+  8. If the distribution and/or use of the Program is restricted in
+certain countries either by patents or by copyrighted interfaces, the
+original copyright holder who places the Program under this License
+may add an explicit geographical distribution limitation excluding
+those countries, so that distribution is permitted only in or among
+countries not thus excluded.  In such case, this License incorporates
+the limitation as if written in the body of this License.
+
+  9. The Free Software Foundation may publish revised and/or new versions
+of the General Public License from time to time.  Such new versions will
+be similar in spirit to the present version, but may differ in detail to
+address new problems or concerns.
+
+Each version is given a distinguishing version number.  If the Program
+specifies a version number of this License which applies to it and "any
+later version", you have the option of following the terms and conditions
+either of that version or of any later version published by the Free
+Software Foundation.  If the Program does not specify a version number of
+this License, you may choose any version ever published by the Free Software
+Foundation.
+
+  10. If you wish to incorporate parts of the Program into other free
+programs whose distribution conditions are different, write to the author
+to ask for permission.  For software which is copyrighted by the Free
+Software Foundation, write to the Free Software Foundation; we sometimes
+make exceptions for this.  Our decision will be guided by the two goals
+of preserving the free status of all derivatives of our free software and
+of promoting the sharing and reuse of software generally.
+
+			    NO WARRANTY
+
+  11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
+FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW.  EXCEPT WHEN
+OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
+PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
+OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.  THE ENTIRE RISK AS
+TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU.  SHOULD THE
+PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,
+REPAIR OR CORRECTION.
+
+  12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
+WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
+REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
+INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
+OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED
+TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY
+YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER
+PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
+POSSIBILITY OF SUCH DAMAGES.
+
+		     END OF TERMS AND CONDITIONS
+
+	    How to Apply These Terms to Your New Programs
+
+  If you develop a new program, and you want it to be of the greatest
+possible use to the public, the best way to achieve this is to make it
+free software which everyone can redistribute and change under these terms.
+
+  To do so, attach the following notices to the program.  It is safest
+to attach them to the start of each source file to most effectively
+convey the exclusion of warranty; and each file should have at least
+the "copyright" line and a pointer to where the full notice is found.
+
+    <one line to give the program's name and a brief idea of what it does.>
+    Copyright (C) 19yy  <name of author>
+
+    This program is free software; you can redistribute it and/or modify
+    it under the terms of the GNU General Public License as published by
+    the Free Software Foundation; either version 2 of the License, or
+    (at your option) any later version.
+
+    This program is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU General Public License for more details.
+
+    You should have received a copy of the GNU General Public License
+    along with this program; see the file COPYING.  If not, write to
+    the Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+    Boston, MA 02111-1307, USA.
+
+Also add information on how to contact you by electronic and paper mail.
+
+If the program is interactive, make it output a short notice like this
+when it starts in an interactive mode:
+
+    Gnomovision version 69, Copyright (C) 19yy name of author
+    Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
+    This is free software, and you are welcome to redistribute it
+    under certain conditions; type `show c' for details.
+
+The hypothetical commands `show w' and `show c' should show the appropriate
+parts of the General Public License.  Of course, the commands you use may
+be called something other than `show w' and `show c'; they could even be
+mouse-clicks or menu items--whatever suits your program.
+
+You should also get your employer (if you work as a programmer) or your
+school, if any, to sign a "copyright disclaimer" for the program, if
+necessary.  Here is a sample; alter the names:
+
+  Yoyodyne, Inc., hereby disclaims all copyright interest in the program
+  `Gnomovision' (which makes passes at compilers) written by James Hacker.
+
+  <signature of Ty Coon>, 1 April 1989
+  Ty Coon, President of Vice
+
+This General Public License does not permit incorporating your program into
+proprietary programs.  If your program is a subroutine library, you may
+consider it more useful to permit linking proprietary applications with the
+library.  If this is what you want to do, use the GNU Library General
+Public License instead of this License.
diff --git a/Carpet/CarpetIOHDF5/README b/Carpet/CarpetIOHDF5/README
new file mode 100644
index 000000000..64520a2da
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/README
@@ -0,0 +1,8 @@
+Cactus Code Thorn CarpetIOHDF5
+Authors    : Erik Schnetter <schnetter@aei.mpg.de>
+CVS info   : $Header: /home/eschnett/C/carpet/Carpet/Carpet/CarpetIOHDF5/README,v 1.1 2004/03/03 09:44:59 schnetter Exp $
+--------------------------------------------------------------------------
+
+Purpose of the thorn:
+
+This thorn provides HDF5 based file I/O for Carpet.
diff --git a/Carpet/CarpetIOHDF5/configuration.ccl b/Carpet/CarpetIOHDF5/configuration.ccl
new file mode 100644
index 000000000..0323a73bd
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/configuration.ccl
@@ -0,0 +1,6 @@
+# Configuration definitions for thorn CarpetIOHDF5
+# $Header: /home/eschnett/C/carpet/Carpet/Carpet/CarpetIOHDF5/configuration.ccl,v 1.3 2004/06/08 21:09:25 schnetter Exp $
+
+REQUIRES Carpet CarpetLib IOUtil
+
+REQUIRES THORNS: Carpet CarpetLib IOUtil
diff --git a/Carpet/CarpetIOHDF5/doc/documentation.tex b/Carpet/CarpetIOHDF5/doc/documentation.tex
new file mode 100644
index 000000000..b07dd73d1
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/doc/documentation.tex
@@ -0,0 +1,314 @@
+% *======================================================================*
+%  Cactus Thorn template for ThornGuide documentation
+%  Author: Ian Kelley
+%  Date: Sun Jun 02, 2002
+%  $Header: /home/eschnett/C/carpet/Carpet/Carpet/CarpetIOHDF5/doc/documentation.tex,v 1.4 2004/06/22 11:56:20 tradke Exp $                                                             
+%
+%  Thorn documentation in the latex file doc/documentation.tex 
+%  will be included in ThornGuides built with the Cactus make system.
+%  The scripts employed by the make system automatically include 
+%  pages about variables, parameters and scheduling parsed from the 
+%  relevent thorn CCL files.
+%  
+%  This template contains guidelines which help to assure that your     
+%  documentation will be correctly added to ThornGuides. More 
+%  information is available in the Cactus UsersGuide.
+%                                                    
+%  Guidelines:
+%   - Do not change anything before the line
+%       % START CACTUS THORNGUIDE",
+%     except for filling in the title, author, date etc. fields.
+%        - Each of these fields should only be on ONE line.
+%        - Author names should be sparated with a \\ or a comma
+%   - You can define your own macros are OK, but they must appear after
+%     the START CACTUS THORNGUIDE line, and do not redefine standard 
+%     latex commands.
+%   - To avoid name clashes with other thorns, 'labels', 'citations', 
+%     'references', and 'image' names should conform to the following 
+%     convention:          
+%       ARRANGEMENT_THORN_LABEL
+%     For example, an image wave.eps in the arrangement CactusWave and 
+%     thorn WaveToyC should be renamed to CactusWave_WaveToyC_wave.eps
+%   - Graphics should only be included using the graphix package. 
+%     More specifically, with the "includegraphics" command. Do
+%     not specify any graphic file extensions in your .tex file. This 
+%     will allow us (later) to create a PDF version of the ThornGuide
+%     via pdflatex. |
+%   - References should be included with the latex "bibitem" command. 
+%   - use \begin{abstract}...\end{abstract} instead of \abstract{...}
+%   - For the benefit of our Perl scripts, and for future extensions, 
+%     please use simple latex.     
+%
+% *======================================================================* 
+% 
+% Example of including a graphic image:
+%    \begin{figure}[ht]
+% 	\begin{center}
+%    	   \includegraphics[width=6cm]{MyArrangement_MyThorn_MyFigure}
+% 	\end{center}
+% 	\caption{Illustration of this and that}
+% 	\label{MyArrangement_MyThorn_MyLabel}
+%    \end{figure}
+%
+% Example of using a label:
+%   \label{MyArrangement_MyThorn_MyLabel}
+%
+% Example of a citation:
+%    \cite{MyArrangement_MyThorn_Author99}
+%
+% Example of including a reference
+%   \bibitem{MyArrangement_MyThorn_Author99}
+%   {J. Author, {\em The Title of the Book, Journal, or periodical}, 1 (1999), 
+%   1--16. {\tt http://www.nowhere.com/}}
+%
+% *======================================================================* 
+
+% If you are using CVS use this line to give version information
+% $Header: /home/eschnett/C/carpet/Carpet/Carpet/CarpetIOHDF5/doc/documentation.tex,v 1.4 2004/06/22 11:56:20 tradke Exp $
+
+\documentclass{article}
+
+% Use the Cactus ThornGuide style file
+% (Automatically used from Cactus distribution, if you have a 
+%  thorn without the Cactus Flesh download this from the Cactus
+%  homepage at www.cactuscode.org)
+\usepackage{../../../../doc/ThornGuide/cactus}
+
+\begin{document}
+
+% The author of the documentation
+\author{Erik Schnetter $<$schnetter@uni-tuebingen.de$>$, Christian D. Ott $<$cott@aei.mpg.de$>$}
+
+% The title of the document (not necessarily the name of the Thorn)
+\title{CarpetIOHDF5}
+
+% the date your document was last changed, if your document is in CVS, 
+% please use:
+%    \date{$ $Date: 2004/06/22 11:56:20 $ $}
+\date{March 18, 2004}
+
+\maketitle
+
+% Do not delete next line
+% START CACTUS THORNGUIDE
+
+% Add all definitions used in this documentation here 
+%   \def\mydef etc
+
+% Add an abstract for this thorn's documentation
+\begin{abstract}
+CarpetIOHDF5 provides HDF5 based 3-D output to the Cactus mesh refinement driver Carpet.
+This document explains CarpetIOHDF5's usage and contains a specification of the
+CarpetIOHDF5 file format that was adapted from John Shalf's FlexIO library.
+\end{abstract}
+
+% The following sections are suggestive only.
+% Remove them or add your own.
+
+\section{Introduction}
+
+Having encountered various problems with CarpetIOFlexIO and the underlying FlexIO library,
+Erik Schnetter decided to create CarpetIOHDF5. CarpetIOHDF5 provides 3-D output for the
+Carpet Mesh Refinement driver within the Cactus Code. Christian D. Ott added  
+file reader (analogous to Erik Schnetter's implementation present in CarpetIOFlexIO) 
+and checkpoint/recovery features to CarpetIOHDF5.
+
+Right now, CarpetIOHDF5 only uses serial I/O - all data are copied to processor 0 for I/O.
+
+This document aims at giving the user a first handle on how to use
+CarpetIOHDF5. It also documents the HDF5 file layout used.
+
+
+\section{Using This Thorn}
+
+
+\subsection{Obtaining This Thorn}
+
+You can get a checkout from 
+
+{\tt cvs.carpetcode.org:/home/cvs Carpet/CarpetIOHDF5}
+
+\subsection{Basic Usage}
+
+First, you have to activate the thorn in your Cactus parameter file:
+
+{\tt ActiveThorns = "CarpetIOHDF5"}
+
+\subsubsection{3-D Output}
+
+\begin{itemize}
+  \item {\tt iohdf5::out3D\_vars = "your list of Cactus grid functions to output"}
+  \item {\tt iohdf5::out3D\_every = n} : Output every {\tt n} time steps
+  \item {\tt iohdf5::out3D\_dir = "your preferred 3-D output directory"}
+  \item {\tt IO::out\_single\_precision = "yes/no (output double-precision data in single precision)"}
+\end{itemize}
+
+\subsubsection{3-D Input}
+
+There are two ways to use the 3-D input capabilities:
+
+\begin{enumerate}
+  \item For evolutions using ADMBase, one may use the thorn IDFileADM and the following parameter settings:
+    \begin{itemize}
+      \item {\tt ADMBase::initial\_data  = "read from file"}
+      \item {\tt IO::filereader\_ID\_files = "space separated list of files containing the ADM variables"}
+      \item {\tt IO::filereader\_ID\_vars = "space separated list of variables to be read in"}
+    \end{itemize}
+  \item For evolutions not using ADMBase one may try to read in data by setting
+    \begin{itemize}
+      \item {\tt iohdf5::in3D\_dir = "directory from where to read data"}
+      \item {\tt iohdf5::in3D\_vars = "space separated list of variables to be read in"}
+    \end{itemize}
+\end{enumerate}
+
+
+\subsubsection{Checkpointing}
+
+CarpetIOHDF5 uses the Cactus checkpoint/recovery infrastructure provided by
+CactusBase/IOUtil.
+
+\begin{itemize}
+  \item {\tt iohdf5::checkpoint = "yes"} : Turns on checkpointing
+  \item {\tt iohdf5::checkpoint\_every = n} : Checkpointing every {\tt n} time steps
+  \item {\tt iohdf5::checkpoint\_ID = "yes"} : Turns on checkpointing after initial data
+  \item {\tt io::checkpoint\_dir = "your preferred checkpoint directory"} 
+  \item  {\tt iohdf5::checkpoint\_keep = n} : Keep {\tt n} checkpoint files around
+\end{itemize}
+
+
+\subsubsection{Recover}
+
+CarpetIOHDF5 uses the Cactus checkpoint/recovery infrastructure provided by
+CactusBase/IOUtil. Currently all the checkpoint information is copied onto processor 0 and
+ written into a single file whose name is invented by IOUtil. Unfortunately, single cpu
+checkpoint files have a different name (a missing \_file\_0 tag) than checkpoint
+files from multi-cpu runs. Somebody should tweek IOBase... 
+
+In principle, CarpetIOHDF5 is able to restart on any number
+of cpus  from a checkpoint file of a run using any (other or same) number of cpus.
+
+\begin{itemize}
+  \item {\tt iohdf5::recover = "auto"} : Recover from the most recent Checkpoint file. This bombs,
+    if no checkpoint file is found.
+  \item {\tt iohdf5::recover = "autoprobe"} : Recover from the most recent Checkpoint file. This continues
+    without recovering if no checkpoint file is found.
+  \item {\tt iohdf5::recover\_dir = "directory containing the checkpoint file"} 
+  \item {\tt iohdf5::recover = "manual"} : Recover from a file specified by {\tt iohdf5::recover\_file}. This
+     bombs if the file is not found.
+  \item {\tt iohdf5::recover\_file = "file you want to recover from"} : Only needs to be set if
+    {\tt iohdf5::recover = "manual"}.
+
+\end{itemize}
+
+
+
+\subsection{Special Behaviour}
+
+\begin{itemize}
+  \item {\tt iohdf5::h5verbose = "yes"} : Makes CarpetIOHDF5 very talkative.
+\end{itemize}
+
+
+\section{CarpetIOHDF5's HDF5 file layout}
+
+The HDF5 file layout of CarpetIOHDF5 is quite simple. There are no groups besides the 
+standard HDF5 root data object group:
+
+\begin{itemize}
+  \item Each dataset is named according to this template: \\
+    {\tt \small [Full Variable Name] it=[Iteration] tl=[Timelevel] ml=[mglevel] rl=[reflevel] m=[map] c=[component]}
+    
+  \item Each dataset has the following (largely redundant!) attributes associated with it:
+
+    \begin{itemize}
+    \item {\tt level} : Carpet::reflevel
+    \item {\tt origin} : 1-D array of length Carpet::dim. \\ origin[d]=CCTK\_ORIGIN\_SPACE(d) + 
+      cctk\_lbnd[d] * delta[d]
+    \item {\tt delta} : 1-D array of length Carpet::dim. \\ delta[d] = CCTK\_DELTA\_SPACE(d)
+    \item {\tt min\_ext} : 1-D array of length Carpet::dim. \\ min\_ext[d] = delta[d] 
+    \item {\tt max\_ext} : 1-D array of length Carpet::dim. \\ origin[d] + cctk\_lsh[d] * delta[d]
+    \item {\tt time} : cctk\_time
+    \item {\tt timestep} : cctk\_iteration
+    \item {\tt level\_timestep} : cctk\_iteration / Carpet::reflevelfact
+    \item {\tt persistence} : cctk\_iteration / Carpet::reflevelfact
+    \item {\tt time\_refinement} : Carpet::time\_refinement
+    \item {\tt spatial\_refinement} : 1-D array of length Carpet::dim. \\ spatial\_refinement[d] = Carpet::reflevelfact
+    \item {\tt grid\_placement\_refinement} : 1-D array of length Carpet::dim. \\ grid\_placement\_refinement[d] = Carpet::reflevelfact
+    \item {\tt iorigin} : 1-D array of length Carpet::dim. \\ iorigin[d] = (Carpet::ext.lower() / Carpet::ext.stride())[d]
+    \item {\tt name} : CCTK\_FullName(variable index)
+    \item {\tt group\_version} : 1
+    \item {\tt group\_fullname} : CCTK\_FullName(variable index)
+    \item {\tt group\_varname} : CCTK\_VarName(variable index)
+    \item {\tt group\_groupname} : CCTK\_GroupName(group index)
+    \item {\tt group\_grouptype} : CCTK\_GF, CCTK\_ARRAY or CCTK\_SCALAR
+    \item {\tt group\_dim} : CCTK\_GroupDimI(group index)
+    \item {\tt group\_timelevel} : tl (current timelevel)
+    \item {\tt group\_numtimelevels} : CCTK\_NumTimeLevelsI(group index)
+    \item {\tt cctk\_version} : 1
+    \item {\tt cctk\_dim} : cctk\_dim
+    \item {\tt cctk\_iteration} : cctk\_iteration
+    \item {\tt cctk\_gsh} : 1-D array of length Carpet::dim. cctk\_gsh
+    \item {\tt cctk\_lsh} : 1-D array of length Carpet::dim. cctk\_lsh
+    \item {\tt cctk\_lbnd} : 1-D array of length Carpet::dim. cctk\_lbnd
+    \item {\tt cctk\_delta\_time} : 1-D array of length Carpet::dim. cctk\_delta\_time
+    \item {\tt cctk\_delta\_space} : 1-D array of length Carpet::dim. cctk\_delta\_space
+    \item {\tt cctk\_origin\_space} : 1-D array of length Carpet::dim. cctk\_origin\_space
+    \item {\tt cctk\_bbox} : 1-D array of length 2*Carpet::dim. cctk\_box
+    \item {\tt cctk\_levfac} : 1-D array of length Carpet::dim. cctk\_levfac
+    \item {\tt cctk\_levoff} : 1-D array of length Carpet::dim. cctk\_levoff
+    \item {\tt cctk\_levoffdenom} : 1-D array of length Carpet::dim. cctk\_levoffdenom
+    \item {\tt cctk\_timefac} : cctk\_timefac
+    \item {\tt cctk\_convlevel} : cctk\_convlevel
+    \item {\tt cctk\_convfac} : cctk\_convfac
+    \item {\tt cctk\_nghostzones} : 1-D array of length Carpet::dim. cctk\_nghostzones
+    \item {\tt cctk\_time} : cctk\_time
+    \item {\tt carpet\_version} : 1
+    \item {\tt carpet\_dim} : Carpet::dim
+    \item {\tt carpet\_basemglevel} : Carpet::basemglevel
+    \item {\tt carpet\_mglevel} : Carpet::mglevel
+    \item {\tt carpet\_mglevels} : Carpet::mglevels
+    \item {\tt carpet\_mgface} : Carpet::mgfact
+    \item {\tt carpet\_reflevel} : Carpet::reflevel
+    \item {\tt carpet\_reflevels} : Carpet::reflevels
+    \item {\tt carpet\_reffact} : Carpet::reffact
+    \item {\tt carpet\_map} : Carpet::map
+    \item {\tt carpet\_maps} : Carpet::maps
+    \item {\tt carpet\_component} : Carpet::component
+    \item {\tt carpet\_components} : Carpet::vhh.at(Carpet::map)->components(reflevel))
+
+    \end{itemize}
+\end{itemize}
+
+\subsection{Attributes needed by the file reader}
+
+The number of attributes needed by the CarpetIOHDF5 file reader is much smaller then the total
+number of attributes attached to each dataset:
+
+\begin{itemize}
+  \item {\tt name}
+  \item {\tt level}
+  \item {\tt iorigin}
+\end{itemize}
+
+
+%\subsection{Interaction With Other Thorns}
+%
+%\subsection{Support and Feedback}
+%
+%\section{History}
+%
+%\subsection{Thorn Source Code}
+%
+%\subsection{Thorn Documentation}
+%
+%\subsection{Acknowledgements}
+%
+%
+%\begin{thebibliography}{9}
+%
+%\end{thebibliography}
+%
+% Do not delete next line
+% END CACTUS THORNGUIDE
+
+\end{document}
diff --git a/Carpet/CarpetIOHDF5/interface.ccl b/Carpet/CarpetIOHDF5/interface.ccl
new file mode 100644
index 000000000..b8d6a5da8
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/interface.ccl
@@ -0,0 +1,20 @@
+# Interface definition for thorn CarpetIOHDF5
+# $Header: /home/eschnett/C/carpet/Carpet/Carpet/CarpetIOHDF5/interface.ccl,v 1.5 2004/06/08 21:09:25 schnetter Exp $
+
+IMPLEMENTS: IOHDF5
+
+uses include header: carpet.hh
+
+uses include header: bbox.hh
+uses include header: vect.hh
+
+uses include header: data.hh
+uses include header: gdata.hh
+
+uses include header: ggf.hh
+
+
+
+CCTK_INT next_output_iteration TYPE=scalar
+CCTK_REAL next_output_time TYPE=scalar
+CCTK_INT this_iteration TYPE=scalar
diff --git a/Carpet/CarpetIOHDF5/par/CarpetWaveToyCheckpointTest.par b/Carpet/CarpetIOHDF5/par/CarpetWaveToyCheckpointTest.par
new file mode 100644
index 000000000..57cf0e59b
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/par/CarpetWaveToyCheckpointTest.par
@@ -0,0 +1,43 @@
+ActiveThorns = "Boundary CoordBase CartGrid3D SymBase Time Carpet CarpetReduce CarpetSlab CarpetLib CarpetRegrid IOUtil IOBasic CarpetIOASCII CarpetIOHDF5 IDScalarWaveC WaveToyC"
+
+
+Cactus::cctk_itlast = 12
+
+Time::dtfac = 0.5
+
+IDScalarWave::initial_data = "gaussian"
+IDScalarWave::sigma        = 2.8
+IDScalarWave::radius       = 0.0
+
+Grid::avoid_origin = "no"
+Grid::type         = "byrange"
+Grid::dxyz         = 1
+
+Driver::global_nsize = 20
+Driver::ghost_size   = 2
+
+Carpet::prolongation_order_space = 3
+Carpet::prolongation_order_time  = 2
+Carpet::init_each_timelevel      = "yes"
+Carpet::buffer_width             = 6
+Carpet::max_refinement_levels    = 2
+
+CarpetRegrid::refinement_levels  = 2
+
+# Output
+IO::out_dir = $parfile
+
+IOBasic::outInfo_every = 1
+IOBasic::outInfo_vars  = "wavetoy::phi"
+
+IOASCII::out1D_vars  = "wavetoy::phi"
+IOASCII::out1D_every = 12
+IOASCII::out3D_vars  = "wavetoy::phi"
+IOASCII::out3D_every = 12
+
+# Checkpoint
+IO::checkpoint_every = 12
+IO::checkpoint_keep  = 2
+
+IOHDF5::checkpoint = "yes"
+IOHDF5::verbose    = "yes"
diff --git a/Carpet/CarpetIOHDF5/par/CarpetWaveToyRecoverTest.par b/Carpet/CarpetIOHDF5/par/CarpetWaveToyRecoverTest.par
new file mode 100644
index 000000000..5972c76d8
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/par/CarpetWaveToyRecoverTest.par
@@ -0,0 +1,12 @@
+ActiveThorns = "Boundary CoordBase CartGrid3D SymBase Time Carpet CarpetReduce CarpetSlab CarpetLib CarpetRegrid IOUtil IOBasic CarpetIOASCII CarpetIOHDF5 IDScalarWaveC WaveToyC"
+
+
+Cactus::cctk_itlast = 24
+
+# Output
+IO::out_dir = $parfile
+
+# Recovery
+IO::recover = "auto"
+
+IOHDF5::checkpoint = "no"
diff --git a/Carpet/CarpetIOHDF5/param.ccl b/Carpet/CarpetIOHDF5/param.ccl
new file mode 100644
index 000000000..c80b383c5
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/param.ccl
@@ -0,0 +1,113 @@
+# Parameter definitions for thorn CarpetIOHDF5
+# $Header: /home/eschnett/C/carpet/Carpet/Carpet/CarpetIOHDF5/param.ccl,v 1.15 2004/06/25 12:54:30 schnetter Exp $
+
+
+
+shares: IO
+
+USES STRING out_dir
+USES BOOLEAN out_single_precision
+
+USES KEYWORD out_criterion
+USES CCTK_INT out_every
+USES CCTK_REAL out_dt
+
+USES INT     checkpoint_every
+USES INT     checkpoint_keep
+USES BOOLEAN checkpoint_ID
+USES BOOLEAN recover_and_remove
+USES BOOLEAN checkpoint_on_terminate
+USES BOOLEAN strict_io_parameter_check
+USES KEYWORD recover
+USES STRING  checkpoint_dir
+USES STRING  checkpoint_ID_file
+USES STRING  checkpoint_file
+USES STRING  recover_dir
+USES STRING  recover_file
+
+
+shares: Cactus
+
+USES CCTK_REAL cctk_initial_time
+
+
+private:
+
+
+
+BOOLEAN verbose "Produce log output" STEERABLE = ALWAYS
+{
+} "no"
+
+
+
+CCTK_STRING out3D_dir "Name of 3D CarpetIOHDF5 output directory, overrides IO::out_dir" STEERABLE = ALWAYS
+{
+  "^$" :: "Empty: use IO::out_dir"
+  ".+" :: "Not empty: directory name"
+} ""
+
+CCTK_STRING out3D_vars "Variables to output in 3D CarpetIOHDF5 file format" STEERABLE = ALWAYS
+{
+  .* :: "List of group and variable names"
+} ""
+
+CCTK_STRING out3D_extension "File extension to use for 3D CarpetIOHDF5 output" STEERABLE = ALWAYS
+{
+  ".*" :: "File extension (including a leading dot, if desired)"
+} ".h5"
+
+KEYWORD out3D_criterion "Criterion to select 3D CarpetIOHDF5 output intervals, overrides out_every" STEERABLE = ALWAYS
+{
+  "default"   :: "Use IO::out_criterion"
+  "never"     :: "Never output"
+  "iteration" :: "Output every so many iterations"
+  "divisor"   :: "Output if (iteration % out_every) == 0."
+  "time"      :: "Output every that much coordinate time"
+} "default"
+
+CCTK_INT out3D_every "How often to do 3D CarpetIOHDF5 output, overrides out_every" STEERABLE = ALWAYS
+{
+  1:*  :: "Output every so many time steps"
+  -1:0 :: "No output"
+  -2   :: "Use IO::out_every"
+} -2
+
+REAL out3D_dt "How often to do 3D CarpetIOHDF5 output, overrides IO::out_dt" STEERABLE = ALWAYS
+{
+  (0:* :: "In intervals of that much coordinate time"
+   0   :: "As often as possible"
+  -1   :: "Disable output"
+  -2   :: "Default to IO::out_dt"
+} -2
+
+
+
+CCTK_STRING in3D_dir "Name of 3D CarpetIOHDF5 input directory" STEERABLE = ALWAYS
+{
+  ".+" :: "Directory name"
+} "."
+
+CCTK_STRING in3D_vars "Variables to input in 3D CarpetIOHDF5 file format" STEERABLE = ALWAYS
+{
+  .* :: "List of group and variable names"
+} ""
+
+CCTK_STRING in3D_extension "File extension to use for 3D CarpetIOHDF5 input" STEERABLE = ALWAYS
+{
+  ".*" :: "File extension (including a leading dot, if desired)"
+} ".h5"
+
+
+BOOLEAN checkpoint "Do checkpointing with CarpetIOHDF5" STEERABLE = ALWAYS
+{
+} "no"
+
+BOOLEAN checkpoint_next "Checkpoint at next iteration" STEERABLE = ALWAYS
+{
+} "no"
+
+
+BOOLEAN use_reflevels_from_checkpoint "Use CarpetRegrid::refinement_levels from the checkpoint file rather than from the parameter file." STEERABLE = ALWAYS
+{
+} "no"
diff --git a/Carpet/CarpetIOHDF5/schedule.ccl b/Carpet/CarpetIOHDF5/schedule.ccl
new file mode 100644
index 000000000..fc3f69751
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/schedule.ccl
@@ -0,0 +1,63 @@
+# Schedule definitions for thorn CarpetIOHDF5
+# $Header: /home/eschnett/C/carpet/Carpet/Carpet/CarpetIOHDF5/schedule.ccl,v 1.12 2004/08/18 16:02:56 tradke Exp $
+
+storage: next_output_iteration next_output_time this_iteration
+
+schedule CarpetIOHDF5Startup at STARTUP after IOUtil_Startup
+{
+  LANG: C
+} "Startup routine"
+
+schedule CarpetIOHDF5Init at BASEGRID
+{
+  LANG: C
+  OPTIONS: global
+} "Initialisation routine"
+
+schedule CarpetIOHDF5ReadData at INITIAL
+{
+  LANG: C
+  OPTIONS: level
+} "Read initial data from file"
+
+if (checkpoint && checkpoint_ID)
+{
+  schedule CarpetIOHDF5_InitialDataCheckpoint at CPINITIAL
+  {
+    LANG: C
+    OPTIONS: meta
+  } "Initial data checkpoint routine"
+}
+
+schedule CarpetIOHDF5_EvolutionCheckpoint at CHECKPOINT
+{
+  LANG: C
+  OPTIONS: meta
+} "Evolution checkpoint routine"
+
+schedule CarpetIOHDF5_TerminationCheckpoint at TERMINATE
+{
+  LANG: C
+  OPTIONS: meta
+} "Termination checkpoint routine"
+
+schedule CarpetIOHDF5_CloseFile at POSTINITIAL
+{
+  LANG: C
+  OPTIONS: global
+} "Close an input file opened by the filereader"
+
+if (! CCTK_Equals (recover, "no") && *recover_file)
+{
+  schedule CarpetIOHDF5_RecoverParameters at RECOVER_PARAMETERS
+  {
+    LANG:C
+    OPTIONS: meta
+  } "Parameter recovery routine"
+
+  schedule CarpetIOHDF5_CloseFile at POST_RECOVER_VARIABLES
+  {
+    LANG: C
+    OPTIONS: global
+  } "Close an initial data checkpoint file"
+}
diff --git a/Carpet/CarpetIOHDF5/src/Checkpoint.cc b/Carpet/CarpetIOHDF5/src/Checkpoint.cc
new file mode 100644
index 000000000..04483c18a
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/src/Checkpoint.cc
@@ -0,0 +1,437 @@
+#include <assert.h>
+#include <limits.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+
+#include <algorithm>
+#include <fstream>
+#include <sstream>
+#include <string>
+#include <vector>
+
+#include <hdf5.h>
+
+#include "cctk.h"
+#include "cctk_Parameters.h"
+#include "cctk_Version.h"
+
+extern "C" {
+static const char* rcsid = "$Header: /home/eschnett/C/carpet/Carpet/Carpet/CarpetIOHDF5/src/Checkpoint.cc,v 1.2 2004/08/18 16:02:56 tradke Exp $";
+CCTK_FILEVERSION(Carpet_CarpetIOHDF5_Checkpoint_cc);
+}
+
+#include "CactusBase/IOUtil/src/ioGH.h"
+#include "CactusBase/IOUtil/src/ioutil_CheckpointRecovery.h"
+
+#include "bbox.hh"
+#include "data.hh"
+#include "gdata.hh"
+#include "ggf.hh"
+#include "vect.hh"
+
+#include "carpet.hh"
+
+#include "iohdf5.hh"
+#include "iohdf5GH.h"
+
+
+namespace CarpetIOHDF5
+{
+
+using namespace std;
+using namespace Carpet;
+
+// when was the last checkpoint written ?
+static int last_checkpoint_iteration = -1;
+
+
+static int Checkpoint (const cGH* const cctkGH, int called_from);
+static int DumpParametersGHExtentions (const cGH *cctkGH, int all, hid_t writer);
+
+
+int CarpetIOHDF5_InitialDataCheckpoint (const cGH* const cctkGH)
+{
+  CCTK_INFO ("---------------------------------------------------------");
+  CCTK_INFO ("Dumping initial data checkpoint");
+  CCTK_INFO ("---------------------------------------------------------");
+
+  int retval = Checkpoint (cctkGH, CP_INITIAL_DATA);
+
+  return (retval);
+}
+
+
+int CarpetIOHDF5_EvolutionCheckpoint (const cGH* const cctkGH)
+{
+  int retval = 0;
+  DECLARE_CCTK_PARAMETERS
+
+  // Test checkpoint_every, and adjust it. This is necessary until
+  // recovery is more flexible.
+  int every_full = 1 << (maxreflevels-1);
+  if (checkpoint_every > 0 && (checkpoint_every % every_full) != 0)
+  {
+    char new_checkpoint_every[32];
+
+    snprintf (new_checkpoint_every, sizeof (new_checkpoint_every), "%d",
+              (checkpoint_every / every_full + 1) * every_full);
+    CCTK_ParameterSet ("checkpoint_every", "IOUtil", new_checkpoint_every);
+    CCTK_VInfo (CCTK_THORNSTRING,"I have adjusted your checkpoint_every to %s.",
+                new_checkpoint_every);
+  }
+
+  if (checkpoint &&
+    ((checkpoint_every > 0 && cctkGH->cctk_iteration % checkpoint_every == 0) ||
+     checkpoint_next))
+  {
+    CCTK_INFO ("---------------------------------------------------------");
+    CCTK_VInfo (CCTK_THORNSTRING, "Dumping periodic checkpoint at "
+                "iteration %d", cctkGH->cctk_iteration);
+    CCTK_INFO ("---------------------------------------------------------");
+
+    retval = Checkpoint (cctkGH, CP_EVOLUTION_DATA);
+
+    if (checkpoint_next)
+    {
+      CCTK_ParameterSet ("checkpoint_next", CCTK_THORNSTRING, "no");
+    }
+  }
+
+  return (retval);
+}
+
+
+int CarpetIOHDF5_TerminationCheckpoint (const cGH *const GH)
+{
+  int retval = 0;
+  DECLARE_CCTK_PARAMETERS
+
+
+  if (checkpoint && checkpoint_on_terminate)
+  {
+    if (last_checkpoint_iteration < GH->cctk_iteration)
+    {
+      CCTK_INFO ("---------------------------------------------------------");
+      CCTK_VInfo (CCTK_THORNSTRING, "Dumping termination checkpoint at "
+                  "iteration %d", GH->cctk_iteration);
+      CCTK_INFO ("---------------------------------------------------------");
+
+      retval = Checkpoint (GH, CP_EVOLUTION_DATA);
+    }
+    else if (verbose)
+    {
+      CCTK_INFO ("---------------------------------------------------------");
+      CCTK_VInfo (CCTK_THORNSTRING, "Termination checkpoint already dumped "
+                  "as last evolution checkpoint at iteration %d",
+                  last_checkpoint_iteration);
+      CCTK_INFO ("---------------------------------------------------------");
+    }
+  }
+
+  return (retval);
+}
+
+
+static int Checkpoint (const cGH* const cctkGH, int called_from)
+{
+  DECLARE_CCTK_ARGUMENTS;
+  DECLARE_CCTK_PARAMETERS;
+
+  int retval = 0;
+
+  ioRequest *request;
+
+  CarpetIOHDF5GH *myGH = (CarpetIOHDF5GH *) CCTK_GHExtension (cctkGH,
+                                                              "CarpetIOHDF5");
+
+  // check if CarpetIOHDF5 was registered as I/O method
+  if (myGH == NULL)
+  {
+    CCTK_WARN (0, "No CarpetIOHDF5 I/O methods registered");
+    return -1;
+  }
+
+  int const myproc = CCTK_MyProc (cctkGH);
+
+
+  /* get the filenames for both the temporary and real checkpoint file */
+  char *filename = IOUtil_AssembleFilename (cctkGH, NULL, "", ".h5",
+                                            called_from, 0, 1);
+  char *tempname = IOUtil_AssembleFilename (cctkGH, NULL, ".tmp", ".h5",
+                                            called_from, 0, 1);
+
+  hid_t writer = -1;
+
+  if (myproc == 0)
+  {
+    if (verbose)
+    {
+      CCTK_VInfo (CCTK_THORNSTRING, "Creating temporary checkpoint file '%s'",
+                  tempname);
+    }
+
+    HDF5_ERROR (writer = H5Fcreate (tempname, H5F_ACC_TRUNC, H5P_DEFAULT,
+                                    H5P_DEFAULT));
+
+    // Dump all parameters and GHExtentions
+    retval = DumpParametersGHExtentions (cctkGH, 1, writer);
+
+  }
+
+  // now dump the grid variables on all mglevels, reflevels, maps and components
+  BEGIN_MGLEVEL_LOOP (cctkGH)
+  {
+    BEGIN_REFLEVEL_LOOP (cctkGH)
+    {
+      if (verbose)
+      {
+        CCTK_VInfo (CCTK_THORNSTRING,
+                    "Dumping grid variables on mglevel %d reflevel %d ...",
+                    mglevel, reflevel);
+      }
+
+      for (int group = CCTK_NumGroups () - 1; group >= 0; group--)
+      {
+        /* only dump groups which have storage assigned */
+        if (CCTK_QueryGroupStorageI (cctkGH, group) <= 0)
+        {
+          continue;
+        }
+
+        const int grouptype = CCTK_GroupTypeI (group);
+
+        // scalars and grid arrays only have one reflevel
+        if (grouptype != CCTK_GF && reflevel > 0)
+        {
+          continue;
+        }
+
+        // now check if there is any memory allocated
+        // for GFs and GAs. GSs should always have
+        // memory allocated and there is at this point
+        // no CCTK function to check this :/
+        if (grouptype == CCTK_GF || grouptype == CCTK_ARRAY)
+        {
+          const int gpdim = CCTK_GroupDimI (group);
+          int gtotalsize = 1;
+          vector<int> tlsh (gpdim);
+          assert(!CCTK_GrouplshGI(cctkGH,gpdim,&tlsh[0],group));
+          for(int i=0;i<gpdim;i++)
+          {
+            gtotalsize=tlsh[i];
+          }
+          if(gtotalsize == 0)
+          {
+            if (verbose)
+            {
+              CCTK_VInfo(CCTK_THORNSTRING, "Group '%s' is zero-sized. "
+                         "No checkpoint info written", CCTK_GroupName (group));
+            }
+            continue;
+          }
+        }
+
+        /* get the number of allocated timelevels */
+        cGroup gdata;
+        CCTK_GroupData (group, &gdata);
+        gdata.numtimelevels = 0;
+        gdata.numtimelevels = CCTK_GroupStorageIncrease (cctkGH, 1, &group,
+                                                         &gdata.numtimelevels,NULL);
+
+        int first_vindex = CCTK_FirstVarIndexI (group);
+
+        /* get the default I/O request for this group */
+        request = IOUtil_DefaultIORequest (cctkGH, first_vindex, 1);
+
+        /* disable checking for old data objects, disable datatype conversion
+           and downsampling */
+        request->check_exist = 0;
+        request->hdatatype = gdata.vartype;
+        for (request->hdim = 0; request->hdim < request->vdim; request->hdim++)
+        {
+          request->downsample[request->hdim] = 1;
+        }
+
+        /* loop over all variables in this group */
+        for (request->vindex = first_vindex;
+             request->vindex < first_vindex + gdata.numvars;
+             request->vindex++)
+        {
+          char *fullname = CCTK_FullName (request->vindex);
+
+          /* loop over all timelevels of this variable */
+          for (request->timelevel = 0;
+               request->timelevel < gdata.numtimelevels;
+               request->timelevel++)
+          {
+            if (verbose)
+            {
+              if (fullname != NULL)
+              {
+                CCTK_VInfo (CCTK_THORNSTRING, "  %s (timelevel %d)",
+                            fullname, request->timelevel);
+              }
+              else
+              {
+                CCTK_VWarn(1, __LINE__, __FILE__, CCTK_THORNSTRING,
+                           "Invalid variable with varindex %d", request->vindex);
+              }
+            }
+            // write the var
+
+            if (grouptype == CCTK_ARRAY || grouptype == CCTK_GF || grouptype == CCTK_SCALAR)
+            {
+              if (verbose)
+              {
+                CCTK_VInfo (CCTK_THORNSTRING,"%s: reflevel: %d map: %d component: %d grouptype: %d ",
+                            fullname,reflevel,Carpet::map,component,grouptype);
+              }
+              retval += WriteVar(cctkGH,writer,request,1);
+            }
+            else
+            {
+              CCTK_VWarn (1, __LINE__, __FILE__, CCTK_THORNSTRING,
+                          "Invalid group type %d for variable '%s'", grouptype, fullname);
+              retval = -1;
+            }
+          }
+          free(fullname);
+        } /* end of loop over all variables */
+      } /* end of loop over all groups */
+    } END_REFLEVEL_LOOP;
+
+  } END_MGLEVEL_LOOP;
+
+
+  // Close the file
+  if (writer >= 0)
+  {
+    HDF5_ERROR (H5Fclose(writer));
+  }
+
+  if (retval == 0 && myproc == 0)
+  {
+    if (rename (tempname, filename))
+    {
+      CCTK_VWarn (1, __LINE__, __FILE__, CCTK_THORNSTRING,
+                  "Could not rename temporary checkpoint file '%s' to '%s'",
+                  tempname, filename);
+      retval = -1;
+    }
+    else
+    {
+      if (myGH->cp_filename_list[myGH->cp_filename_index])
+      {
+        if (checkpoint_keep > 0)
+        {
+          remove (myGH->cp_filename_list[myGH->cp_filename_index]);
+        }
+        free (myGH->cp_filename_list[myGH->cp_filename_index]);
+      }
+      myGH->cp_filename_list[myGH->cp_filename_index] = strdup (filename);
+      myGH->cp_filename_index = (myGH->cp_filename_index+1) % abs (checkpoint_keep);
+    }
+  }
+
+  // save the iteration number of this checkpoint
+  last_checkpoint_iteration = cctkGH->cctk_iteration;
+
+  // free allocated resources
+  free (tempname);
+  free (filename);
+
+  return retval;
+
+} // Checkpoint
+
+
+static int DumpParametersGHExtentions (const cGH *cctkGH, int all, hid_t writer)
+{
+  // large parts of this routine were taken from
+  // Thomas Radke's IOHDF5Util. Thanks Thomas!
+
+  DECLARE_CCTK_PARAMETERS;
+
+  char *parameters;
+  hid_t group, dspace, dset;
+  hsize_t size;
+
+  int itmp;
+  double dtmp;
+  const char *version;
+  const ioGH *ioUtilGH;
+
+  if (verbose)
+  {
+    CCTK_INFO ("Dumping Parameters and GH Extentions...");
+  }
+
+  /* get the parameter string buffer */
+  parameters = IOUtil_GetAllParameters (cctkGH, all);
+  if (parameters)
+  {
+    size = strlen (parameters) + 1;
+    HDF5_ERROR (group = H5Gcreate (writer, METADATA_GROUP, 0));
+    HDF5_ERROR (dspace = H5Screate_simple (1, &size, NULL));
+    HDF5_ERROR (dset = H5Dcreate (group, ALL_PARAMETERS, H5T_NATIVE_UCHAR,
+                                  dspace, H5P_DEFAULT));
+    HDF5_ERROR (H5Dwrite (dset, H5T_NATIVE_CHAR, H5S_ALL, H5S_ALL,
+                          H5P_DEFAULT, parameters));
+
+    // now dump the GH Extentions
+
+    /* get the handle for IOUtil extensions */
+    ioUtilGH = (const ioGH *) CCTK_GHExtension (cctkGH, "IO");
+
+    itmp = CCTK_MainLoopIndex ();
+    WriteAttribute(dset,"main loop index",itmp);
+
+    itmp = cctkGH->cctk_iteration;
+    WriteAttribute(dset,"GH$iteration",itmp);
+
+    itmp = ioUtilGH->ioproc_every;
+    WriteAttribute(dset,"GH$ioproc_every",itmp);
+
+    itmp = CCTK_nProcs (cctkGH);
+    WriteAttribute(dset,"GH$nprocs",itmp);
+
+    dtmp = cctkGH->cctk_time;
+    WriteAttribute(dset,"GH$time", dtmp);
+
+    dtmp = global_time;
+    WriteAttribute(dset,"carpet_global_time", dtmp);
+
+    itmp = reflevels;
+    WriteAttribute(dset,"carpet_reflevels", itmp);
+
+    dtmp = delta_time;
+    WriteAttribute(dset,"carpet_delta_time", dtmp);
+
+    version = CCTK_FullVersion();
+    WriteAttribute(dset,"Cactus version", version);
+
+    /* finally, we need all the times on the individual refinement levels */
+
+    const int numberofmgtimes=mglevels;
+    WriteAttribute(dset,"numberofmgtimes",numberofmgtimes);
+    for(int i=0;i < numberofmgtimes;i++)
+    {
+      char buffer[100];
+      snprintf (buffer, sizeof (buffer), "mgleveltimes %d", i);
+      WriteAttribute(dset,buffer,(double *) &leveltimes.at(i).at(0), reflevels);
+    }
+
+    HDF5_ERROR (H5Dclose (dset));
+    HDF5_ERROR (H5Sclose (dspace));
+    HDF5_ERROR (H5Gclose (group));
+
+    free (parameters);
+  }
+
+  return 0;
+} // DumpParametersGHExtentions
+
+} // namespace CarpetIOHDF5
diff --git a/Carpet/CarpetIOHDF5/src/Recover.cc b/Carpet/CarpetIOHDF5/src/Recover.cc
new file mode 100644
index 000000000..0f5349bb8
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/src/Recover.cc
@@ -0,0 +1,523 @@
+#include <assert.h>
+#include <limits.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+
+#include <algorithm>
+#include <fstream>
+#include <sstream>
+#include <string>
+#include <vector>
+
+#include <hdf5.h>
+
+#include "cctk.h"
+#include "cctk_Parameters.h"
+#include "cctk_Version.h"
+
+extern "C" {
+static const char* rcsid = "$Header: /home/eschnett/C/carpet/Carpet/Carpet/CarpetIOHDF5/src/Recover.cc,v 1.2 2004/07/09 15:38:18 tradke Exp $";
+CCTK_FILEVERSION(Carpet_CarpetIOHDF5_iohdf5chckpt_recover_cc);
+}
+
+#include "CactusBase/IOUtil/src/ioGH.h"
+#include "CactusBase/IOUtil/src/ioutil_CheckpointRecovery.h"
+
+#include "bbox.hh"
+#include "data.hh"
+#include "gdata.hh"
+#include "ggf.hh"
+#include "vect.hh"
+
+#include "carpet.hh"
+
+#include "iohdf5.hh"
+#include "iohdf5GH.h"
+
+/* some macros for HDF5 group names */
+#define METADATA_GROUP "Parameters and Global Attributes"
+#define ALL_PARAMETERS "All Parameters"
+
+
+
+namespace CarpetIOHDF5
+{
+
+using namespace std;
+using namespace Carpet;
+
+// structure describing a single dataset of an HDF5 file to read from
+typedef struct
+{
+  char *datasetname;
+
+  int vindex;
+  int timelevel;
+  int mglevel;
+  int reflevel;
+  int rank;
+  int *shape;    // [rank]
+  int *iorigin;  // [rank]
+} dataset_t;
+
+// structure describing the contents of an HDF5 file to read from
+typedef struct
+{
+  int num_mglevels;
+  int num_reflevels;
+  int parameter_len;
+  int cctk_iteration;
+  int main_loop_index;
+  double global_time;
+  double delta_time;
+  double *mgleveltimes;  // [num_mglevels*num_reflevels]
+
+  char *filename;
+  hid_t file;
+  int num_datasets;
+  list<dataset_t> datasets;  // [num_datasets]
+  int num_ints;              // total number of integers in datasets[]
+} file_t;
+
+static file_t infile = {0, 0, 0, 0, 0, 0, 0, NULL, NULL, -1, -1,
+                        list<dataset_t> (), 0};
+
+
+static int OpenFile (const char *basefilename, file_t *file, int called_from);
+static int RecoverVariables (cGH* cctkGH, file_t *file);
+static herr_t ReadMetadata (hid_t group, const char *objectname, void *arg);
+
+
+// Register with the Cactus Recovery Interface
+int CarpetIOHDF5_RecoverParameters (void)
+{
+  int retval = IOUtil_RecoverParameters (Recover, ".h5", "HDF5");
+
+  return (retval);
+}
+
+
+// close a checkpoint/filereader file after recovering grid variables
+int CarpetIOHDF5_CloseFile (void)
+{
+  DECLARE_CCTK_PARAMETERS
+
+
+  if (infile.num_datasets <= 0)
+  {
+    return (0);
+  }
+  infile.num_datasets = -1;
+
+  if (verbose)
+  {
+    CCTK_VInfo (CCTK_THORNSTRING, "closing file '%s' after recovery",
+                infile.filename);
+  }
+
+  if (infile.file >= 0)
+  {
+    HDF5_ERROR (H5Fclose (infile.file));
+    infile.file = -1;
+  }
+  free (infile.filename);
+  delete[] infile.mgleveltimes;
+
+  for (list<dataset_t>::iterator dataset = infile.datasets.begin ();
+       dataset != infile.datasets.end ();
+       dataset++)
+  {
+    delete[] dataset->shape;
+    delete[] dataset->iorigin;
+  }
+  infile.datasets.clear ();
+
+  return (0);
+}
+
+static int OpenFile (const char *basefilename, file_t *file, int called_from)
+{
+  hid_t dset = -1;
+  const int myproc = CCTK_MyProc (NULL);
+  DECLARE_CCTK_PARAMETERS
+
+
+  // generate filename for an unchunked checkpoint file */
+  file->filename = IOUtil_AssembleFilename (NULL, basefilename, "", ".h5",
+                                            called_from, 0, 1);
+  if (verbose)
+  {
+    CCTK_VInfo (CCTK_THORNSTRING, "opening %s file '%s'",
+                called_from == CP_RECOVER_PARAMETERS ? "checkpoint" : "input",
+                file->filename);
+  }
+
+  if (myproc == 0)
+  {
+    // try to open the file (prevent HDF5 error messages if it fails)
+    H5E_BEGIN_TRY
+    {
+      file->file = H5Fopen (file->filename, H5F_ACC_RDONLY, H5P_DEFAULT);
+    } H5E_END_TRY;
+
+    if (file->file >= 0)
+    {
+      if (called_from == CP_RECOVER_PARAMETERS)
+      {
+        HDF5_ERROR (dset = H5Dopen (file->file,
+                                    METADATA_GROUP"/"ALL_PARAMETERS));
+        ReadAttribute (dset, "carpet_reflevels", file->num_reflevels);
+        ReadAttribute (dset, "numberofmgtimes", file->num_mglevels);
+        ReadAttribute (dset, "GH$iteration", file->cctk_iteration);
+        ReadAttribute (dset, "main loop index", file->main_loop_index);
+        ReadAttribute (dset, "carpet_global_time", file->global_time);
+        ReadAttribute (dset, "carpet_delta_time", file->delta_time);
+        file->parameter_len = H5Dget_storage_size (dset) + 1;
+        assert (file->parameter_len > 1);
+      }
+
+      file->num_ints = 0;
+      HDF5_ERROR (H5Giterate (file->file, "/", NULL, ReadMetadata, file));
+    }
+    file->num_datasets = file->datasets.size ();
+    if (file->num_datasets <= 0)
+    {
+      CCTK_VWarn (1, __LINE__, __FILE__, CCTK_THORNSTRING,
+                  "No valid HDF5 file '%s' found", file->filename);
+      free (file->filename);
+    }
+  }
+
+  // broadcast integer variables
+  int *intbuffer = new int[7];
+  intbuffer[0] = file->num_datasets;
+  intbuffer[1] = file->num_mglevels;
+  intbuffer[2] = file->num_reflevels;
+  intbuffer[3] = file->cctk_iteration;
+  intbuffer[4] = file->main_loop_index;
+  intbuffer[5] = file->parameter_len;
+  intbuffer[6] = file->num_ints;
+  MPI_Bcast (intbuffer, 7, MPI_INT, 0, MPI_COMM_WORLD);
+  file->num_datasets    = intbuffer[0];
+  file->num_mglevels    = intbuffer[1];
+  file->num_reflevels   = intbuffer[2];
+  file->cctk_iteration  = intbuffer[3];
+  file->main_loop_index = intbuffer[4];
+  file->parameter_len   = intbuffer[5];
+  file->num_ints        = intbuffer[6];
+  delete[] intbuffer;
+
+  // return if no valid checkpoint could be found
+  if (file->num_datasets <= 0)
+  {
+    return (-1);
+  }
+
+  // serialize the dataset list metadata into a single MPI_INT buffer
+  intbuffer = new int[file->num_ints];
+  if (myproc == 0)
+  {
+    for (list<dataset_t>::iterator dataset = file->datasets.begin ();
+         dataset != file->datasets.end ();
+         dataset++)
+    {
+      *intbuffer++ = dataset->vindex;
+      *intbuffer++ = dataset->timelevel;
+      *intbuffer++ = dataset->mglevel;
+      *intbuffer++ = dataset->reflevel;
+      *intbuffer++ = dataset->rank;
+
+      for (int i = 0; i < dataset->rank; i++)
+      {
+        *intbuffer++ = dataset->shape[i];
+        *intbuffer++ = dataset->iorigin[i];
+      }
+    }
+    intbuffer -= file->num_ints;
+  }
+
+  // broadcast the serialized dataset list metadata
+  MPI_Bcast (intbuffer, file->num_ints, MPI_INT, 0, MPI_COMM_WORLD);
+
+  // build the dataset list on non-I/O processors
+  if (myproc != 0)
+  {
+    for (int i = 0; i < file->num_datasets; i++)
+    {
+      dataset_t dataset;
+
+
+      dataset.vindex    = *intbuffer++;
+      dataset.timelevel = *intbuffer++;
+      dataset.mglevel   = *intbuffer++;
+      dataset.reflevel  = *intbuffer++;
+      dataset.rank      = *intbuffer++;
+
+      dataset.shape     = new int[dataset.rank];
+      dataset.iorigin   = new int[dataset.rank];
+      for (int j = 0; j < dataset.rank; j++)
+      {
+        dataset.shape[j]   = *intbuffer++;
+        dataset.iorigin[j] = *intbuffer++;
+      }
+
+      file->datasets.push_back (dataset);
+    }
+    intbuffer -= file->num_ints;
+  }
+  delete[] intbuffer;
+
+  if (called_from == FILEREADER_DATA)
+  {
+    return (0);
+  }
+
+  // Use refinement levels parameter from checkpointing file ?
+  if (use_reflevels_from_checkpoint)
+  {
+    char buffer[32];
+
+    snprintf (buffer, sizeof (buffer), "%d", file->num_reflevels);
+    CCTK_ParameterSet ("refinement_levels", "CarpetRegrid", buffer);
+
+    CCTK_VInfo (CCTK_THORNSTRING, "Using %i reflevels read from checkpoint "
+                "file. Ignoring value in parameter file.", file->num_reflevels);
+  }
+
+  // leave space at the end for global_time and delta_time
+  // so that all double variables can be broadcasted in one go
+  int num_times = file->num_mglevels*file->num_reflevels + 2;
+  file->mgleveltimes = new double[num_times];
+  if (myproc == 0)
+  {
+    // FIXME: should store all mgleveltimes in a single contiguous array
+    //        to get rid of this loop and save some attributes
+    for (int i = 0; i < file->num_mglevels; i++)
+    {
+      char buffer[32];
+
+      snprintf (buffer, sizeof (buffer), "mgleveltimes %d", i);
+      ReadAttribute (dset, buffer, file->mgleveltimes + i*file->num_reflevels,
+                     file->num_reflevels);
+    }
+  }
+
+  // broadcast double variables
+  file->mgleveltimes[num_times - 2] = file->global_time;
+  file->mgleveltimes[num_times - 1] = file->delta_time;
+  MPI_Bcast (file->mgleveltimes, num_times, MPI_DOUBLE, 0, MPI_COMM_WORLD);
+  file->global_time = file->mgleveltimes[num_times - 2];
+  file->delta_time = file->mgleveltimes[num_times - 1];
+
+  char *parameters = new char[file->parameter_len];
+  if (myproc == 0)
+  {
+    CCTK_VInfo (CCTK_THORNSTRING, "Recovering parameters from checkpoint");
+
+    HDF5_ERROR (H5Dread (dset, H5T_NATIVE_CHAR, H5S_ALL, H5S_ALL, H5P_DEFAULT,
+                         parameters));
+  }
+  if (dset >= 0)
+  {
+    HDF5_ERROR (H5Dclose (dset));
+  }
+
+  // broadcast char variables
+  MPI_Bcast (parameters, file->parameter_len, MPI_CHAR, 0, MPI_COMM_WORLD);
+
+  // recover parameters
+  IOUtil_SetAllParameters (parameters);
+  delete[] parameters;
+
+  return (0);
+}
+
+
+int Recover (cGH* cctkGH, const char *basefilename, int called_from)
+{
+  DECLARE_CCTK_PARAMETERS
+  int retval = 0;
+
+
+  assert (called_from == CP_RECOVER_PARAMETERS ||
+          called_from == CP_RECOVER_DATA ||
+          called_from == FILEREADER_DATA);
+
+  if (called_from == CP_RECOVER_PARAMETERS ||
+      called_from == FILEREADER_DATA)
+  {
+    // open the file, read and broadcast its metadata information
+    // for CP_RECOVER_PARAMETERS: also recover all parameters
+    retval = OpenFile (basefilename, &infile, called_from);
+
+    if (called_from == CP_RECOVER_PARAMETERS || retval)
+    {
+      return (retval == 0 ? 1 : -1);
+    }
+  }
+
+  // can only proceed with a valid checkpoint file from here on
+  assert (infile.num_datasets > 0);
+
+  // set global Cactus/Carpet variables
+  if (called_from == CP_RECOVER_DATA)
+  {
+    global_time = infile.global_time;
+    delta_time = infile.delta_time;
+    CCTK_SetMainLoopIndex (infile.main_loop_index);
+
+    cctkGH->cctk_iteration = infile.cctk_iteration;
+    cctkGH->cctk_time = infile.mgleveltimes[mglevel*infile.num_reflevels +
+                                            reflevel];
+  }
+
+  // now recover all grid variables on current mglevel and reflevel
+  retval = RecoverVariables (cctkGH, &infile);
+
+  if (called_from == CP_RECOVER_DATA)
+  {
+    CCTK_VInfo (CCTK_THORNSTRING,
+                "restarting simulation at iteration %d (physical time %g)",
+                cctkGH->cctk_iteration, (double) cctkGH->cctk_time);
+  }
+  else
+  {
+    // FIXME: keep filereader input file open for all mglevels, reflevels
+    //        this doesn't work now because we don't know the number of
+    //        mglevels and reflevels
+    //        also: there may be multiple files to be read in, and they
+    //              must not share the same data structures
+    CarpetIOHDF5_CloseFile ();
+  }
+
+  return (retval);
+}
+
+
+static herr_t ReadMetadata (hid_t group, const char *objectname, void *arg)
+{
+  file_t *file = (file_t *) arg;
+  dataset_t dataset;
+  hid_t dset, dspace;
+  H5G_stat_t object_info;
+
+
+  // we are interested only in datasets
+  HDF5_ERROR (H5Gget_objinfo (group, objectname, 0, &object_info));
+  if (object_info.type != H5G_DATASET)
+  {
+    return (0);
+  }
+
+  dataset.datasetname = strdup (objectname);
+  assert (dataset.datasetname);
+
+  HDF5_ERROR (dset = H5Dopen (group, objectname));
+  char *varname = NULL;
+  ReadAttribute (dset, "name", varname);
+  dataset.vindex = CCTK_VarIndex (varname);
+  free (varname);
+  ReadAttribute (dset, "name", varname);
+  ReadAttribute (dset, "level", dataset.timelevel);
+  ReadAttribute (dset, "carpet_mglevel", dataset.mglevel);
+  ReadAttribute (dset, "carpet_reflevel", dataset.reflevel);
+  HDF5_ERROR (dspace = H5Dget_space (dset));
+  HDF5_ERROR (dataset.rank = H5Sget_simple_extent_ndims (dspace));
+  dataset.iorigin = new int[dataset.rank];
+  ReadAttribute (dset, "iorigin", dataset.iorigin, dataset.rank);
+  hsize_t *shape = new hsize_t[dataset.rank];
+  HDF5_ERROR (H5Sget_simple_extent_dims (dspace, shape, NULL));
+  dataset.shape = new int[dataset.rank];
+  for (int i = 0; i < dataset.rank; i++)
+  {
+    dataset.shape[i] = shape[i];
+  }
+  HDF5_ERROR (H5Sclose (dspace));
+  HDF5_ERROR (H5Dclose (dset));
+
+  // add this dataset to our list and count the number of int elements
+  file->datasets.push_back (dataset);
+  file->num_ints += 5 + 2*dataset.rank;
+
+  return (0);
+}
+
+
+static int RecoverVariables (cGH* cctkGH, file_t *file)
+{
+  DECLARE_CCTK_PARAMETERS
+  int myproc = CCTK_MyProc (cctkGH);
+  hid_t dset = -1;
+
+
+#if 0
+  double leveltime = MPI_Wtime();
+  double comparetime = MPI_Wtime();
+  static double totaltime;
+  if (reflevel == 0) totaltime = 0;
+
+  comparetime = MPI_Wtime() - comparetime;
+  // cout << "Time for string comparison: " << comparetime << endl;
+  // cout << "I have for this reflevel " << refleveldatasetnamelist.size() << endl;
+#endif
+
+  CCTK_VInfo (CCTK_THORNSTRING,
+              "reading grid variables on mglevel %d reflevel %d",
+              mglevel, reflevel);
+
+  int num_vars = CCTK_NumVars ();
+  for (list<dataset_t>::iterator dataset = file->datasets.begin ();
+       dataset != file->datasets.end ();
+       dataset++)
+  {
+    // only recover grid variables for the current mglevel/reflevel
+    if (dataset->mglevel != mglevel || dataset->reflevel != reflevel)
+    {
+      continue;
+    }
+
+    if (dataset->vindex <  0 || dataset->vindex >= num_vars)
+    {
+      if (myproc == 0)
+      {
+        CCTK_VWarn (1, __LINE__, __FILE__, CCTK_THORNSTRING,
+                    "Ignoring dataset '%s' (invalid variable name)",
+                    dataset->datasetname);
+      }
+      continue;
+    }
+
+    if (myproc == 0)
+    {
+      HDF5_ERROR (dset = H5Dopen (file->file, dataset->datasetname));
+      assert (dset >= 0);
+    }
+
+    vector<ibset> regions_read (Carpet::maps);
+    int did_read_something = ReadVar (cctkGH, dataset->vindex, dset,
+                                      regions_read, 1);
+    MPI_Bcast (&did_read_something, 1, MPI_INT, 0, dist::comm);
+
+    if (dset >= 0)
+    {
+      HDF5_ERROR (H5Dclose (dset));
+    }
+
+  }
+
+#if 0
+  leveltime = MPI_Wtime() - leveltime;
+  totaltime += leveltime;
+
+  if (verbose)
+    cout << "Timers: leveltime: " << leveltime << " totaltime: " << totaltime << endl;
+#endif
+
+  return (0);
+}
+
+
+} // namespace CarpetIOHDF5
diff --git a/Carpet/CarpetIOHDF5/src/iohdf5.cc b/Carpet/CarpetIOHDF5/src/iohdf5.cc
new file mode 100644
index 000000000..6a7ba3b46
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/src/iohdf5.cc
@@ -0,0 +1,1181 @@
+#include <assert.h>
+#include <limits.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+
+#include <algorithm>
+#include <fstream>
+#include <sstream>
+#include <vector>
+
+#include <hdf5.h>
+
+#include "cctk.h"
+#include "cctk_Parameters.h"
+
+extern "C" {
+  static const char* rcsid = "$Header: /home/eschnett/C/carpet/Carpet/Carpet/CarpetIOHDF5/src/iohdf5.cc,v 1.41 2004/07/09 15:38:18 tradke Exp $";
+  CCTK_FILEVERSION(Carpet_CarpetIOHDF5_iohdf5_cc);
+}
+
+#include "CactusBase/IOUtil/src/ioGH.h"
+#include "CactusBase/IOUtil/src/ioutil_CheckpointRecovery.h"
+
+#include "bbox.hh"
+#include "data.hh"
+#include "gdata.hh"
+#include "ggf.hh"
+#include "vect.hh"
+
+#include "carpet.hh"
+
+#include "iohdf5.hh"
+#include "iohdf5GH.h"
+
+
+namespace CarpetIOHDF5 {
+
+using namespace std;
+using namespace Carpet;
+
+
+// Variable definitions
+vector<bool> do_truncate;     // [var]
+vector<vector<vector<int> > > last_output; // [ml][rl][var]
+
+
+void CarpetIOHDF5Startup (void)
+{
+  DECLARE_CCTK_PARAMETERS
+
+
+  CCTK_RegisterBanner ("AMR 3D HDF5 I/O provided by CarpetIOHDF5");
+
+  int GHExtension = CCTK_RegisterGHExtension ("CarpetIOHDF5");
+  CCTK_RegisterGHExtensionSetupGH (GHExtension, SetupGH);
+
+  int IOMethod = CCTK_RegisterIOMethod ("IOHDF5");
+  CCTK_RegisterIOMethodOutputGH (IOMethod, OutputGH);
+  CCTK_RegisterIOMethodOutputVarAs (IOMethod, OutputVarAs);
+  CCTK_RegisterIOMethodTimeToOutput (IOMethod, TimeToOutput);
+  CCTK_RegisterIOMethodTriggerOutput (IOMethod, TriggerOutput);
+
+  /* initial I/O parameter check */
+  int numvars = CCTK_NumVars ();
+  vector<bool> flags(numvars);
+
+  if (CCTK_TraverseString (out3D_vars, SetFlag, &flags,CCTK_GROUP_OR_VAR) < 0)
+  {
+    CCTK_VWarn (strict_io_parameter_check ? 0 : 1,
+                __LINE__, __FILE__, CCTK_THORNSTRING,
+                "error while parsing parameter 'IOHDF5::out3D_vars'");
+  }
+
+#if 0
+  // Christian's Recovery routine
+  if ( !(CCTK_Equals(recover,"no")) ) {
+    ierr = IOUtil_RegisterRecover ("CarpetIOHDF5 recovery", Recover);
+    assert (! ierr);
+  } else {
+    // Erik's Recovery routine
+    ierr = IOUtil_RegisterRecover ("CarpetIOHDF5", Recover);
+    assert (! ierr);
+  }
+#else
+  if (IOUtil_RegisterRecover ("CarpetIOHDF5 recovery", Recover) < 0)
+  {
+    CCTK_WARN (1, "Failed to register IOFlexIO recovery routine");
+  }
+#endif
+}
+
+
+
+int CarpetIOHDF5Init (const cGH* const cctkGH)
+{
+  DECLARE_CCTK_ARGUMENTS;
+
+  *this_iteration = -1;
+  *next_output_iteration = 0;
+  *next_output_time = cctk_time;
+
+  return (0);
+}
+
+
+
+void* SetupGH (tFleshConfig* const fc,
+               const int convLevel, cGH* const cctkGH)
+{
+  DECLARE_CCTK_PARAMETERS;
+
+  CarpetIOHDF5GH* myGH;
+
+  const void *dummy;
+  dummy = &fc;
+  dummy = &convLevel;
+  dummy = &cctkGH;
+  dummy = &dummy;
+
+  // Truncate all files if this is not a restart
+  do_truncate.resize(CCTK_NumVars(), true);
+
+  // No iterations have yet been output
+  last_output.resize(mglevels);
+  for (int ml=0; ml<mglevels; ++ml) {
+    last_output.at(ml).resize(maxreflevels);
+    for (int rl=0; rl<maxreflevels; ++rl) {
+      last_output.at(ml).at(rl).resize(CCTK_NumVars(), INT_MIN);
+    }
+  }
+
+  // We register only once, ergo we get only one handle.  We store
+  // that statically, so there is no need to pass anything to
+  // Cactus.
+
+  /* allocate a new GH extension structure */
+
+  CCTK_INT numvars = CCTK_NumVars ();
+
+  myGH            = (CarpetIOHDF5GH*) malloc (sizeof (CarpetIOHDF5GH));
+  myGH->out_last  = (int *) malloc (numvars * sizeof (int));
+  myGH->requests  = (ioRequest **) calloc (numvars, sizeof (ioRequest *));
+  myGH->cp_filename_list = (char **) calloc (abs (checkpoint_keep), sizeof (char *));
+  myGH->cp_filename_index = 0;
+  myGH->out_vars = strdup ("");
+  myGH->out_every_default = out_every - 1;
+
+  for (int i = 0; i < numvars; i++)
+  {
+    myGH->out_last [i] = -1;
+  }
+
+  myGH->open_output_files = NULL;
+
+  // Now set hdf5 complex datatypes
+  // Stolen from Thomas Radke
+  HDF5_ERROR (myGH->HDF5_COMPLEX =
+            H5Tcreate (H5T_COMPOUND, sizeof (CCTK_COMPLEX)));
+  HDF5_ERROR (H5Tinsert (myGH->HDF5_COMPLEX, "real",
+                       offsetof (CCTK_COMPLEX, Re), HDF5_REAL));
+  HDF5_ERROR (H5Tinsert (myGH->HDF5_COMPLEX, "imag",
+                       offsetof (CCTK_COMPLEX, Im), HDF5_REAL));
+#ifdef CCTK_REAL4
+  HDF5_ERROR (myGH->HDF5_COMPLEX8 =
+              H5Tcreate (H5T_COMPOUND, sizeof (CCTK_COMPLEX8)));
+  HDF5_ERROR (H5Tinsert (myGH->HDF5_COMPLEX8, "real",
+                         offsetof (CCTK_COMPLEX8, Re), HDF5_REAL4));
+  HDF5_ERROR (H5Tinsert (myGH->HDF5_COMPLEX8, "imag",
+                         offsetof (CCTK_COMPLEX8, Im), HDF5_REAL4));
+#endif
+#ifdef CCTK_REAL8
+  HDF5_ERROR (myGH->HDF5_COMPLEX16 =
+              H5Tcreate (H5T_COMPOUND, sizeof (CCTK_COMPLEX16)));
+  HDF5_ERROR (H5Tinsert (myGH->HDF5_COMPLEX16, "real",
+                         offsetof (CCTK_COMPLEX16, Re), HDF5_REAL8));
+  HDF5_ERROR (H5Tinsert (myGH->HDF5_COMPLEX16, "imag",
+                         offsetof (CCTK_COMPLEX16, Im), HDF5_REAL8));
+#endif
+#ifdef CCTK_REAL16
+  HDF5_ERROR (myGH->HDF5_COMPLEX32 =
+              H5Tcreate (H5T_COMPOUND, sizeof (CCTK_COMPLEX32)));
+  HDF5_ERROR (H5Tinsert (myGH->HDF5_COMPLEX32, "real",
+                         offsetof (CCTK_COMPLEX32, Re), HDF5_REAL16));
+  HDF5_ERROR (H5Tinsert (myGH->HDF5_COMPLEX32, "imag",
+                         offsetof (CCTK_COMPLEX32, Im), HDF5_REAL16));
+#endif
+  return (myGH);
+}
+
+int OutputGH (const cGH* const cctkGH) {
+  for (int vindex=0; vindex<CCTK_NumVars(); ++vindex) {
+    if (TimeToOutput(cctkGH, vindex)) {
+      TriggerOutput(cctkGH, vindex);
+    }
+  }
+  return 0;
+}
+
+
+
+int OutputVarAs (const cGH* const cctkGH, const char* const varname,
+                 const char* const alias)
+{
+  DECLARE_CCTK_ARGUMENTS
+  DECLARE_CCTK_PARAMETERS
+
+  int numvars = CCTK_NumVars ();
+  vector<bool> flags (numvars);
+
+  if (CCTK_TraverseString (varname, SetFlag, &flags, CCTK_VAR) < 0)
+  {
+    CCTK_VWarn (1, __LINE__, __FILE__, CCTK_THORNSTRING,
+                "error while parsing variable name '%s' (alias name '%s')",
+                varname, alias);
+    return (-1);
+  }
+
+  int vindex = 0;
+  while (! flags.at (vindex) && vindex < numvars) vindex++;
+  if (vindex >= numvars)
+  {
+    return (-1);
+  }
+
+  const int group = CCTK_GroupIndexFromVarI (vindex);
+  assert (group>=0 && group<(int)Carpet::arrdata.size());
+
+  // Check for storage
+  if (! CCTK_QueryGroupStorageI(cctkGH, group))
+  {
+    CCTK_VWarn (1, __LINE__, __FILE__, CCTK_THORNSTRING,
+                "Cannot output variable '%s' because it has no storage",
+                varname);
+    return (0);
+  }
+
+  const int grouptype = CCTK_GroupTypeI(group);
+  if (grouptype == CCTK_SCALAR || grouptype == CCTK_ARRAY)
+  {
+    assert (do_global_mode);
+  }
+
+  /* get the default I/O request for this variable */
+  ioRequest* request = IOUtil_DefaultIORequest (cctkGH, vindex, 1);
+
+  // Get grid hierarchy extentsion from IOUtil
+  const ioGH * const iogh = (const ioGH *)CCTK_GHExtension (cctkGH, "IO");
+  assert (iogh);
+
+  // Create the output directory
+  const char* myoutdir = out3D_dir;
+  if (CCTK_EQUALS(myoutdir, "")) {
+    myoutdir = out_dir;
+  }
+  if (CCTK_MyProc(cctkGH)==0) {
+    CCTK_CreateDirectory (0755, myoutdir);
+  }
+
+  // Invent a file name
+  ostringstream filenamebuf;
+  filenamebuf << myoutdir << "/" << alias << out3D_extension;
+  string filenamestr = filenamebuf.str();
+  const char * const filename = filenamestr.c_str();
+
+  hid_t writer = -1;
+
+  // Write the file only on the root processor
+  if (CCTK_MyProc (cctkGH) == 0)
+  {
+
+    // If this is the first time, then create and truncate the file
+    if (do_truncate.at(vindex))
+    {
+      struct stat fileinfo;
+      if (! iogh->recovered || stat(filename, &fileinfo)!=0)
+      {
+        HDF5_ERROR (writer = H5Fcreate (filename, H5F_ACC_TRUNC, H5P_DEFAULT,
+                                        H5P_DEFAULT));
+        assert (writer>=0);
+        HDF5_ERROR (H5Fclose (writer));
+        writer = -1;
+      }
+    }
+
+    // Open the file
+    HDF5_ERROR (writer = H5Fopen (filename, H5F_ACC_RDWR, H5P_DEFAULT));
+  }
+
+  if (verbose)
+  {
+    CCTK_VInfo (CCTK_THORNSTRING,
+                "Writing variable '%s' on mglevel %d reflevel %d",
+                varname, mglevel, reflevel);
+  }
+  WriteVar (cctkGH, writer, request, 0);
+
+  // Close the file
+  if (writer >= 0)
+  {
+    HDF5_ERROR (H5Fclose (writer));
+  }
+
+  // Don't truncate again
+  do_truncate.at(vindex) = false;
+
+  return (0);
+}
+
+int WriteVar (const cGH* const cctkGH, const hid_t writer, const ioRequest* request,
+                 const int called_from_checkpoint) {
+
+  DECLARE_CCTK_ARGUMENTS;
+  DECLARE_CCTK_PARAMETERS;
+
+  void * h5data=NULL;
+
+  const int n = request->vindex;
+  assert (n>=0 && n<CCTK_NumVars());
+  const char * varname = CCTK_FullName(n);
+  const int group = CCTK_GroupIndexFromVarI (n);
+  assert (group>=0 && group<(int)Carpet::arrdata.size());
+  const int n0 = CCTK_FirstVarIndexI(group);
+  assert (n0>=0 && n0<CCTK_NumVars());
+  const int var = n - n0;
+  assert (var>=0 && var<CCTK_NumVars());
+  int tl = 0;
+
+  const int gpdim = CCTK_GroupDimI(group);
+
+  // Check for storage
+  if (! CCTK_QueryGroupStorageI(cctkGH, group)) {
+    CCTK_VWarn (1, __LINE__, __FILE__, CCTK_THORNSTRING,
+                "Cannot output variable \"%s\" because it has no storage",
+                varname);
+    return 0;
+  }
+
+  const int grouptype = CCTK_GroupTypeI(group);
+  switch (grouptype) {
+  case CCTK_SCALAR:
+  case CCTK_ARRAY:
+    assert (do_global_mode);
+    break;
+  case CCTK_GF:
+    /* do nothing */
+    break;
+  default:
+    assert (0);
+  }
+  const int rl = grouptype==CCTK_GF ? reflevel : 0;
+
+  cGroup cgdata;
+  int ierr = CCTK_GroupData(group,&cgdata);
+  assert(ierr==0);
+
+  // Select memory (source) and file (destination) datatypes
+  int cctkDataType = CCTK_VarTypeI(n);
+  const hid_t memdatatype = h5DataType(cctkGH, cctkDataType);
+  assert(memdatatype >= 0);
+  if (out_single_precision && ! called_from_checkpoint)
+  {
+    if (cctkDataType == CCTK_VARIABLE_REAL)
+    {
+      cctkDataType = CCTK_VARIABLE_REAL4;
+    }
+    else if (cctkDataType == CCTK_VARIABLE_COMPLEX)
+    {
+      cctkDataType = CCTK_VARIABLE_COMPLEX8;
+    }
+#ifdef CCTK_INT2
+    else if (cctkDataType == CCTK_VARIABLE_INT)
+    {
+      cctkDataType = CCTK_VARIABLE_INT2;
+    }
+#endif
+  }
+  const hid_t filedatatype = h5DataType(cctkGH, cctkDataType);
+  assert(filedatatype >= 0);
+
+  // let's get the correct Carpet time level (which is the (-1) * Cactus timelevel):
+  tl = - request->timelevel;
+
+  // Traverse all components
+  BEGIN_MAP_LOOP(cctkGH, grouptype) {
+    BEGIN_COMPONENT_LOOP(cctkGH, grouptype) {
+
+      const ggf<dim>* ff = 0;
+
+      ff = (ggf<dim>*)arrdata.at(group).at(Carpet::map).data.at(var);
+
+      const gdata<dim>* const data = (*ff) (tl, rl, component, mglevel);
+
+      // Make temporary copy on processor 0
+      const ibbox ext = data->extent();
+//         vect<int,dim> lo = ext.lower();
+//         vect<int,dim> hi = ext.upper();
+//         vect<int,dim> str = ext.stride();
+
+      gdata<dim>* const tmp = data->make_typed (n);
+      tmp->allocate (ext, 0);
+
+      if ( !((cgdata.disttype == CCTK_DISTRIB_CONSTANT) &&
+             (arrdata.at(group).at(Carpet::map).hh->processors.at(reflevel).at(component)!=0))) {
+
+        if (cgdata.disttype == CCTK_DISTRIB_CONSTANT) {
+          assert(grouptype == CCTK_ARRAY || grouptype == CCTK_SCALAR);
+          if(component!=0) goto skip;
+          h5data = CCTK_VarDataPtrI(cctkGH,tl,n);
+        } else {
+          for (comm_state<dim> state; !state.done(); state.step()) {
+            tmp->copy_from (state, data, ext);
+          }
+        }
+          // Write data
+          if (CCTK_MyProc(cctkGH)==0) {
+
+            int ldim=0;
+            if ( grouptype==CCTK_SCALAR ) {
+              ldim = 1;
+            } else {
+              ldim = gpdim;
+            }
+
+              //              hsize_t shape[ldim];
+
+              vector<hsize_t> shape(ldim);
+
+              for (int d=0; d<ldim; ++d) {
+                shape[ldim-1-d] = (ext.shape() / ext.stride())[d];
+              }
+              hid_t dataspace;
+              HDF5_ERROR (dataspace = H5Screate_simple (ldim, &shape[0], NULL));
+              assert (dataspace>=0);
+
+
+              ostringstream datasetnamebuf;
+              datasetnamebuf << varname
+                             << " it=" << cctk_iteration
+                             << " tl=" << tl
+                             << " ml=" << mglevel
+                             << " rl=" << rl
+                             << " m=" << Carpet::map
+                             << " c=" << component;
+              string datasetnamestr = datasetnamebuf.str();
+              assert (datasetnamestr.size() <= 256); // limit dataset name size
+              const char * const datasetname = datasetnamestr.c_str();
+              hid_t dataset;
+              HDF5_ERROR (dataset = H5Dcreate (writer, datasetname, filedatatype, dataspace, H5P_DEFAULT));
+
+              if (dataset>=0) {
+
+                  if (cgdata.disttype != CCTK_DISTRIB_CONSTANT) {
+                      h5data = (void*)tmp->storage();
+                  }
+
+                  HDF5_ERROR (H5Dwrite (dataset, memdatatype, H5S_ALL, H5S_ALL, H5P_DEFAULT, h5data));
+
+                  // Write FlexIO attributes
+                  WriteAttribute (dataset, "level", rl);
+                  {
+                      CCTK_REAL origin[dim], delta[dim];
+                      CCTK_REAL min_ext[dim], max_ext[dim];
+                      for (int d=0; d<dim; ++d) {
+                          origin[d] = CCTK_ORIGIN_SPACE(d);
+                          delta[d] = CCTK_DELTA_SPACE(d);
+                          min_ext[d] = origin[d] + cctk_lbnd[d] * delta[d];
+                          max_ext[d] = origin[d] + cctk_ubnd[d] * delta[d];
+                      }
+                      WriteAttribute (dataset, "origin", min_ext, dim);
+                      WriteAttribute (dataset, "delta", delta, dim);
+                      WriteAttribute (dataset, "min_ext", min_ext, dim);
+                      WriteAttribute (dataset, "max_ext", max_ext, dim);
+                  }
+                  WriteAttribute (dataset, "time", cctk_time);
+                  WriteAttribute (dataset, "timestep", cctk_iteration);
+                  WriteAttribute (dataset, "level_timestep", cctk_iteration / reflevelfact);
+                  WriteAttribute (dataset, "persistence", maxreflevelfact / reflevelfact);
+                  {
+                      int time_refinement=0;
+                      int spatial_refinement[dim];
+                      int grid_placement_refinement[dim];
+                      time_refinement = reflevelfact;
+                      for (int d=0; d<dim; ++d) {
+                          spatial_refinement[d] = reflevelfact;
+                          grid_placement_refinement[d] = reflevelfact;
+                      }
+                      WriteAttribute (dataset, "time_refinement", time_refinement);
+                      WriteAttribute (dataset, "spatial_refinement", spatial_refinement, dim);
+                      WriteAttribute (dataset, "grid_placement_refinement", grid_placement_refinement, dim);
+                  }
+                  {
+                      int iorigin[dim];
+                      for (int d=0; d<dim; ++d) {
+                          iorigin[d] = (ext.lower() / ext.stride())[d];
+                      }
+                      WriteAttribute (dataset, "iorigin", iorigin, dim);
+                  }
+
+                  // Write some additional attributes
+
+                  // Legacy arguments
+                  {
+                      char * fullname = CCTK_FullName(n);
+                      assert (fullname);
+                      WriteAttribute (dataset, "name", fullname);
+                      free (fullname);
+                  }
+
+                  // Group arguments
+                  WriteAttribute (dataset, "group_version", 1);
+                  {
+                      char * fullname = CCTK_FullName(n);
+                      assert (fullname);
+                      WriteAttribute (dataset, "group_fullname", fullname);
+                      free (fullname);
+                  }
+                  WriteAttribute (dataset, "group_varname", CCTK_VarName(n));
+                  {
+                      char * groupname = CCTK_GroupName(group);
+                      assert (groupname);
+                      WriteAttribute (dataset, "group_groupname", groupname);
+                      free (groupname);
+                  }
+                  switch (grouptype) {
+                      case CCTK_GF:
+                          WriteAttribute (dataset, "group_grouptype", "CCTK_GF");
+                          break;
+                      case CCTK_ARRAY:
+                          WriteAttribute (dataset, "group_grouptype", "CCTK_ARRAY");
+                          break;
+                      case CCTK_SCALAR:
+                          WriteAttribute (dataset, "group_grouptype", "CCTK_SCALAR");
+                          break;
+                      default:
+                          assert (0);
+                  }
+                  WriteAttribute (dataset, "group_dim", CCTK_GroupDimI(group));
+                  WriteAttribute (dataset, "group_timelevel", tl);
+                  WriteAttribute (dataset, "group_numtimelevels", CCTK_NumTimeLevelsI(group));
+
+                  // Cactus arguments
+                  WriteAttribute (dataset, "cctk_version", 1);
+                  WriteAttribute (dataset, "cctk_dim", cctk_dim);
+                  WriteAttribute (dataset, "cctk_iteration", cctk_iteration);
+                  WriteAttribute (dataset, "cctk_gsh", cctk_gsh, dim);
+                  WriteAttribute (dataset, "cctk_lsh", cctk_lsh, dim);
+                  WriteAttribute (dataset, "cctk_lbnd", cctk_lbnd, dim);
+                  WriteAttribute (dataset, "cctk_delta_time", cctk_delta_time);
+                  WriteAttribute (dataset, "cctk_delta_space", cctk_delta_space, dim);
+                  WriteAttribute (dataset, "cctk_origin_space", cctk_origin_space, dim);
+                  WriteAttribute (dataset, "cctk_bbox", cctk_bbox, 2*dim);
+                  WriteAttribute (dataset, "cctk_levfac", cctk_levfac, dim);
+                  WriteAttribute (dataset, "cctk_levoff", cctk_levoff, dim);
+                  WriteAttribute (dataset, "cctk_levoffdenom", cctk_levoffdenom, dim);
+                  WriteAttribute (dataset, "cctk_timefac", cctk_timefac);
+                  WriteAttribute (dataset, "cctk_convlevel", cctk_convlevel);
+                  WriteAttribute (dataset, "cctk_convfac", cctk_convfac);
+                  WriteAttribute (dataset, "cctk_nghostzones", cctk_nghostzones, dim);
+                  WriteAttribute (dataset, "cctk_time", cctk_time);
+
+                  // Carpet arguments
+                  WriteAttribute (dataset, "carpet_version", 1);
+                  WriteAttribute (dataset, "carpet_dim", dim);
+                  WriteAttribute (dataset, "carpet_basemglevel", basemglevel);
+                  WriteAttribute (dataset, "carpet_mglevel", mglevel);
+                  WriteAttribute (dataset, "carpet_mglevels", mglevels);
+                  WriteAttribute (dataset, "carpet_mgface", mgfact);
+                  WriteAttribute (dataset, "carpet_reflevel", reflevel);
+                  WriteAttribute (dataset, "carpet_reflevels", reflevels);
+                  WriteAttribute (dataset, "carpet_reffact", reffact);
+                  WriteAttribute (dataset, "carpet_map", Carpet::map);
+                  WriteAttribute (dataset, "carpet_maps", maps);
+                  WriteAttribute (dataset, "carpet_component", component);
+                  WriteAttribute (dataset, "carpet_components", vhh.at(Carpet::map)->components(reflevel));
+
+                  HDF5_ERROR (H5Dclose (dataset));
+              }
+
+            HDF5_ERROR (H5Sclose (dataspace));
+
+          } // if on root processor
+      } // if ! CCTK_DISTRIB_BLAH
+
+    skip:
+      // Delete temporary copy
+      delete tmp;
+
+    } END_COMPONENT_LOOP;
+  } END_MAP_LOOP;
+
+  return 0;
+}
+
+
+
+int TimeToOutput (const cGH* const cctkGH, const int vindex)
+{
+  DECLARE_CCTK_ARGUMENTS;
+  DECLARE_CCTK_PARAMETERS;
+
+  assert (vindex>=0 && vindex<CCTK_NumVars());
+
+
+
+  const int grouptype = CCTK_GroupTypeFromVarI(vindex);
+  switch (grouptype) {
+  case CCTK_SCALAR:
+  case CCTK_ARRAY:
+    if (! do_global_mode) return 0;
+    break;
+  case CCTK_GF:
+    // do nothing
+    break;
+  default:
+    assert (0);
+  }
+
+
+
+  // check whether to output at this iteration
+  bool output_this_iteration;
+
+  const char* myoutcriterion = out3D_criterion;
+  if (CCTK_EQUALS(myoutcriterion, "default")) {
+    myoutcriterion = out_criterion;
+  }
+
+  if (CCTK_EQUALS (myoutcriterion, "never")) {
+
+    // Never output
+    output_this_iteration = false;
+
+  } else if (CCTK_EQUALS (myoutcriterion, "iteration")) {
+
+    int myoutevery = out3D_every;
+    if (myoutevery == -2) {
+      myoutevery = out_every;
+    }
+    if (myoutevery <= 0) {
+      // output is disabled
+      output_this_iteration = false;
+    } else if (cctk_iteration == *this_iteration) {
+      // we already decided to output this iteration
+      output_this_iteration = true;
+    } else if (cctk_iteration >= *next_output_iteration) {
+      // it is time for the next output
+      output_this_iteration = true;
+      *next_output_iteration = cctk_iteration + myoutevery;
+      *this_iteration = cctk_iteration;
+    } else {
+      // we want no output at this iteration
+      output_this_iteration = false;
+    }
+
+  } else if (CCTK_EQUALS (myoutcriterion, "divisor")) {
+
+    int myoutevery = out3D_every;
+    if (myoutevery == -2) {
+      myoutevery = out_every;
+    }
+    if (myoutevery <= 0) {
+      // output is disabled
+      output_this_iteration = false;
+    } else if ((cctk_iteration % myoutevery) == 0) {
+      // we already decided to output this iteration
+      output_this_iteration = true;
+    } else {
+      // we want no output at this iteration
+      output_this_iteration = false;
+    }
+
+  } else if (CCTK_EQUALS (myoutcriterion, "time")) {
+
+    CCTK_REAL myoutdt = out3D_dt;
+    if (myoutdt == -2) {
+      myoutdt = out_dt;
+    }
+    if (myoutdt < 0) {
+      // output is disabled
+      output_this_iteration = false;
+    } else if (myoutdt == 0) {
+      // output all iterations
+      output_this_iteration = true;
+    } else if (cctk_iteration == *this_iteration) {
+      // we already decided to output this iteration
+      output_this_iteration = true;
+    } else if (cctk_time / cctk_delta_time
+               >= *next_output_time / cctk_delta_time - 1.0e-12) {
+      // it is time for the next output
+      output_this_iteration = true;
+      *next_output_time = cctk_time + myoutdt;
+      *this_iteration = cctk_iteration;
+    } else {
+      // we want no output at this iteration
+      output_this_iteration = false;
+    }
+
+  } else {
+
+    assert (0);
+
+  } // select output criterion
+
+  if (! output_this_iteration) return 0;
+
+
+
+  if (! CheckForVariable(out3D_vars, vindex)) {
+    // This variable should not be output
+    return 0;
+  }
+
+
+
+  if (last_output.at(mglevel).at(reflevel).at(vindex) == cctk_iteration) {
+    // Has already been output during this iteration
+    char* varname = CCTK_FullName(vindex);
+    CCTK_VWarn (5, __LINE__, __FILE__, CCTK_THORNSTRING,
+                "Skipping output for variable \"%s\", because this variable "
+                "has already been output during the current iteration -- "
+                "probably via a trigger during the analysis stage",
+                varname);
+    free (varname);
+    return 0;
+  }
+
+  assert (last_output.at(mglevel).at(reflevel).at(vindex) < cctk_iteration);
+
+  // Should be output during this iteration
+  return 1;
+}
+
+
+
+int TriggerOutput (const cGH* const cctkGH, const int vindex) {
+  assert (vindex>=0 && vindex<CCTK_NumVars());
+
+  char* varname = CCTK_FullName(vindex);
+  const int retval = OutputVarAs (cctkGH, varname, CCTK_VarName(vindex));
+  free (varname);
+
+  last_output.at(mglevel).at(reflevel).at(vindex) = cctkGH->cctk_iteration;
+
+  return retval;
+}
+
+
+
+
+int ReadVar (const cGH* const cctkGH, const int vindex,
+             const hid_t dataset, vector<ibset> &regions_read,
+             const int called_from_recovery)
+{
+
+  DECLARE_CCTK_PARAMETERS;
+
+  const int group = CCTK_GroupIndexFromVarI (vindex);
+  assert (group>=0 && group<(int)Carpet::arrdata.size());
+  char *fullname = CCTK_FullName (vindex);
+  const int n0 = CCTK_FirstVarIndexI(group);
+  assert (n0>=0 && n0<CCTK_NumVars());
+  const int var = vindex - n0;
+  assert (var>=0 && var<CCTK_NumVars());
+  int tl = 0;
+
+  bool did_read_something = false;
+
+  // Stuff needed for Recovery
+
+  void *h5data = NULL;
+
+  if (verbose)
+  {
+    CCTK_VInfo (CCTK_THORNSTRING, "  reading '%s'", fullname);
+  }
+
+  // Check for storage
+  if (! CCTK_QueryGroupStorageI(cctkGH, group))
+  {
+    CCTK_VWarn (1, __LINE__, __FILE__, CCTK_THORNSTRING,
+                "Cannot input variable \"%s\" because it has no storage",
+                fullname);
+    free (fullname);
+    return 0;
+  }
+
+  const int grouptype = CCTK_GroupTypeI(group);
+  if ((grouptype == CCTK_SCALAR || grouptype == CCTK_ARRAY) && reflevel > 0)
+  {
+    free (fullname);
+    return 0;
+  }
+
+  const int gpdim = CCTK_GroupDimI(group);
+
+
+  int intbuffer[2 + 2*dim];
+  int &group_timelevel = intbuffer[0],
+      &amr_level       = intbuffer[1];
+  int *amr_origin      = &intbuffer[2],
+      *amr_dims        = &intbuffer[2+dim];
+
+  if (CCTK_MyProc(cctkGH)==0)
+  {
+    // get dataset dimensions
+    hid_t dataspace;
+    HDF5_ERROR (dataspace = H5Dget_space (dataset));
+    int rank = (int) H5Sget_simple_extent_ndims (dataspace);
+    assert (0 < rank && rank <= dim);
+    assert ((grouptype == CCTK_SCALAR ? gpdim+1 : gpdim) == rank);
+    vector<hsize_t> shape(rank);
+    HDF5_ERROR (H5Sget_simple_extent_dims (dataspace, &shape[0], NULL));
+    HDF5_ERROR (H5Sclose (dataspace));
+
+    for (int i = 0; i < dim; i++)
+    {
+      amr_dims[i]   = 1;
+      amr_origin[i] = 0;
+    }
+    int datalength = 1;
+    for (int i = 0; i < rank; i++)
+    {
+      datalength *= shape[i];
+      amr_dims[i] = shape[rank-i-1];
+    }
+
+    const int cctkDataType = CCTK_VarTypeI(vindex);
+    const hid_t datatype = h5DataType(cctkGH,cctkDataType);
+
+    //cout << "datalength: " << datalength << " rank: " << rank << "\n";
+    //cout << shape[0] << " " << shape[1] << " " << shape[2] << "\n";
+
+    // to do: read in an allocate with correct datatype
+
+    h5data = malloc (CCTK_VarTypeSize (cctkDataType) * datalength);
+    HDF5_ERROR (H5Dread (dataset, datatype, H5S_ALL, H5S_ALL, H5P_DEFAULT,
+                         h5data));
+
+    ReadAttribute (dataset, "level", amr_level);
+    ReadAttribute (dataset, "iorigin", amr_origin, rank);
+
+    if(called_from_recovery)
+    {
+      ReadAttribute(dataset,"group_timelevel", group_timelevel);
+    }
+  } // MyProc == 0
+
+  MPI_Bcast (intbuffer, sizeof (intbuffer) / sizeof (*intbuffer), MPI_INT, 0, dist::comm);
+
+#ifdef CARPETIOHDF5_DEBUG
+  cout << "amr_level: " << amr_level << " reflevel: " << reflevel << endl;
+#endif
+
+  if (amr_level == reflevel)
+  {
+    // Traverse all components on all levels
+    BEGIN_MAP_LOOP(cctkGH, grouptype)
+    {
+      BEGIN_COMPONENT_LOOP(cctkGH, grouptype)
+      {
+        did_read_something = true;
+
+        ggf<dim>* ff = 0;
+
+        assert (var < (int)arrdata.at(group).at(Carpet::map).data.size());
+        ff = (ggf<dim>*)arrdata.at(group).at(Carpet::map).data.at(var);
+
+        if(called_from_recovery) tl = group_timelevel;
+
+        gdata<dim>* const data = (*ff) (tl, reflevel, component, mglevel);
+
+        // Create temporary data storage on processor 0
+        vect<int,dim> str = vect<int,dim>(maxreflevelfact/reflevelfact);
+
+        if(grouptype == CCTK_SCALAR || grouptype == CCTK_ARRAY)
+          str = vect<int,dim> (1);
+
+        vect<int,dim> lb = vect<int,dim>::ref(amr_origin) * str;
+        vect<int,dim> ub
+          = lb + (vect<int,dim>::ref(amr_dims) - 1) * str;
+
+        gdata<dim>* const tmp = data->make_typed (vindex);
+
+
+        cGroup cgdata;
+        int ierr = CCTK_GroupData(group,&cgdata);
+        assert(ierr==0);
+        //cout << "lb_before: " << lb << endl;
+        //cout << "ub_before: " << ub << endl;
+        if (cgdata.disttype == CCTK_DISTRIB_CONSTANT) {
+#ifdef CARPETIOHDF5_DEBUG
+          cout << "CCTK_DISTRIB_CONSTANT: " << fullname << endl;
+#endif
+          assert(grouptype == CCTK_ARRAY || grouptype == CCTK_SCALAR);
+          if (grouptype == CCTK_SCALAR) {
+            lb[0] = arrdata.at(group).at(Carpet::map).hh->processors.at(reflevel).at(component);
+            ub[0] = arrdata.at(group).at(Carpet::map).hh->processors.at(reflevel).at(component);
+            for(int i=1;i<dim;i++) {
+              lb[i]=0;
+              ub[i]=0;
+            }
+          } else {
+            const int newlb = lb[gpdim-1] +
+              (ub[gpdim-1]-lb[gpdim-1]+1)*
+              (arrdata.at(group).at(Carpet::map).hh->processors.at(reflevel).at(component));
+            const int newub = ub[gpdim-1] +
+              (ub[gpdim-1]-lb[gpdim-1]+1)*
+              (arrdata.at(group).at(Carpet::map).hh->processors.at(reflevel).at(component));
+            lb[gpdim-1] = newlb;
+            ub[gpdim-1] = newub;
+          }
+#ifdef CARPETIOHDF5_DEBUG
+          cout << "lb: " << lb << endl;
+          cout << "ub: " << ub << endl;
+#endif
+        }
+        const bbox<int,dim> ext(lb,ub,str);
+
+#ifdef CARPETIOHDF5_DEBUG
+        cout << "ext: " << ext << endl;
+#endif
+
+        if (CCTK_MyProc(cctkGH)==0) {
+          tmp->allocate (ext, 0, h5data);
+        } else {
+          tmp->allocate (ext, 0);
+        }
+
+        // Initialise with what is found in the file -- this does
+        // not guarantee that everything is initialised.
+        const bbox<int,dim> overlap = tmp->extent() & data->extent();
+        regions_read.at(Carpet::map) |= overlap;
+
+#ifdef CARPETIOHDF5_DEBUG
+        cout << "working on component: " << component << endl;
+        cout << "tmp->extent " << tmp->extent() << endl;
+        cout << "data->extent " << data->extent() << endl;
+        cout << "overlap " << overlap << endl;
+        cout << "-----------------------------------------------------" << endl;
+#endif
+
+        // FIXME: is this barrier really necessary ??
+        MPI_Barrier(MPI_COMM_WORLD);
+
+        // Copy into grid function
+        for (comm_state<dim> state; !state.done(); state.step()) {
+          data->copy_from (state, tmp, overlap);
+        }
+
+
+        // Delete temporary copy
+        delete tmp;
+
+
+      } END_COMPONENT_LOOP;
+
+      if (called_from_recovery) {
+        arrdata.at(group).at(Carpet::map).tt->set_time(reflevel,mglevel,
+                (CCTK_REAL) ((cctkGH->cctk_time - cctk_initial_time)
+                             / (delta_time * mglevelfact)) );
+      }
+
+
+    } END_MAP_LOOP;
+
+  } // if amr_level == reflevel
+
+  free (h5data);
+  free (fullname);
+
+  return did_read_something;
+}
+
+
+
+static int InputVarAs (const cGH* const cctkGH, const int vindex,
+                       const char* const alias)
+{
+  DECLARE_CCTK_PARAMETERS;
+
+  char *fullname = CCTK_FullName (vindex);
+  const int group = CCTK_GroupIndexFromVarI (vindex);
+  assert (group>=0 && group<(int)Carpet::arrdata.size());
+
+  int want_dataset = 0;
+  bool did_read_something = false;
+  int ndatasets = 0;
+  hid_t dataset = 0;
+
+  char datasetname[1024];
+
+  // Check for storage
+  if (! CCTK_QueryGroupStorageI(cctkGH, group)) {
+    CCTK_VWarn (1, __LINE__, __FILE__, CCTK_THORNSTRING,
+                "Cannot input variable \"%s\" because it has no storage",
+                fullname);
+    free (fullname);
+    return 0;
+  }
+
+  const int grouptype = CCTK_GroupTypeI(group);
+  const int rl = grouptype==CCTK_GF ? reflevel : 0;
+  //cout << "want level " << rl << " reflevel " << reflevel << endl;
+
+  // Find the input directory
+  const char* const myindir = in3D_dir;
+
+  // Invent a file name
+  ostringstream filenamebuf;
+  filenamebuf << myindir << "/" << alias << in3D_extension;
+  string filenamestr = filenamebuf.str();
+  const char * const filename = filenamestr.c_str();
+
+  hid_t reader = -1;
+
+  // Read the file only on the root processor
+  if (CCTK_MyProc(cctkGH)==0) {
+
+    // Open the file
+    if (verbose) CCTK_VInfo (CCTK_THORNSTRING, "Opening file \"%s\"", filename);
+    reader = H5Fopen (filename, H5F_ACC_RDONLY, H5P_DEFAULT);
+    if (reader<0) {
+      CCTK_VWarn (0, __LINE__, __FILE__, CCTK_THORNSTRING,
+                  "Could not open file \"%s\" for reading", filename);
+    }
+    assert (reader>=0);
+    // get the number of datasets in the file
+    ndatasets=GetnDatasets(reader);
+  }
+
+  vector<ibset> regions_read(Carpet::maps);
+
+  // Broadcast number of datasets
+  MPI_Bcast (&ndatasets, 1, MPI_INT, 0, dist::comm);
+  assert (ndatasets>=0);
+
+
+  for (int datasetid=0; datasetid<ndatasets; ++datasetid) {
+    if (verbose) CCTK_VInfo (CCTK_THORNSTRING, "Handling dataset #%d", datasetid);
+
+
+    // Read data
+    if (CCTK_MyProc(cctkGH)==0) {
+      GetDatasetName(reader,datasetid,datasetname);
+      //         cout << datasetname << "\n";
+
+      HDF5_ERROR (dataset = H5Dopen (reader, datasetname));
+    }
+
+
+#if 0
+    int amr_level;
+    int amr_origin[dim];
+    int amr_dims[dim];
+#endif
+
+    if (CCTK_MyProc(cctkGH)==0) {
+
+     // Read data
+     char * name;
+     ReadAttribute (dataset, "name", name);
+     //        cout << "dataset name is " << name << endl;
+     if (verbose && name) {
+       CCTK_VInfo (CCTK_THORNSTRING, "Dataset name is \"%s\"", name);
+     }
+     want_dataset = name && CCTK_EQUALS(name, fullname);
+     free (name);
+    } // myproc == 0
+
+    MPI_Bcast (&want_dataset, 1, MPI_INT, 0, dist::comm);
+
+    if(want_dataset) {
+      did_read_something = ReadVar(cctkGH,vindex,dataset,regions_read,0);
+    } // want_dataset
+
+  } // loop over datasets
+
+  // Close the file
+  if (CCTK_MyProc(cctkGH)==0) {
+    if (verbose) CCTK_VInfo (CCTK_THORNSTRING, "Closing file");
+    HDF5_ERROR (H5Fclose(reader));
+    reader=-1;
+  }
+
+  // Was everything initialised?
+  if (did_read_something) {
+    for (int m=0; m<Carpet::maps; ++m) {
+      dh<dim>& thedd = *arrdata.at(group).at(m).dd;
+      ibset all_exterior;
+      for (size_t c=0; c<thedd.boxes.at(rl).size(); ++c) {
+        all_exterior |= thedd.boxes.at(rl).at(c).at(mglevel).exterior;
+      }
+      if (regions_read.at(m) != all_exterior) {
+#ifdef CARPETIOHDF5_DEBUG
+        cout << "read: " << regions_read.at(m) << endl
+             << "want: " << all_exterior << endl;
+#endif
+        CCTK_VWarn (0, __LINE__, __FILE__, CCTK_THORNSTRING,
+                    "Variable \"%s\" could not be initialised from file -- the file may be missing data",
+                    fullname);
+      }
+    }
+  } // if did_read_something
+  //        CCTK_VWarn (0, __LINE__, __FILE__, CCTK_THORNSTRING,"stop!");
+
+  return did_read_something ? 0 : -1;
+
+}
+
+
+int CarpetIOHDF5ReadData (const cGH* const cctkGH)
+{
+  int retval = 0;
+  DECLARE_CCTK_PARAMETERS
+
+
+  int numvars = CCTK_NumVars ();
+  vector<bool> flags (numvars);
+
+  if (CCTK_TraverseString (in3D_vars, SetFlag, &flags, CCTK_GROUP_OR_VAR) < 0)
+  {
+    CCTK_VWarn (strict_io_parameter_check ? 0 : 1,
+                __LINE__, __FILE__, CCTK_THORNSTRING,
+                "error while parsing parameter 'IOHDF5::in3D_vars'");
+  }
+
+  for (int vindex = 0; vindex < numvars; vindex++)
+  {
+    if (flags.at (vindex))
+    {
+      retval = InputVarAs (cctkGH, vindex, CCTK_VarName (vindex));
+      if (retval)
+      {
+        break;
+      }
+    }
+  }
+
+  return (retval);
+}
+
+
+
+#if 0
+/** returns the number of recovered variables */
+int Recover (cGH* const cctkGH, const char *basefilename,
+             const int called_from)
+{
+  assert (cctkGH);
+  assert (basefilename);
+  assert (called_from == CP_INITIAL_DATA
+          || called_from == CP_EVOLUTION_DATA
+          || called_from == CP_RECOVER_PARAMETERS
+          || called_from == CP_RECOVER_DATA
+          || called_from == FILEREADER_DATA);
+
+  // the other modes are not supported yet
+  assert (called_from == FILEREADER_DATA);
+
+  const ioGH * const iogh = (const ioGH *) CCTK_GHExtension (cctkGH, "IO");
+  assert (iogh);
+
+  int num_vars_read = 0;
+  assert (iogh->do_inVars);
+  for (int n=0; n<CCTK_NumVars(); ++n) {
+    if (iogh->do_inVars[n]) {
+      const int ierr = InputVarAs (cctkGH, vindex, basefilename);
+      if (! ierr) {
+        ++ num_vars_read;
+      }
+    }
+  }
+
+  return num_vars_read;
+}
+#endif
+
+} // namespace CarpetIOHDF5
diff --git a/Carpet/CarpetIOHDF5/src/iohdf5.h b/Carpet/CarpetIOHDF5/src/iohdf5.h
new file mode 100644
index 000000000..17ea9d98a
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/src/iohdf5.h
@@ -0,0 +1,44 @@
+/* $Header: /home/eschnett/C/carpet/Carpet/Carpet/CarpetIOHDF5/src/iohdf5.h,v 1.7 2004/08/18 16:02:56 tradke Exp $ */
+
+#ifndef CARPETIOHDF5_H
+#define CARPETIOHDF5_H
+
+#ifdef __cplusplus
+namespace CarpetIOHDF5 {
+  extern "C" {
+#endif
+    
+#ifdef USE_CHRISTIANS_ROUTINES
+#include "cctk_Arguments.h"
+
+int CarpetIOHDF5Startup (void);
+void CarpetIOHDF5Init (CCTK_ARGUMENTS);
+void CarpetIOHDF5ReadData (CCTK_ARGUMENTS);
+void CarpetIOHDF5_EvolutionCheckpoint (const cGH*);
+void CarpetIOHDF5_InitialDataCheckpoint (const cGH*);
+
+#else
+
+/* Scheduled functions */
+void CarpetIOHDF5Startup (void);
+int CarpetIOHDF5Init (const cGH* const);
+int CarpetIOHDF5ReadData (const cGH* const);
+int CarpetIOHDF5_InitialDataCheckpoint (const cGH* const);
+int CarpetIOHDF5_EvolutionCheckpoint (const cGH* const);
+int CarpetIOHDF5_TerminationCheckpoint (const cGH* const);
+void CarpetIOHDF5EvolutionCheckpoint (const cGH* const);
+void CarpetIOHDF5InitialDataCheckpoint (const cGH* const);
+    
+#endif
+
+int CarpetIOHDF5_Recover (cGH* cgh, const char *basefilename, int called_from);
+
+int CarpetIOHDF5_RecoverParameters (void);
+int CarpetIOHDF5_CloseFile (void);
+
+#ifdef __cplusplus
+  } /* extern "C" */
+} /* namespace CarpetIOHDF5 */
+#endif
+
+#endif /* !defined(CARPETIOHDF5_H) */
diff --git a/Carpet/CarpetIOHDF5/src/iohdf5.hh b/Carpet/CarpetIOHDF5/src/iohdf5.hh
new file mode 100644
index 000000000..bfb49386b
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/src/iohdf5.hh
@@ -0,0 +1,198 @@
+// $Header: /home/eschnett/C/carpet/Carpet/Carpet/CarpetIOHDF5/src/iohdf5.hh,v 1.12 2004/07/09 15:38:18 tradke Exp $
+
+#ifndef CARPETIOHDF5_HH
+#define CARPETIOHDF5_HH
+
+#include <vector>
+
+#include "cctk.h"
+#include "cctk_Arguments.h"
+
+#include "carpet.hh"
+
+#include "iohdf5.h"
+#include "CactusBase/IOUtil/src/ioutil_Utils.h"
+
+/* some macros for HDF5 group names */
+#define METADATA_GROUP "Parameters and Global Attributes"
+#define ALL_PARAMETERS "All Parameters"
+
+// Some MPI Datatypes we need for Recovery
+// Originally written by Thomas Radke.
+
+#ifdef  CCTK_INT4
+#define CARPET_MPI_INT4  (sizeof (CCTK_INT4) == sizeof (int) ? MPI_INT :        \
+                        sizeof (CCTK_INT4) == sizeof (short) ? MPI_SHORT :    \
+                        MPI_DATATYPE_NULL)
+#endif
+
+#define CARPET_MPI_CHAR      MPI_CHAR
+
+/* floating point types are architecture-independent,
+   ie. a float has always 4 bytes, and a double has 8 bytes
+
+   PUGH_MPI_REAL  is used for communicating reals of the generic CCTK_REAL type
+   PUGH_MPI_REALn is used to explicitely communicate n-byte reals */
+#ifdef  CCTK_REAL4
+#define CARPET_MPI_REAL4  MPI_FLOAT
+#endif
+#ifdef  CCTK_REAL8
+#define CARPET_MPI_REAL8  MPI_DOUBLE
+#endif
+#ifdef  CCTK_REAL16
+#define CARPET_MPI_REAL16  (sizeof (CCTK_REAL16) == sizeof (long double) ?      \
+                          MPI_LONG_DOUBLE : MPI_DATATYPE_NULL)
+#endif
+
+
+#ifdef  CCTK_REAL_PRECISION_16
+#define CARPET_MPI_REAL   CARPET_MPI_REAL16
+#elif   CCTK_REAL_PRECISION_8
+#define CARPET_MPI_REAL   CARPET_MPI_REAL8
+#elif   CCTK_REAL_PRECISION_4
+#define CARPET_MPI_REAL   CARPET_MPI_REAL4
+#endif
+
+
+/*** Define the different datatypes used for HDF5 I/O
+     NOTE: the complex datatype SHOULD be [is] defined dynamically at runtime in Startup.c
+     100% of the definitions below were taken from Thomas Radke's IOHDF5Util thorn for PUGH
+ ***/
+/* char type is easy */
+#define HDF5_CHAR   H5T_NATIVE_CHAR
+
+/* floating point types are architecture-independent,
+   ie. a float has always 4 bytes, and a double has 8 bytes
+     HDF5_REAL  is used for storing reals of the generic CCTK_REAL type
+     HDF5_REALn is used to explicitely store n-byte reals */
+#ifdef  CCTK_REAL4
+#define HDF5_REAL4  H5T_NATIVE_FLOAT
+#endif
+#ifdef  CCTK_REAL8
+#define HDF5_REAL8  H5T_NATIVE_DOUBLE
+#endif
+#ifdef  CCTK_REAL16
+#define HDF5_REAL16 (sizeof (CCTK_REAL16) == sizeof (long double) ?           \
+                     H5T_NATIVE_LDOUBLE : -1)
+#endif
+
+
+#ifdef  CCTK_REAL_PRECISION_16
+#define HDF5_REAL   HDF5_REAL16
+#elif   CCTK_REAL_PRECISION_8
+#define HDF5_REAL   HDF5_REAL8
+#elif   CCTK_REAL_PRECISION_4
+#define HDF5_REAL   HDF5_REAL4
+#endif
+
+
+/* integer types are architecture-dependent:
+     HDF5_INT  is used for communicating integers of the generic CCTK_INT type
+     HDF5_INTn is used to explicitely communicate n-byte integers */
+#ifdef  CCTK_INT8
+#define HDF5_INT8   (sizeof (CCTK_INT8) == sizeof (int) ? H5T_NATIVE_INT :    \
+                     sizeof (CCTK_INT8) == sizeof (long) ? H5T_NATIVE_LONG :  \
+                     sizeof (CCTK_INT8) == sizeof (long long) ?               \
+                     H5T_NATIVE_LLONG : -1)
+#endif
+
+#ifdef  CCTK_INT4
+#define HDF5_INT4   (sizeof (CCTK_INT4) == sizeof (int) ? H5T_NATIVE_INT :    \
+                     sizeof (CCTK_INT4) == sizeof (short) ?                   \
+                     H5T_NATIVE_SHORT : -1)
+#endif
+
+#ifdef  CCTK_INT2
+#define HDF5_INT2   (sizeof (CCTK_INT2) == sizeof (short) ?                   \
+                     H5T_NATIVE_SHORT : -1)
+#endif
+
+#ifdef  CCTK_INT1
+#define HDF5_INT1   H5T_NATIVE_CHAR
+#endif
+
+#ifdef  CCTK_INTEGER_PRECISION_8
+#define HDF5_INT    HDF5_INT8
+#elif   CCTK_INTEGER_PRECISION_4
+#define HDF5_INT    HDF5_INT4
+#elif   CCTK_INTEGER_PRECISION_2
+#define HDF5_INT    HDF5_INT2
+#elif   CCTK_INTEGER_PRECISION_1
+#define HDF5_INT    HDF5_INT1
+#endif
+
+/* Nice error handling. Stolen from Thomas Radke */
+/* check return code of HDF5 call and print a warning in case of an error */
+#define HDF5_ERROR(fn_call)                                                   \
+        {                                                                     \
+          int _error_code = fn_call;                                          \
+                                                                              \
+                                                                              \
+          if (_error_code < 0)                                                \
+          {                                                                   \
+            CCTK_VWarn (1, __LINE__, __FILE__, CCTK_THORNSTRING,              \
+                        "HDF5 call '%s' returned error code %d",              \
+                        #fn_call, _error_code);                               \
+          }                                                                   \
+        }
+
+
+
+
+namespace CarpetIOHDF5 {
+  
+  using namespace std;
+  using namespace Carpet;
+  
+  // Variable definitions
+  extern vector<bool> do_truncate; // [var]
+  extern vector<vector<vector<int> > > last_output; // [ml][rl][var]
+  
+  void* SetupGH (tFleshConfig* const fc,
+		 const int convLevel, cGH* const cctkGH);
+  
+  int OutputGH (const cGH* const cctkGH);
+  int WriteVar (const cGH* const cctkGH, const hid_t writer, const ioRequest* request,
+		   const int called_from_checkpoint);
+  int OutputVarAs (const cGH* const cctkGH, const char* const varname,
+		   const char* const alias);
+
+  int TimeToOutput (const cGH* const cctkGH, const int vindex);
+  int TriggerOutput (const cGH* const cctkGH, const int vindex);
+  
+  int InputGH (const cGH* const cctkGH);
+  int ReadVar (const cGH* const cctkGH, const int vindex,
+	       const hid_t currdataset, vector<ibset> &regions_read, 
+	       const int called_from_recovery);
+
+  int Recover (cGH* cgh, const char *basefilename, int called_from);
+
+  // auxiliary functions defined in iohdf5utils.cc
+
+  bool CheckForVariable (const char* const varlist, const int vindex);
+  void SetFlag (int index, const char* optstring, void* arg);
+  
+  void WriteAttribute (const hid_t dataset, const char* name, int value);
+  void WriteAttribute (const hid_t dataset, const char* name, const int* values, int nvalues);
+  void WriteAttribute (const hid_t dataset, const char* name, double value);
+  void WriteAttribute (const hid_t dataset, const char* name, const double* values, int nvalues);
+  void WriteAttribute (const hid_t dataset, const char* name, char value);
+  void WriteAttribute (const hid_t dataset, const char* name, const char* values);
+  void WriteAttribute (const hid_t dataset, const char* name, const char* values, int nvalues);
+  
+  int ReadAttribute (const hid_t dataset, const char* name, int& value);
+  int ReadAttribute (const hid_t dataset, const char* name, int* values, int nvalues);
+  int ReadAttribute (const hid_t dataset, const char* name, double& value);
+  int ReadAttribute (const hid_t dataset, const char* name, double* values, int nvalues);
+  int ReadAttribute (const hid_t dataset, const char* name, char& value);
+  int ReadAttribute (const hid_t dataset, const char* name, char*& values);
+  int ReadAttribute (const hid_t dataset, const char* name, char* values, int nvalues);
+  
+  int GetnDatasets (const hid_t reader);
+  void GetDatasetName (const hid_t reader, const int _index, char* name);
+
+  hid_t h5DataType(const cGH* const cctkGH, int cctk_type);
+
+} // namespace CarpetIOHDF5
+
+#endif // !defined(CARPETIOHDF5_HH)
diff --git a/Carpet/CarpetIOHDF5/src/iohdf5GH.h b/Carpet/CarpetIOHDF5/src/iohdf5GH.h
new file mode 100644
index 000000000..0516201c2
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/src/iohdf5GH.h
@@ -0,0 +1,56 @@
+// This was adopted from Thomas Radke's IOHDF5 thorn.
+// Thanks, Thomas!
+
+#ifndef _CARPETIOHDF5_IOHDF5GH_H_
+#define _CARPETIOHDF5_IOHDF5GH_H_ 1
+
+#include "StoreNamedData.h"
+
+/* I took basically everything in this file from Thomas' IOHDF5; much of
+   below is still unused.. */
+
+/* CARPET IOHDF5 GH extension structure */
+typedef struct
+{
+  /* default number of times to output */
+  int out_every_default;
+
+  /* number of times to output for each variable */
+  CCTK_INT *out_every;
+
+  /* the last iteration output for each variable */
+  int *out_last;
+
+  /* list of variables to output */
+  char *out_vars;
+
+  /* I/O request description list (for all variables) */
+  ioRequest **requests;
+
+  /* directory in which to output */
+  char *out_dir;
+
+  /* filename database for opened files */
+  pNamedData *open_output_files;
+
+  /* timer array for checkpointing/recovery */
+  // int timers[IOHDF5_NUM_TIMERS];
+
+  /* flag to indicate request for timer output */
+  // int print_timing_info;
+
+  /* ring buffer for list of successfully created cp files */
+  int    cp_filename_index;
+  char **cp_filename_list;
+
+  /* iteration number of the last checkpoint */
+  int last_checkpoint_iteration;
+
+  /* hdf5 datatype for stupid complex variables; to be set at run time */
+  hid_t HDF5_COMPLEX, HDF5_COMPLEX8, HDF5_COMPLEX16, HDF5_COMPLEX32;
+  
+
+} CarpetIOHDF5GH;
+
+
+#endif  
diff --git a/Carpet/CarpetIOHDF5/src/iohdf5chckpt_recover.cc b/Carpet/CarpetIOHDF5/src/iohdf5chckpt_recover.cc
new file mode 100644
index 000000000..f8e6b9a3d
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/src/iohdf5chckpt_recover.cc
@@ -0,0 +1,914 @@
+#include <assert.h>
+#include <limits.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+
+#include <algorithm>
+#include <fstream>
+#include <sstream>
+#include <string>
+#include <vector>
+
+#include <hdf5.h>
+
+#include "cctk.h"
+#include "cctk_Parameters.h"
+#include "cctk_Version.h"
+
+extern "C" {
+  static const char* rcsid = "$Header: /home/eschnett/C/carpet/Carpet/Carpet/CarpetIOHDF5/src/iohdf5chckpt_recover.cc,v 1.36 2004/07/07 11:01:05 tradke Exp $";
+  CCTK_FILEVERSION(Carpet_CarpetIOHDF5_iohdf5chckpt_recover_cc);
+}
+
+#include "CactusBase/IOUtil/src/ioGH.h"
+#include "CactusBase/IOUtil/src/ioutil_CheckpointRecovery.h"
+
+#include "bbox.hh"
+#include "data.hh"
+#include "gdata.hh"
+#include "ggf.hh"
+#include "vect.hh"
+
+#include "carpet.hh"
+
+#include "iohdf5.hh"
+#include "iohdf5GH.h"
+
+/* some macros for HDF5 group names */
+#define PARAMETERS_GLOBAL_ATTRIBUTES_GROUP "Parameters and Global Attributes"
+#define ALL_PARAMETERS "All Parameters"
+
+
+
+namespace CarpetIOHDF5 {
+  
+  using namespace std;
+  using namespace Carpet;
+
+  // linked list for reading in the checkpoint file
+
+  list<string> datasetnamelist;
+  
+  int Checkpoint (const cGH* const cctkGH, int called_from);
+  int DumpParametersGHExtentions (const cGH *cctkGH, int all, hid_t writer);  
+  
+  int RecoverParameters (hid_t reader);
+  int RecoverGHextensions (cGH* cctkGH, hid_t reader);
+  int RecoverVariables (cGH* cctkGH, hid_t reader);
+
+  void CarpetIOHDF5InitialDataCheckpoint( const cGH* const cgh){
+    
+    DECLARE_CCTK_PARAMETERS;
+    
+    if (checkpoint && checkpoint_ID)
+      {
+	CCTK_INFO ("---------------------------------------------------------");
+	CCTK_INFO ("Dumping initial data checkpoint");
+	CCTK_INFO ("---------------------------------------------------------");
+	
+	Checkpoint (cgh, CP_INITIAL_DATA);
+      }
+  } // CarpetIOHDF5_InitialDataCheckpoint
+
+
+  void CarpetIOHDF5EvolutionCheckpoint( const cGH* const cgh){
+    
+    DECLARE_CCTK_PARAMETERS;
+
+    // Test checkpoint_every, and adjust it. This is necessary until 
+    // recovery is more flexible.
+    int every_full = pow(2.0,maxreflevels-1);
+    if (checkpoint_every>0 && (checkpoint_every % every_full) != 0) {
+      int every = (checkpoint_every / every_full + 1) * every_full;
+      ostringstream outevery;
+      outevery << every;
+      CCTK_ParameterSet("checkpoint_every","IOUtil",
+			outevery.str().c_str());
+      CCTK_VInfo (CCTK_THORNSTRING,"I have adjusted your checkpoint_every to %i.",every);
+      
+    }
+
+      if (checkpoint &&
+      ((checkpoint_every > 0 && cgh->cctk_iteration % checkpoint_every == 0) ||
+       checkpoint_next))
+      {
+	CCTK_INFO ("---------------------------------------------------------");
+	CCTK_VInfo (CCTK_THORNSTRING, "Dumping periodic checkpoint at "
+		    "iteration %d", cgh->cctk_iteration);
+	CCTK_INFO ("---------------------------------------------------------");
+
+	Checkpoint (cgh, CP_EVOLUTION_DATA);
+
+       	if (checkpoint_next)
+	{
+	  CCTK_ParameterSet ("checkpoint_next", CCTK_THORNSTRING, "no");
+	}
+      }
+  } // CarpetIOHDF5_EvolutionCheckpoint
+
+
+  int CarpetIOHDF5RecoverParameters(void){
+    // Register with the Cactus Recovery Interface
+    return (IOUtil_RecoverParameters (CarpetIOHDF5_Recover, ".h5", "HDF5"));
+  }
+
+  int CarpetIOHDF5_Recover (cGH* cctkGH, const char *basefilename, int called_from) {
+    
+    DECLARE_CCTK_PARAMETERS;
+
+    int result=0;
+    int myproc=-1;
+
+    CarpetIOHDF5GH *myGH;
+
+    
+    static hid_t reader=0; //this thing absolutely needs to be static!!!
+
+    myGH = NULL;
+
+    myproc = CCTK_MyProc (cctkGH);
+    
+  
+    if (called_from == CP_RECOVER_PARAMETERS) {
+      // Okay, let's see what we can do about the parameters
+    
+       // Invent a file name
+      ostringstream filenamebuf;
+
+      //      if(CCTK_nProcs(cctkGH) == 1)
+      filenamebuf << recover_dir << "/" << basefilename << ".h5";
+      //else
+      //	filenamebuf << recover_dir << "/" << basefilename << ".file_0.h5";
+
+      string filenamestr = filenamebuf.str();
+      const char * const filename = filenamestr.c_str();
+
+      if (myproc == 0) {
+	// First, open the file
+	if (verbose) 
+	  CCTK_VInfo(CCTK_THORNSTRING, "Opening Checkpoint file %s for recovery",filename);
+	reader = H5Fopen (filename, H5F_ACC_RDONLY, H5P_DEFAULT);
+	if (reader<0) {
+	  CCTK_VWarn (0, __LINE__, __FILE__, CCTK_THORNSTRING,
+		    "Could not recover from \"%s\"", filename);
+	}
+      } // myproc == 0
+    }
+    else {
+	/* This is the case for CP_RECOVER_DATA.
+	   CCTK_RECOVER_PARAMETERS must have been called before
+	   and set up the file info structure. */
+	if (myproc == 0) {
+	  assert(reader>=0);
+	}
+    }
+  
+    if (called_from == CP_RECOVER_PARAMETERS)
+      {
+	return (RecoverParameters (reader));
+      }
+  
+    if (called_from == CP_RECOVER_DATA) {
+      CCTK_INT4 numberofmgtimes=0;
+
+      CCTK_VInfo(CCTK_THORNSTRING,"Starting to recover data on reflevel %d!!!",reflevel);
+
+      if (myproc == 0) {
+	
+	// Use refinement levels parameter from checkpointing file?
+	if (use_reflevels_from_checkpoint && reflevel==0) {
+	  
+	  herr_t herr;
+	  
+	  hid_t group = H5Gopen (reader, PARAMETERS_GLOBAL_ATTRIBUTES_GROUP);
+	  assert(group >= 0);
+	  hid_t dataset = H5Dopen (group, ALL_PARAMETERS);
+	  assert(dataset>= 0);
+	  
+	  int reflevels_chkpt;
+	
+	  ReadAttribute (dataset, "carpet_reflevels", reflevels_chkpt);
+      
+	  herr = H5Dclose(dataset);
+	  assert(!herr);
+	  herr = H5Gclose(group);
+	  assert(!herr);
+      
+	  ostringstream reflevels_str;
+	  reflevels_str << reflevels_chkpt;
+	  CCTK_ParameterSet ("refinement_levels","CarpetRegrid",reflevels_str.str().c_str());
+	  	 
+	  CCTK_VInfo (CCTK_THORNSTRING, "Using %i reflevels read from checkpoint file. Ignoring value in parameter file.", reflevels_chkpt);
+	  
+	}     
+	
+	/* we need all the times on the individual levels */
+	// these are a bit messy to extract
+
+	// Actually, we do have to do this only once
+
+	  hid_t group = H5Gopen (reader, PARAMETERS_GLOBAL_ATTRIBUTES_GROUP);
+	  assert(group>=0);
+	  hid_t dataset = H5Dopen (group, ALL_PARAMETERS);
+	  assert(dataset >= 0);
+	  hid_t attr = H5Aopen_name (dataset, "numberofmgtimes");
+	  assert(attr >= 0);
+	  hid_t atype = H5Aget_type (attr);
+	  if(H5Tequal(atype, H5T_NATIVE_INT)) {
+	    herr_t herr = H5Aread(attr, atype, &numberofmgtimes);
+	    assert(!herr);
+	    herr = H5Aclose(attr);
+	    assert(numberofmgtimes==mglevels);
+	    char buffer[100];
+	    for(int lcv=0;lcv<numberofmgtimes;lcv++) {
+	      sprintf(buffer,"mgleveltimes %d",lcv);
+	      attr = H5Aopen_name(dataset, buffer);
+	      assert (attr>=0);
+	      atype = H5Aget_type (attr);
+	      assert (atype>=0);
+	      herr = H5Aread (attr, atype, &leveltimes.at(lcv).at(0));
+	      assert(!herr);
+	      herr = H5Aclose(attr);
+	      assert(!herr);
+	    }  
+	    herr = H5Dclose(dataset);
+	    assert(!herr);
+	    herr = H5Gclose(group);
+	    assert(!herr);
+	  } else {
+	    CCTK_WARN(0,"BAD BAD BAD! Can't read leveltimes!!");
+	  }
+
+      } // myproc == 0  	
+
+
+      // communicate the time on all the mglevels and reflevels
+
+      int mpierr = MPI_Bcast (&numberofmgtimes, 1, CARPET_MPI_INT4, 0,MPI_COMM_WORLD);
+      assert(!mpierr);
+
+
+      for(int i=0;i<numberofmgtimes;i++) {
+	mpierr = MPI_Bcast (&(leveltimes.at(i).at(0)), reflevels, CARPET_MPI_REAL, 0, MPI_COMM_WORLD);
+	assert(!mpierr);
+      }
+
+      if (verbose) {
+	cout << "leveltimes: " << leveltimes << endl;
+      }
+
+      cctkGH->cctk_time = leveltimes.at(mglevel).at(reflevel);
+
+      result += RecoverGHextensions(cctkGH,reader);
+
+	  
+      if (verbose) {
+	cout << "reflevel: " << reflevel << endl;
+      }
+
+      result += RecoverVariables (cctkGH,reader);
+
+
+      CCTK_VInfo (CCTK_THORNSTRING,
+		  "Restarting simulation at iteration %d (physical time %g)",
+		  cctkGH->cctk_iteration, (double) cctkGH->cctk_time);
+
+    } // called_from == CP_RECOVER_DATA
+  
+    if (reflevel==maxreflevels) {	
+      if(myproc == 0) {
+	H5Fclose(reader);
+      }
+    }
+
+    return (result);
+  } // CarpetIOHDF5_Recover
+
+  
+  int RecoverVariables (cGH* cctkGH, hid_t reader) {
+
+    DECLARE_CCTK_PARAMETERS;
+
+    int retval = 0;
+    int myproc = CCTK_MyProc (cctkGH);
+    char * name;
+
+    int ndatasets;
+
+    int varindex; 
+
+    double datasettime;
+    double leveltime;
+    static double totaltime;
+
+    hid_t dataset;
+    herr_t herr;
+
+    list<string> refleveldatasetnamelist;
+
+    if (reflevel==0) {
+      totaltime = 0;
+    }
+
+    leveltime = MPI_Wtime();
+
+
+    if(myproc==0) {
+      ndatasets=GetnDatasets(reader);
+      assert (ndatasets>=0);
+    }
+
+    // Broadcast number of datasets
+    MPI_Bcast (&ndatasets, 1, MPI_INT, 0, dist::comm);
+
+    assert ((ndatasets)>=0);
+    
+    //if (verbose && reflevel==0) cout << "ndatasets: " << ndatasets << endl;
+    if ( reflevel == 0) {
+      cout << "ndatasets: " << ndatasets << endl;
+    }
+
+    if (reflevel==0) {
+      for (int currdataset=0;currdataset<ndatasets+1;currdataset++){
+	char datasetname[256];
+	if (myproc==0) {
+	  GetDatasetName(reader,currdataset,datasetname);
+	  datasetnamelist.push_back(datasetname);
+	} //myproc = 0
+	else {
+	  datasetnamelist.push_back("blah");
+	}
+      }
+    } 
+
+    cout << "I have " << datasetnamelist.size() << endl;
+
+    double comparetime = MPI_Wtime();
+      
+    if(myproc==0) {
+      //      for(currdataset=datasetnamelist.begin();
+      //	  currdataset!=datasetnamelist.end();
+      //	  ++currdataset) {
+
+      list<string>::iterator currdataset;
+      currdataset=datasetnamelist.begin();
+       while(currdataset!=datasetnamelist.end()) {
+	char tempstr[10];
+	sprintf(tempstr,"rl=%d m=",reflevel);
+	if ( (*currdataset).find(tempstr) < (*currdataset).size() ) {
+	  refleveldatasetnamelist.push_back((*currdataset).c_str());
+	  currdataset = datasetnamelist.erase(currdataset);
+	} else {
+	  ++currdataset;
+	} // if ...
+      } // while ...
+    } // if(myproc==0)
+
+
+
+   
+    static long reflevelnamenum; // static keeps this thing local
+
+    if(myproc==0) {
+	reflevelnamenum=refleveldatasetnamelist.size();
+      }
+
+    MPI_Bcast (&reflevelnamenum, 1, MPI_LONG, 0, dist::comm);
+    assert ((reflevelnamenum)>=0);
+    
+    // fill bogus namelist for non-IO cpus
+    if(myproc !=0) {
+      for(long i=0;i < reflevelnamenum;i++) {
+	refleveldatasetnamelist.push_back("blah");
+      }
+    }
+
+    comparetime = MPI_Wtime() - comparetime;
+    cout << "Time for string comparison: " << comparetime << endl;
+    cout << "I have for this reflevel " << refleveldatasetnamelist.size() << endl;
+
+    list<string>::iterator currdataset;
+
+      currdataset=refleveldatasetnamelist.begin();
+ 
+      while(currdataset!=refleveldatasetnamelist.end()) {
+
+	//	cout << "name: " << (*currdataset).c_str() << endl;
+
+      if (myproc==0) {
+	dataset = H5Dopen (reader, (*currdataset).c_str());
+	assert(dataset);
+	// Read data
+	ReadAttribute (dataset, "name", name);
+	varindex = CCTK_VarIndex(name);
+      }
+
+      MPI_Bcast (&varindex, 1, MPI_INT, 0, dist::comm);
+
+      name = CCTK_FullName(varindex);
+
+      if (verbose) {
+	cout << name << "  rl: " << reflevel << endl;
+      }
+      vector<ibset> regions_read(Carpet::maps);
+
+      assert (varindex>=0 && varindex<CCTK_NumVars());
+      const int group = CCTK_GroupIndexFromVarI (varindex);
+      const int grouptype = CCTK_GroupTypeI(group);
+
+      int did_read_something = ReadVar(cctkGH,name,dataset,regions_read,1);
+
+      MPI_Bcast (&did_read_something, 1, MPI_INT, 0, dist::comm);
+
+      if (did_read_something) {
+      	currdataset = refleveldatasetnamelist.erase(currdataset);
+      } else {
+	++currdataset;
+      }
+
+      if(myproc==0) {
+	herr = H5Dclose(dataset);
+	assert(!herr);
+      }
+      free(name);
+
+    }  // for (currdataset ... )
+    leveltime = MPI_Wtime() - leveltime;
+    totaltime = totaltime + leveltime;
+
+    cout << "Timers: leveltime: " << leveltime << " totaltime: " << totaltime << endl; 
+
+    return retval;
+  }
+   
+  
+  int RecoverGHextensions (cGH *cctkGH, hid_t reader)
+  {
+    const int myproc = CCTK_MyProc(cctkGH);
+    CCTK_INT4 int4Buffer[3];
+    CCTK_REAL realBuffer;
+    CCTK_REAL realBuffer2;
+    CCTK_INT4 intbuffer;
+
+    int mpierr = 0;
+
+    if (myproc==0)
+      {
+		
+	// First open group and dataset
+	hid_t group = H5Gopen (reader, PARAMETERS_GLOBAL_ATTRIBUTES_GROUP);
+	assert(group>=0);
+	hid_t dataset = H5Dopen (group, ALL_PARAMETERS);
+	assert(dataset >= 0);
+
+	ReadAttribute(dataset,"GH$iteration",int4Buffer[0]);
+	ReadAttribute(dataset,"main loop index",int4Buffer[1]);
+	ReadAttribute(dataset,"carpet_global_time",realBuffer);
+	//	ReadAttribute(dataset,"carpet_reflevels",int4Buffer[2]);
+	ReadAttribute(dataset,"carpet_delta_time",realBuffer2);
+
+	herr_t herr = H5Dclose(dataset);
+	assert(!herr);
+	herr = H5Gclose(group);
+	assert(!herr);
+	
+      }
+  /* Broadcast the GH extensions to all processors */
+  /* NOTE: We have to use MPI_COMM_WORLD here
+     because PUGH_COMM_WORLD is not yet set up at parameter recovery time.
+     We also assume that PUGH_MPI_INT4 is a compile-time defined datatype. */
+
+    mpierr = MPI_Bcast (int4Buffer, 3, CARPET_MPI_INT4, 0,MPI_COMM_WORLD);
+    assert(!mpierr);
+    mpierr = MPI_Bcast (int4Buffer, 3, CARPET_MPI_INT4, 0,MPI_COMM_WORLD);
+    assert(!mpierr);
+    mpierr = MPI_Bcast (&realBuffer, 1, CARPET_MPI_REAL,0,MPI_COMM_WORLD);
+    assert(!mpierr);
+    mpierr = MPI_Bcast (&realBuffer2, 1, CARPET_MPI_REAL,0,MPI_COMM_WORLD);
+    assert(!mpierr);
+
+    global_time = (CCTK_REAL) realBuffer;
+    delta_time = (CCTK_REAL) realBuffer2;
+    //    reflevels = (int) int4Buffer[2];
+    cctkGH->cctk_iteration = (int) int4Buffer[0];
+    CCTK_SetMainLoopIndex ((int) int4Buffer[1]);
+
+
+    return (0);
+
+  } // RecoverGHExtensions
+
+  int RecoverParameters(hid_t reader){
+
+    DECLARE_CCTK_PARAMETERS; 
+    
+    int myproc, retval;
+    char *parameters;
+    CCTK_INT4 parameterSize;
+
+    hid_t group,dataset;
+    herr_t herr;
+
+    int mpierr;
+
+    myproc = CCTK_MyProc (NULL);
+
+    if (myproc == 0){
+      CCTK_VInfo (CCTK_THORNSTRING, "Recovering parameters from checkpoint ");
+      
+      group = H5Gopen (reader, PARAMETERS_GLOBAL_ATTRIBUTES_GROUP);
+      assert(group >= 0);
+      dataset = H5Dopen (group, ALL_PARAMETERS);
+      assert(dataset>= 0);
+
+      parameterSize = H5Dget_storage_size (dataset);
+      parameters = (char *) malloc (parameterSize);
+      herr = H5Dread (dataset, H5T_NATIVE_CHAR, H5S_ALL, H5S_ALL,
+                             H5P_DEFAULT, parameters);
+      assert(!herr);
+      herr = H5Dclose(dataset);
+      assert(!herr);
+      herr = H5Gclose(group);
+      assert(!herr);
+
+      if(verbose) {
+	CCTK_VInfo (CCTK_THORNSTRING, "\n%s\n",parameters);
+      }
+      
+      CCTK_VInfo(CCTK_THORNSTRING, "Successfully recovered parameters!");
+    } // myproc == 0
+
+    /* Broadcast the parameter buffer size to all processors */
+    /* NOTE: We have to use MPI_COMM_WORLD here
+       because CARPET_COMM_WORLD is not yet set up at parameter recovery time.
+       We also assume that CARPET_MPI_INT4 is a compile-time defined datatype. */
+    mpierr = MPI_Bcast (&parameterSize, 1, CARPET_MPI_INT4, 0,
+			MPI_COMM_WORLD);
+    assert(!mpierr);
+
+    if (parameterSize > 0) {
+      if (myproc) {
+	parameters = (char*) malloc (parameterSize + 1);
+      }
+	
+      mpierr = MPI_Bcast (parameters, parameterSize + 1, CARPET_MPI_CHAR,
+			  0,MPI_COMM_WORLD);
+      assert(!mpierr);
+      
+      IOUtil_SetAllParameters (parameters);
+    
+      free (parameters);
+    }
+  
+    /* return positive value for success otherwise negative */
+    retval = (parameterSize > 0 ? 1 : -1);
+
+    return (retval);
+
+  } // RecoverParameters
+
+
+
+  int Checkpoint (const cGH* const cctkGH, int called_from)
+  {
+    DECLARE_CCTK_ARGUMENTS;
+    DECLARE_CCTK_PARAMETERS;
+
+    herr_t herr;
+    
+    int retval = 0;
+
+    const ioGH *ioUtilGH;
+    
+    ioRequest *request;
+
+    CarpetIOHDF5GH *myGH;
+    myGH = (CarpetIOHDF5GH *) CCTK_GHExtension (cctkGH, "CarpetIOHDF5");
+
+    // check if CarpetIOHDF5 was registered as I/O method
+    if (myGH == NULL) {
+      CCTK_WARN (0, "No CarpetIOHDF5 I/O methods registered");
+      return -1;
+    }
+    
+    int const myproc = CCTK_MyProc (cctkGH);
+    
+    
+    
+    // Invent a file name
+    ostringstream cp_tempname_buf;
+    ostringstream cp_filename_buf;
+    
+    switch (called_from)
+    {
+    case CP_INITIAL_DATA:
+      cp_tempname_buf << checkpoint_dir << "/tmp_" << checkpoint_ID_file << ".it_" << cctkGH->cctk_iteration << ".h5";
+      cp_filename_buf << checkpoint_dir << "/" << checkpoint_ID_file << ".it_" << cctkGH->cctk_iteration << ".h5";
+      break;
+      
+    case CP_EVOLUTION_DATA:
+      cp_tempname_buf << checkpoint_dir << "/tmp_" << checkpoint_file << ".it_" << cctkGH->cctk_iteration << ".h5";
+      cp_filename_buf << checkpoint_dir << "/" << checkpoint_file << ".it_" << cctkGH->cctk_iteration << ".h5";
+      break;
+      
+    case CP_RECOVER_DATA:
+    case CP_RECOVER_PARAMETERS:
+    case FILEREADER_DATA:
+      cp_tempname_buf << recover_dir << "/tmp_" << recover_file << ".h5";
+      cp_filename_buf << recover_dir << "/" << recover_file << ".h5";
+      break;
+      
+    default:
+      CCTK_VWarn (0, __LINE__, __FILE__, CCTK_THORNSTRING,
+                  "IOUtil_PrepareFilename: Unknown calling mode %d",
+                  called_from);
+      break;
+    }
+    
+    string const cp_tempname_str = cp_tempname_buf.str();
+    string const cp_filename_str = cp_filename_buf.str();
+    char const * const cp_tempname = cp_tempname_str.c_str();
+    char const * const cp_filename = cp_filename_str.c_str();
+    
+    
+    
+    hid_t writer = -1;
+
+    if (myproc == 0) {
+
+      if (verbose) {
+	CCTK_VInfo (CCTK_THORNSTRING, "Creating temporary checkpoint file '%s'", cp_tempname);
+      }
+
+      CCTK_VInfo (CCTK_THORNSTRING, "Creating temporary checkpoint file '%s'", cp_tempname);
+      
+      CCTK_VInfo (CCTK_THORNSTRING, "Verbose = %d", verbose);
+
+      writer = H5Fcreate (cp_tempname, H5F_ACC_TRUNC, H5P_DEFAULT, H5P_DEFAULT);
+      assert (writer>=0);
+      herr = H5Fclose (writer);
+      assert (!herr);
+      writer = -1;
+
+      // Now open the file
+
+      writer = H5Fopen (cp_tempname, H5F_ACC_RDWR, H5P_DEFAULT);
+      assert (writer>=0);
+
+      // Dump all parameters and GHExtentions
+      retval += DumpParametersGHExtentions (cctkGH, 1, writer);
+      assert(!retval);
+
+    } // myproc == 0 
+
+
+    // now dump the grid variables on all mglevels, reflevels, maps and components
+    BEGIN_MGLEVEL_LOOP(cctkGH) {
+
+      BEGIN_REFLEVEL_LOOP(cctkGH) {
+
+	if (verbose)
+	  {
+	    CCTK_INFO ("Dumping Grid Variables ...");
+	  }
+	for (int group = CCTK_NumGroups () - 1; group >= 0; group--)
+	  {
+	    /* only dump groups which have storage assigned */
+
+	    if (CCTK_QueryGroupStorageI (cctkGH, group) <= 0)
+	      {
+		continue;
+	      }
+
+	    const int grouptype = CCTK_GroupTypeI(group);
+
+	    /* scalars and grid arrays only have 1 reflevel: */
+	    if ( (grouptype != CCTK_GF) && (reflevel != 0) ) {
+	      continue;
+	    }
+	    /* now check if there is any memory allocated
+	       for GFs and GAs. GSs should always have 
+	       memory allocated and there is at this point
+	       no CCTK function to check this :/
+	    */
+
+	    if ( (grouptype == CCTK_GF) || (grouptype == CCTK_ARRAY)){
+	      const int gpdim = CCTK_GroupDimI(group);
+	      int gtotalsize=1;
+	      vector<int> tlsh(gpdim);
+	      assert(!CCTK_GrouplshGI(cctkGH,gpdim,&tlsh[0],group));
+	      for(int i=0;i<gpdim;i++) {
+		gtotalsize=tlsh[i];		  
+	      }
+	      if(gtotalsize == 0){
+		if (verbose) CCTK_VInfo(CCTK_THORNSTRING, 
+		    "Group %s is zero-sized. No checkpoint info written",CCTK_GroupName(group));
+		continue;
+	      }
+	    }
+	    
+	    /* get the number of allocated timelevels */
+            cGroup gdata;
+	    CCTK_GroupData (group, &gdata);
+	    gdata.numtimelevels = 0;
+	    gdata.numtimelevels = CCTK_GroupStorageIncrease (cctkGH, 1, &group,
+							     &gdata.numtimelevels,NULL);
+	    
+	    int first_vindex = CCTK_FirstVarIndexI (group);
+
+	    /* get the default I/O request for this group */
+	    request = IOUtil_DefaultIORequest (cctkGH, first_vindex, 1);
+	    
+	    /* disable checking for old data objects, disable datatype conversion
+	       and downsampling */
+	    request->check_exist = 0;
+	    request->hdatatype = gdata.vartype;
+	    for (request->hdim = 0; request->hdim < request->vdim; request->hdim++)
+	      {
+		request->downsample[request->hdim] = 1;
+	      }
+	    
+	    /* loop over all variables in this group */
+	    for (request->vindex = first_vindex;
+		 request->vindex < first_vindex + gdata.numvars;
+		 request->vindex++)
+	      {
+		/* loop over all timelevels of this variable */
+		for (request->timelevel = 0;
+		     request->timelevel < gdata.numtimelevels;
+		     request->timelevel++)
+		  {
+		    if (verbose)
+		      {
+			char *fullname = CCTK_FullName (request->vindex);
+			if (fullname != NULL) {
+			    CCTK_VInfo (CCTK_THORNSTRING, "  %s (timelevel %d)",
+					fullname, request->timelevel);
+			    free (fullname);
+			}
+			else {
+			  CCTK_VWarn(1, __LINE__, __FILE__, CCTK_THORNSTRING,
+				     "Invalid variable with varindex %d", request->vindex);
+			}
+		      }
+		    // write the var
+		    
+		    if (grouptype == CCTK_ARRAY || grouptype == CCTK_GF || grouptype == CCTK_SCALAR)
+		      {
+			char* fullname = CCTK_FullName (request->vindex);
+			if (verbose)
+			  CCTK_VInfo (CCTK_THORNSTRING,"%s: reflevel: %d map: %d component: %d grouptype: %d ",
+				      fullname,reflevel,Carpet::map,component,grouptype);
+			free(fullname);
+		    retval += WriteVar(cctkGH,writer,request,1);
+		      }
+		    else
+		      {
+			char *fullname = CCTK_FullName (request->vindex);
+			CCTK_VWarn (1, __LINE__, __FILE__, CCTK_THORNSTRING,
+				    "Invalid group type %d for variable '%s'", grouptype, fullname);
+			free(fullname);
+			retval = -1;
+		      }
+		    
+		  }
+	      } /* end of loop over all variables */
+	  } /* end of loop over all groups */
+      } END_REFLEVEL_LOOP;
+      
+    } END_MGLEVEL_LOOP;
+  
+
+    if (myproc==0) {
+      // Close the file
+      herr = H5Fclose(writer);
+      assert(!herr);
+    }
+    
+    if (retval == 0) {
+      if (myproc==0) {
+	if (rename (cp_tempname, cp_filename))
+	  {
+	    CCTK_VWarn (1, __LINE__, __FILE__, CCTK_THORNSTRING,
+			"Could not rename temporary checkpoint file '%s' to '%s'",
+			cp_tempname, cp_filename);
+	    retval = -1;
+	  }
+	else {
+	  if (myGH->cp_filename_list[myGH->cp_filename_index]) {
+	    if (checkpoint_keep > 0) {
+	      remove (myGH->cp_filename_list[myGH->cp_filename_index]);
+	    }
+	    free (myGH->cp_filename_list[myGH->cp_filename_index]);
+	  }
+	  myGH->cp_filename_list[myGH->cp_filename_index] = strdup (cp_filename);
+	  myGH->cp_filename_index = (myGH->cp_filename_index+1) % abs (checkpoint_keep);
+	} // else
+      } // myproc == 0
+    } // retval == 0
+
+    return retval;
+
+  } // Checkpoint
+
+  
+  int DumpParametersGHExtentions (const cGH *cctkGH, int all, hid_t writer)
+  {
+    // large parts of this routine were taken from
+    // Thomas Radke's IOHDF5Util. Thanks Thomas!
+
+    DECLARE_CCTK_PARAMETERS;
+
+    char *parameters;
+    hid_t group, dataspace, dataset;
+    hsize_t size;
+    herr_t herr;
+    
+    CCTK_INT4 itmp;
+    CCTK_REAL dtmp;
+    const char *version;
+    ioGH *ioUtilGH;
+  
+    if (verbose) {
+      CCTK_INFO ("Dumping Parameters and GH Extentions...");
+    }
+  
+    /* get the parameter string buffer */
+    parameters = IOUtil_GetAllParameters (cctkGH, all);
+  
+    if (parameters)
+    {
+      size = strlen (parameters) + 1;
+      group = H5Gcreate (writer, PARAMETERS_GLOBAL_ATTRIBUTES_GROUP, 0);
+      assert(group>=0);
+      dataspace = H5Screate_simple (1, &size, NULL);
+      assert(dataspace>=0);
+      dataset = H5Dcreate (group, ALL_PARAMETERS, H5T_NATIVE_UCHAR,
+                                       dataspace, H5P_DEFAULT);
+      assert(dataset>=0);
+      herr = H5Dwrite (dataset, H5T_NATIVE_CHAR, H5S_ALL, H5S_ALL,
+                            H5P_DEFAULT, parameters);
+      assert(!herr);
+
+      // now dump the GH Extentions
+
+      /* get the handle for IOUtil extensions */
+      ioUtilGH = (ioGH *) CCTK_GHExtension (cctkGH, "IO");
+      
+      itmp = CCTK_MainLoopIndex ();
+      WriteAttribute(dataset,"main loop index",itmp);
+      
+      itmp = cctkGH->cctk_iteration;
+      WriteAttribute(dataset,"GH$iteration",itmp);
+      
+      itmp = ioUtilGH->ioproc_every;
+      WriteAttribute(dataset,"GH$ioproc_every",itmp);
+      
+      itmp = CCTK_nProcs (cctkGH);
+      WriteAttribute(dataset,"GH$nprocs",itmp);
+      
+      dtmp = cctkGH->cctk_time;
+      WriteAttribute(dataset,"GH$time", dtmp);
+
+      dtmp = global_time;
+      WriteAttribute(dataset,"carpet_global_time", dtmp);
+
+      itmp = reflevels;
+      WriteAttribute(dataset,"carpet_reflevels", itmp);
+
+      dtmp = delta_time;
+      WriteAttribute(dataset,"carpet_delta_time", dtmp);
+
+      version = CCTK_FullVersion();
+      WriteAttribute(dataset,"Cactus version", version);
+
+      /* finally, we need all the times on the individual refinement levels */
+
+      const int numberofmgtimes=mglevels;
+      WriteAttribute(dataset,"numberofmgtimes",numberofmgtimes);
+      for(int i=0;i < numberofmgtimes;i++) {
+	char buffer[100];
+	sprintf(buffer,"mgleveltimes %d",i);
+	WriteAttribute(dataset,buffer,(double *) &leveltimes.at(i).at(0), reflevels);
+      }
+
+      herr = H5Dclose (dataset);
+      assert(!herr);
+      herr = H5Sclose (dataspace);
+      assert(!herr);
+      herr = H5Gclose (group);
+      assert(!herr);
+  
+      free (parameters);
+    }
+
+    return 0;
+  } // DumpParametersGHExtentions
+  
+
+    
+  
+} // namespace CarpetIOHDF5
diff --git a/Carpet/CarpetIOHDF5/src/iohdf5utils.cc b/Carpet/CarpetIOHDF5/src/iohdf5utils.cc
new file mode 100644
index 000000000..5545a3563
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/src/iohdf5utils.cc
@@ -0,0 +1,521 @@
+#include <assert.h>
+#include <limits.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+
+#include <algorithm>
+#include <fstream>
+#include <sstream>
+#include <vector>
+
+#include <hdf5.h>
+
+#include "cctk.h"
+#include "cctk_Parameters.h"
+
+extern "C" {
+  static const char* rcsid = "$Header: /home/eschnett/C/carpet/Carpet/Carpet/CarpetIOHDF5/src/iohdf5utils.cc,v 1.8 2004/07/07 17:09:17 tradke Exp $";
+  CCTK_FILEVERSION(Carpet_CarpetIOHDF5_iohdf5utils_cc);
+}
+
+#include "CactusBase/IOUtil/src/ioGH.h"
+#include "CactusBase/IOUtil/src/ioutil_CheckpointRecovery.h"
+
+#include "bbox.hh"
+#include "data.hh"
+#include "gdata.hh"
+#include "ggf.hh"
+#include "vect.hh"
+
+#include "carpet.hh"
+
+#include "iohdf5.hh"
+#include "iohdf5GH.h"
+
+
+
+namespace CarpetIOHDF5 {
+  
+  using namespace std;
+  using namespace Carpet;
+  
+  
+  
+  bool CheckForVariable (const char* const varlist, const int vindex)
+  {
+    const int numvars = CCTK_NumVars();
+    assert (vindex>=0 && vindex<numvars);
+    
+    vector<bool> flags(numvars);
+    
+    CCTK_TraverseString (varlist, SetFlag, &flags, CCTK_GROUP_OR_VAR);
+    
+    return flags.at(vindex);
+  }
+  
+  void SetFlag (int vindex, const char* optstring, void* arg)
+  {
+    if (optstring)
+    {
+      char *fullname = CCTK_FullName (vindex);
+      CCTK_VWarn (2, __LINE__, __FILE__, CCTK_THORNSTRING,
+                  "Option string '%s' will be ignored for HDF5 output of "
+                  "variable '%s'", optstring, fullname);
+      free (fullname);
+    }
+    vector<bool>& flags = *(vector<bool>*)arg;
+    flags.at(vindex) = true;
+  }
+  
+  
+  
+  void WriteAttribute (const hid_t dataset, const char* const name, const int value)
+  {
+    WriteAttribute (dataset, name, &value, 1);
+  }
+  
+  void WriteAttribute (const hid_t dataset, const char* const name, const int* const values, const int nvalues)
+  {
+    assert (dataset>=0);
+    assert (name);
+    assert (values);
+    assert (nvalues>=0);
+    
+    herr_t herr;
+    
+    hsize_t shape[1];
+    shape[0] = nvalues;
+    const hid_t dataspace = nvalues==1 ? H5Screate (H5S_SCALAR) : H5Screate_simple (1, shape, NULL);
+    assert (dataspace>=0);
+    
+    const hid_t datatype = H5T_NATIVE_INT;
+    
+    const hid_t attribute = H5Acreate (dataset, name, datatype, dataspace, H5P_DEFAULT);
+    assert (attribute>=0);
+    herr = H5Awrite (attribute, datatype, values);
+    assert (!herr);
+    herr = H5Aclose (attribute);
+    assert (!herr);
+    
+    herr = H5Sclose (dataspace);
+    assert (!herr);
+  }
+  
+  
+  
+  void WriteAttribute (const hid_t dataset, const char* const name, const double value)
+  {
+    WriteAttribute (dataset, name, &value, 1);
+  }
+  
+  void WriteAttribute (const hid_t dataset, const char* const name, const double* const values, const int nvalues)
+  {
+    assert (dataset>=0);
+    assert (name);
+    assert (values);
+    assert (nvalues>=0);
+    
+    herr_t herr;
+    
+    hsize_t shape[1];
+    shape[0] = nvalues;
+    const hid_t dataspace = nvalues==1 ? H5Screate (H5S_SCALAR) : H5Screate_simple (1, shape, NULL);
+    assert (dataspace>=0);
+    
+    const hid_t datatype = H5T_NATIVE_DOUBLE;
+    
+    const hid_t attribute = H5Acreate (dataset, name, datatype, dataspace, H5P_DEFAULT);
+    assert (attribute>=0);
+    herr = H5Awrite (attribute, datatype, values);
+    assert (!herr);
+    herr = H5Aclose (attribute);
+    assert (!herr);
+    
+    herr = H5Sclose (dataspace);
+    assert (!herr);
+  }
+  
+  
+  
+  void WriteAttribute (const hid_t dataset, const char* const name, const char value)
+  {
+    WriteAttribute (dataset, name, &value, 1);
+  }
+  
+  void WriteAttribute (const hid_t dataset, const char* const name, const char* const values)
+  {
+    WriteAttribute (dataset, name, values, strlen(values));
+  }
+  
+  void WriteAttribute (const hid_t dataset, const char* const name, const char* const values, const int nvalues)
+  {
+    assert (dataset>=0);
+    assert (name);
+    assert (values);
+    assert (nvalues>=0);
+    
+    herr_t herr;
+    
+    const hid_t dataspace = H5Screate (H5S_SCALAR);
+    assert (dataspace>=0);
+    
+    const hid_t datatype = H5Tcopy (H5T_C_S1);
+    assert (datatype>=0);
+    herr = H5Tset_size (datatype, nvalues);
+    assert (!herr);
+    
+    const hid_t attribute = H5Acreate (dataset, name, datatype, dataspace, H5P_DEFAULT);
+    assert (attribute>=0);
+    herr = H5Awrite (attribute, datatype, values);
+    assert (!herr);
+    herr = H5Aclose (attribute);
+    assert (!herr);
+    
+    herr = H5Tclose (datatype);
+    assert (!herr);
+    
+    herr = H5Sclose (dataspace);
+    assert (!herr);
+  }
+  
+  
+  
+  int ReadAttribute (const hid_t dataset, const char* const name, int& value)
+  {
+    return ReadAttribute (dataset, name, &value, 1);
+  }
+  
+  int ReadAttribute (const hid_t dataset, const char* name, int* const values, const int nvalues)
+  {
+    assert (dataset>=0);
+    assert (name);
+    assert (values);
+    assert (nvalues>=0);
+    
+    herr_t herr;
+    
+    const hid_t attribute = H5Aopen_name (dataset, name);
+    if (attribute<0) return attribute;
+    
+    const hid_t dataspace = H5Aget_space (attribute);
+    assert (dataspace>=0);
+    
+    //    cout << "reading int attribute " << name << endl;
+
+    hsize_t rank = H5Sget_simple_extent_ndims (dataspace);
+    hsize_t shape[1];
+    if (rank==0) {
+      shape[0] = 1;
+    } else if (rank==1) {
+      herr = H5Sget_simple_extent_dims (dataspace, shape, NULL);
+      assert (herr >= 0);
+    } else {
+      assert (0);
+    }
+    const int length = shape[0];
+    
+    const hid_t datatype = H5Aget_type (attribute);
+    assert (datatype>=0);
+
+    assert(H5Tequal(datatype, H5T_NATIVE_INT));
+    
+    vector<int> values1(length);
+    
+    herr = H5Aread (attribute, datatype, &values1.at(0));
+    assert (!herr);
+    
+    for (int i=0; i<min(length, nvalues); ++i) {
+      values[i] = values1[i];
+    }
+
+    herr = H5Tclose (datatype);
+    assert (!herr);
+    
+    herr = H5Sclose (dataspace);
+    assert (!herr);
+    
+    herr = H5Aclose (attribute);
+    assert (!herr);
+    
+    return length;
+  }
+  
+  
+  
+  int ReadAttribute (const hid_t dataset, const char* const name, double& value)
+  {
+    return ReadAttribute (dataset, name, &value, 1);
+  }
+  
+  int ReadAttribute (const hid_t dataset, const char* const name, double* const values, const int nvalues)
+  {
+    assert (dataset>=0);
+    assert (name);
+    assert (values);
+    assert (nvalues>=0);
+    
+    herr_t herr;
+    
+    const hid_t attribute = H5Aopen_name (dataset, name);
+    if (attribute<0) return attribute;
+    
+    const hid_t dataspace = H5Aget_space (attribute);
+    assert (dataspace>=0);
+    
+    hsize_t rank = H5Sget_simple_extent_ndims (dataspace);
+    hsize_t shape[1];
+    if (rank==0) {
+      shape[0] = 1;
+    } else if (rank==1) {
+      rank = H5Sget_simple_extent_dims (dataspace, shape, NULL);
+      assert (rank == 1);
+    } else {
+      assert (0);
+    }
+    const int length = shape[0];
+    
+    const hid_t datatype = H5Aget_type (attribute);
+    assert (datatype>=0);
+    assert(H5Tequal(datatype, H5T_NATIVE_DOUBLE));
+    
+    vector<double> values1(length);
+    
+    herr = H5Aread (attribute, datatype, &values1.at(0));
+    assert (!herr);
+    
+    for (int i=0; i<min(length, nvalues); ++i) {
+      values[i] = values1[i];
+    }
+    
+    herr = H5Tclose (datatype);
+    assert (!herr);
+    
+    herr = H5Sclose (dataspace);
+    assert (!herr);
+    
+    herr = H5Aclose (attribute);
+    assert (!herr);
+    
+    return length;
+  }
+  
+  
+  
+  int ReadAttribute (const hid_t dataset, const char* const name, char& value)
+  {
+    return ReadAttribute (dataset, name, &value, 1);
+  }
+  
+  int ReadAttribute (const hid_t dataset, const char* const name, char*& values)
+  {
+    assert (dataset>=0);
+    assert (name);
+    
+    herr_t herr;
+    
+    const hid_t attribute = H5Aopen_name (dataset, name);
+    if (attribute<0) return attribute;
+    
+    const hid_t dataspace = H5Aget_space (attribute);
+    assert (dataspace>=0);
+    
+    hsize_t rank = H5Sget_simple_extent_ndims (dataspace);
+    assert (rank==0);
+    
+    const hid_t datatype = H5Aget_type (attribute);
+    assert (datatype>=0);
+    assert (H5Tget_class (datatype) == H5T_STRING);
+    const int length = H5Tget_size (datatype);
+    assert (length>=0);
+    
+    values = (char*) malloc (length+1);
+    assert (values);
+    
+    herr = H5Aread (attribute, datatype, values);
+    assert (!herr);
+    values[length] = '\0';
+    
+    herr = H5Tclose (datatype);
+    assert (!herr);
+    
+    herr = H5Sclose (dataspace);
+    assert (!herr);
+    
+    herr = H5Aclose (attribute);
+    assert (!herr);
+    
+    return length;
+  }
+  
+  int ReadAttribute (const hid_t dataset, const char* const name, char* const values, const int nvalues)
+  {
+    assert (dataset>=0);
+    assert (name);
+    assert (values);
+    assert (nvalues>=0);
+    
+    herr_t herr;
+    
+    const hid_t attribute = H5Aopen_name (dataset, name);
+    if (attribute<0) return attribute;
+    
+    const hid_t dataspace = H5Aget_space (attribute);
+    assert (dataspace>=0);
+    
+    hsize_t rank = H5Sget_simple_extent_ndims (dataspace);
+    assert (rank==0);
+    
+    const hid_t datatype = H5Aget_type (attribute);
+    assert (datatype>=0);
+    assert(H5Tget_class (datatype) == H5T_STRING); 
+    const int length = H5Tget_size (datatype);
+    assert (length>=0);
+    
+    vector<char> values1(length);
+    
+    herr = H5Aread (attribute, datatype, &values1.at(0));
+    assert (!herr);
+    
+    for (int i=0; i<min(length, nvalues); ++i) {
+      values[i] = values1[i];
+    }
+    
+    herr = H5Tclose (datatype);
+    assert (!herr);
+    
+    herr = H5Sclose (dataspace);
+    assert (!herr);
+    
+    herr = H5Aclose (attribute);
+    assert (!herr);
+    
+    return length;
+  }
+
+  herr_t DatasetCounter(hid_t group_id, const char *member_name, void *operator_data)
+    /* Counts datasets. Used by GetnDatasets; straight from John Shalf's FlexIO library */
+  {
+    int *count = (int*)operator_data;
+    H5G_stat_t objinfo;
+    // request info about the type of objects in root group
+    if(H5Gget_objinfo(group_id,member_name,1 /* follow links */,&objinfo)<0) {
+      return 0; 
+    }
+    // only count objects that are datasets (not subgroups)
+    if(objinfo.type==H5G_DATASET) {
+      (*count)++;
+    }
+  return 0;
+  }
+
+  
+  int GetnDatasets(const hid_t reader)
+  {
+    //this is straight from John Shalf's FlexIO library
+
+    int count=0;
+    int idx=0;
+    while(H5Giterate(reader, /* hid_t loc_id, */
+		     "/", /*const char *name, */
+		     &idx, /* int *idx, */
+		     DatasetCounter,
+		     &count)<0){}
+    return count;
+  }
+
+  struct H5IO_getname_t {
+    //this is straight from John Shalf's FlexIO library
+    int index,count;
+    char *name;
+  };
+
+
+  herr_t GetName(hid_t group_id, const char *member_name, void *operator_data)
+  {
+    //this is straight from John Shalf's FlexIO library
+    H5IO_getname_t *getn = (H5IO_getname_t*)operator_data;
+    // check type first (only respond if it is a dataset)
+    H5G_stat_t objinfo;
+    // request info about the type of objects in root group
+    if(H5Gget_objinfo(group_id,
+		      member_name,
+		    1 /* follow links */,
+		      &objinfo)<0) return 0; // error (probably bad symlink)
+    // only count objects that are datasets (not subgroups)
+    if(objinfo.type!=H5G_DATASET)
+      return 0; // do not increment count if it isn't a dataset.
+    if(getn->index==getn->count){
+      strcpy(getn->name,member_name);
+      return 1; // success
+    }
+    getn->count++;
+    return 0;
+  }
+
+
+  void GetDatasetName(const hid_t reader, const int _index,  char *name) {
+    //this is straight from John Shalf's FlexIO library
+    H5IO_getname_t getn;
+    int idx=_index;
+    getn.index=_index; getn.name=name; getn.count=_index;
+    while(H5Giterate(reader, /* hid_t loc_id, */
+		     "/", /*const char *name, */
+		     &idx, /* int *idx, */
+		     GetName,
+		     &getn)<0){}
+  }
+
+  hid_t h5DataType (const cGH* const cctkGH, int cctk_type) {
+
+    hid_t retval;  
+
+    CarpetIOHDF5GH *myGH;
+    myGH = (CarpetIOHDF5GH *) CCTK_GHExtension (cctkGH, "CarpetIOHDF5");
+  
+    // this is adapted from Thomas Radke's IOHDF5Util. Thanks, Thomas!
+
+    switch (cctk_type)
+    {
+      case CCTK_VARIABLE_CHAR:      retval = HDF5_CHAR; break;
+      case CCTK_VARIABLE_INT:       retval = HDF5_INT; break;
+      case CCTK_VARIABLE_REAL:      retval = HDF5_REAL; break;
+      case CCTK_VARIABLE_COMPLEX:   retval = myGH->HDF5_COMPLEX; break;
+#ifdef CCTK_INT1
+      case CCTK_VARIABLE_INT1:      retval = HDF5_INT1; break;
+#endif
+#ifdef CCTK_INT2
+      case CCTK_VARIABLE_INT2:      retval = HDF5_INT2; break;
+#endif
+#ifdef CCTK_INT4
+      case CCTK_VARIABLE_INT4:      retval = HDF5_INT4; break;
+#endif
+#ifdef CCTK_INT8
+      case CCTK_VARIABLE_INT8:      retval = HDF5_INT8; break;
+#endif
+#ifdef CCTK_REAL4
+      case CCTK_VARIABLE_REAL4:     retval = HDF5_REAL4; break;
+      case CCTK_VARIABLE_COMPLEX8:  retval = myGH->HDF5_COMPLEX8; break;
+#endif
+#ifdef CCTK_REAL8
+      case CCTK_VARIABLE_REAL8:     retval = HDF5_REAL8; break;
+      case CCTK_VARIABLE_COMPLEX16: retval = myGH->HDF5_COMPLEX16; break;
+#endif
+#ifdef CCTK_REAL16
+      case CCTK_VARIABLE_REAL16:    retval = HDF5_REAL16; break;
+      case CCTK_VARIABLE_COMPLEX32: retval = myGH->HDF5_COMPLEX32; break;
+#endif
+  
+      default: CCTK_VWarn (1, __LINE__, __FILE__, CCTK_THORNSTRING,
+                           "Unsupported CCTK variable datatype %d", cctk_type);
+               retval = -1;
+    }
+  
+    return (retval);
+  }
+  
+    
+  
+} // namespace CarpetIOHDF5
diff --git a/Carpet/CarpetIOHDF5/src/make.code.defn b/Carpet/CarpetIOHDF5/src/make.code.defn
new file mode 100644
index 000000000..a8600e812
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/src/make.code.defn
@@ -0,0 +1,5 @@
+# Main make.code.defn file for thorn CarpetIOHDF5
+# $Header: /home/eschnett/C/carpet/Carpet/Carpet/CarpetIOHDF5/src/make.code.defn,v 1.5 2004/07/09 15:38:18 tradke Exp $
+
+# Source files in this directory
+SRCS = iohdf5.cc iohdf5utils.cc Checkpoint.cc Recover.cc
diff --git a/Carpet/CarpetIOHDF5/src/make.configuration.defn b/Carpet/CarpetIOHDF5/src/make.configuration.defn
new file mode 100644
index 000000000..3de9eea95
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/src/make.configuration.defn
@@ -0,0 +1,7 @@
+# Main make.configuration.defn file for thorn CarpetIOHDF5   -*-Makefile-*-
+# $Header: /home/eschnett/C/carpet/Carpet/Carpet/CarpetIOHDF5/src/make.configuration.defn,v 1.1 2004/05/04 22:08:37 schnetter Exp $
+
+# Ensure that MPI is available
+ifeq ($(strip $(HAVE_HDF5)), )
+    $(error Configuration error: The CarpetIOHDF5 thorn requires HDF5.  Please configure with HDF5, or remove the CarpetIOHDF5 thorn from the ThornList.)
+endif
diff --git a/Carpet/CarpetIOHDF5/test/CarpetWaveToyCheckpoint_test.it_64.h5 b/Carpet/CarpetIOHDF5/test/CarpetWaveToyCheckpoint_test.it_64.h5
new file mode 100644
index 000000000..bed1b5bb5
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/test/CarpetWaveToyCheckpoint_test.it_64.h5
diff --git a/Carpet/CarpetIOHDF5/test/CarpetWaveToyCheckpoint_test.parfile b/Carpet/CarpetIOHDF5/test/CarpetWaveToyCheckpoint_test.parfile
new file mode 100644
index 000000000..f5d898aa1
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/test/CarpetWaveToyCheckpoint_test.parfile
@@ -0,0 +1,44 @@
+ActiveThorns = "Boundary CartGrid3D Time CoordBase SymBase Carpet CarpetLib CarpetRegrid CarpetReduce CarpetSlab IOUtil IOBasic CarpetIOASCII CarpetIOHDF5 IDScalarWaveC WaveToyC"
+
+
+Cactus::cctk_itlast = 64
+
+Time::dtfac = 0.5
+
+Grid::avoid_origin = "no"
+Grid::type         = "byrange"
+Grid::dxyz         = 1
+
+IDScalarWave::initial_data = "gaussian"
+IDScalarWave::sigma        = 2.8
+IDScalarWave::radius       = 0.0
+
+Driver::global_nsize = 10
+Driver::ghost_size   = 2
+
+Carpet::prolongation_order_space = 3
+Carpet::prolongation_order_time  = 2
+Carpet::init_each_timelevel      = "yes"
+Carpet::buffer_width             = 6
+Carpet::max_refinement_levels    = 4
+CarpetRegrid::refinement_levels  = 4
+
+# Output
+IO::out_dir       = $parfile
+IO::out_fileinfo  = "none"
+IO::parfile_write = "no"
+
+IOBasic::outInfo_every   = 1
+IOBasic::outInfo_vars    = "wavetoy::phi"
+IOBasic::outScalar_every = 32
+IOBasic::outScalar_vars  = "wavetoy::phi"
+
+IOASCII::out1D_vars  = "wavetoy::phi"
+IOASCII::out1D_every = 32
+
+# Checkpointing
+IO::checkpoint_file  = $parfile
+IO::checkpoint_every = 32
+
+IOHDF5::checkpoint   = "yes"
+IOHDF5::verbose      = "yes"
diff --git a/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_1proc.par b/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_1proc.par
new file mode 100644
index 000000000..cb72b9b60
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_1proc.par
@@ -0,0 +1,14 @@
+ActiveThorns = "Boundary CartGrid3D Time CoordBase SymBase Carpet CarpetLib CarpetRegrid CarpetReduce CarpetSlab IOUtil IOBasic CarpetIOASCII CarpetIOHDF5 IDScalarWaveC WaveToyC"
+
+
+Cactus::cctk_itlast = 128
+
+# Output
+IO::out_dir = $parfile
+
+# Recovery
+IO::recover_dir  = "../../../arrangements/Carpet/CarpetIOHDF5/test"
+IO::recover      = "manual"
+IO::recover_file = "CarpetWaveToyCheckpoint_test.it_64"
+
+IOHDF5::checkpoint = "no"
diff --git a/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_1proc/phi.x.asc b/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_1proc/phi.x.asc
new file mode 100644
index 000000000..05526bfeb
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_1proc/phi.x.asc
@@ -0,0 +1,227 @@
+# WAVETOY::phi x (phi)
+#
+# iteration 64
+# refinement level 0   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 0 0 0	0 40 40	0.888888888888889	-1 0.111111111111111 0.111111111111111	0.873700996777018
+64	0 0 0 0	8 40 40	0.888888888888889	-0.777777777777778 0.111111111111111 0.111111111111111	0.807676282684182
+64	0 0 0 0	16 40 40	0.888888888888889	-0.555555555555556 0.111111111111111 0.111111111111111	0.756699260944861
+64	0 0 0 0	24 40 40	0.888888888888889	-0.333333333333333 0.111111111111111 0.111111111111111	0.730294733882952
+64	0 0 0 0	32 40 40	0.888888888888889	-0.111111111111111 0.111111111111111 0.111111111111111	0.725539249473295
+64	0 0 0 0	40 40 40	0.888888888888889	0.111111111111111 0.111111111111111 0.111111111111111	0.725539249473295
+64	0 0 0 0	48 40 40	0.888888888888889	0.333333333333333 0.111111111111111 0.111111111111111	0.730294733882952
+64	0 0 0 0	56 40 40	0.888888888888889	0.555555555555556 0.111111111111111 0.111111111111111	0.756699260944861
+64	0 0 0 0	64 40 40	0.888888888888889	0.777777777777778 0.111111111111111 0.111111111111111	0.807676282684182
+64	0 0 0 0	72 40 40	0.888888888888889	1 0.111111111111111 0.111111111111111	0.873700996777018
+
+
+# iteration 64
+# refinement level 1   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 1 0 0	8 36 36	0.888888888888889	-0.777777777777778 -5.55111512312578e-17 -5.55111512312578e-17	0.808379332677938
+64	0 1 0 0	12 36 36	0.888888888888889	-0.666666666666667 -5.55111512312578e-17 -5.55111512312578e-17	0.779665123251561
+64	0 1 0 0	16 36 36	0.888888888888889	-0.555555555555556 -5.55111512312578e-17 -5.55111512312578e-17	0.756097245006156
+64	0 1 0 0	20 36 36	0.888888888888889	-0.444444444444444 -5.55111512312578e-17 -5.55111512312578e-17	0.739694852518638
+64	0 1 0 0	24 36 36	0.888888888888889	-0.333333333333333 -5.55111512312578e-17 -5.55111512312578e-17	0.729218296316213
+64	0 1 0 0	28 36 36	0.888888888888889	-0.222222222222222 -5.55111512312578e-17 -5.55111512312578e-17	0.725100663327574
+64	0 1 0 0	32 36 36	0.888888888888889	-0.111111111111111 -5.55111512312578e-17 -5.55111512312578e-17	0.724342898925177
+64	0 1 0 0	36 36 36	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.723733474251298
+64	0 1 0 0	40 36 36	0.888888888888889	0.111111111111111 -5.55111512312578e-17 -5.55111512312578e-17	0.724342898925177
+64	0 1 0 0	44 36 36	0.888888888888889	0.222222222222222 -5.55111512312578e-17 -5.55111512312578e-17	0.725100663327574
+64	0 1 0 0	48 36 36	0.888888888888889	0.333333333333333 -5.55111512312578e-17 -5.55111512312578e-17	0.729218296316213
+64	0 1 0 0	52 36 36	0.888888888888889	0.444444444444444 -5.55111512312578e-17 -5.55111512312578e-17	0.739694852518638
+64	0 1 0 0	56 36 36	0.888888888888889	0.555555555555555 -5.55111512312578e-17 -5.55111512312578e-17	0.756097245006155
+64	0 1 0 0	60 36 36	0.888888888888889	0.666666666666667 -5.55111512312578e-17 -5.55111512312578e-17	0.779665123251561
+64	0 1 0 0	64 36 36	0.888888888888889	0.777777777777778 -5.55111512312578e-17 -5.55111512312578e-17	0.808379332677938
+
+
+# iteration 64
+# refinement level 2   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 2 0 0	22 36 36	0.888888888888889	-0.388888888888889 -5.55111512312578e-17 -5.55111512312578e-17	0.733688776948746
+64	0 2 0 0	24 36 36	0.888888888888889	-0.333333333333333 -5.55111512312578e-17 -5.55111512312578e-17	0.729218296316213
+64	0 2 0 0	26 36 36	0.888888888888889	-0.277777777777778 -5.55111512312578e-17 -5.55111512312578e-17	0.726552055334392
+64	0 2 0 0	28 36 36	0.888888888888889	-0.222222222222222 -5.55111512312578e-17 -5.55111512312578e-17	0.725100663327574
+64	0 2 0 0	30 36 36	0.888888888888889	-0.166666666666667 -5.55111512312578e-17 -5.55111512312578e-17	0.724502518106703
+64	0 2 0 0	32 36 36	0.888888888888889	-0.111111111111111 -5.55111512312578e-17 -5.55111512312578e-17	0.724342898925177
+64	0 2 0 0	34 36 36	0.888888888888889	-0.0555555555555556 -5.55111512312578e-17 -5.55111512312578e-17	0.72395273727097
+64	0 2 0 0	36 36 36	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.723733474251298
+64	0 2 0 0	38 36 36	0.888888888888889	0.0555555555555555 -5.55111512312578e-17 -5.55111512312578e-17	0.72395273727097
+64	0 2 0 0	40 36 36	0.888888888888889	0.111111111111111 -5.55111512312578e-17 -5.55111512312578e-17	0.724342898925177
+64	0 2 0 0	42 36 36	0.888888888888889	0.166666666666667 -5.55111512312578e-17 -5.55111512312578e-17	0.724502518106703
+64	0 2 0 0	44 36 36	0.888888888888889	0.222222222222222 -5.55111512312578e-17 -5.55111512312578e-17	0.725100663327574
+64	0 2 0 0	46 36 36	0.888888888888889	0.277777777777778 -5.55111512312578e-17 -5.55111512312578e-17	0.726552055334392
+64	0 2 0 0	48 36 36	0.888888888888889	0.333333333333333 -5.55111512312578e-17 -5.55111512312578e-17	0.729218296316213
+64	0 2 0 0	50 36 36	0.888888888888889	0.388888888888889 -5.55111512312578e-17 -5.55111512312578e-17	0.733688776948745
+
+
+# iteration 64
+# refinement level 3   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 3 0 0	29 36 36	0.888888888888889	-0.194444444444445 -5.55111512312578e-17 -5.55111512312578e-17	0.724720854915558
+64	0 3 0 0	30 36 36	0.888888888888889	-0.166666666666667 -5.55111512312578e-17 -5.55111512312578e-17	0.724502518106703
+64	0 3 0 0	31 36 36	0.888888888888889	-0.138888888888889 -5.55111512312578e-17 -5.55111512312578e-17	0.724409709543024
+64	0 3 0 0	32 36 36	0.888888888888889	-0.111111111111111 -5.55111512312578e-17 -5.55111512312578e-17	0.724342898925177
+64	0 3 0 0	33 36 36	0.888888888888889	-0.0833333333333334 -5.55111512312578e-17 -5.55111512312578e-17	0.724151545837958
+64	0 3 0 0	34 36 36	0.888888888888889	-0.0555555555555556 -5.55111512312578e-17 -5.55111512312578e-17	0.72395273727097
+64	0 3 0 0	35 36 36	0.888888888888889	-0.0277777777777778 -5.55111512312578e-17 -5.55111512312578e-17	0.723805016719017
+64	0 3 0 0	36 36 36	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.723733474251298
+64	0 3 0 0	37 36 36	0.888888888888889	0.0277777777777777 -5.55111512312578e-17 -5.55111512312578e-17	0.723805016719017
+64	0 3 0 0	38 36 36	0.888888888888889	0.0555555555555555 -5.55111512312578e-17 -5.55111512312578e-17	0.72395273727097
+64	0 3 0 0	39 36 36	0.888888888888889	0.0833333333333333 -5.55111512312578e-17 -5.55111512312578e-17	0.724151545837958
+64	0 3 0 0	40 36 36	0.888888888888889	0.111111111111111 -5.55111512312578e-17 -5.55111512312578e-17	0.724342898925177
+64	0 3 0 0	41 36 36	0.888888888888889	0.138888888888889 -5.55111512312578e-17 -5.55111512312578e-17	0.724409709543024
+64	0 3 0 0	42 36 36	0.888888888888889	0.166666666666667 -5.55111512312578e-17 -5.55111512312578e-17	0.724502518106703
+64	0 3 0 0	43 36 36	0.888888888888889	0.194444444444444 -5.55111512312578e-17 -5.55111512312578e-17	0.724720854915558
+
+
+# iteration 96
+# refinement level 0   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 0 0 0	0 40 40	1.33333333333333	-1 0.111111111111111 0.111111111111111	0.877480603359996
+96	0 0 0 0	8 40 40	1.33333333333333	-0.777777777777778 0.111111111111111 0.111111111111111	0.798008089475601
+96	0 0 0 0	16 40 40	1.33333333333333	-0.555555555555556 0.111111111111111 0.111111111111111	0.735090597558599
+96	0 0 0 0	24 40 40	1.33333333333333	-0.333333333333333 0.111111111111111 0.111111111111111	0.677899959110667
+96	0 0 0 0	32 40 40	1.33333333333333	-0.111111111111111 0.111111111111111 0.111111111111111	0.632828813666694
+96	0 0 0 0	40 40 40	1.33333333333333	0.111111111111111 0.111111111111111 0.111111111111111	0.632828813666694
+96	0 0 0 0	48 40 40	1.33333333333333	0.333333333333333 0.111111111111111 0.111111111111111	0.677899959110667
+96	0 0 0 0	56 40 40	1.33333333333333	0.555555555555556 0.111111111111111 0.111111111111111	0.735090597558599
+96	0 0 0 0	64 40 40	1.33333333333333	0.777777777777778 0.111111111111111 0.111111111111111	0.798008089475601
+96	0 0 0 0	72 40 40	1.33333333333333	1 0.111111111111111 0.111111111111111	0.877480603359996
+
+
+# iteration 96
+# refinement level 1   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 1 0 0	8 36 36	1.33333333333333	-0.777777777777778 -5.55111512312578e-17 -5.55111512312578e-17	0.796688784792057
+96	0 1 0 0	12 36 36	1.33333333333333	-0.666666666666667 -5.55111512312578e-17 -5.55111512312578e-17	0.76190136189151
+96	0 1 0 0	16 36 36	1.33333333333333	-0.555555555555556 -5.55111512312578e-17 -5.55111512312578e-17	0.729928937348657
+96	0 1 0 0	20 36 36	1.33333333333333	-0.444444444444444 -5.55111512312578e-17 -5.55111512312578e-17	0.698215010923196
+96	0 1 0 0	24 36 36	1.33333333333333	-0.333333333333333 -5.55111512312578e-17 -5.55111512312578e-17	0.668908746807157
+96	0 1 0 0	28 36 36	1.33333333333333	-0.222222222222222 -5.55111512312578e-17 -5.55111512312578e-17	0.641345710414296
+96	0 1 0 0	32 36 36	1.33333333333333	-0.111111111111111 -5.55111512312578e-17 -5.55111512312578e-17	0.621410197839124
+96	0 1 0 0	36 36 36	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.61547287921812
+96	0 1 0 0	40 36 36	1.33333333333333	0.111111111111111 -5.55111512312578e-17 -5.55111512312578e-17	0.621410197839125
+96	0 1 0 0	44 36 36	1.33333333333333	0.222222222222222 -5.55111512312578e-17 -5.55111512312578e-17	0.641345710414296
+96	0 1 0 0	48 36 36	1.33333333333333	0.333333333333333 -5.55111512312578e-17 -5.55111512312578e-17	0.668908746807157
+96	0 1 0 0	52 36 36	1.33333333333333	0.444444444444444 -5.55111512312578e-17 -5.55111512312578e-17	0.698215010923196
+96	0 1 0 0	56 36 36	1.33333333333333	0.555555555555555 -5.55111512312578e-17 -5.55111512312578e-17	0.729928937348657
+96	0 1 0 0	60 36 36	1.33333333333333	0.666666666666667 -5.55111512312578e-17 -5.55111512312578e-17	0.76190136189151
+96	0 1 0 0	64 36 36	1.33333333333333	0.777777777777778 -5.55111512312578e-17 -5.55111512312578e-17	0.796688784792057
+
+
+# iteration 96
+# refinement level 2   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 2 0 0	22 36 36	1.33333333333333	-0.388888888888889 -5.55111512312578e-17 -5.55111512312578e-17	0.683302448238139
+96	0 2 0 0	24 36 36	1.33333333333333	-0.333333333333333 -5.55111512312578e-17 -5.55111512312578e-17	0.668908746807157
+96	0 2 0 0	26 36 36	1.33333333333333	-0.277777777777778 -5.55111512312578e-17 -5.55111512312578e-17	0.654541556639422
+96	0 2 0 0	28 36 36	1.33333333333333	-0.222222222222222 -5.55111512312578e-17 -5.55111512312578e-17	0.641345710414296
+96	0 2 0 0	30 36 36	1.33333333333333	-0.166666666666667 -5.55111512312578e-17 -5.55111512312578e-17	0.630026346765969
+96	0 2 0 0	32 36 36	1.33333333333333	-0.111111111111111 -5.55111512312578e-17 -5.55111512312578e-17	0.621410197839124
+96	0 2 0 0	34 36 36	1.33333333333333	-0.0555555555555556 -5.55111512312578e-17 -5.55111512312578e-17	0.616824486578861
+96	0 2 0 0	36 36 36	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.61547287921812
+96	0 2 0 0	38 36 36	1.33333333333333	0.0555555555555555 -5.55111512312578e-17 -5.55111512312578e-17	0.616824486578861
+96	0 2 0 0	40 36 36	1.33333333333333	0.111111111111111 -5.55111512312578e-17 -5.55111512312578e-17	0.621410197839125
+96	0 2 0 0	42 36 36	1.33333333333333	0.166666666666667 -5.55111512312578e-17 -5.55111512312578e-17	0.63002634676597
+96	0 2 0 0	44 36 36	1.33333333333333	0.222222222222222 -5.55111512312578e-17 -5.55111512312578e-17	0.641345710414296
+96	0 2 0 0	46 36 36	1.33333333333333	0.277777777777778 -5.55111512312578e-17 -5.55111512312578e-17	0.654541556639422
+96	0 2 0 0	48 36 36	1.33333333333333	0.333333333333333 -5.55111512312578e-17 -5.55111512312578e-17	0.668908746807157
+96	0 2 0 0	50 36 36	1.33333333333333	0.388888888888889 -5.55111512312578e-17 -5.55111512312578e-17	0.683302448238139
+
+
+# iteration 96
+# refinement level 3   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 3 0 0	29 36 36	1.33333333333333	-0.194444444444445 -5.55111512312578e-17 -5.55111512312578e-17	0.63539979750899
+96	0 3 0 0	30 36 36	1.33333333333333	-0.166666666666667 -5.55111512312578e-17 -5.55111512312578e-17	0.630026346765969
+96	0 3 0 0	31 36 36	1.33333333333333	-0.138888888888889 -5.55111512312578e-17 -5.55111512312578e-17	0.625297419028293
+96	0 3 0 0	32 36 36	1.33333333333333	-0.111111111111111 -5.55111512312578e-17 -5.55111512312578e-17	0.621410197839124
+96	0 3 0 0	33 36 36	1.33333333333333	-0.0833333333333334 -5.55111512312578e-17 -5.55111512312578e-17	0.618663308361111
+96	0 3 0 0	34 36 36	1.33333333333333	-0.0555555555555556 -5.55111512312578e-17 -5.55111512312578e-17	0.616824486578861
+96	0 3 0 0	35 36 36	1.33333333333333	-0.0277777777777778 -5.55111512312578e-17 -5.55111512312578e-17	0.615777600484678
+96	0 3 0 0	36 36 36	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.61547287921812
+96	0 3 0 0	37 36 36	1.33333333333333	0.0277777777777777 -5.55111512312578e-17 -5.55111512312578e-17	0.615777600484678
+96	0 3 0 0	38 36 36	1.33333333333333	0.0555555555555555 -5.55111512312578e-17 -5.55111512312578e-17	0.616824486578861
+96	0 3 0 0	39 36 36	1.33333333333333	0.0833333333333333 -5.55111512312578e-17 -5.55111512312578e-17	0.618663308361111
+96	0 3 0 0	40 36 36	1.33333333333333	0.111111111111111 -5.55111512312578e-17 -5.55111512312578e-17	0.621410197839125
+96	0 3 0 0	41 36 36	1.33333333333333	0.138888888888889 -5.55111512312578e-17 -5.55111512312578e-17	0.625297419028293
+96	0 3 0 0	42 36 36	1.33333333333333	0.166666666666667 -5.55111512312578e-17 -5.55111512312578e-17	0.63002634676597
+96	0 3 0 0	43 36 36	1.33333333333333	0.194444444444444 -5.55111512312578e-17 -5.55111512312578e-17	0.63539979750899
+
+
+# iteration 128
+# refinement level 0   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 0 0 0	0 40 40	1.77777777777778	-1 0.111111111111111 0.111111111111111	0.873700996777018
+128	0 0 0 0	8 40 40	1.77777777777778	-0.777777777777778 0.111111111111111 0.111111111111111	0.859471717622308
+128	0 0 0 0	16 40 40	1.77777777777778	-0.555555555555556 0.111111111111111 0.111111111111111	0.863004424640887
+128	0 0 0 0	24 40 40	1.77777777777778	-0.333333333333333 0.111111111111111 0.111111111111111	0.883486952153519
+128	0 0 0 0	32 40 40	1.77777777777778	-0.111111111111111 0.111111111111111 0.111111111111111	0.915799135771738
+128	0 0 0 0	40 40 40	1.77777777777778	0.111111111111111 0.111111111111111 0.111111111111111	0.915799135771738
+128	0 0 0 0	48 40 40	1.77777777777778	0.333333333333333 0.111111111111111 0.111111111111111	0.883486952153519
+128	0 0 0 0	56 40 40	1.77777777777778	0.555555555555556 0.111111111111111 0.111111111111111	0.863004424640887
+128	0 0 0 0	64 40 40	1.77777777777778	0.777777777777778 0.111111111111111 0.111111111111111	0.859471717622308
+128	0 0 0 0	72 40 40	1.77777777777778	1 0.111111111111111 0.111111111111111	0.873700996777018
+
+
+# iteration 128
+# refinement level 1   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 1 0 0	8 36 36	1.77777777777778	-0.777777777777778 -5.55111512312578e-17 -5.55111512312578e-17	0.862653663591007
+128	0 1 0 0	12 36 36	1.77777777777778	-0.666666666666667 -5.55111512312578e-17 -5.55111512312578e-17	0.862560028606905
+128	0 1 0 0	16 36 36	1.77777777777778	-0.555555555555556 -5.55111512312578e-17 -5.55111512312578e-17	0.866965759785718
+128	0 1 0 0	20 36 36	1.77777777777778	-0.444444444444444 -5.55111512312578e-17 -5.55111512312578e-17	0.876167092595071
+128	0 1 0 0	24 36 36	1.77777777777778	-0.333333333333333 -5.55111512312578e-17 -5.55111512312578e-17	0.889185421864397
+128	0 1 0 0	28 36 36	1.77777777777778	-0.222222222222222 -5.55111512312578e-17 -5.55111512312578e-17	0.907998184681559
+128	0 1 0 0	32 36 36	1.77777777777778	-0.111111111111111 -5.55111512312578e-17 -5.55111512312578e-17	0.924033489738888
+128	0 1 0 0	36 36 36	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.928389498223199
+128	0 1 0 0	40 36 36	1.77777777777778	0.111111111111111 -5.55111512312578e-17 -5.55111512312578e-17	0.924033489738888
+128	0 1 0 0	44 36 36	1.77777777777778	0.222222222222222 -5.55111512312578e-17 -5.55111512312578e-17	0.90799818468156
+128	0 1 0 0	48 36 36	1.77777777777778	0.333333333333333 -5.55111512312578e-17 -5.55111512312578e-17	0.889185421864397
+128	0 1 0 0	52 36 36	1.77777777777778	0.444444444444444 -5.55111512312578e-17 -5.55111512312578e-17	0.876167092595071
+128	0 1 0 0	56 36 36	1.77777777777778	0.555555555555555 -5.55111512312578e-17 -5.55111512312578e-17	0.866965759785718
+128	0 1 0 0	60 36 36	1.77777777777778	0.666666666666667 -5.55111512312578e-17 -5.55111512312578e-17	0.862560028606905
+128	0 1 0 0	64 36 36	1.77777777777778	0.777777777777778 -5.55111512312578e-17 -5.55111512312578e-17	0.862653663591007
+
+
+# iteration 128
+# refinement level 2   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 2 0 0	22 36 36	1.77777777777778	-0.388888888888889 -5.55111512312578e-17 -5.55111512312578e-17	0.882075542854246
+128	0 2 0 0	24 36 36	1.77777777777778	-0.333333333333333 -5.55111512312578e-17 -5.55111512312578e-17	0.889185421864397
+128	0 2 0 0	26 36 36	1.77777777777778	-0.277777777777778 -5.55111512312578e-17 -5.55111512312578e-17	0.898403242286228
+128	0 2 0 0	28 36 36	1.77777777777778	-0.222222222222222 -5.55111512312578e-17 -5.55111512312578e-17	0.907998184681559
+128	0 2 0 0	30 36 36	1.77777777777778	-0.166666666666667 -5.55111512312578e-17 -5.55111512312578e-17	0.916919384356027
+128	0 2 0 0	32 36 36	1.77777777777778	-0.111111111111111 -5.55111512312578e-17 -5.55111512312578e-17	0.924033489738888
+128	0 2 0 0	34 36 36	1.77777777777778	-0.0555555555555556 -5.55111512312578e-17 -5.55111512312578e-17	0.927485951077396
+128	0 2 0 0	36 36 36	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.928389498223199
+128	0 2 0 0	38 36 36	1.77777777777778	0.0555555555555555 -5.55111512312578e-17 -5.55111512312578e-17	0.927485951077396
+128	0 2 0 0	40 36 36	1.77777777777778	0.111111111111111 -5.55111512312578e-17 -5.55111512312578e-17	0.924033489738888
+128	0 2 0 0	42 36 36	1.77777777777778	0.166666666666667 -5.55111512312578e-17 -5.55111512312578e-17	0.916919384356027
+128	0 2 0 0	44 36 36	1.77777777777778	0.222222222222222 -5.55111512312578e-17 -5.55111512312578e-17	0.90799818468156
+128	0 2 0 0	46 36 36	1.77777777777778	0.277777777777778 -5.55111512312578e-17 -5.55111512312578e-17	0.898403242286228
+128	0 2 0 0	48 36 36	1.77777777777778	0.333333333333333 -5.55111512312578e-17 -5.55111512312578e-17	0.889185421864397
+128	0 2 0 0	50 36 36	1.77777777777778	0.388888888888889 -5.55111512312578e-17 -5.55111512312578e-17	0.882075542854246
+
+
+# iteration 128
+# refinement level 3   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 3 0 0	29 36 36	1.77777777777778	-0.194444444444445 -5.55111512312578e-17 -5.55111512312578e-17	0.912613836832073
+128	0 3 0 0	30 36 36	1.77777777777778	-0.166666666666667 -5.55111512312578e-17 -5.55111512312578e-17	0.916919384356027
+128	0 3 0 0	31 36 36	1.77777777777778	-0.138888888888889 -5.55111512312578e-17 -5.55111512312578e-17	0.920818233193455
+128	0 3 0 0	32 36 36	1.77777777777778	-0.111111111111111 -5.55111512312578e-17 -5.55111512312578e-17	0.924033489738888
+128	0 3 0 0	33 36 36	1.77777777777778	-0.0833333333333334 -5.55111512312578e-17 -5.55111512312578e-17	0.926147880297958
+128	0 3 0 0	34 36 36	1.77777777777778	-0.0555555555555556 -5.55111512312578e-17 -5.55111512312578e-17	0.927485951077396
+128	0 3 0 0	35 36 36	1.77777777777778	-0.0277777777777778 -5.55111512312578e-17 -5.55111512312578e-17	0.928209975180567
+128	0 3 0 0	36 36 36	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.928389498223199
+128	0 3 0 0	37 36 36	1.77777777777778	0.0277777777777777 -5.55111512312578e-17 -5.55111512312578e-17	0.928209975180567
+128	0 3 0 0	38 36 36	1.77777777777778	0.0555555555555555 -5.55111512312578e-17 -5.55111512312578e-17	0.927485951077396
+128	0 3 0 0	39 36 36	1.77777777777778	0.0833333333333333 -5.55111512312578e-17 -5.55111512312578e-17	0.926147880297958
+128	0 3 0 0	40 36 36	1.77777777777778	0.111111111111111 -5.55111512312578e-17 -5.55111512312578e-17	0.924033489738888
+128	0 3 0 0	41 36 36	1.77777777777778	0.138888888888889 -5.55111512312578e-17 -5.55111512312578e-17	0.920818233193455
+128	0 3 0 0	42 36 36	1.77777777777778	0.166666666666667 -5.55111512312578e-17 -5.55111512312578e-17	0.916919384356027
+128	0 3 0 0	43 36 36	1.77777777777778	0.194444444444444 -5.55111512312578e-17 -5.55111512312578e-17	0.912613836832073
+
+
diff --git a/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_1proc/phi.y.asc b/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_1proc/phi.y.asc
new file mode 100644
index 000000000..305349452
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_1proc/phi.y.asc
@@ -0,0 +1,227 @@
+# WAVETOY::phi y (phi)
+#
+# iteration 64
+# refinement level 0   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 0 0 0	40 0 40	0.888888888888889	0.111111111111111 -1 0.111111111111111	0.873700996777018
+64	0 0 0 0	40 8 40	0.888888888888889	0.111111111111111 -0.777777777777778 0.111111111111111	0.807676282684182
+64	0 0 0 0	40 16 40	0.888888888888889	0.111111111111111 -0.555555555555556 0.111111111111111	0.756699260944861
+64	0 0 0 0	40 24 40	0.888888888888889	0.111111111111111 -0.333333333333333 0.111111111111111	0.730294733882952
+64	0 0 0 0	40 32 40	0.888888888888889	0.111111111111111 -0.111111111111111 0.111111111111111	0.725539249473295
+64	0 0 0 0	40 40 40	0.888888888888889	0.111111111111111 0.111111111111111 0.111111111111111	0.725539249473295
+64	0 0 0 0	40 48 40	0.888888888888889	0.111111111111111 0.333333333333333 0.111111111111111	0.730294733882952
+64	0 0 0 0	40 56 40	0.888888888888889	0.111111111111111 0.555555555555556 0.111111111111111	0.756699260944861
+64	0 0 0 0	40 64 40	0.888888888888889	0.111111111111111 0.777777777777778 0.111111111111111	0.807676282684182
+64	0 0 0 0	40 72 40	0.888888888888889	0.111111111111111 1 0.111111111111111	0.873700996777018
+
+
+# iteration 64
+# refinement level 1   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 1 0 0	36 8 36	0.888888888888889	-5.55111512312578e-17 -0.777777777777778 -5.55111512312578e-17	0.808379332677938
+64	0 1 0 0	36 12 36	0.888888888888889	-5.55111512312578e-17 -0.666666666666667 -5.55111512312578e-17	0.779665123251562
+64	0 1 0 0	36 16 36	0.888888888888889	-5.55111512312578e-17 -0.555555555555556 -5.55111512312578e-17	0.756097245006156
+64	0 1 0 0	36 20 36	0.888888888888889	-5.55111512312578e-17 -0.444444444444444 -5.55111512312578e-17	0.739694852518638
+64	0 1 0 0	36 24 36	0.888888888888889	-5.55111512312578e-17 -0.333333333333333 -5.55111512312578e-17	0.729218296316213
+64	0 1 0 0	36 28 36	0.888888888888889	-5.55111512312578e-17 -0.222222222222222 -5.55111512312578e-17	0.725100663327575
+64	0 1 0 0	36 32 36	0.888888888888889	-5.55111512312578e-17 -0.111111111111111 -5.55111512312578e-17	0.724342898925177
+64	0 1 0 0	36 36 36	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.723733474251298
+64	0 1 0 0	36 40 36	0.888888888888889	-5.55111512312578e-17 0.111111111111111 -5.55111512312578e-17	0.724342898925177
+64	0 1 0 0	36 44 36	0.888888888888889	-5.55111512312578e-17 0.222222222222222 -5.55111512312578e-17	0.725100663327574
+64	0 1 0 0	36 48 36	0.888888888888889	-5.55111512312578e-17 0.333333333333333 -5.55111512312578e-17	0.729218296316213
+64	0 1 0 0	36 52 36	0.888888888888889	-5.55111512312578e-17 0.444444444444444 -5.55111512312578e-17	0.739694852518638
+64	0 1 0 0	36 56 36	0.888888888888889	-5.55111512312578e-17 0.555555555555555 -5.55111512312578e-17	0.756097245006155
+64	0 1 0 0	36 60 36	0.888888888888889	-5.55111512312578e-17 0.666666666666667 -5.55111512312578e-17	0.779665123251561
+64	0 1 0 0	36 64 36	0.888888888888889	-5.55111512312578e-17 0.777777777777778 -5.55111512312578e-17	0.808379332677938
+
+
+# iteration 64
+# refinement level 2   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 2 0 0	36 22 36	0.888888888888889	-5.55111512312578e-17 -0.388888888888889 -5.55111512312578e-17	0.733688776948746
+64	0 2 0 0	36 24 36	0.888888888888889	-5.55111512312578e-17 -0.333333333333333 -5.55111512312578e-17	0.729218296316213
+64	0 2 0 0	36 26 36	0.888888888888889	-5.55111512312578e-17 -0.277777777777778 -5.55111512312578e-17	0.726552055334392
+64	0 2 0 0	36 28 36	0.888888888888889	-5.55111512312578e-17 -0.222222222222222 -5.55111512312578e-17	0.725100663327575
+64	0 2 0 0	36 30 36	0.888888888888889	-5.55111512312578e-17 -0.166666666666667 -5.55111512312578e-17	0.724502518106703
+64	0 2 0 0	36 32 36	0.888888888888889	-5.55111512312578e-17 -0.111111111111111 -5.55111512312578e-17	0.724342898925177
+64	0 2 0 0	36 34 36	0.888888888888889	-5.55111512312578e-17 -0.0555555555555556 -5.55111512312578e-17	0.72395273727097
+64	0 2 0 0	36 36 36	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.723733474251298
+64	0 2 0 0	36 38 36	0.888888888888889	-5.55111512312578e-17 0.0555555555555555 -5.55111512312578e-17	0.72395273727097
+64	0 2 0 0	36 40 36	0.888888888888889	-5.55111512312578e-17 0.111111111111111 -5.55111512312578e-17	0.724342898925177
+64	0 2 0 0	36 42 36	0.888888888888889	-5.55111512312578e-17 0.166666666666667 -5.55111512312578e-17	0.724502518106703
+64	0 2 0 0	36 44 36	0.888888888888889	-5.55111512312578e-17 0.222222222222222 -5.55111512312578e-17	0.725100663327574
+64	0 2 0 0	36 46 36	0.888888888888889	-5.55111512312578e-17 0.277777777777778 -5.55111512312578e-17	0.726552055334392
+64	0 2 0 0	36 48 36	0.888888888888889	-5.55111512312578e-17 0.333333333333333 -5.55111512312578e-17	0.729218296316213
+64	0 2 0 0	36 50 36	0.888888888888889	-5.55111512312578e-17 0.388888888888889 -5.55111512312578e-17	0.733688776948745
+
+
+# iteration 64
+# refinement level 3   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 3 0 0	36 29 36	0.888888888888889	-5.55111512312578e-17 -0.194444444444445 -5.55111512312578e-17	0.724720854915558
+64	0 3 0 0	36 30 36	0.888888888888889	-5.55111512312578e-17 -0.166666666666667 -5.55111512312578e-17	0.724502518106703
+64	0 3 0 0	36 31 36	0.888888888888889	-5.55111512312578e-17 -0.138888888888889 -5.55111512312578e-17	0.724409709543024
+64	0 3 0 0	36 32 36	0.888888888888889	-5.55111512312578e-17 -0.111111111111111 -5.55111512312578e-17	0.724342898925177
+64	0 3 0 0	36 33 36	0.888888888888889	-5.55111512312578e-17 -0.0833333333333334 -5.55111512312578e-17	0.724151545837958
+64	0 3 0 0	36 34 36	0.888888888888889	-5.55111512312578e-17 -0.0555555555555556 -5.55111512312578e-17	0.72395273727097
+64	0 3 0 0	36 35 36	0.888888888888889	-5.55111512312578e-17 -0.0277777777777778 -5.55111512312578e-17	0.723805016719017
+64	0 3 0 0	36 36 36	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.723733474251298
+64	0 3 0 0	36 37 36	0.888888888888889	-5.55111512312578e-17 0.0277777777777777 -5.55111512312578e-17	0.723805016719017
+64	0 3 0 0	36 38 36	0.888888888888889	-5.55111512312578e-17 0.0555555555555555 -5.55111512312578e-17	0.72395273727097
+64	0 3 0 0	36 39 36	0.888888888888889	-5.55111512312578e-17 0.0833333333333333 -5.55111512312578e-17	0.724151545837958
+64	0 3 0 0	36 40 36	0.888888888888889	-5.55111512312578e-17 0.111111111111111 -5.55111512312578e-17	0.724342898925177
+64	0 3 0 0	36 41 36	0.888888888888889	-5.55111512312578e-17 0.138888888888889 -5.55111512312578e-17	0.724409709543024
+64	0 3 0 0	36 42 36	0.888888888888889	-5.55111512312578e-17 0.166666666666667 -5.55111512312578e-17	0.724502518106703
+64	0 3 0 0	36 43 36	0.888888888888889	-5.55111512312578e-17 0.194444444444444 -5.55111512312578e-17	0.724720854915558
+
+
+# iteration 96
+# refinement level 0   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 0 0 0	40 0 40	1.33333333333333	0.111111111111111 -1 0.111111111111111	0.877480603359996
+96	0 0 0 0	40 8 40	1.33333333333333	0.111111111111111 -0.777777777777778 0.111111111111111	0.798008089475601
+96	0 0 0 0	40 16 40	1.33333333333333	0.111111111111111 -0.555555555555556 0.111111111111111	0.735090597558599
+96	0 0 0 0	40 24 40	1.33333333333333	0.111111111111111 -0.333333333333333 0.111111111111111	0.677899959110667
+96	0 0 0 0	40 32 40	1.33333333333333	0.111111111111111 -0.111111111111111 0.111111111111111	0.632828813666694
+96	0 0 0 0	40 40 40	1.33333333333333	0.111111111111111 0.111111111111111 0.111111111111111	0.632828813666694
+96	0 0 0 0	40 48 40	1.33333333333333	0.111111111111111 0.333333333333333 0.111111111111111	0.677899959110667
+96	0 0 0 0	40 56 40	1.33333333333333	0.111111111111111 0.555555555555556 0.111111111111111	0.735090597558599
+96	0 0 0 0	40 64 40	1.33333333333333	0.111111111111111 0.777777777777778 0.111111111111111	0.798008089475601
+96	0 0 0 0	40 72 40	1.33333333333333	0.111111111111111 1 0.111111111111111	0.877480603359996
+
+
+# iteration 96
+# refinement level 1   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 1 0 0	36 8 36	1.33333333333333	-5.55111512312578e-17 -0.777777777777778 -5.55111512312578e-17	0.796688784792057
+96	0 1 0 0	36 12 36	1.33333333333333	-5.55111512312578e-17 -0.666666666666667 -5.55111512312578e-17	0.76190136189151
+96	0 1 0 0	36 16 36	1.33333333333333	-5.55111512312578e-17 -0.555555555555556 -5.55111512312578e-17	0.729928937348657
+96	0 1 0 0	36 20 36	1.33333333333333	-5.55111512312578e-17 -0.444444444444444 -5.55111512312578e-17	0.698215010923196
+96	0 1 0 0	36 24 36	1.33333333333333	-5.55111512312578e-17 -0.333333333333333 -5.55111512312578e-17	0.668908746807157
+96	0 1 0 0	36 28 36	1.33333333333333	-5.55111512312578e-17 -0.222222222222222 -5.55111512312578e-17	0.641345710414296
+96	0 1 0 0	36 32 36	1.33333333333333	-5.55111512312578e-17 -0.111111111111111 -5.55111512312578e-17	0.621410197839124
+96	0 1 0 0	36 36 36	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.61547287921812
+96	0 1 0 0	36 40 36	1.33333333333333	-5.55111512312578e-17 0.111111111111111 -5.55111512312578e-17	0.621410197839125
+96	0 1 0 0	36 44 36	1.33333333333333	-5.55111512312578e-17 0.222222222222222 -5.55111512312578e-17	0.641345710414297
+96	0 1 0 0	36 48 36	1.33333333333333	-5.55111512312578e-17 0.333333333333333 -5.55111512312578e-17	0.668908746807157
+96	0 1 0 0	36 52 36	1.33333333333333	-5.55111512312578e-17 0.444444444444444 -5.55111512312578e-17	0.698215010923196
+96	0 1 0 0	36 56 36	1.33333333333333	-5.55111512312578e-17 0.555555555555555 -5.55111512312578e-17	0.729928937348657
+96	0 1 0 0	36 60 36	1.33333333333333	-5.55111512312578e-17 0.666666666666667 -5.55111512312578e-17	0.76190136189151
+96	0 1 0 0	36 64 36	1.33333333333333	-5.55111512312578e-17 0.777777777777778 -5.55111512312578e-17	0.796688784792057
+
+
+# iteration 96
+# refinement level 2   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 2 0 0	36 22 36	1.33333333333333	-5.55111512312578e-17 -0.388888888888889 -5.55111512312578e-17	0.683302448238139
+96	0 2 0 0	36 24 36	1.33333333333333	-5.55111512312578e-17 -0.333333333333333 -5.55111512312578e-17	0.668908746807157
+96	0 2 0 0	36 26 36	1.33333333333333	-5.55111512312578e-17 -0.277777777777778 -5.55111512312578e-17	0.654541556639422
+96	0 2 0 0	36 28 36	1.33333333333333	-5.55111512312578e-17 -0.222222222222222 -5.55111512312578e-17	0.641345710414296
+96	0 2 0 0	36 30 36	1.33333333333333	-5.55111512312578e-17 -0.166666666666667 -5.55111512312578e-17	0.630026346765969
+96	0 2 0 0	36 32 36	1.33333333333333	-5.55111512312578e-17 -0.111111111111111 -5.55111512312578e-17	0.621410197839124
+96	0 2 0 0	36 34 36	1.33333333333333	-5.55111512312578e-17 -0.0555555555555556 -5.55111512312578e-17	0.616824486578861
+96	0 2 0 0	36 36 36	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.61547287921812
+96	0 2 0 0	36 38 36	1.33333333333333	-5.55111512312578e-17 0.0555555555555555 -5.55111512312578e-17	0.616824486578861
+96	0 2 0 0	36 40 36	1.33333333333333	-5.55111512312578e-17 0.111111111111111 -5.55111512312578e-17	0.621410197839125
+96	0 2 0 0	36 42 36	1.33333333333333	-5.55111512312578e-17 0.166666666666667 -5.55111512312578e-17	0.63002634676597
+96	0 2 0 0	36 44 36	1.33333333333333	-5.55111512312578e-17 0.222222222222222 -5.55111512312578e-17	0.641345710414297
+96	0 2 0 0	36 46 36	1.33333333333333	-5.55111512312578e-17 0.277777777777778 -5.55111512312578e-17	0.654541556639422
+96	0 2 0 0	36 48 36	1.33333333333333	-5.55111512312578e-17 0.333333333333333 -5.55111512312578e-17	0.668908746807157
+96	0 2 0 0	36 50 36	1.33333333333333	-5.55111512312578e-17 0.388888888888889 -5.55111512312578e-17	0.683302448238139
+
+
+# iteration 96
+# refinement level 3   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 3 0 0	36 29 36	1.33333333333333	-5.55111512312578e-17 -0.194444444444445 -5.55111512312578e-17	0.63539979750899
+96	0 3 0 0	36 30 36	1.33333333333333	-5.55111512312578e-17 -0.166666666666667 -5.55111512312578e-17	0.630026346765969
+96	0 3 0 0	36 31 36	1.33333333333333	-5.55111512312578e-17 -0.138888888888889 -5.55111512312578e-17	0.625297419028293
+96	0 3 0 0	36 32 36	1.33333333333333	-5.55111512312578e-17 -0.111111111111111 -5.55111512312578e-17	0.621410197839124
+96	0 3 0 0	36 33 36	1.33333333333333	-5.55111512312578e-17 -0.0833333333333334 -5.55111512312578e-17	0.618663308361111
+96	0 3 0 0	36 34 36	1.33333333333333	-5.55111512312578e-17 -0.0555555555555556 -5.55111512312578e-17	0.616824486578861
+96	0 3 0 0	36 35 36	1.33333333333333	-5.55111512312578e-17 -0.0277777777777778 -5.55111512312578e-17	0.615777600484678
+96	0 3 0 0	36 36 36	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.61547287921812
+96	0 3 0 0	36 37 36	1.33333333333333	-5.55111512312578e-17 0.0277777777777777 -5.55111512312578e-17	0.615777600484678
+96	0 3 0 0	36 38 36	1.33333333333333	-5.55111512312578e-17 0.0555555555555555 -5.55111512312578e-17	0.616824486578861
+96	0 3 0 0	36 39 36	1.33333333333333	-5.55111512312578e-17 0.0833333333333333 -5.55111512312578e-17	0.618663308361111
+96	0 3 0 0	36 40 36	1.33333333333333	-5.55111512312578e-17 0.111111111111111 -5.55111512312578e-17	0.621410197839125
+96	0 3 0 0	36 41 36	1.33333333333333	-5.55111512312578e-17 0.138888888888889 -5.55111512312578e-17	0.625297419028293
+96	0 3 0 0	36 42 36	1.33333333333333	-5.55111512312578e-17 0.166666666666667 -5.55111512312578e-17	0.63002634676597
+96	0 3 0 0	36 43 36	1.33333333333333	-5.55111512312578e-17 0.194444444444444 -5.55111512312578e-17	0.63539979750899
+
+
+# iteration 128
+# refinement level 0   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 0 0 0	40 0 40	1.77777777777778	0.111111111111111 -1 0.111111111111111	0.873700996777018
+128	0 0 0 0	40 8 40	1.77777777777778	0.111111111111111 -0.777777777777778 0.111111111111111	0.859471717622308
+128	0 0 0 0	40 16 40	1.77777777777778	0.111111111111111 -0.555555555555556 0.111111111111111	0.863004424640887
+128	0 0 0 0	40 24 40	1.77777777777778	0.111111111111111 -0.333333333333333 0.111111111111111	0.883486952153519
+128	0 0 0 0	40 32 40	1.77777777777778	0.111111111111111 -0.111111111111111 0.111111111111111	0.915799135771738
+128	0 0 0 0	40 40 40	1.77777777777778	0.111111111111111 0.111111111111111 0.111111111111111	0.915799135771738
+128	0 0 0 0	40 48 40	1.77777777777778	0.111111111111111 0.333333333333333 0.111111111111111	0.883486952153519
+128	0 0 0 0	40 56 40	1.77777777777778	0.111111111111111 0.555555555555556 0.111111111111111	0.863004424640887
+128	0 0 0 0	40 64 40	1.77777777777778	0.111111111111111 0.777777777777778 0.111111111111111	0.859471717622308
+128	0 0 0 0	40 72 40	1.77777777777778	0.111111111111111 1 0.111111111111111	0.873700996777018
+
+
+# iteration 128
+# refinement level 1   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 1 0 0	36 8 36	1.77777777777778	-5.55111512312578e-17 -0.777777777777778 -5.55111512312578e-17	0.862653663591007
+128	0 1 0 0	36 12 36	1.77777777777778	-5.55111512312578e-17 -0.666666666666667 -5.55111512312578e-17	0.862560028606905
+128	0 1 0 0	36 16 36	1.77777777777778	-5.55111512312578e-17 -0.555555555555556 -5.55111512312578e-17	0.866965759785718
+128	0 1 0 0	36 20 36	1.77777777777778	-5.55111512312578e-17 -0.444444444444444 -5.55111512312578e-17	0.876167092595071
+128	0 1 0 0	36 24 36	1.77777777777778	-5.55111512312578e-17 -0.333333333333333 -5.55111512312578e-17	0.889185421864397
+128	0 1 0 0	36 28 36	1.77777777777778	-5.55111512312578e-17 -0.222222222222222 -5.55111512312578e-17	0.907998184681559
+128	0 1 0 0	36 32 36	1.77777777777778	-5.55111512312578e-17 -0.111111111111111 -5.55111512312578e-17	0.924033489738888
+128	0 1 0 0	36 36 36	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.928389498223199
+128	0 1 0 0	36 40 36	1.77777777777778	-5.55111512312578e-17 0.111111111111111 -5.55111512312578e-17	0.924033489738888
+128	0 1 0 0	36 44 36	1.77777777777778	-5.55111512312578e-17 0.222222222222222 -5.55111512312578e-17	0.907998184681559
+128	0 1 0 0	36 48 36	1.77777777777778	-5.55111512312578e-17 0.333333333333333 -5.55111512312578e-17	0.889185421864397
+128	0 1 0 0	36 52 36	1.77777777777778	-5.55111512312578e-17 0.444444444444444 -5.55111512312578e-17	0.876167092595071
+128	0 1 0 0	36 56 36	1.77777777777778	-5.55111512312578e-17 0.555555555555555 -5.55111512312578e-17	0.866965759785718
+128	0 1 0 0	36 60 36	1.77777777777778	-5.55111512312578e-17 0.666666666666667 -5.55111512312578e-17	0.862560028606905
+128	0 1 0 0	36 64 36	1.77777777777778	-5.55111512312578e-17 0.777777777777778 -5.55111512312578e-17	0.862653663591007
+
+
+# iteration 128
+# refinement level 2   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 2 0 0	36 22 36	1.77777777777778	-5.55111512312578e-17 -0.388888888888889 -5.55111512312578e-17	0.882075542854246
+128	0 2 0 0	36 24 36	1.77777777777778	-5.55111512312578e-17 -0.333333333333333 -5.55111512312578e-17	0.889185421864397
+128	0 2 0 0	36 26 36	1.77777777777778	-5.55111512312578e-17 -0.277777777777778 -5.55111512312578e-17	0.898403242286228
+128	0 2 0 0	36 28 36	1.77777777777778	-5.55111512312578e-17 -0.222222222222222 -5.55111512312578e-17	0.907998184681559
+128	0 2 0 0	36 30 36	1.77777777777778	-5.55111512312578e-17 -0.166666666666667 -5.55111512312578e-17	0.916919384356027
+128	0 2 0 0	36 32 36	1.77777777777778	-5.55111512312578e-17 -0.111111111111111 -5.55111512312578e-17	0.924033489738888
+128	0 2 0 0	36 34 36	1.77777777777778	-5.55111512312578e-17 -0.0555555555555556 -5.55111512312578e-17	0.927485951077396
+128	0 2 0 0	36 36 36	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.928389498223199
+128	0 2 0 0	36 38 36	1.77777777777778	-5.55111512312578e-17 0.0555555555555555 -5.55111512312578e-17	0.927485951077396
+128	0 2 0 0	36 40 36	1.77777777777778	-5.55111512312578e-17 0.111111111111111 -5.55111512312578e-17	0.924033489738888
+128	0 2 0 0	36 42 36	1.77777777777778	-5.55111512312578e-17 0.166666666666667 -5.55111512312578e-17	0.916919384356027
+128	0 2 0 0	36 44 36	1.77777777777778	-5.55111512312578e-17 0.222222222222222 -5.55111512312578e-17	0.907998184681559
+128	0 2 0 0	36 46 36	1.77777777777778	-5.55111512312578e-17 0.277777777777778 -5.55111512312578e-17	0.898403242286228
+128	0 2 0 0	36 48 36	1.77777777777778	-5.55111512312578e-17 0.333333333333333 -5.55111512312578e-17	0.889185421864397
+128	0 2 0 0	36 50 36	1.77777777777778	-5.55111512312578e-17 0.388888888888889 -5.55111512312578e-17	0.882075542854246
+
+
+# iteration 128
+# refinement level 3   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 3 0 0	36 29 36	1.77777777777778	-5.55111512312578e-17 -0.194444444444445 -5.55111512312578e-17	0.912613836832073
+128	0 3 0 0	36 30 36	1.77777777777778	-5.55111512312578e-17 -0.166666666666667 -5.55111512312578e-17	0.916919384356027
+128	0 3 0 0	36 31 36	1.77777777777778	-5.55111512312578e-17 -0.138888888888889 -5.55111512312578e-17	0.920818233193455
+128	0 3 0 0	36 32 36	1.77777777777778	-5.55111512312578e-17 -0.111111111111111 -5.55111512312578e-17	0.924033489738888
+128	0 3 0 0	36 33 36	1.77777777777778	-5.55111512312578e-17 -0.0833333333333334 -5.55111512312578e-17	0.926147880297958
+128	0 3 0 0	36 34 36	1.77777777777778	-5.55111512312578e-17 -0.0555555555555556 -5.55111512312578e-17	0.927485951077396
+128	0 3 0 0	36 35 36	1.77777777777778	-5.55111512312578e-17 -0.0277777777777778 -5.55111512312578e-17	0.928209975180567
+128	0 3 0 0	36 36 36	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.928389498223199
+128	0 3 0 0	36 37 36	1.77777777777778	-5.55111512312578e-17 0.0277777777777777 -5.55111512312578e-17	0.928209975180567
+128	0 3 0 0	36 38 36	1.77777777777778	-5.55111512312578e-17 0.0555555555555555 -5.55111512312578e-17	0.927485951077396
+128	0 3 0 0	36 39 36	1.77777777777778	-5.55111512312578e-17 0.0833333333333333 -5.55111512312578e-17	0.926147880297958
+128	0 3 0 0	36 40 36	1.77777777777778	-5.55111512312578e-17 0.111111111111111 -5.55111512312578e-17	0.924033489738888
+128	0 3 0 0	36 41 36	1.77777777777778	-5.55111512312578e-17 0.138888888888889 -5.55111512312578e-17	0.920818233193455
+128	0 3 0 0	36 42 36	1.77777777777778	-5.55111512312578e-17 0.166666666666667 -5.55111512312578e-17	0.916919384356027
+128	0 3 0 0	36 43 36	1.77777777777778	-5.55111512312578e-17 0.194444444444444 -5.55111512312578e-17	0.912613836832073
+
+
diff --git a/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_1proc/phi.z.asc b/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_1proc/phi.z.asc
new file mode 100644
index 000000000..8868db22c
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_1proc/phi.z.asc
@@ -0,0 +1,227 @@
+# WAVETOY::phi z (phi)
+#
+# iteration 64
+# refinement level 0   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 0 0 0	40 40 0	0.888888888888889	0.111111111111111 0.111111111111111 -1	0.873700996777018
+64	0 0 0 0	40 40 8	0.888888888888889	0.111111111111111 0.111111111111111 -0.777777777777778	0.807676282684182
+64	0 0 0 0	40 40 16	0.888888888888889	0.111111111111111 0.111111111111111 -0.555555555555556	0.756699260944861
+64	0 0 0 0	40 40 24	0.888888888888889	0.111111111111111 0.111111111111111 -0.333333333333333	0.730294733882952
+64	0 0 0 0	40 40 32	0.888888888888889	0.111111111111111 0.111111111111111 -0.111111111111111	0.725539249473295
+64	0 0 0 0	40 40 40	0.888888888888889	0.111111111111111 0.111111111111111 0.111111111111111	0.725539249473295
+64	0 0 0 0	40 40 48	0.888888888888889	0.111111111111111 0.111111111111111 0.333333333333333	0.730294733882952
+64	0 0 0 0	40 40 56	0.888888888888889	0.111111111111111 0.111111111111111 0.555555555555556	0.756699260944861
+64	0 0 0 0	40 40 64	0.888888888888889	0.111111111111111 0.111111111111111 0.777777777777778	0.807676282684182
+64	0 0 0 0	40 40 72	0.888888888888889	0.111111111111111 0.111111111111111 1	0.873700996777018
+
+
+# iteration 64
+# refinement level 1   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 1 0 0	36 36 8	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -0.777777777777778	0.808379332677938
+64	0 1 0 0	36 36 12	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -0.666666666666667	0.779665123251562
+64	0 1 0 0	36 36 16	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -0.555555555555556	0.756097245006156
+64	0 1 0 0	36 36 20	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -0.444444444444444	0.739694852518638
+64	0 1 0 0	36 36 24	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -0.333333333333333	0.729218296316213
+64	0 1 0 0	36 36 28	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -0.222222222222222	0.725100663327574
+64	0 1 0 0	36 36 32	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -0.111111111111111	0.724342898925177
+64	0 1 0 0	36 36 36	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.723733474251298
+64	0 1 0 0	36 36 40	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 0.111111111111111	0.724342898925177
+64	0 1 0 0	36 36 44	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 0.222222222222222	0.725100663327574
+64	0 1 0 0	36 36 48	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 0.333333333333333	0.729218296316213
+64	0 1 0 0	36 36 52	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 0.444444444444444	0.739694852518638
+64	0 1 0 0	36 36 56	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 0.555555555555555	0.756097245006155
+64	0 1 0 0	36 36 60	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 0.666666666666667	0.779665123251561
+64	0 1 0 0	36 36 64	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 0.777777777777778	0.808379332677938
+
+
+# iteration 64
+# refinement level 2   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 2 0 0	36 36 22	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -0.388888888888889	0.733688776948746
+64	0 2 0 0	36 36 24	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -0.333333333333333	0.729218296316213
+64	0 2 0 0	36 36 26	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -0.277777777777778	0.726552055334392
+64	0 2 0 0	36 36 28	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -0.222222222222222	0.725100663327574
+64	0 2 0 0	36 36 30	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -0.166666666666667	0.724502518106703
+64	0 2 0 0	36 36 32	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -0.111111111111111	0.724342898925177
+64	0 2 0 0	36 36 34	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -0.0555555555555556	0.72395273727097
+64	0 2 0 0	36 36 36	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.723733474251298
+64	0 2 0 0	36 36 38	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 0.0555555555555555	0.72395273727097
+64	0 2 0 0	36 36 40	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 0.111111111111111	0.724342898925177
+64	0 2 0 0	36 36 42	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 0.166666666666667	0.724502518106703
+64	0 2 0 0	36 36 44	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 0.222222222222222	0.725100663327574
+64	0 2 0 0	36 36 46	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 0.277777777777778	0.726552055334391
+64	0 2 0 0	36 36 48	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 0.333333333333333	0.729218296316213
+64	0 2 0 0	36 36 50	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 0.388888888888889	0.733688776948745
+
+
+# iteration 64
+# refinement level 3   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 3 0 0	36 36 29	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -0.194444444444445	0.724720854915558
+64	0 3 0 0	36 36 30	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -0.166666666666667	0.724502518106703
+64	0 3 0 0	36 36 31	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -0.138888888888889	0.724409709543024
+64	0 3 0 0	36 36 32	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -0.111111111111111	0.724342898925177
+64	0 3 0 0	36 36 33	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -0.0833333333333334	0.724151545837958
+64	0 3 0 0	36 36 34	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -0.0555555555555556	0.72395273727097
+64	0 3 0 0	36 36 35	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -0.0277777777777778	0.723805016719017
+64	0 3 0 0	36 36 36	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.723733474251298
+64	0 3 0 0	36 36 37	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 0.0277777777777777	0.723805016719017
+64	0 3 0 0	36 36 38	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 0.0555555555555555	0.72395273727097
+64	0 3 0 0	36 36 39	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 0.0833333333333333	0.724151545837958
+64	0 3 0 0	36 36 40	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 0.111111111111111	0.724342898925177
+64	0 3 0 0	36 36 41	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 0.138888888888889	0.724409709543024
+64	0 3 0 0	36 36 42	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 0.166666666666667	0.724502518106703
+64	0 3 0 0	36 36 43	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 0.194444444444444	0.724720854915557
+
+
+# iteration 96
+# refinement level 0   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 0 0 0	40 40 0	1.33333333333333	0.111111111111111 0.111111111111111 -1	0.877480603359996
+96	0 0 0 0	40 40 8	1.33333333333333	0.111111111111111 0.111111111111111 -0.777777777777778	0.798008089475601
+96	0 0 0 0	40 40 16	1.33333333333333	0.111111111111111 0.111111111111111 -0.555555555555556	0.735090597558599
+96	0 0 0 0	40 40 24	1.33333333333333	0.111111111111111 0.111111111111111 -0.333333333333333	0.677899959110667
+96	0 0 0 0	40 40 32	1.33333333333333	0.111111111111111 0.111111111111111 -0.111111111111111	0.632828813666694
+96	0 0 0 0	40 40 40	1.33333333333333	0.111111111111111 0.111111111111111 0.111111111111111	0.632828813666694
+96	0 0 0 0	40 40 48	1.33333333333333	0.111111111111111 0.111111111111111 0.333333333333333	0.677899959110667
+96	0 0 0 0	40 40 56	1.33333333333333	0.111111111111111 0.111111111111111 0.555555555555556	0.735090597558599
+96	0 0 0 0	40 40 64	1.33333333333333	0.111111111111111 0.111111111111111 0.777777777777778	0.798008089475601
+96	0 0 0 0	40 40 72	1.33333333333333	0.111111111111111 0.111111111111111 1	0.877480603359996
+
+
+# iteration 96
+# refinement level 1   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 1 0 0	36 36 8	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -0.777777777777778	0.796688784792057
+96	0 1 0 0	36 36 12	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -0.666666666666667	0.76190136189151
+96	0 1 0 0	36 36 16	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -0.555555555555556	0.729928937348657
+96	0 1 0 0	36 36 20	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -0.444444444444444	0.698215010923196
+96	0 1 0 0	36 36 24	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -0.333333333333333	0.668908746807157
+96	0 1 0 0	36 36 28	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -0.222222222222222	0.641345710414296
+96	0 1 0 0	36 36 32	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -0.111111111111111	0.621410197839124
+96	0 1 0 0	36 36 36	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.61547287921812
+96	0 1 0 0	36 36 40	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 0.111111111111111	0.621410197839124
+96	0 1 0 0	36 36 44	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 0.222222222222222	0.641345710414297
+96	0 1 0 0	36 36 48	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 0.333333333333333	0.668908746807157
+96	0 1 0 0	36 36 52	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 0.444444444444444	0.698215010923196
+96	0 1 0 0	36 36 56	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 0.555555555555555	0.729928937348657
+96	0 1 0 0	36 36 60	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 0.666666666666667	0.76190136189151
+96	0 1 0 0	36 36 64	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 0.777777777777778	0.796688784792057
+
+
+# iteration 96
+# refinement level 2   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 2 0 0	36 36 22	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -0.388888888888889	0.683302448238139
+96	0 2 0 0	36 36 24	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -0.333333333333333	0.668908746807157
+96	0 2 0 0	36 36 26	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -0.277777777777778	0.654541556639422
+96	0 2 0 0	36 36 28	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -0.222222222222222	0.641345710414296
+96	0 2 0 0	36 36 30	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -0.166666666666667	0.630026346765969
+96	0 2 0 0	36 36 32	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -0.111111111111111	0.621410197839124
+96	0 2 0 0	36 36 34	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -0.0555555555555556	0.616824486578861
+96	0 2 0 0	36 36 36	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.61547287921812
+96	0 2 0 0	36 36 38	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 0.0555555555555555	0.616824486578861
+96	0 2 0 0	36 36 40	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 0.111111111111111	0.621410197839124
+96	0 2 0 0	36 36 42	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 0.166666666666667	0.63002634676597
+96	0 2 0 0	36 36 44	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 0.222222222222222	0.641345710414297
+96	0 2 0 0	36 36 46	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 0.277777777777778	0.654541556639422
+96	0 2 0 0	36 36 48	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 0.333333333333333	0.668908746807157
+96	0 2 0 0	36 36 50	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 0.388888888888889	0.683302448238139
+
+
+# iteration 96
+# refinement level 3   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 3 0 0	36 36 29	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -0.194444444444445	0.63539979750899
+96	0 3 0 0	36 36 30	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -0.166666666666667	0.630026346765969
+96	0 3 0 0	36 36 31	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -0.138888888888889	0.625297419028293
+96	0 3 0 0	36 36 32	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -0.111111111111111	0.621410197839124
+96	0 3 0 0	36 36 33	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -0.0833333333333334	0.618663308361111
+96	0 3 0 0	36 36 34	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -0.0555555555555556	0.616824486578861
+96	0 3 0 0	36 36 35	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -0.0277777777777778	0.615777600484678
+96	0 3 0 0	36 36 36	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.61547287921812
+96	0 3 0 0	36 36 37	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 0.0277777777777777	0.615777600484678
+96	0 3 0 0	36 36 38	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 0.0555555555555555	0.616824486578861
+96	0 3 0 0	36 36 39	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 0.0833333333333333	0.618663308361111
+96	0 3 0 0	36 36 40	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 0.111111111111111	0.621410197839124
+96	0 3 0 0	36 36 41	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 0.138888888888889	0.625297419028293
+96	0 3 0 0	36 36 42	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 0.166666666666667	0.63002634676597
+96	0 3 0 0	36 36 43	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 0.194444444444444	0.63539979750899
+
+
+# iteration 128
+# refinement level 0   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 0 0 0	40 40 0	1.77777777777778	0.111111111111111 0.111111111111111 -1	0.873700996777018
+128	0 0 0 0	40 40 8	1.77777777777778	0.111111111111111 0.111111111111111 -0.777777777777778	0.859471717622308
+128	0 0 0 0	40 40 16	1.77777777777778	0.111111111111111 0.111111111111111 -0.555555555555556	0.863004424640887
+128	0 0 0 0	40 40 24	1.77777777777778	0.111111111111111 0.111111111111111 -0.333333333333333	0.883486952153519
+128	0 0 0 0	40 40 32	1.77777777777778	0.111111111111111 0.111111111111111 -0.111111111111111	0.915799135771738
+128	0 0 0 0	40 40 40	1.77777777777778	0.111111111111111 0.111111111111111 0.111111111111111	0.915799135771738
+128	0 0 0 0	40 40 48	1.77777777777778	0.111111111111111 0.111111111111111 0.333333333333333	0.883486952153519
+128	0 0 0 0	40 40 56	1.77777777777778	0.111111111111111 0.111111111111111 0.555555555555556	0.863004424640887
+128	0 0 0 0	40 40 64	1.77777777777778	0.111111111111111 0.111111111111111 0.777777777777778	0.859471717622308
+128	0 0 0 0	40 40 72	1.77777777777778	0.111111111111111 0.111111111111111 1	0.873700996777018
+
+
+# iteration 128
+# refinement level 1   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 1 0 0	36 36 8	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -0.777777777777778	0.862653663591007
+128	0 1 0 0	36 36 12	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -0.666666666666667	0.862560028606905
+128	0 1 0 0	36 36 16	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -0.555555555555556	0.866965759785718
+128	0 1 0 0	36 36 20	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -0.444444444444444	0.876167092595071
+128	0 1 0 0	36 36 24	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -0.333333333333333	0.889185421864397
+128	0 1 0 0	36 36 28	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -0.222222222222222	0.90799818468156
+128	0 1 0 0	36 36 32	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -0.111111111111111	0.924033489738888
+128	0 1 0 0	36 36 36	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.928389498223199
+128	0 1 0 0	36 36 40	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 0.111111111111111	0.924033489738888
+128	0 1 0 0	36 36 44	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 0.222222222222222	0.90799818468156
+128	0 1 0 0	36 36 48	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 0.333333333333333	0.889185421864397
+128	0 1 0 0	36 36 52	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 0.444444444444444	0.876167092595071
+128	0 1 0 0	36 36 56	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 0.555555555555555	0.866965759785718
+128	0 1 0 0	36 36 60	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 0.666666666666667	0.862560028606905
+128	0 1 0 0	36 36 64	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 0.777777777777778	0.862653663591007
+
+
+# iteration 128
+# refinement level 2   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 2 0 0	36 36 22	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -0.388888888888889	0.882075542854246
+128	0 2 0 0	36 36 24	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -0.333333333333333	0.889185421864397
+128	0 2 0 0	36 36 26	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -0.277777777777778	0.898403242286228
+128	0 2 0 0	36 36 28	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -0.222222222222222	0.90799818468156
+128	0 2 0 0	36 36 30	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -0.166666666666667	0.916919384356027
+128	0 2 0 0	36 36 32	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -0.111111111111111	0.924033489738888
+128	0 2 0 0	36 36 34	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -0.0555555555555556	0.927485951077396
+128	0 2 0 0	36 36 36	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.928389498223199
+128	0 2 0 0	36 36 38	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 0.0555555555555555	0.927485951077396
+128	0 2 0 0	36 36 40	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 0.111111111111111	0.924033489738888
+128	0 2 0 0	36 36 42	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 0.166666666666667	0.916919384356027
+128	0 2 0 0	36 36 44	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 0.222222222222222	0.90799818468156
+128	0 2 0 0	36 36 46	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 0.277777777777778	0.898403242286228
+128	0 2 0 0	36 36 48	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 0.333333333333333	0.889185421864397
+128	0 2 0 0	36 36 50	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 0.388888888888889	0.882075542854246
+
+
+# iteration 128
+# refinement level 3   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 3 0 0	36 36 29	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -0.194444444444445	0.912613836832073
+128	0 3 0 0	36 36 30	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -0.166666666666667	0.916919384356027
+128	0 3 0 0	36 36 31	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -0.138888888888889	0.920818233193455
+128	0 3 0 0	36 36 32	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -0.111111111111111	0.924033489738888
+128	0 3 0 0	36 36 33	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -0.0833333333333334	0.926147880297958
+128	0 3 0 0	36 36 34	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -0.0555555555555556	0.927485951077396
+128	0 3 0 0	36 36 35	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -0.0277777777777778	0.928209975180567
+128	0 3 0 0	36 36 36	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.928389498223199
+128	0 3 0 0	36 36 37	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 0.0277777777777777	0.928209975180567
+128	0 3 0 0	36 36 38	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 0.0555555555555555	0.927485951077396
+128	0 3 0 0	36 36 39	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 0.0833333333333333	0.926147880297958
+128	0 3 0 0	36 36 40	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 0.111111111111111	0.924033489738888
+128	0 3 0 0	36 36 41	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 0.138888888888889	0.920818233193455
+128	0 3 0 0	36 36 42	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 0.166666666666667	0.916919384356027
+128	0 3 0 0	36 36 43	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 0.194444444444444	0.912613836832073
+
+
diff --git a/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_1proc/phi_maximum.xg b/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_1proc/phi_maximum.xg
new file mode 100644
index 000000000..e25d17d43
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_1proc/phi_maximum.xg
@@ -0,0 +1,4 @@
+"phi v time
+0.8888888888889	0.8737009967770
+1.3333333333333	0.8774806033600
+1.7777777777778	0.9157991357717
diff --git a/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_1proc/phi_minimum.xg b/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_1proc/phi_minimum.xg
new file mode 100644
index 000000000..973172b6d
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_1proc/phi_minimum.xg
@@ -0,0 +1,4 @@
+"phi v time
+0.8888888888889	0.6796516032334
+1.3333333333333	0.6328288136667
+1.7777777777778	0.6796516032334
diff --git a/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_1proc/phi_norm1.xg b/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_1proc/phi_norm1.xg
new file mode 100644
index 000000000..6e8792bca
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_1proc/phi_norm1.xg
@@ -0,0 +1,4 @@
+"phi v time
+0.8888888888889	0.7927895664365
+1.3333333333333	0.7914772512610
+1.7777777777778	0.8208628581415
diff --git a/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_1proc/phi_norm2.xg b/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_1proc/phi_norm2.xg
new file mode 100644
index 000000000..124c22eda
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_1proc/phi_norm2.xg
@@ -0,0 +1,4 @@
+"phi v time
+0.8888888888889	0.7937177688028
+1.3333333333333	0.7927587735574
+1.7777777777778	0.8218200985038
diff --git a/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_4proc.par b/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_4proc.par
new file mode 100644
index 000000000..cb72b9b60
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_4proc.par
@@ -0,0 +1,14 @@
+ActiveThorns = "Boundary CartGrid3D Time CoordBase SymBase Carpet CarpetLib CarpetRegrid CarpetReduce CarpetSlab IOUtil IOBasic CarpetIOASCII CarpetIOHDF5 IDScalarWaveC WaveToyC"
+
+
+Cactus::cctk_itlast = 128
+
+# Output
+IO::out_dir = $parfile
+
+# Recovery
+IO::recover_dir  = "../../../arrangements/Carpet/CarpetIOHDF5/test"
+IO::recover      = "manual"
+IO::recover_file = "CarpetWaveToyCheckpoint_test.it_64"
+
+IOHDF5::checkpoint = "no"
diff --git a/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_4proc/phi.x.asc b/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_4proc/phi.x.asc
new file mode 100644
index 000000000..f0d2110bb
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_4proc/phi.x.asc
@@ -0,0 +1,632 @@
+# WAVETOY::phi x (phi)
+#
+# iteration 64
+# refinement level 0   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 0 0 0	0 40 40	0.888888888888889	-1 0.111111111111111 0.111111111111111	0.873700996777018
+64	0 0 0 0	8 40 40	0.888888888888889	-0.777777777777778 0.111111111111111 0.111111111111111	0.807676282684182
+64	0 0 0 0	16 40 40	0.888888888888889	-0.555555555555556 0.111111111111111 0.111111111111111	0.756699260944861
+64	0 0 0 0	24 40 40	0.888888888888889	-0.333333333333333 0.111111111111111 0.111111111111111	0.730294733882952
+64	0 0 0 0	32 40 40	0.888888888888889	-0.111111111111111 0.111111111111111 0.111111111111111	0.725539249473295
+64	0 0 0 0	40 40 40	0.888888888888889	0.111111111111111 0.111111111111111 0.111111111111111	0.725539249473295
+64	0 0 0 0	48 40 40	0.888888888888889	0.333333333333333 0.111111111111111 0.111111111111111	0.730294733882952
+
+# iteration 64
+# refinement level 0   multigrid level 0   map 0   component 1   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 0 1 0	24 40 40	0.888888888888889	-0.333333333333333 0.111111111111111 0.111111111111111	0.730294733882952
+64	0 0 1 0	32 40 40	0.888888888888889	-0.111111111111111 0.111111111111111 0.111111111111111	0.725539249473295
+64	0 0 1 0	40 40 40	0.888888888888889	0.111111111111111 0.111111111111111 0.111111111111111	0.725539249473295
+64	0 0 1 0	48 40 40	0.888888888888889	0.333333333333333 0.111111111111111 0.111111111111111	0.730294733882952
+64	0 0 1 0	56 40 40	0.888888888888889	0.555555555555556 0.111111111111111 0.111111111111111	0.756699260944861
+64	0 0 1 0	64 40 40	0.888888888888889	0.777777777777778 0.111111111111111 0.111111111111111	0.807676282684182
+64	0 0 1 0	72 40 40	0.888888888888889	1 0.111111111111111 0.111111111111111	0.873700996777018
+
+# iteration 64
+# refinement level 0   multigrid level 0   map 0   component 2   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 0 2 0	0 40 40	0.888888888888889	-1 0.111111111111111 0.111111111111111	0.873700996777018
+64	0 0 2 0	8 40 40	0.888888888888889	-0.777777777777778 0.111111111111111 0.111111111111111	0.807676282684182
+64	0 0 2 0	16 40 40	0.888888888888889	-0.555555555555556 0.111111111111111 0.111111111111111	0.756699260944861
+64	0 0 2 0	24 40 40	0.888888888888889	-0.333333333333333 0.111111111111111 0.111111111111111	0.730294733882952
+64	0 0 2 0	32 40 40	0.888888888888889	-0.111111111111111 0.111111111111111 0.111111111111111	0.725539249473295
+64	0 0 2 0	40 40 40	0.888888888888889	0.111111111111111 0.111111111111111 0.111111111111111	0.725539249473295
+64	0 0 2 0	48 40 40	0.888888888888889	0.333333333333333 0.111111111111111 0.111111111111111	0.730294733882952
+
+# iteration 64
+# refinement level 0   multigrid level 0   map 0   component 3   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 0 3 0	24 40 40	0.888888888888889	-0.333333333333333 0.111111111111111 0.111111111111111	0.730294733882952
+64	0 0 3 0	32 40 40	0.888888888888889	-0.111111111111111 0.111111111111111 0.111111111111111	0.725539249473295
+64	0 0 3 0	40 40 40	0.888888888888889	0.111111111111111 0.111111111111111 0.111111111111111	0.725539249473295
+64	0 0 3 0	48 40 40	0.888888888888889	0.333333333333333 0.111111111111111 0.111111111111111	0.730294733882952
+64	0 0 3 0	56 40 40	0.888888888888889	0.555555555555556 0.111111111111111 0.111111111111111	0.756699260944861
+64	0 0 3 0	64 40 40	0.888888888888889	0.777777777777778 0.111111111111111 0.111111111111111	0.807676282684182
+64	0 0 3 0	72 40 40	0.888888888888889	1 0.111111111111111 0.111111111111111	0.873700996777018
+
+
+# iteration 64
+# refinement level 1   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 1 0 0	8 36 36	0.888888888888889	-0.777777777777778 -5.55111512312578e-17 -9.86623976961809e-17	0.808379332677938
+64	0 1 0 0	12 36 36	0.888888888888889	-0.666666666666667 -5.55111512312578e-17 -9.86623976961809e-17	0.779665123251561
+64	0 1 0 0	16 36 36	0.888888888888889	-0.555555555555556 -5.55111512312578e-17 -9.86623976961809e-17	0.756097245006156
+64	0 1 0 0	20 36 36	0.888888888888889	-0.444444444444444 -5.55111512312578e-17 -9.86623976961809e-17	0.739694852518638
+64	0 1 0 0	24 36 36	0.888888888888889	-0.333333333333333 -5.55111512312578e-17 -9.86623976961809e-17	0.729218296316213
+64	0 1 0 0	28 36 36	0.888888888888889	-0.222222222222222 -5.55111512312578e-17 -9.86623976961809e-17	0.725100663327574
+64	0 1 0 0	32 36 36	0.888888888888889	-0.111111111111111 -5.55111512312578e-17 -9.86623976961809e-17	0.724342898925177
+64	0 1 0 0	36 36 36	0.888888888888889	-9.86623976961809e-17 -5.55111512312578e-17 -9.86623976961809e-17	0.723733474251298
+64	0 1 0 0	40 36 36	0.888888888888889	0.111111111111111 -5.55111512312578e-17 -9.86623976961809e-17	0.724342898925177
+64	0 1 0 0	44 36 36	0.888888888888889	0.222222222222222 -5.55111512312578e-17 -9.86623976961809e-17	0.725100663327574
+
+# iteration 64
+# refinement level 1   multigrid level 0   map 0   component 1   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 1 1 0	32 36 36	0.888888888888889	-0.111111111111111 -5.55111512312578e-17 -9.86623976961809e-17	0.724342898925177
+64	0 1 1 0	36 36 36	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -9.86623976961809e-17	0.723733474251298
+64	0 1 1 0	40 36 36	0.888888888888889	0.111111111111111 -5.55111512312578e-17 -9.86623976961809e-17	0.724342898925177
+64	0 1 1 0	44 36 36	0.888888888888889	0.222222222222222 -5.55111512312578e-17 -9.86623976961809e-17	0.725100663327574
+64	0 1 1 0	48 36 36	0.888888888888889	0.333333333333333 -5.55111512312578e-17 -9.86623976961809e-17	0.729218296316213
+64	0 1 1 0	52 36 36	0.888888888888889	0.444444444444444 -5.55111512312578e-17 -9.86623976961809e-17	0.739694852518638
+64	0 1 1 0	56 36 36	0.888888888888889	0.555555555555555 -5.55111512312578e-17 -9.86623976961809e-17	0.756097245006155
+64	0 1 1 0	60 36 36	0.888888888888889	0.666666666666667 -5.55111512312578e-17 -9.86623976961809e-17	0.779665123251561
+64	0 1 1 0	64 36 36	0.888888888888889	0.777777777777778 -5.55111512312578e-17 -9.86623976961809e-17	0.808379332677938
+
+# iteration 64
+# refinement level 1   multigrid level 0   map 0   component 2   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 1 2 0	8 36 36	0.888888888888889	-0.777777777777778 -5.55111512312578e-17 -5.55111512312578e-17	0.808379332677938
+64	0 1 2 0	12 36 36	0.888888888888889	-0.666666666666667 -5.55111512312578e-17 -5.55111512312578e-17	0.779665123251561
+64	0 1 2 0	16 36 36	0.888888888888889	-0.555555555555556 -5.55111512312578e-17 -5.55111512312578e-17	0.756097245006156
+64	0 1 2 0	20 36 36	0.888888888888889	-0.444444444444444 -5.55111512312578e-17 -5.55111512312578e-17	0.739694852518638
+64	0 1 2 0	24 36 36	0.888888888888889	-0.333333333333333 -5.55111512312578e-17 -5.55111512312578e-17	0.729218296316213
+64	0 1 2 0	28 36 36	0.888888888888889	-0.222222222222222 -5.55111512312578e-17 -5.55111512312578e-17	0.725100663327574
+64	0 1 2 0	32 36 36	0.888888888888889	-0.111111111111111 -5.55111512312578e-17 -5.55111512312578e-17	0.724342898925177
+64	0 1 2 0	36 36 36	0.888888888888889	-9.86623976961809e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.723733474251298
+64	0 1 2 0	40 36 36	0.888888888888889	0.111111111111111 -5.55111512312578e-17 -5.55111512312578e-17	0.724342898925177
+64	0 1 2 0	44 36 36	0.888888888888889	0.222222222222222 -5.55111512312578e-17 -5.55111512312578e-17	0.725100663327574
+
+# iteration 64
+# refinement level 1   multigrid level 0   map 0   component 3   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 1 3 0	32 36 36	0.888888888888889	-0.111111111111111 -5.55111512312578e-17 -5.55111512312578e-17	0.724342898925177
+64	0 1 3 0	36 36 36	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.723733474251298
+64	0 1 3 0	40 36 36	0.888888888888889	0.111111111111111 -5.55111512312578e-17 -5.55111512312578e-17	0.724342898925177
+64	0 1 3 0	44 36 36	0.888888888888889	0.222222222222222 -5.55111512312578e-17 -5.55111512312578e-17	0.725100663327574
+64	0 1 3 0	48 36 36	0.888888888888889	0.333333333333333 -5.55111512312578e-17 -5.55111512312578e-17	0.729218296316213
+64	0 1 3 0	52 36 36	0.888888888888889	0.444444444444444 -5.55111512312578e-17 -5.55111512312578e-17	0.739694852518638
+64	0 1 3 0	56 36 36	0.888888888888889	0.555555555555555 -5.55111512312578e-17 -5.55111512312578e-17	0.756097245006155
+64	0 1 3 0	60 36 36	0.888888888888889	0.666666666666667 -5.55111512312578e-17 -5.55111512312578e-17	0.779665123251561
+64	0 1 3 0	64 36 36	0.888888888888889	0.777777777777778 -5.55111512312578e-17 -5.55111512312578e-17	0.808379332677938
+
+
+# iteration 64
+# refinement level 2   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 2 0 0	22 36 36	0.888888888888889	-0.388888888888889 -5.55111512312578e-17 2.46655994240452e-17	0.733688776948746
+64	0 2 0 0	24 36 36	0.888888888888889	-0.333333333333333 -5.55111512312578e-17 2.46655994240452e-17	0.729218296316213
+64	0 2 0 0	26 36 36	0.888888888888889	-0.277777777777778 -5.55111512312578e-17 2.46655994240452e-17	0.726552055334392
+64	0 2 0 0	28 36 36	0.888888888888889	-0.222222222222222 -5.55111512312578e-17 2.46655994240452e-17	0.725100663327574
+64	0 2 0 0	30 36 36	0.888888888888889	-0.166666666666667 -5.55111512312578e-17 2.46655994240452e-17	0.724502518106703
+64	0 2 0 0	32 36 36	0.888888888888889	-0.111111111111111 -5.55111512312578e-17 2.46655994240452e-17	0.724342898925177
+64	0 2 0 0	34 36 36	0.888888888888889	-0.0555555555555555 -5.55111512312578e-17 2.46655994240452e-17	0.72395273727097
+64	0 2 0 0	36 36 36	0.888888888888889	2.46655994240452e-17 -5.55111512312578e-17 2.46655994240452e-17	0.723733474251298
+64	0 2 0 0	38 36 36	0.888888888888889	0.0555555555555556 -5.55111512312578e-17 2.46655994240452e-17	0.72395273727097
+64	0 2 0 0	40 36 36	0.888888888888889	0.111111111111111 -5.55111512312578e-17 2.46655994240452e-17	0.724342898925177
+
+# iteration 64
+# refinement level 2   multigrid level 0   map 0   component 1   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 2 1 0	34 36 36	0.888888888888889	-0.0555555555555556 -5.55111512312578e-17 2.46655994240452e-17	0.72395273727097
+64	0 2 1 0	36 36 36	0.888888888888889	-2.77555756156289e-17 -5.55111512312578e-17 2.46655994240452e-17	0.723733474251298
+64	0 2 1 0	38 36 36	0.888888888888889	0.0555555555555555 -5.55111512312578e-17 2.46655994240452e-17	0.72395273727097
+64	0 2 1 0	40 36 36	0.888888888888889	0.111111111111111 -5.55111512312578e-17 2.46655994240452e-17	0.724342898925177
+64	0 2 1 0	42 36 36	0.888888888888889	0.166666666666667 -5.55111512312578e-17 2.46655994240452e-17	0.724502518106703
+64	0 2 1 0	44 36 36	0.888888888888889	0.222222222222222 -5.55111512312578e-17 2.46655994240452e-17	0.725100663327574
+64	0 2 1 0	46 36 36	0.888888888888889	0.277777777777778 -5.55111512312578e-17 2.46655994240452e-17	0.726552055334392
+64	0 2 1 0	48 36 36	0.888888888888889	0.333333333333333 -5.55111512312578e-17 2.46655994240452e-17	0.729218296316213
+64	0 2 1 0	50 36 36	0.888888888888889	0.388888888888889 -5.55111512312578e-17 2.46655994240452e-17	0.733688776948745
+
+# iteration 64
+# refinement level 2   multigrid level 0   map 0   component 2   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 2 2 0	22 36 36	0.888888888888889	-0.388888888888889 -5.55111512312578e-17 -2.77555756156289e-17	0.733688776948746
+64	0 2 2 0	24 36 36	0.888888888888889	-0.333333333333333 -5.55111512312578e-17 -2.77555756156289e-17	0.729218296316213
+64	0 2 2 0	26 36 36	0.888888888888889	-0.277777777777778 -5.55111512312578e-17 -2.77555756156289e-17	0.726552055334392
+64	0 2 2 0	28 36 36	0.888888888888889	-0.222222222222222 -5.55111512312578e-17 -2.77555756156289e-17	0.725100663327574
+64	0 2 2 0	30 36 36	0.888888888888889	-0.166666666666667 -5.55111512312578e-17 -2.77555756156289e-17	0.724502518106703
+64	0 2 2 0	32 36 36	0.888888888888889	-0.111111111111111 -5.55111512312578e-17 -2.77555756156289e-17	0.724342898925177
+64	0 2 2 0	34 36 36	0.888888888888889	-0.0555555555555555 -5.55111512312578e-17 -2.77555756156289e-17	0.72395273727097
+64	0 2 2 0	36 36 36	0.888888888888889	2.46655994240452e-17 -5.55111512312578e-17 -2.77555756156289e-17	0.723733474251298
+64	0 2 2 0	38 36 36	0.888888888888889	0.0555555555555556 -5.55111512312578e-17 -2.77555756156289e-17	0.72395273727097
+64	0 2 2 0	40 36 36	0.888888888888889	0.111111111111111 -5.55111512312578e-17 -2.77555756156289e-17	0.724342898925177
+
+# iteration 64
+# refinement level 2   multigrid level 0   map 0   component 3   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 2 3 0	34 36 36	0.888888888888889	-0.0555555555555556 -5.55111512312578e-17 -2.77555756156289e-17	0.72395273727097
+64	0 2 3 0	36 36 36	0.888888888888889	-2.77555756156289e-17 -5.55111512312578e-17 -2.77555756156289e-17	0.723733474251298
+64	0 2 3 0	38 36 36	0.888888888888889	0.0555555555555555 -5.55111512312578e-17 -2.77555756156289e-17	0.72395273727097
+64	0 2 3 0	40 36 36	0.888888888888889	0.111111111111111 -5.55111512312578e-17 -2.77555756156289e-17	0.724342898925177
+64	0 2 3 0	42 36 36	0.888888888888889	0.166666666666667 -5.55111512312578e-17 -2.77555756156289e-17	0.724502518106703
+64	0 2 3 0	44 36 36	0.888888888888889	0.222222222222222 -5.55111512312578e-17 -2.77555756156289e-17	0.725100663327574
+64	0 2 3 0	46 36 36	0.888888888888889	0.277777777777778 -5.55111512312578e-17 -2.77555756156289e-17	0.726552055334392
+64	0 2 3 0	48 36 36	0.888888888888889	0.333333333333333 -5.55111512312578e-17 -2.77555756156289e-17	0.729218296316213
+64	0 2 3 0	50 36 36	0.888888888888889	0.388888888888889 -5.55111512312578e-17 -2.77555756156289e-17	0.733688776948745
+
+
+# iteration 64
+# refinement level 3   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 3 0 0	29 36 36	0.888888888888889	-0.194444444444445 -5.55111512312578e-17 0	0.724720854915558
+64	0 3 0 0	30 36 36	0.888888888888889	-0.166666666666667 -5.55111512312578e-17 0	0.724502518106703
+64	0 3 0 0	31 36 36	0.888888888888889	-0.138888888888889 -5.55111512312578e-17 0	0.724409709543024
+64	0 3 0 0	32 36 36	0.888888888888889	-0.111111111111111 -5.55111512312578e-17 0	0.724342898925177
+64	0 3 0 0	33 36 36	0.888888888888889	-0.0833333333333334 -5.55111512312578e-17 0	0.724151545837958
+64	0 3 0 0	34 36 36	0.888888888888889	-0.0555555555555556 -5.55111512312578e-17 0	0.72395273727097
+64	0 3 0 0	35 36 36	0.888888888888889	-0.0277777777777778 -5.55111512312578e-17 0	0.723805016719017
+64	0 3 0 0	36 36 36	0.888888888888889	0 -5.55111512312578e-17 0	0.723733474251298
+64	0 3 0 0	37 36 36	0.888888888888889	0.0277777777777778 -5.55111512312578e-17 0	0.723805016719017
+64	0 3 0 0	38 36 36	0.888888888888889	0.0555555555555556 -5.55111512312578e-17 0	0.72395273727097
+
+# iteration 64
+# refinement level 3   multigrid level 0   map 0   component 1   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 3 1 0	35 36 36	0.888888888888889	-0.0277777777777778 -5.55111512312578e-17 0	0.723805016719017
+64	0 3 1 0	36 36 36	0.888888888888889	-1.38777878078145e-17 -5.55111512312578e-17 0	0.723733474251298
+64	0 3 1 0	37 36 36	0.888888888888889	0.0277777777777778 -5.55111512312578e-17 0	0.723805016719017
+64	0 3 1 0	38 36 36	0.888888888888889	0.0555555555555555 -5.55111512312578e-17 0	0.72395273727097
+64	0 3 1 0	39 36 36	0.888888888888889	0.0833333333333333 -5.55111512312578e-17 0	0.724151545837958
+64	0 3 1 0	40 36 36	0.888888888888889	0.111111111111111 -5.55111512312578e-17 0	0.724342898925177
+64	0 3 1 0	41 36 36	0.888888888888889	0.138888888888889 -5.55111512312578e-17 0	0.724409709543024
+64	0 3 1 0	42 36 36	0.888888888888889	0.166666666666667 -5.55111512312578e-17 0	0.724502518106703
+64	0 3 1 0	43 36 36	0.888888888888889	0.194444444444444 -5.55111512312578e-17 0	0.724720854915558
+
+# iteration 64
+# refinement level 3   multigrid level 0   map 0   component 2   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 3 2 0	29 36 36	0.888888888888889	-0.194444444444445 -5.55111512312578e-17 -1.38777878078145e-17	0.724720854915558
+64	0 3 2 0	30 36 36	0.888888888888889	-0.166666666666667 -5.55111512312578e-17 -1.38777878078145e-17	0.724502518106703
+64	0 3 2 0	31 36 36	0.888888888888889	-0.138888888888889 -5.55111512312578e-17 -1.38777878078145e-17	0.724409709543024
+64	0 3 2 0	32 36 36	0.888888888888889	-0.111111111111111 -5.55111512312578e-17 -1.38777878078145e-17	0.724342898925177
+64	0 3 2 0	33 36 36	0.888888888888889	-0.0833333333333334 -5.55111512312578e-17 -1.38777878078145e-17	0.724151545837958
+64	0 3 2 0	34 36 36	0.888888888888889	-0.0555555555555556 -5.55111512312578e-17 -1.38777878078145e-17	0.72395273727097
+64	0 3 2 0	35 36 36	0.888888888888889	-0.0277777777777778 -5.55111512312578e-17 -1.38777878078145e-17	0.723805016719017
+64	0 3 2 0	36 36 36	0.888888888888889	0 -5.55111512312578e-17 -1.38777878078145e-17	0.723733474251298
+64	0 3 2 0	37 36 36	0.888888888888889	0.0277777777777778 -5.55111512312578e-17 -1.38777878078145e-17	0.723805016719017
+64	0 3 2 0	38 36 36	0.888888888888889	0.0555555555555556 -5.55111512312578e-17 -1.38777878078145e-17	0.72395273727097
+
+# iteration 64
+# refinement level 3   multigrid level 0   map 0   component 3   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 3 3 0	35 36 36	0.888888888888889	-0.0277777777777778 -5.55111512312578e-17 -1.38777878078145e-17	0.723805016719017
+64	0 3 3 0	36 36 36	0.888888888888889	-1.38777878078145e-17 -5.55111512312578e-17 -1.38777878078145e-17	0.723733474251298
+64	0 3 3 0	37 36 36	0.888888888888889	0.0277777777777778 -5.55111512312578e-17 -1.38777878078145e-17	0.723805016719017
+64	0 3 3 0	38 36 36	0.888888888888889	0.0555555555555555 -5.55111512312578e-17 -1.38777878078145e-17	0.72395273727097
+64	0 3 3 0	39 36 36	0.888888888888889	0.0833333333333333 -5.55111512312578e-17 -1.38777878078145e-17	0.724151545837958
+64	0 3 3 0	40 36 36	0.888888888888889	0.111111111111111 -5.55111512312578e-17 -1.38777878078145e-17	0.724342898925177
+64	0 3 3 0	41 36 36	0.888888888888889	0.138888888888889 -5.55111512312578e-17 -1.38777878078145e-17	0.724409709543024
+64	0 3 3 0	42 36 36	0.888888888888889	0.166666666666667 -5.55111512312578e-17 -1.38777878078145e-17	0.724502518106703
+64	0 3 3 0	43 36 36	0.888888888888889	0.194444444444444 -5.55111512312578e-17 -1.38777878078145e-17	0.724720854915558
+
+
+# iteration 96
+# refinement level 0   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 0 0 0	0 40 40	1.33333333333333	-1 0.111111111111111 0.111111111111111	0.877480603359996
+96	0 0 0 0	8 40 40	1.33333333333333	-0.777777777777778 0.111111111111111 0.111111111111111	0.798008089475601
+96	0 0 0 0	16 40 40	1.33333333333333	-0.555555555555556 0.111111111111111 0.111111111111111	0.735090597558599
+96	0 0 0 0	24 40 40	1.33333333333333	-0.333333333333333 0.111111111111111 0.111111111111111	0.677899959110667
+96	0 0 0 0	32 40 40	1.33333333333333	-0.111111111111111 0.111111111111111 0.111111111111111	0.632828813666694
+96	0 0 0 0	40 40 40	1.33333333333333	0.111111111111111 0.111111111111111 0.111111111111111	0.632828813666694
+96	0 0 0 0	48 40 40	1.33333333333333	0.333333333333333 0.111111111111111 0.111111111111111	0.677899959110667
+
+# iteration 96
+# refinement level 0   multigrid level 0   map 0   component 1   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 0 1 0	24 40 40	1.33333333333333	-0.333333333333333 0.111111111111111 0.111111111111111	0.677899959110667
+96	0 0 1 0	32 40 40	1.33333333333333	-0.111111111111111 0.111111111111111 0.111111111111111	0.632828813666694
+96	0 0 1 0	40 40 40	1.33333333333333	0.111111111111111 0.111111111111111 0.111111111111111	0.632828813666694
+96	0 0 1 0	48 40 40	1.33333333333333	0.333333333333333 0.111111111111111 0.111111111111111	0.677899959110667
+96	0 0 1 0	56 40 40	1.33333333333333	0.555555555555556 0.111111111111111 0.111111111111111	0.735090597558599
+96	0 0 1 0	64 40 40	1.33333333333333	0.777777777777778 0.111111111111111 0.111111111111111	0.798008089475601
+96	0 0 1 0	72 40 40	1.33333333333333	1 0.111111111111111 0.111111111111111	0.877480603359996
+
+# iteration 96
+# refinement level 0   multigrid level 0   map 0   component 2   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 0 2 0	0 40 40	1.33333333333333	-1 0.111111111111111 0.111111111111111	0.877480603359996
+96	0 0 2 0	8 40 40	1.33333333333333	-0.777777777777778 0.111111111111111 0.111111111111111	0.798008089475601
+96	0 0 2 0	16 40 40	1.33333333333333	-0.555555555555556 0.111111111111111 0.111111111111111	0.735090597558599
+96	0 0 2 0	24 40 40	1.33333333333333	-0.333333333333333 0.111111111111111 0.111111111111111	0.677899959110667
+96	0 0 2 0	32 40 40	1.33333333333333	-0.111111111111111 0.111111111111111 0.111111111111111	0.632828813666694
+96	0 0 2 0	40 40 40	1.33333333333333	0.111111111111111 0.111111111111111 0.111111111111111	0.632828813666694
+96	0 0 2 0	48 40 40	1.33333333333333	0.333333333333333 0.111111111111111 0.111111111111111	0.677899959110667
+
+# iteration 96
+# refinement level 0   multigrid level 0   map 0   component 3   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 0 3 0	24 40 40	1.33333333333333	-0.333333333333333 0.111111111111111 0.111111111111111	0.677899959110667
+96	0 0 3 0	32 40 40	1.33333333333333	-0.111111111111111 0.111111111111111 0.111111111111111	0.632828813666694
+96	0 0 3 0	40 40 40	1.33333333333333	0.111111111111111 0.111111111111111 0.111111111111111	0.632828813666694
+96	0 0 3 0	48 40 40	1.33333333333333	0.333333333333333 0.111111111111111 0.111111111111111	0.677899959110667
+96	0 0 3 0	56 40 40	1.33333333333333	0.555555555555556 0.111111111111111 0.111111111111111	0.735090597558599
+96	0 0 3 0	64 40 40	1.33333333333333	0.777777777777778 0.111111111111111 0.111111111111111	0.798008089475601
+96	0 0 3 0	72 40 40	1.33333333333333	1 0.111111111111111 0.111111111111111	0.877480603359996
+
+
+# iteration 96
+# refinement level 1   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 1 0 0	8 36 36	1.33333333333333	-0.777777777777778 -5.55111512312578e-17 -9.86623976961809e-17	0.796688784792057
+96	0 1 0 0	12 36 36	1.33333333333333	-0.666666666666667 -5.55111512312578e-17 -9.86623976961809e-17	0.76190136189151
+96	0 1 0 0	16 36 36	1.33333333333333	-0.555555555555556 -5.55111512312578e-17 -9.86623976961809e-17	0.729928937348657
+96	0 1 0 0	20 36 36	1.33333333333333	-0.444444444444444 -5.55111512312578e-17 -9.86623976961809e-17	0.698215010923196
+96	0 1 0 0	24 36 36	1.33333333333333	-0.333333333333333 -5.55111512312578e-17 -9.86623976961809e-17	0.668908746807157
+96	0 1 0 0	28 36 36	1.33333333333333	-0.222222222222222 -5.55111512312578e-17 -9.86623976961809e-17	0.641345710414296
+96	0 1 0 0	32 36 36	1.33333333333333	-0.111111111111111 -5.55111512312578e-17 -9.86623976961809e-17	0.621410197839124
+96	0 1 0 0	36 36 36	1.33333333333333	-9.86623976961809e-17 -5.55111512312578e-17 -9.86623976961809e-17	0.61547287921812
+96	0 1 0 0	40 36 36	1.33333333333333	0.111111111111111 -5.55111512312578e-17 -9.86623976961809e-17	0.621410197839125
+96	0 1 0 0	44 36 36	1.33333333333333	0.222222222222222 -5.55111512312578e-17 -9.86623976961809e-17	0.641345710414296
+
+# iteration 96
+# refinement level 1   multigrid level 0   map 0   component 1   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 1 1 0	32 36 36	1.33333333333333	-0.111111111111111 -5.55111512312578e-17 -9.86623976961809e-17	0.621410197839124
+96	0 1 1 0	36 36 36	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -9.86623976961809e-17	0.61547287921812
+96	0 1 1 0	40 36 36	1.33333333333333	0.111111111111111 -5.55111512312578e-17 -9.86623976961809e-17	0.621410197839125
+96	0 1 1 0	44 36 36	1.33333333333333	0.222222222222222 -5.55111512312578e-17 -9.86623976961809e-17	0.641345710414296
+96	0 1 1 0	48 36 36	1.33333333333333	0.333333333333333 -5.55111512312578e-17 -9.86623976961809e-17	0.668908746807157
+96	0 1 1 0	52 36 36	1.33333333333333	0.444444444444444 -5.55111512312578e-17 -9.86623976961809e-17	0.698215010923196
+96	0 1 1 0	56 36 36	1.33333333333333	0.555555555555555 -5.55111512312578e-17 -9.86623976961809e-17	0.729928937348657
+96	0 1 1 0	60 36 36	1.33333333333333	0.666666666666667 -5.55111512312578e-17 -9.86623976961809e-17	0.76190136189151
+96	0 1 1 0	64 36 36	1.33333333333333	0.777777777777778 -5.55111512312578e-17 -9.86623976961809e-17	0.796688784792057
+
+# iteration 96
+# refinement level 1   multigrid level 0   map 0   component 2   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 1 2 0	8 36 36	1.33333333333333	-0.777777777777778 -5.55111512312578e-17 -5.55111512312578e-17	0.796688784792057
+96	0 1 2 0	12 36 36	1.33333333333333	-0.666666666666667 -5.55111512312578e-17 -5.55111512312578e-17	0.76190136189151
+96	0 1 2 0	16 36 36	1.33333333333333	-0.555555555555556 -5.55111512312578e-17 -5.55111512312578e-17	0.729928937348657
+96	0 1 2 0	20 36 36	1.33333333333333	-0.444444444444444 -5.55111512312578e-17 -5.55111512312578e-17	0.698215010923196
+96	0 1 2 0	24 36 36	1.33333333333333	-0.333333333333333 -5.55111512312578e-17 -5.55111512312578e-17	0.668908746807157
+96	0 1 2 0	28 36 36	1.33333333333333	-0.222222222222222 -5.55111512312578e-17 -5.55111512312578e-17	0.641345710414296
+96	0 1 2 0	32 36 36	1.33333333333333	-0.111111111111111 -5.55111512312578e-17 -5.55111512312578e-17	0.621410197839124
+96	0 1 2 0	36 36 36	1.33333333333333	-9.86623976961809e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.61547287921812
+96	0 1 2 0	40 36 36	1.33333333333333	0.111111111111111 -5.55111512312578e-17 -5.55111512312578e-17	0.621410197839125
+96	0 1 2 0	44 36 36	1.33333333333333	0.222222222222222 -5.55111512312578e-17 -5.55111512312578e-17	0.641345710414296
+
+# iteration 96
+# refinement level 1   multigrid level 0   map 0   component 3   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 1 3 0	32 36 36	1.33333333333333	-0.111111111111111 -5.55111512312578e-17 -5.55111512312578e-17	0.621410197839124
+96	0 1 3 0	36 36 36	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.61547287921812
+96	0 1 3 0	40 36 36	1.33333333333333	0.111111111111111 -5.55111512312578e-17 -5.55111512312578e-17	0.621410197839125
+96	0 1 3 0	44 36 36	1.33333333333333	0.222222222222222 -5.55111512312578e-17 -5.55111512312578e-17	0.641345710414296
+96	0 1 3 0	48 36 36	1.33333333333333	0.333333333333333 -5.55111512312578e-17 -5.55111512312578e-17	0.668908746807157
+96	0 1 3 0	52 36 36	1.33333333333333	0.444444444444444 -5.55111512312578e-17 -5.55111512312578e-17	0.698215010923196
+96	0 1 3 0	56 36 36	1.33333333333333	0.555555555555555 -5.55111512312578e-17 -5.55111512312578e-17	0.729928937348657
+96	0 1 3 0	60 36 36	1.33333333333333	0.666666666666667 -5.55111512312578e-17 -5.55111512312578e-17	0.76190136189151
+96	0 1 3 0	64 36 36	1.33333333333333	0.777777777777778 -5.55111512312578e-17 -5.55111512312578e-17	0.796688784792057
+
+
+# iteration 96
+# refinement level 2   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 2 0 0	22 36 36	1.33333333333333	-0.388888888888889 -5.55111512312578e-17 2.46655994240452e-17	0.683302448238139
+96	0 2 0 0	24 36 36	1.33333333333333	-0.333333333333333 -5.55111512312578e-17 2.46655994240452e-17	0.668908746807157
+96	0 2 0 0	26 36 36	1.33333333333333	-0.277777777777778 -5.55111512312578e-17 2.46655994240452e-17	0.654541556639422
+96	0 2 0 0	28 36 36	1.33333333333333	-0.222222222222222 -5.55111512312578e-17 2.46655994240452e-17	0.641345710414296
+96	0 2 0 0	30 36 36	1.33333333333333	-0.166666666666667 -5.55111512312578e-17 2.46655994240452e-17	0.630026346765969
+96	0 2 0 0	32 36 36	1.33333333333333	-0.111111111111111 -5.55111512312578e-17 2.46655994240452e-17	0.621410197839124
+96	0 2 0 0	34 36 36	1.33333333333333	-0.0555555555555555 -5.55111512312578e-17 2.46655994240452e-17	0.616824486578861
+96	0 2 0 0	36 36 36	1.33333333333333	2.46655994240452e-17 -5.55111512312578e-17 2.46655994240452e-17	0.61547287921812
+96	0 2 0 0	38 36 36	1.33333333333333	0.0555555555555556 -5.55111512312578e-17 2.46655994240452e-17	0.616824486578861
+96	0 2 0 0	40 36 36	1.33333333333333	0.111111111111111 -5.55111512312578e-17 2.46655994240452e-17	0.621410197839125
+
+# iteration 96
+# refinement level 2   multigrid level 0   map 0   component 1   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 2 1 0	34 36 36	1.33333333333333	-0.0555555555555556 -5.55111512312578e-17 2.46655994240452e-17	0.616824486578861
+96	0 2 1 0	36 36 36	1.33333333333333	-2.77555756156289e-17 -5.55111512312578e-17 2.46655994240452e-17	0.61547287921812
+96	0 2 1 0	38 36 36	1.33333333333333	0.0555555555555555 -5.55111512312578e-17 2.46655994240452e-17	0.616824486578861
+96	0 2 1 0	40 36 36	1.33333333333333	0.111111111111111 -5.55111512312578e-17 2.46655994240452e-17	0.621410197839125
+96	0 2 1 0	42 36 36	1.33333333333333	0.166666666666667 -5.55111512312578e-17 2.46655994240452e-17	0.63002634676597
+96	0 2 1 0	44 36 36	1.33333333333333	0.222222222222222 -5.55111512312578e-17 2.46655994240452e-17	0.641345710414296
+96	0 2 1 0	46 36 36	1.33333333333333	0.277777777777778 -5.55111512312578e-17 2.46655994240452e-17	0.654541556639422
+96	0 2 1 0	48 36 36	1.33333333333333	0.333333333333333 -5.55111512312578e-17 2.46655994240452e-17	0.668908746807157
+96	0 2 1 0	50 36 36	1.33333333333333	0.388888888888889 -5.55111512312578e-17 2.46655994240452e-17	0.683302448238139
+
+# iteration 96
+# refinement level 2   multigrid level 0   map 0   component 2   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 2 2 0	22 36 36	1.33333333333333	-0.388888888888889 -5.55111512312578e-17 -2.77555756156289e-17	0.683302448238139
+96	0 2 2 0	24 36 36	1.33333333333333	-0.333333333333333 -5.55111512312578e-17 -2.77555756156289e-17	0.668908746807157
+96	0 2 2 0	26 36 36	1.33333333333333	-0.277777777777778 -5.55111512312578e-17 -2.77555756156289e-17	0.654541556639422
+96	0 2 2 0	28 36 36	1.33333333333333	-0.222222222222222 -5.55111512312578e-17 -2.77555756156289e-17	0.641345710414296
+96	0 2 2 0	30 36 36	1.33333333333333	-0.166666666666667 -5.55111512312578e-17 -2.77555756156289e-17	0.630026346765969
+96	0 2 2 0	32 36 36	1.33333333333333	-0.111111111111111 -5.55111512312578e-17 -2.77555756156289e-17	0.621410197839124
+96	0 2 2 0	34 36 36	1.33333333333333	-0.0555555555555555 -5.55111512312578e-17 -2.77555756156289e-17	0.616824486578861
+96	0 2 2 0	36 36 36	1.33333333333333	2.46655994240452e-17 -5.55111512312578e-17 -2.77555756156289e-17	0.61547287921812
+96	0 2 2 0	38 36 36	1.33333333333333	0.0555555555555556 -5.55111512312578e-17 -2.77555756156289e-17	0.616824486578861
+96	0 2 2 0	40 36 36	1.33333333333333	0.111111111111111 -5.55111512312578e-17 -2.77555756156289e-17	0.621410197839125
+
+# iteration 96
+# refinement level 2   multigrid level 0   map 0   component 3   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 2 3 0	34 36 36	1.33333333333333	-0.0555555555555556 -5.55111512312578e-17 -2.77555756156289e-17	0.616824486578861
+96	0 2 3 0	36 36 36	1.33333333333333	-2.77555756156289e-17 -5.55111512312578e-17 -2.77555756156289e-17	0.61547287921812
+96	0 2 3 0	38 36 36	1.33333333333333	0.0555555555555555 -5.55111512312578e-17 -2.77555756156289e-17	0.616824486578861
+96	0 2 3 0	40 36 36	1.33333333333333	0.111111111111111 -5.55111512312578e-17 -2.77555756156289e-17	0.621410197839125
+96	0 2 3 0	42 36 36	1.33333333333333	0.166666666666667 -5.55111512312578e-17 -2.77555756156289e-17	0.63002634676597
+96	0 2 3 0	44 36 36	1.33333333333333	0.222222222222222 -5.55111512312578e-17 -2.77555756156289e-17	0.641345710414296
+96	0 2 3 0	46 36 36	1.33333333333333	0.277777777777778 -5.55111512312578e-17 -2.77555756156289e-17	0.654541556639422
+96	0 2 3 0	48 36 36	1.33333333333333	0.333333333333333 -5.55111512312578e-17 -2.77555756156289e-17	0.668908746807157
+96	0 2 3 0	50 36 36	1.33333333333333	0.388888888888889 -5.55111512312578e-17 -2.77555756156289e-17	0.683302448238139
+
+
+# iteration 96
+# refinement level 3   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 3 0 0	29 36 36	1.33333333333333	-0.194444444444445 -5.55111512312578e-17 0	0.63539979750899
+96	0 3 0 0	30 36 36	1.33333333333333	-0.166666666666667 -5.55111512312578e-17 0	0.630026346765969
+96	0 3 0 0	31 36 36	1.33333333333333	-0.138888888888889 -5.55111512312578e-17 0	0.625297419028293
+96	0 3 0 0	32 36 36	1.33333333333333	-0.111111111111111 -5.55111512312578e-17 0	0.621410197839124
+96	0 3 0 0	33 36 36	1.33333333333333	-0.0833333333333334 -5.55111512312578e-17 0	0.618663308361111
+96	0 3 0 0	34 36 36	1.33333333333333	-0.0555555555555556 -5.55111512312578e-17 0	0.616824486578861
+96	0 3 0 0	35 36 36	1.33333333333333	-0.0277777777777778 -5.55111512312578e-17 0	0.615777600484678
+96	0 3 0 0	36 36 36	1.33333333333333	0 -5.55111512312578e-17 0	0.61547287921812
+96	0 3 0 0	37 36 36	1.33333333333333	0.0277777777777778 -5.55111512312578e-17 0	0.615777600484678
+96	0 3 0 0	38 36 36	1.33333333333333	0.0555555555555556 -5.55111512312578e-17 0	0.616824486578861
+
+# iteration 96
+# refinement level 3   multigrid level 0   map 0   component 1   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 3 1 0	35 36 36	1.33333333333333	-0.0277777777777778 -5.55111512312578e-17 0	0.615777600484678
+96	0 3 1 0	36 36 36	1.33333333333333	-1.38777878078145e-17 -5.55111512312578e-17 0	0.61547287921812
+96	0 3 1 0	37 36 36	1.33333333333333	0.0277777777777778 -5.55111512312578e-17 0	0.615777600484678
+96	0 3 1 0	38 36 36	1.33333333333333	0.0555555555555555 -5.55111512312578e-17 0	0.616824486578861
+96	0 3 1 0	39 36 36	1.33333333333333	0.0833333333333333 -5.55111512312578e-17 0	0.618663308361111
+96	0 3 1 0	40 36 36	1.33333333333333	0.111111111111111 -5.55111512312578e-17 0	0.621410197839125
+96	0 3 1 0	41 36 36	1.33333333333333	0.138888888888889 -5.55111512312578e-17 0	0.625297419028293
+96	0 3 1 0	42 36 36	1.33333333333333	0.166666666666667 -5.55111512312578e-17 0	0.63002634676597
+96	0 3 1 0	43 36 36	1.33333333333333	0.194444444444444 -5.55111512312578e-17 0	0.63539979750899
+
+# iteration 96
+# refinement level 3   multigrid level 0   map 0   component 2   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 3 2 0	29 36 36	1.33333333333333	-0.194444444444445 -5.55111512312578e-17 -1.38777878078145e-17	0.63539979750899
+96	0 3 2 0	30 36 36	1.33333333333333	-0.166666666666667 -5.55111512312578e-17 -1.38777878078145e-17	0.630026346765969
+96	0 3 2 0	31 36 36	1.33333333333333	-0.138888888888889 -5.55111512312578e-17 -1.38777878078145e-17	0.625297419028293
+96	0 3 2 0	32 36 36	1.33333333333333	-0.111111111111111 -5.55111512312578e-17 -1.38777878078145e-17	0.621410197839124
+96	0 3 2 0	33 36 36	1.33333333333333	-0.0833333333333334 -5.55111512312578e-17 -1.38777878078145e-17	0.618663308361111
+96	0 3 2 0	34 36 36	1.33333333333333	-0.0555555555555556 -5.55111512312578e-17 -1.38777878078145e-17	0.616824486578861
+96	0 3 2 0	35 36 36	1.33333333333333	-0.0277777777777778 -5.55111512312578e-17 -1.38777878078145e-17	0.615777600484678
+96	0 3 2 0	36 36 36	1.33333333333333	0 -5.55111512312578e-17 -1.38777878078145e-17	0.61547287921812
+96	0 3 2 0	37 36 36	1.33333333333333	0.0277777777777778 -5.55111512312578e-17 -1.38777878078145e-17	0.615777600484678
+96	0 3 2 0	38 36 36	1.33333333333333	0.0555555555555556 -5.55111512312578e-17 -1.38777878078145e-17	0.616824486578861
+
+# iteration 96
+# refinement level 3   multigrid level 0   map 0   component 3   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 3 3 0	35 36 36	1.33333333333333	-0.0277777777777778 -5.55111512312578e-17 -1.38777878078145e-17	0.615777600484678
+96	0 3 3 0	36 36 36	1.33333333333333	-1.38777878078145e-17 -5.55111512312578e-17 -1.38777878078145e-17	0.61547287921812
+96	0 3 3 0	37 36 36	1.33333333333333	0.0277777777777778 -5.55111512312578e-17 -1.38777878078145e-17	0.615777600484678
+96	0 3 3 0	38 36 36	1.33333333333333	0.0555555555555555 -5.55111512312578e-17 -1.38777878078145e-17	0.616824486578861
+96	0 3 3 0	39 36 36	1.33333333333333	0.0833333333333333 -5.55111512312578e-17 -1.38777878078145e-17	0.618663308361111
+96	0 3 3 0	40 36 36	1.33333333333333	0.111111111111111 -5.55111512312578e-17 -1.38777878078145e-17	0.621410197839125
+96	0 3 3 0	41 36 36	1.33333333333333	0.138888888888889 -5.55111512312578e-17 -1.38777878078145e-17	0.625297419028293
+96	0 3 3 0	42 36 36	1.33333333333333	0.166666666666667 -5.55111512312578e-17 -1.38777878078145e-17	0.63002634676597
+96	0 3 3 0	43 36 36	1.33333333333333	0.194444444444444 -5.55111512312578e-17 -1.38777878078145e-17	0.63539979750899
+
+
+# iteration 128
+# refinement level 0   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 0 0 0	0 40 40	1.77777777777778	-1 0.111111111111111 0.111111111111111	0.873700996777018
+128	0 0 0 0	8 40 40	1.77777777777778	-0.777777777777778 0.111111111111111 0.111111111111111	0.859471717622308
+128	0 0 0 0	16 40 40	1.77777777777778	-0.555555555555556 0.111111111111111 0.111111111111111	0.863004424640887
+128	0 0 0 0	24 40 40	1.77777777777778	-0.333333333333333 0.111111111111111 0.111111111111111	0.883486952153519
+128	0 0 0 0	32 40 40	1.77777777777778	-0.111111111111111 0.111111111111111 0.111111111111111	0.915799135771738
+128	0 0 0 0	40 40 40	1.77777777777778	0.111111111111111 0.111111111111111 0.111111111111111	0.915799135771738
+128	0 0 0 0	48 40 40	1.77777777777778	0.333333333333333 0.111111111111111 0.111111111111111	0.883486952153519
+
+# iteration 128
+# refinement level 0   multigrid level 0   map 0   component 1   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 0 1 0	24 40 40	1.77777777777778	-0.333333333333333 0.111111111111111 0.111111111111111	0.883486952153519
+128	0 0 1 0	32 40 40	1.77777777777778	-0.111111111111111 0.111111111111111 0.111111111111111	0.915799135771738
+128	0 0 1 0	40 40 40	1.77777777777778	0.111111111111111 0.111111111111111 0.111111111111111	0.915799135771738
+128	0 0 1 0	48 40 40	1.77777777777778	0.333333333333333 0.111111111111111 0.111111111111111	0.883486952153519
+128	0 0 1 0	56 40 40	1.77777777777778	0.555555555555556 0.111111111111111 0.111111111111111	0.863004424640887
+128	0 0 1 0	64 40 40	1.77777777777778	0.777777777777778 0.111111111111111 0.111111111111111	0.859471717622308
+128	0 0 1 0	72 40 40	1.77777777777778	1 0.111111111111111 0.111111111111111	0.873700996777018
+
+# iteration 128
+# refinement level 0   multigrid level 0   map 0   component 2   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 0 2 0	0 40 40	1.77777777777778	-1 0.111111111111111 0.111111111111111	0.873700996777018
+128	0 0 2 0	8 40 40	1.77777777777778	-0.777777777777778 0.111111111111111 0.111111111111111	0.859471717622308
+128	0 0 2 0	16 40 40	1.77777777777778	-0.555555555555556 0.111111111111111 0.111111111111111	0.863004424640887
+128	0 0 2 0	24 40 40	1.77777777777778	-0.333333333333333 0.111111111111111 0.111111111111111	0.883486952153519
+128	0 0 2 0	32 40 40	1.77777777777778	-0.111111111111111 0.111111111111111 0.111111111111111	0.915799135771738
+128	0 0 2 0	40 40 40	1.77777777777778	0.111111111111111 0.111111111111111 0.111111111111111	0.915799135771738
+128	0 0 2 0	48 40 40	1.77777777777778	0.333333333333333 0.111111111111111 0.111111111111111	0.883486952153519
+
+# iteration 128
+# refinement level 0   multigrid level 0   map 0   component 3   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 0 3 0	24 40 40	1.77777777777778	-0.333333333333333 0.111111111111111 0.111111111111111	0.883486952153519
+128	0 0 3 0	32 40 40	1.77777777777778	-0.111111111111111 0.111111111111111 0.111111111111111	0.915799135771738
+128	0 0 3 0	40 40 40	1.77777777777778	0.111111111111111 0.111111111111111 0.111111111111111	0.915799135771738
+128	0 0 3 0	48 40 40	1.77777777777778	0.333333333333333 0.111111111111111 0.111111111111111	0.883486952153519
+128	0 0 3 0	56 40 40	1.77777777777778	0.555555555555556 0.111111111111111 0.111111111111111	0.863004424640887
+128	0 0 3 0	64 40 40	1.77777777777778	0.777777777777778 0.111111111111111 0.111111111111111	0.859471717622308
+128	0 0 3 0	72 40 40	1.77777777777778	1 0.111111111111111 0.111111111111111	0.873700996777018
+
+
+# iteration 128
+# refinement level 1   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 1 0 0	8 36 36	1.77777777777778	-0.777777777777778 -5.55111512312578e-17 -9.86623976961809e-17	0.862653663591007
+128	0 1 0 0	12 36 36	1.77777777777778	-0.666666666666667 -5.55111512312578e-17 -9.86623976961809e-17	0.862560028606905
+128	0 1 0 0	16 36 36	1.77777777777778	-0.555555555555556 -5.55111512312578e-17 -9.86623976961809e-17	0.866965759785718
+128	0 1 0 0	20 36 36	1.77777777777778	-0.444444444444444 -5.55111512312578e-17 -9.86623976961809e-17	0.876167092595071
+128	0 1 0 0	24 36 36	1.77777777777778	-0.333333333333333 -5.55111512312578e-17 -9.86623976961809e-17	0.889185421864397
+128	0 1 0 0	28 36 36	1.77777777777778	-0.222222222222222 -5.55111512312578e-17 -9.86623976961809e-17	0.907998184681559
+128	0 1 0 0	32 36 36	1.77777777777778	-0.111111111111111 -5.55111512312578e-17 -9.86623976961809e-17	0.924033489738888
+128	0 1 0 0	36 36 36	1.77777777777778	-9.86623976961809e-17 -5.55111512312578e-17 -9.86623976961809e-17	0.928389498223199
+128	0 1 0 0	40 36 36	1.77777777777778	0.111111111111111 -5.55111512312578e-17 -9.86623976961809e-17	0.924033489738888
+128	0 1 0 0	44 36 36	1.77777777777778	0.222222222222222 -5.55111512312578e-17 -9.86623976961809e-17	0.90799818468156
+
+# iteration 128
+# refinement level 1   multigrid level 0   map 0   component 1   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 1 1 0	32 36 36	1.77777777777778	-0.111111111111111 -5.55111512312578e-17 -9.86623976961809e-17	0.924033489738888
+128	0 1 1 0	36 36 36	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -9.86623976961809e-17	0.928389498223199
+128	0 1 1 0	40 36 36	1.77777777777778	0.111111111111111 -5.55111512312578e-17 -9.86623976961809e-17	0.924033489738888
+128	0 1 1 0	44 36 36	1.77777777777778	0.222222222222222 -5.55111512312578e-17 -9.86623976961809e-17	0.90799818468156
+128	0 1 1 0	48 36 36	1.77777777777778	0.333333333333333 -5.55111512312578e-17 -9.86623976961809e-17	0.889185421864397
+128	0 1 1 0	52 36 36	1.77777777777778	0.444444444444444 -5.55111512312578e-17 -9.86623976961809e-17	0.876167092595071
+128	0 1 1 0	56 36 36	1.77777777777778	0.555555555555555 -5.55111512312578e-17 -9.86623976961809e-17	0.866965759785718
+128	0 1 1 0	60 36 36	1.77777777777778	0.666666666666667 -5.55111512312578e-17 -9.86623976961809e-17	0.862560028606905
+128	0 1 1 0	64 36 36	1.77777777777778	0.777777777777778 -5.55111512312578e-17 -9.86623976961809e-17	0.862653663591007
+
+# iteration 128
+# refinement level 1   multigrid level 0   map 0   component 2   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 1 2 0	8 36 36	1.77777777777778	-0.777777777777778 -5.55111512312578e-17 -5.55111512312578e-17	0.862653663591007
+128	0 1 2 0	12 36 36	1.77777777777778	-0.666666666666667 -5.55111512312578e-17 -5.55111512312578e-17	0.862560028606905
+128	0 1 2 0	16 36 36	1.77777777777778	-0.555555555555556 -5.55111512312578e-17 -5.55111512312578e-17	0.866965759785718
+128	0 1 2 0	20 36 36	1.77777777777778	-0.444444444444444 -5.55111512312578e-17 -5.55111512312578e-17	0.876167092595071
+128	0 1 2 0	24 36 36	1.77777777777778	-0.333333333333333 -5.55111512312578e-17 -5.55111512312578e-17	0.889185421864397
+128	0 1 2 0	28 36 36	1.77777777777778	-0.222222222222222 -5.55111512312578e-17 -5.55111512312578e-17	0.907998184681559
+128	0 1 2 0	32 36 36	1.77777777777778	-0.111111111111111 -5.55111512312578e-17 -5.55111512312578e-17	0.924033489738888
+128	0 1 2 0	36 36 36	1.77777777777778	-9.86623976961809e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.928389498223199
+128	0 1 2 0	40 36 36	1.77777777777778	0.111111111111111 -5.55111512312578e-17 -5.55111512312578e-17	0.924033489738888
+128	0 1 2 0	44 36 36	1.77777777777778	0.222222222222222 -5.55111512312578e-17 -5.55111512312578e-17	0.90799818468156
+
+# iteration 128
+# refinement level 1   multigrid level 0   map 0   component 3   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 1 3 0	32 36 36	1.77777777777778	-0.111111111111111 -5.55111512312578e-17 -5.55111512312578e-17	0.924033489738888
+128	0 1 3 0	36 36 36	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.928389498223199
+128	0 1 3 0	40 36 36	1.77777777777778	0.111111111111111 -5.55111512312578e-17 -5.55111512312578e-17	0.924033489738888
+128	0 1 3 0	44 36 36	1.77777777777778	0.222222222222222 -5.55111512312578e-17 -5.55111512312578e-17	0.90799818468156
+128	0 1 3 0	48 36 36	1.77777777777778	0.333333333333333 -5.55111512312578e-17 -5.55111512312578e-17	0.889185421864397
+128	0 1 3 0	52 36 36	1.77777777777778	0.444444444444444 -5.55111512312578e-17 -5.55111512312578e-17	0.876167092595071
+128	0 1 3 0	56 36 36	1.77777777777778	0.555555555555555 -5.55111512312578e-17 -5.55111512312578e-17	0.866965759785718
+128	0 1 3 0	60 36 36	1.77777777777778	0.666666666666667 -5.55111512312578e-17 -5.55111512312578e-17	0.862560028606905
+128	0 1 3 0	64 36 36	1.77777777777778	0.777777777777778 -5.55111512312578e-17 -5.55111512312578e-17	0.862653663591007
+
+
+# iteration 128
+# refinement level 2   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 2 0 0	22 36 36	1.77777777777778	-0.388888888888889 -5.55111512312578e-17 2.46655994240452e-17	0.882075542854246
+128	0 2 0 0	24 36 36	1.77777777777778	-0.333333333333333 -5.55111512312578e-17 2.46655994240452e-17	0.889185421864397
+128	0 2 0 0	26 36 36	1.77777777777778	-0.277777777777778 -5.55111512312578e-17 2.46655994240452e-17	0.898403242286228
+128	0 2 0 0	28 36 36	1.77777777777778	-0.222222222222222 -5.55111512312578e-17 2.46655994240452e-17	0.907998184681559
+128	0 2 0 0	30 36 36	1.77777777777778	-0.166666666666667 -5.55111512312578e-17 2.46655994240452e-17	0.916919384356027
+128	0 2 0 0	32 36 36	1.77777777777778	-0.111111111111111 -5.55111512312578e-17 2.46655994240452e-17	0.924033489738888
+128	0 2 0 0	34 36 36	1.77777777777778	-0.0555555555555555 -5.55111512312578e-17 2.46655994240452e-17	0.927485951077396
+128	0 2 0 0	36 36 36	1.77777777777778	2.46655994240452e-17 -5.55111512312578e-17 2.46655994240452e-17	0.928389498223199
+128	0 2 0 0	38 36 36	1.77777777777778	0.0555555555555556 -5.55111512312578e-17 2.46655994240452e-17	0.927485951077396
+128	0 2 0 0	40 36 36	1.77777777777778	0.111111111111111 -5.55111512312578e-17 2.46655994240452e-17	0.924033489738888
+
+# iteration 128
+# refinement level 2   multigrid level 0   map 0   component 1   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 2 1 0	34 36 36	1.77777777777778	-0.0555555555555556 -5.55111512312578e-17 2.46655994240452e-17	0.927485951077396
+128	0 2 1 0	36 36 36	1.77777777777778	-2.77555756156289e-17 -5.55111512312578e-17 2.46655994240452e-17	0.928389498223199
+128	0 2 1 0	38 36 36	1.77777777777778	0.0555555555555555 -5.55111512312578e-17 2.46655994240452e-17	0.927485951077396
+128	0 2 1 0	40 36 36	1.77777777777778	0.111111111111111 -5.55111512312578e-17 2.46655994240452e-17	0.924033489738888
+128	0 2 1 0	42 36 36	1.77777777777778	0.166666666666667 -5.55111512312578e-17 2.46655994240452e-17	0.916919384356027
+128	0 2 1 0	44 36 36	1.77777777777778	0.222222222222222 -5.55111512312578e-17 2.46655994240452e-17	0.90799818468156
+128	0 2 1 0	46 36 36	1.77777777777778	0.277777777777778 -5.55111512312578e-17 2.46655994240452e-17	0.898403242286228
+128	0 2 1 0	48 36 36	1.77777777777778	0.333333333333333 -5.55111512312578e-17 2.46655994240452e-17	0.889185421864397
+128	0 2 1 0	50 36 36	1.77777777777778	0.388888888888889 -5.55111512312578e-17 2.46655994240452e-17	0.882075542854246
+
+# iteration 128
+# refinement level 2   multigrid level 0   map 0   component 2   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 2 2 0	22 36 36	1.77777777777778	-0.388888888888889 -5.55111512312578e-17 -2.77555756156289e-17	0.882075542854246
+128	0 2 2 0	24 36 36	1.77777777777778	-0.333333333333333 -5.55111512312578e-17 -2.77555756156289e-17	0.889185421864397
+128	0 2 2 0	26 36 36	1.77777777777778	-0.277777777777778 -5.55111512312578e-17 -2.77555756156289e-17	0.898403242286228
+128	0 2 2 0	28 36 36	1.77777777777778	-0.222222222222222 -5.55111512312578e-17 -2.77555756156289e-17	0.907998184681559
+128	0 2 2 0	30 36 36	1.77777777777778	-0.166666666666667 -5.55111512312578e-17 -2.77555756156289e-17	0.916919384356027
+128	0 2 2 0	32 36 36	1.77777777777778	-0.111111111111111 -5.55111512312578e-17 -2.77555756156289e-17	0.924033489738888
+128	0 2 2 0	34 36 36	1.77777777777778	-0.0555555555555555 -5.55111512312578e-17 -2.77555756156289e-17	0.927485951077396
+128	0 2 2 0	36 36 36	1.77777777777778	2.46655994240452e-17 -5.55111512312578e-17 -2.77555756156289e-17	0.928389498223199
+128	0 2 2 0	38 36 36	1.77777777777778	0.0555555555555556 -5.55111512312578e-17 -2.77555756156289e-17	0.927485951077396
+128	0 2 2 0	40 36 36	1.77777777777778	0.111111111111111 -5.55111512312578e-17 -2.77555756156289e-17	0.924033489738888
+
+# iteration 128
+# refinement level 2   multigrid level 0   map 0   component 3   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 2 3 0	34 36 36	1.77777777777778	-0.0555555555555556 -5.55111512312578e-17 -2.77555756156289e-17	0.927485951077396
+128	0 2 3 0	36 36 36	1.77777777777778	-2.77555756156289e-17 -5.55111512312578e-17 -2.77555756156289e-17	0.928389498223199
+128	0 2 3 0	38 36 36	1.77777777777778	0.0555555555555555 -5.55111512312578e-17 -2.77555756156289e-17	0.927485951077396
+128	0 2 3 0	40 36 36	1.77777777777778	0.111111111111111 -5.55111512312578e-17 -2.77555756156289e-17	0.924033489738888
+128	0 2 3 0	42 36 36	1.77777777777778	0.166666666666667 -5.55111512312578e-17 -2.77555756156289e-17	0.916919384356027
+128	0 2 3 0	44 36 36	1.77777777777778	0.222222222222222 -5.55111512312578e-17 -2.77555756156289e-17	0.90799818468156
+128	0 2 3 0	46 36 36	1.77777777777778	0.277777777777778 -5.55111512312578e-17 -2.77555756156289e-17	0.898403242286228
+128	0 2 3 0	48 36 36	1.77777777777778	0.333333333333333 -5.55111512312578e-17 -2.77555756156289e-17	0.889185421864397
+128	0 2 3 0	50 36 36	1.77777777777778	0.388888888888889 -5.55111512312578e-17 -2.77555756156289e-17	0.882075542854246
+
+
+# iteration 128
+# refinement level 3   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 3 0 0	29 36 36	1.77777777777778	-0.194444444444445 -5.55111512312578e-17 0	0.912613836832073
+128	0 3 0 0	30 36 36	1.77777777777778	-0.166666666666667 -5.55111512312578e-17 0	0.916919384356027
+128	0 3 0 0	31 36 36	1.77777777777778	-0.138888888888889 -5.55111512312578e-17 0	0.920818233193455
+128	0 3 0 0	32 36 36	1.77777777777778	-0.111111111111111 -5.55111512312578e-17 0	0.924033489738888
+128	0 3 0 0	33 36 36	1.77777777777778	-0.0833333333333334 -5.55111512312578e-17 0	0.926147880297958
+128	0 3 0 0	34 36 36	1.77777777777778	-0.0555555555555556 -5.55111512312578e-17 0	0.927485951077396
+128	0 3 0 0	35 36 36	1.77777777777778	-0.0277777777777778 -5.55111512312578e-17 0	0.928209975180567
+128	0 3 0 0	36 36 36	1.77777777777778	0 -5.55111512312578e-17 0	0.928389498223199
+128	0 3 0 0	37 36 36	1.77777777777778	0.0277777777777778 -5.55111512312578e-17 0	0.928209975180567
+128	0 3 0 0	38 36 36	1.77777777777778	0.0555555555555556 -5.55111512312578e-17 0	0.927485951077396
+
+# iteration 128
+# refinement level 3   multigrid level 0   map 0   component 1   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 3 1 0	35 36 36	1.77777777777778	-0.0277777777777778 -5.55111512312578e-17 0	0.928209975180567
+128	0 3 1 0	36 36 36	1.77777777777778	-1.38777878078145e-17 -5.55111512312578e-17 0	0.928389498223199
+128	0 3 1 0	37 36 36	1.77777777777778	0.0277777777777778 -5.55111512312578e-17 0	0.928209975180567
+128	0 3 1 0	38 36 36	1.77777777777778	0.0555555555555555 -5.55111512312578e-17 0	0.927485951077396
+128	0 3 1 0	39 36 36	1.77777777777778	0.0833333333333333 -5.55111512312578e-17 0	0.926147880297958
+128	0 3 1 0	40 36 36	1.77777777777778	0.111111111111111 -5.55111512312578e-17 0	0.924033489738888
+128	0 3 1 0	41 36 36	1.77777777777778	0.138888888888889 -5.55111512312578e-17 0	0.920818233193455
+128	0 3 1 0	42 36 36	1.77777777777778	0.166666666666667 -5.55111512312578e-17 0	0.916919384356027
+128	0 3 1 0	43 36 36	1.77777777777778	0.194444444444444 -5.55111512312578e-17 0	0.912613836832073
+
+# iteration 128
+# refinement level 3   multigrid level 0   map 0   component 2   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 3 2 0	29 36 36	1.77777777777778	-0.194444444444445 -5.55111512312578e-17 -1.38777878078145e-17	0.912613836832073
+128	0 3 2 0	30 36 36	1.77777777777778	-0.166666666666667 -5.55111512312578e-17 -1.38777878078145e-17	0.916919384356027
+128	0 3 2 0	31 36 36	1.77777777777778	-0.138888888888889 -5.55111512312578e-17 -1.38777878078145e-17	0.920818233193455
+128	0 3 2 0	32 36 36	1.77777777777778	-0.111111111111111 -5.55111512312578e-17 -1.38777878078145e-17	0.924033489738888
+128	0 3 2 0	33 36 36	1.77777777777778	-0.0833333333333334 -5.55111512312578e-17 -1.38777878078145e-17	0.926147880297958
+128	0 3 2 0	34 36 36	1.77777777777778	-0.0555555555555556 -5.55111512312578e-17 -1.38777878078145e-17	0.927485951077396
+128	0 3 2 0	35 36 36	1.77777777777778	-0.0277777777777778 -5.55111512312578e-17 -1.38777878078145e-17	0.928209975180567
+128	0 3 2 0	36 36 36	1.77777777777778	0 -5.55111512312578e-17 -1.38777878078145e-17	0.928389498223199
+128	0 3 2 0	37 36 36	1.77777777777778	0.0277777777777778 -5.55111512312578e-17 -1.38777878078145e-17	0.928209975180567
+128	0 3 2 0	38 36 36	1.77777777777778	0.0555555555555556 -5.55111512312578e-17 -1.38777878078145e-17	0.927485951077396
+
+# iteration 128
+# refinement level 3   multigrid level 0   map 0   component 3   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 3 3 0	35 36 36	1.77777777777778	-0.0277777777777778 -5.55111512312578e-17 -1.38777878078145e-17	0.928209975180567
+128	0 3 3 0	36 36 36	1.77777777777778	-1.38777878078145e-17 -5.55111512312578e-17 -1.38777878078145e-17	0.928389498223199
+128	0 3 3 0	37 36 36	1.77777777777778	0.0277777777777778 -5.55111512312578e-17 -1.38777878078145e-17	0.928209975180567
+128	0 3 3 0	38 36 36	1.77777777777778	0.0555555555555555 -5.55111512312578e-17 -1.38777878078145e-17	0.927485951077396
+128	0 3 3 0	39 36 36	1.77777777777778	0.0833333333333333 -5.55111512312578e-17 -1.38777878078145e-17	0.926147880297958
+128	0 3 3 0	40 36 36	1.77777777777778	0.111111111111111 -5.55111512312578e-17 -1.38777878078145e-17	0.924033489738888
+128	0 3 3 0	41 36 36	1.77777777777778	0.138888888888889 -5.55111512312578e-17 -1.38777878078145e-17	0.920818233193455
+128	0 3 3 0	42 36 36	1.77777777777778	0.166666666666667 -5.55111512312578e-17 -1.38777878078145e-17	0.916919384356027
+128	0 3 3 0	43 36 36	1.77777777777778	0.194444444444444 -5.55111512312578e-17 -1.38777878078145e-17	0.912613836832073
+
+
diff --git a/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_4proc/phi.y.asc b/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_4proc/phi.y.asc
new file mode 100644
index 000000000..acc2fbf34
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_4proc/phi.y.asc
@@ -0,0 +1,866 @@
+# WAVETOY::phi y (phi)
+#
+# iteration 64
+# refinement level 0   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 0 0 0	40 0 40	0.888888888888889	0.111111111111111 -1 0.111111111111111	0.873700996777018
+64	0 0 0 0	40 8 40	0.888888888888889	0.111111111111111 -0.777777777777778 0.111111111111111	0.807676282684182
+64	0 0 0 0	40 16 40	0.888888888888889	0.111111111111111 -0.555555555555556 0.111111111111111	0.756699260944861
+64	0 0 0 0	40 24 40	0.888888888888889	0.111111111111111 -0.333333333333333 0.111111111111111	0.730294733882952
+64	0 0 0 0	40 32 40	0.888888888888889	0.111111111111111 -0.111111111111111 0.111111111111111	0.725539249473295
+64	0 0 0 0	40 40 40	0.888888888888889	0.111111111111111 0.111111111111111 0.111111111111111	0.725539249473295
+64	0 0 0 0	40 48 40	0.888888888888889	0.111111111111111 0.333333333333333 0.111111111111111	0.730294733882952
+64	0 0 0 0	40 56 40	0.888888888888889	0.111111111111111 0.555555555555556 0.111111111111111	0.756699260944861
+64	0 0 0 0	40 64 40	0.888888888888889	0.111111111111111 0.777777777777778 0.111111111111111	0.807676282684182
+64	0 0 0 0	40 72 40	0.888888888888889	0.111111111111111 1 0.111111111111111	0.873700996777018
+
+# iteration 64
+# refinement level 0   multigrid level 0   map 0   component 1   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 0 1 0	40 0 40	0.888888888888889	0.111111111111111 -1 0.111111111111111	0.873700996777018
+64	0 0 1 0	40 8 40	0.888888888888889	0.111111111111111 -0.777777777777778 0.111111111111111	0.807676282684182
+64	0 0 1 0	40 16 40	0.888888888888889	0.111111111111111 -0.555555555555556 0.111111111111111	0.756699260944861
+64	0 0 1 0	40 24 40	0.888888888888889	0.111111111111111 -0.333333333333333 0.111111111111111	0.730294733882952
+64	0 0 1 0	40 32 40	0.888888888888889	0.111111111111111 -0.111111111111111 0.111111111111111	0.725539249473295
+64	0 0 1 0	40 40 40	0.888888888888889	0.111111111111111 0.111111111111111 0.111111111111111	0.725539249473295
+64	0 0 1 0	40 48 40	0.888888888888889	0.111111111111111 0.333333333333333 0.111111111111111	0.730294733882952
+64	0 0 1 0	40 56 40	0.888888888888889	0.111111111111111 0.555555555555556 0.111111111111111	0.756699260944861
+64	0 0 1 0	40 64 40	0.888888888888889	0.111111111111111 0.777777777777778 0.111111111111111	0.807676282684182
+64	0 0 1 0	40 72 40	0.888888888888889	0.111111111111111 1 0.111111111111111	0.873700996777018
+
+# iteration 64
+# refinement level 0   multigrid level 0   map 0   component 2   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 0 2 0	40 0 40	0.888888888888889	0.111111111111111 -1 0.111111111111111	0.873700996777018
+64	0 0 2 0	40 8 40	0.888888888888889	0.111111111111111 -0.777777777777778 0.111111111111111	0.807676282684182
+64	0 0 2 0	40 16 40	0.888888888888889	0.111111111111111 -0.555555555555556 0.111111111111111	0.756699260944861
+64	0 0 2 0	40 24 40	0.888888888888889	0.111111111111111 -0.333333333333333 0.111111111111111	0.730294733882952
+64	0 0 2 0	40 32 40	0.888888888888889	0.111111111111111 -0.111111111111111 0.111111111111111	0.725539249473295
+64	0 0 2 0	40 40 40	0.888888888888889	0.111111111111111 0.111111111111111 0.111111111111111	0.725539249473295
+64	0 0 2 0	40 48 40	0.888888888888889	0.111111111111111 0.333333333333333 0.111111111111111	0.730294733882952
+64	0 0 2 0	40 56 40	0.888888888888889	0.111111111111111 0.555555555555556 0.111111111111111	0.756699260944861
+64	0 0 2 0	40 64 40	0.888888888888889	0.111111111111111 0.777777777777778 0.111111111111111	0.807676282684182
+64	0 0 2 0	40 72 40	0.888888888888889	0.111111111111111 1 0.111111111111111	0.873700996777018
+
+# iteration 64
+# refinement level 0   multigrid level 0   map 0   component 3   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 0 3 0	40 0 40	0.888888888888889	0.111111111111111 -1 0.111111111111111	0.873700996777018
+64	0 0 3 0	40 8 40	0.888888888888889	0.111111111111111 -0.777777777777778 0.111111111111111	0.807676282684182
+64	0 0 3 0	40 16 40	0.888888888888889	0.111111111111111 -0.555555555555556 0.111111111111111	0.756699260944861
+64	0 0 3 0	40 24 40	0.888888888888889	0.111111111111111 -0.333333333333333 0.111111111111111	0.730294733882952
+64	0 0 3 0	40 32 40	0.888888888888889	0.111111111111111 -0.111111111111111 0.111111111111111	0.725539249473295
+64	0 0 3 0	40 40 40	0.888888888888889	0.111111111111111 0.111111111111111 0.111111111111111	0.725539249473295
+64	0 0 3 0	40 48 40	0.888888888888889	0.111111111111111 0.333333333333333 0.111111111111111	0.730294733882952
+64	0 0 3 0	40 56 40	0.888888888888889	0.111111111111111 0.555555555555556 0.111111111111111	0.756699260944861
+64	0 0 3 0	40 64 40	0.888888888888889	0.111111111111111 0.777777777777778 0.111111111111111	0.807676282684182
+64	0 0 3 0	40 72 40	0.888888888888889	0.111111111111111 1 0.111111111111111	0.873700996777018
+
+
+# iteration 64
+# refinement level 1   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 1 0 0	36 8 36	0.888888888888889	-9.86623976961809e-17 -0.777777777777778 -9.86623976961809e-17	0.808379332677938
+64	0 1 0 0	36 12 36	0.888888888888889	-9.86623976961809e-17 -0.666666666666667 -9.86623976961809e-17	0.779665123251562
+64	0 1 0 0	36 16 36	0.888888888888889	-9.86623976961809e-17 -0.555555555555556 -9.86623976961809e-17	0.756097245006156
+64	0 1 0 0	36 20 36	0.888888888888889	-9.86623976961809e-17 -0.444444444444444 -9.86623976961809e-17	0.739694852518638
+64	0 1 0 0	36 24 36	0.888888888888889	-9.86623976961809e-17 -0.333333333333333 -9.86623976961809e-17	0.729218296316213
+64	0 1 0 0	36 28 36	0.888888888888889	-9.86623976961809e-17 -0.222222222222222 -9.86623976961809e-17	0.725100663327575
+64	0 1 0 0	36 32 36	0.888888888888889	-9.86623976961809e-17 -0.111111111111111 -9.86623976961809e-17	0.724342898925177
+64	0 1 0 0	36 36 36	0.888888888888889	-9.86623976961809e-17 -5.55111512312578e-17 -9.86623976961809e-17	0.723733474251298
+64	0 1 0 0	36 40 36	0.888888888888889	-9.86623976961809e-17 0.111111111111111 -9.86623976961809e-17	0.724342898925177
+64	0 1 0 0	36 44 36	0.888888888888889	-9.86623976961809e-17 0.222222222222222 -9.86623976961809e-17	0.725100663327574
+64	0 1 0 0	36 48 36	0.888888888888889	-9.86623976961809e-17 0.333333333333333 -9.86623976961809e-17	0.729218296316213
+64	0 1 0 0	36 52 36	0.888888888888889	-9.86623976961809e-17 0.444444444444444 -9.86623976961809e-17	0.739694852518638
+64	0 1 0 0	36 56 36	0.888888888888889	-9.86623976961809e-17 0.555555555555555 -9.86623976961809e-17	0.756097245006155
+64	0 1 0 0	36 60 36	0.888888888888889	-9.86623976961809e-17 0.666666666666667 -9.86623976961809e-17	0.779665123251561
+64	0 1 0 0	36 64 36	0.888888888888889	-9.86623976961809e-17 0.777777777777778 -9.86623976961809e-17	0.808379332677938
+
+# iteration 64
+# refinement level 1   multigrid level 0   map 0   component 1   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 1 1 0	36 8 36	0.888888888888889	-5.55111512312578e-17 -0.777777777777778 -9.86623976961809e-17	0.808379332677938
+64	0 1 1 0	36 12 36	0.888888888888889	-5.55111512312578e-17 -0.666666666666667 -9.86623976961809e-17	0.779665123251562
+64	0 1 1 0	36 16 36	0.888888888888889	-5.55111512312578e-17 -0.555555555555556 -9.86623976961809e-17	0.756097245006156
+64	0 1 1 0	36 20 36	0.888888888888889	-5.55111512312578e-17 -0.444444444444444 -9.86623976961809e-17	0.739694852518638
+64	0 1 1 0	36 24 36	0.888888888888889	-5.55111512312578e-17 -0.333333333333333 -9.86623976961809e-17	0.729218296316213
+64	0 1 1 0	36 28 36	0.888888888888889	-5.55111512312578e-17 -0.222222222222222 -9.86623976961809e-17	0.725100663327575
+64	0 1 1 0	36 32 36	0.888888888888889	-5.55111512312578e-17 -0.111111111111111 -9.86623976961809e-17	0.724342898925177
+64	0 1 1 0	36 36 36	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -9.86623976961809e-17	0.723733474251298
+64	0 1 1 0	36 40 36	0.888888888888889	-5.55111512312578e-17 0.111111111111111 -9.86623976961809e-17	0.724342898925177
+64	0 1 1 0	36 44 36	0.888888888888889	-5.55111512312578e-17 0.222222222222222 -9.86623976961809e-17	0.725100663327574
+64	0 1 1 0	36 48 36	0.888888888888889	-5.55111512312578e-17 0.333333333333333 -9.86623976961809e-17	0.729218296316213
+64	0 1 1 0	36 52 36	0.888888888888889	-5.55111512312578e-17 0.444444444444444 -9.86623976961809e-17	0.739694852518638
+64	0 1 1 0	36 56 36	0.888888888888889	-5.55111512312578e-17 0.555555555555555 -9.86623976961809e-17	0.756097245006155
+64	0 1 1 0	36 60 36	0.888888888888889	-5.55111512312578e-17 0.666666666666667 -9.86623976961809e-17	0.779665123251561
+64	0 1 1 0	36 64 36	0.888888888888889	-5.55111512312578e-17 0.777777777777778 -9.86623976961809e-17	0.808379332677938
+
+# iteration 64
+# refinement level 1   multigrid level 0   map 0   component 2   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 1 2 0	36 8 36	0.888888888888889	-9.86623976961809e-17 -0.777777777777778 -5.55111512312578e-17	0.808379332677938
+64	0 1 2 0	36 12 36	0.888888888888889	-9.86623976961809e-17 -0.666666666666667 -5.55111512312578e-17	0.779665123251562
+64	0 1 2 0	36 16 36	0.888888888888889	-9.86623976961809e-17 -0.555555555555556 -5.55111512312578e-17	0.756097245006156
+64	0 1 2 0	36 20 36	0.888888888888889	-9.86623976961809e-17 -0.444444444444444 -5.55111512312578e-17	0.739694852518638
+64	0 1 2 0	36 24 36	0.888888888888889	-9.86623976961809e-17 -0.333333333333333 -5.55111512312578e-17	0.729218296316213
+64	0 1 2 0	36 28 36	0.888888888888889	-9.86623976961809e-17 -0.222222222222222 -5.55111512312578e-17	0.725100663327575
+64	0 1 2 0	36 32 36	0.888888888888889	-9.86623976961809e-17 -0.111111111111111 -5.55111512312578e-17	0.724342898925177
+64	0 1 2 0	36 36 36	0.888888888888889	-9.86623976961809e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.723733474251298
+64	0 1 2 0	36 40 36	0.888888888888889	-9.86623976961809e-17 0.111111111111111 -5.55111512312578e-17	0.724342898925177
+64	0 1 2 0	36 44 36	0.888888888888889	-9.86623976961809e-17 0.222222222222222 -5.55111512312578e-17	0.725100663327574
+64	0 1 2 0	36 48 36	0.888888888888889	-9.86623976961809e-17 0.333333333333333 -5.55111512312578e-17	0.729218296316213
+64	0 1 2 0	36 52 36	0.888888888888889	-9.86623976961809e-17 0.444444444444444 -5.55111512312578e-17	0.739694852518638
+64	0 1 2 0	36 56 36	0.888888888888889	-9.86623976961809e-17 0.555555555555555 -5.55111512312578e-17	0.756097245006155
+64	0 1 2 0	36 60 36	0.888888888888889	-9.86623976961809e-17 0.666666666666667 -5.55111512312578e-17	0.779665123251561
+64	0 1 2 0	36 64 36	0.888888888888889	-9.86623976961809e-17 0.777777777777778 -5.55111512312578e-17	0.808379332677938
+
+# iteration 64
+# refinement level 1   multigrid level 0   map 0   component 3   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 1 3 0	36 8 36	0.888888888888889	-5.55111512312578e-17 -0.777777777777778 -5.55111512312578e-17	0.808379332677938
+64	0 1 3 0	36 12 36	0.888888888888889	-5.55111512312578e-17 -0.666666666666667 -5.55111512312578e-17	0.779665123251562
+64	0 1 3 0	36 16 36	0.888888888888889	-5.55111512312578e-17 -0.555555555555556 -5.55111512312578e-17	0.756097245006156
+64	0 1 3 0	36 20 36	0.888888888888889	-5.55111512312578e-17 -0.444444444444444 -5.55111512312578e-17	0.739694852518638
+64	0 1 3 0	36 24 36	0.888888888888889	-5.55111512312578e-17 -0.333333333333333 -5.55111512312578e-17	0.729218296316213
+64	0 1 3 0	36 28 36	0.888888888888889	-5.55111512312578e-17 -0.222222222222222 -5.55111512312578e-17	0.725100663327575
+64	0 1 3 0	36 32 36	0.888888888888889	-5.55111512312578e-17 -0.111111111111111 -5.55111512312578e-17	0.724342898925177
+64	0 1 3 0	36 36 36	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.723733474251298
+64	0 1 3 0	36 40 36	0.888888888888889	-5.55111512312578e-17 0.111111111111111 -5.55111512312578e-17	0.724342898925177
+64	0 1 3 0	36 44 36	0.888888888888889	-5.55111512312578e-17 0.222222222222222 -5.55111512312578e-17	0.725100663327574
+64	0 1 3 0	36 48 36	0.888888888888889	-5.55111512312578e-17 0.333333333333333 -5.55111512312578e-17	0.729218296316213
+64	0 1 3 0	36 52 36	0.888888888888889	-5.55111512312578e-17 0.444444444444444 -5.55111512312578e-17	0.739694852518638
+64	0 1 3 0	36 56 36	0.888888888888889	-5.55111512312578e-17 0.555555555555555 -5.55111512312578e-17	0.756097245006155
+64	0 1 3 0	36 60 36	0.888888888888889	-5.55111512312578e-17 0.666666666666667 -5.55111512312578e-17	0.779665123251561
+64	0 1 3 0	36 64 36	0.888888888888889	-5.55111512312578e-17 0.777777777777778 -5.55111512312578e-17	0.808379332677938
+
+
+# iteration 64
+# refinement level 2   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 2 0 0	36 22 36	0.888888888888889	2.46655994240452e-17 -0.388888888888889 2.46655994240452e-17	0.733688776948746
+64	0 2 0 0	36 24 36	0.888888888888889	2.46655994240452e-17 -0.333333333333333 2.46655994240452e-17	0.729218296316213
+64	0 2 0 0	36 26 36	0.888888888888889	2.46655994240452e-17 -0.277777777777778 2.46655994240452e-17	0.726552055334392
+64	0 2 0 0	36 28 36	0.888888888888889	2.46655994240452e-17 -0.222222222222222 2.46655994240452e-17	0.725100663327575
+64	0 2 0 0	36 30 36	0.888888888888889	2.46655994240452e-17 -0.166666666666667 2.46655994240452e-17	0.724502518106703
+64	0 2 0 0	36 32 36	0.888888888888889	2.46655994240452e-17 -0.111111111111111 2.46655994240452e-17	0.724342898925177
+64	0 2 0 0	36 34 36	0.888888888888889	2.46655994240452e-17 -0.0555555555555556 2.46655994240452e-17	0.72395273727097
+64	0 2 0 0	36 36 36	0.888888888888889	2.46655994240452e-17 -5.55111512312578e-17 2.46655994240452e-17	0.723733474251298
+64	0 2 0 0	36 38 36	0.888888888888889	2.46655994240452e-17 0.0555555555555555 2.46655994240452e-17	0.72395273727097
+64	0 2 0 0	36 40 36	0.888888888888889	2.46655994240452e-17 0.111111111111111 2.46655994240452e-17	0.724342898925177
+64	0 2 0 0	36 42 36	0.888888888888889	2.46655994240452e-17 0.166666666666667 2.46655994240452e-17	0.724502518106703
+64	0 2 0 0	36 44 36	0.888888888888889	2.46655994240452e-17 0.222222222222222 2.46655994240452e-17	0.725100663327574
+64	0 2 0 0	36 46 36	0.888888888888889	2.46655994240452e-17 0.277777777777778 2.46655994240452e-17	0.726552055334392
+64	0 2 0 0	36 48 36	0.888888888888889	2.46655994240452e-17 0.333333333333333 2.46655994240452e-17	0.729218296316213
+64	0 2 0 0	36 50 36	0.888888888888889	2.46655994240452e-17 0.388888888888889 2.46655994240452e-17	0.733688776948745
+
+# iteration 64
+# refinement level 2   multigrid level 0   map 0   component 1   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 2 1 0	36 22 36	0.888888888888889	-2.77555756156289e-17 -0.388888888888889 2.46655994240452e-17	0.733688776948746
+64	0 2 1 0	36 24 36	0.888888888888889	-2.77555756156289e-17 -0.333333333333333 2.46655994240452e-17	0.729218296316213
+64	0 2 1 0	36 26 36	0.888888888888889	-2.77555756156289e-17 -0.277777777777778 2.46655994240452e-17	0.726552055334392
+64	0 2 1 0	36 28 36	0.888888888888889	-2.77555756156289e-17 -0.222222222222222 2.46655994240452e-17	0.725100663327575
+64	0 2 1 0	36 30 36	0.888888888888889	-2.77555756156289e-17 -0.166666666666667 2.46655994240452e-17	0.724502518106703
+64	0 2 1 0	36 32 36	0.888888888888889	-2.77555756156289e-17 -0.111111111111111 2.46655994240452e-17	0.724342898925177
+64	0 2 1 0	36 34 36	0.888888888888889	-2.77555756156289e-17 -0.0555555555555556 2.46655994240452e-17	0.72395273727097
+64	0 2 1 0	36 36 36	0.888888888888889	-2.77555756156289e-17 -5.55111512312578e-17 2.46655994240452e-17	0.723733474251298
+64	0 2 1 0	36 38 36	0.888888888888889	-2.77555756156289e-17 0.0555555555555555 2.46655994240452e-17	0.72395273727097
+64	0 2 1 0	36 40 36	0.888888888888889	-2.77555756156289e-17 0.111111111111111 2.46655994240452e-17	0.724342898925177
+64	0 2 1 0	36 42 36	0.888888888888889	-2.77555756156289e-17 0.166666666666667 2.46655994240452e-17	0.724502518106703
+64	0 2 1 0	36 44 36	0.888888888888889	-2.77555756156289e-17 0.222222222222222 2.46655994240452e-17	0.725100663327574
+64	0 2 1 0	36 46 36	0.888888888888889	-2.77555756156289e-17 0.277777777777778 2.46655994240452e-17	0.726552055334392
+64	0 2 1 0	36 48 36	0.888888888888889	-2.77555756156289e-17 0.333333333333333 2.46655994240452e-17	0.729218296316213
+64	0 2 1 0	36 50 36	0.888888888888889	-2.77555756156289e-17 0.388888888888889 2.46655994240452e-17	0.733688776948745
+
+# iteration 64
+# refinement level 2   multigrid level 0   map 0   component 2   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 2 2 0	36 22 36	0.888888888888889	2.46655994240452e-17 -0.388888888888889 -2.77555756156289e-17	0.733688776948746
+64	0 2 2 0	36 24 36	0.888888888888889	2.46655994240452e-17 -0.333333333333333 -2.77555756156289e-17	0.729218296316213
+64	0 2 2 0	36 26 36	0.888888888888889	2.46655994240452e-17 -0.277777777777778 -2.77555756156289e-17	0.726552055334392
+64	0 2 2 0	36 28 36	0.888888888888889	2.46655994240452e-17 -0.222222222222222 -2.77555756156289e-17	0.725100663327575
+64	0 2 2 0	36 30 36	0.888888888888889	2.46655994240452e-17 -0.166666666666667 -2.77555756156289e-17	0.724502518106703
+64	0 2 2 0	36 32 36	0.888888888888889	2.46655994240452e-17 -0.111111111111111 -2.77555756156289e-17	0.724342898925177
+64	0 2 2 0	36 34 36	0.888888888888889	2.46655994240452e-17 -0.0555555555555556 -2.77555756156289e-17	0.72395273727097
+64	0 2 2 0	36 36 36	0.888888888888889	2.46655994240452e-17 -5.55111512312578e-17 -2.77555756156289e-17	0.723733474251298
+64	0 2 2 0	36 38 36	0.888888888888889	2.46655994240452e-17 0.0555555555555555 -2.77555756156289e-17	0.72395273727097
+64	0 2 2 0	36 40 36	0.888888888888889	2.46655994240452e-17 0.111111111111111 -2.77555756156289e-17	0.724342898925177
+64	0 2 2 0	36 42 36	0.888888888888889	2.46655994240452e-17 0.166666666666667 -2.77555756156289e-17	0.724502518106703
+64	0 2 2 0	36 44 36	0.888888888888889	2.46655994240452e-17 0.222222222222222 -2.77555756156289e-17	0.725100663327574
+64	0 2 2 0	36 46 36	0.888888888888889	2.46655994240452e-17 0.277777777777778 -2.77555756156289e-17	0.726552055334392
+64	0 2 2 0	36 48 36	0.888888888888889	2.46655994240452e-17 0.333333333333333 -2.77555756156289e-17	0.729218296316213
+64	0 2 2 0	36 50 36	0.888888888888889	2.46655994240452e-17 0.388888888888889 -2.77555756156289e-17	0.733688776948745
+
+# iteration 64
+# refinement level 2   multigrid level 0   map 0   component 3   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 2 3 0	36 22 36	0.888888888888889	-2.77555756156289e-17 -0.388888888888889 -2.77555756156289e-17	0.733688776948746
+64	0 2 3 0	36 24 36	0.888888888888889	-2.77555756156289e-17 -0.333333333333333 -2.77555756156289e-17	0.729218296316213
+64	0 2 3 0	36 26 36	0.888888888888889	-2.77555756156289e-17 -0.277777777777778 -2.77555756156289e-17	0.726552055334392
+64	0 2 3 0	36 28 36	0.888888888888889	-2.77555756156289e-17 -0.222222222222222 -2.77555756156289e-17	0.725100663327575
+64	0 2 3 0	36 30 36	0.888888888888889	-2.77555756156289e-17 -0.166666666666667 -2.77555756156289e-17	0.724502518106703
+64	0 2 3 0	36 32 36	0.888888888888889	-2.77555756156289e-17 -0.111111111111111 -2.77555756156289e-17	0.724342898925177
+64	0 2 3 0	36 34 36	0.888888888888889	-2.77555756156289e-17 -0.0555555555555556 -2.77555756156289e-17	0.72395273727097
+64	0 2 3 0	36 36 36	0.888888888888889	-2.77555756156289e-17 -5.55111512312578e-17 -2.77555756156289e-17	0.723733474251298
+64	0 2 3 0	36 38 36	0.888888888888889	-2.77555756156289e-17 0.0555555555555555 -2.77555756156289e-17	0.72395273727097
+64	0 2 3 0	36 40 36	0.888888888888889	-2.77555756156289e-17 0.111111111111111 -2.77555756156289e-17	0.724342898925177
+64	0 2 3 0	36 42 36	0.888888888888889	-2.77555756156289e-17 0.166666666666667 -2.77555756156289e-17	0.724502518106703
+64	0 2 3 0	36 44 36	0.888888888888889	-2.77555756156289e-17 0.222222222222222 -2.77555756156289e-17	0.725100663327574
+64	0 2 3 0	36 46 36	0.888888888888889	-2.77555756156289e-17 0.277777777777778 -2.77555756156289e-17	0.726552055334392
+64	0 2 3 0	36 48 36	0.888888888888889	-2.77555756156289e-17 0.333333333333333 -2.77555756156289e-17	0.729218296316213
+64	0 2 3 0	36 50 36	0.888888888888889	-2.77555756156289e-17 0.388888888888889 -2.77555756156289e-17	0.733688776948745
+
+
+# iteration 64
+# refinement level 3   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 3 0 0	36 29 36	0.888888888888889	0 -0.194444444444445 0	0.724720854915558
+64	0 3 0 0	36 30 36	0.888888888888889	0 -0.166666666666667 0	0.724502518106703
+64	0 3 0 0	36 31 36	0.888888888888889	0 -0.138888888888889 0	0.724409709543024
+64	0 3 0 0	36 32 36	0.888888888888889	0 -0.111111111111111 0	0.724342898925177
+64	0 3 0 0	36 33 36	0.888888888888889	0 -0.0833333333333334 0	0.724151545837958
+64	0 3 0 0	36 34 36	0.888888888888889	0 -0.0555555555555556 0	0.72395273727097
+64	0 3 0 0	36 35 36	0.888888888888889	0 -0.0277777777777778 0	0.723805016719017
+64	0 3 0 0	36 36 36	0.888888888888889	0 -5.55111512312578e-17 0	0.723733474251298
+64	0 3 0 0	36 37 36	0.888888888888889	0 0.0277777777777777 0	0.723805016719017
+64	0 3 0 0	36 38 36	0.888888888888889	0 0.0555555555555555 0	0.72395273727097
+64	0 3 0 0	36 39 36	0.888888888888889	0 0.0833333333333333 0	0.724151545837958
+64	0 3 0 0	36 40 36	0.888888888888889	0 0.111111111111111 0	0.724342898925177
+64	0 3 0 0	36 41 36	0.888888888888889	0 0.138888888888889 0	0.724409709543024
+64	0 3 0 0	36 42 36	0.888888888888889	0 0.166666666666667 0	0.724502518106703
+64	0 3 0 0	36 43 36	0.888888888888889	0 0.194444444444444 0	0.724720854915558
+
+# iteration 64
+# refinement level 3   multigrid level 0   map 0   component 1   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 3 1 0	36 29 36	0.888888888888889	-1.38777878078145e-17 -0.194444444444445 0	0.724720854915558
+64	0 3 1 0	36 30 36	0.888888888888889	-1.38777878078145e-17 -0.166666666666667 0	0.724502518106703
+64	0 3 1 0	36 31 36	0.888888888888889	-1.38777878078145e-17 -0.138888888888889 0	0.724409709543024
+64	0 3 1 0	36 32 36	0.888888888888889	-1.38777878078145e-17 -0.111111111111111 0	0.724342898925177
+64	0 3 1 0	36 33 36	0.888888888888889	-1.38777878078145e-17 -0.0833333333333334 0	0.724151545837958
+64	0 3 1 0	36 34 36	0.888888888888889	-1.38777878078145e-17 -0.0555555555555556 0	0.72395273727097
+64	0 3 1 0	36 35 36	0.888888888888889	-1.38777878078145e-17 -0.0277777777777778 0	0.723805016719017
+64	0 3 1 0	36 36 36	0.888888888888889	-1.38777878078145e-17 -5.55111512312578e-17 0	0.723733474251298
+64	0 3 1 0	36 37 36	0.888888888888889	-1.38777878078145e-17 0.0277777777777777 0	0.723805016719017
+64	0 3 1 0	36 38 36	0.888888888888889	-1.38777878078145e-17 0.0555555555555555 0	0.72395273727097
+64	0 3 1 0	36 39 36	0.888888888888889	-1.38777878078145e-17 0.0833333333333333 0	0.724151545837958
+64	0 3 1 0	36 40 36	0.888888888888889	-1.38777878078145e-17 0.111111111111111 0	0.724342898925177
+64	0 3 1 0	36 41 36	0.888888888888889	-1.38777878078145e-17 0.138888888888889 0	0.724409709543024
+64	0 3 1 0	36 42 36	0.888888888888889	-1.38777878078145e-17 0.166666666666667 0	0.724502518106703
+64	0 3 1 0	36 43 36	0.888888888888889	-1.38777878078145e-17 0.194444444444444 0	0.724720854915558
+
+# iteration 64
+# refinement level 3   multigrid level 0   map 0   component 2   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 3 2 0	36 29 36	0.888888888888889	0 -0.194444444444445 -1.38777878078145e-17	0.724720854915558
+64	0 3 2 0	36 30 36	0.888888888888889	0 -0.166666666666667 -1.38777878078145e-17	0.724502518106703
+64	0 3 2 0	36 31 36	0.888888888888889	0 -0.138888888888889 -1.38777878078145e-17	0.724409709543024
+64	0 3 2 0	36 32 36	0.888888888888889	0 -0.111111111111111 -1.38777878078145e-17	0.724342898925177
+64	0 3 2 0	36 33 36	0.888888888888889	0 -0.0833333333333334 -1.38777878078145e-17	0.724151545837958
+64	0 3 2 0	36 34 36	0.888888888888889	0 -0.0555555555555556 -1.38777878078145e-17	0.72395273727097
+64	0 3 2 0	36 35 36	0.888888888888889	0 -0.0277777777777778 -1.38777878078145e-17	0.723805016719017
+64	0 3 2 0	36 36 36	0.888888888888889	0 -5.55111512312578e-17 -1.38777878078145e-17	0.723733474251298
+64	0 3 2 0	36 37 36	0.888888888888889	0 0.0277777777777777 -1.38777878078145e-17	0.723805016719017
+64	0 3 2 0	36 38 36	0.888888888888889	0 0.0555555555555555 -1.38777878078145e-17	0.72395273727097
+64	0 3 2 0	36 39 36	0.888888888888889	0 0.0833333333333333 -1.38777878078145e-17	0.724151545837958
+64	0 3 2 0	36 40 36	0.888888888888889	0 0.111111111111111 -1.38777878078145e-17	0.724342898925177
+64	0 3 2 0	36 41 36	0.888888888888889	0 0.138888888888889 -1.38777878078145e-17	0.724409709543024
+64	0 3 2 0	36 42 36	0.888888888888889	0 0.166666666666667 -1.38777878078145e-17	0.724502518106703
+64	0 3 2 0	36 43 36	0.888888888888889	0 0.194444444444444 -1.38777878078145e-17	0.724720854915558
+
+# iteration 64
+# refinement level 3   multigrid level 0   map 0   component 3   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 3 3 0	36 29 36	0.888888888888889	-1.38777878078145e-17 -0.194444444444445 -1.38777878078145e-17	0.724720854915558
+64	0 3 3 0	36 30 36	0.888888888888889	-1.38777878078145e-17 -0.166666666666667 -1.38777878078145e-17	0.724502518106703
+64	0 3 3 0	36 31 36	0.888888888888889	-1.38777878078145e-17 -0.138888888888889 -1.38777878078145e-17	0.724409709543024
+64	0 3 3 0	36 32 36	0.888888888888889	-1.38777878078145e-17 -0.111111111111111 -1.38777878078145e-17	0.724342898925177
+64	0 3 3 0	36 33 36	0.888888888888889	-1.38777878078145e-17 -0.0833333333333334 -1.38777878078145e-17	0.724151545837958
+64	0 3 3 0	36 34 36	0.888888888888889	-1.38777878078145e-17 -0.0555555555555556 -1.38777878078145e-17	0.72395273727097
+64	0 3 3 0	36 35 36	0.888888888888889	-1.38777878078145e-17 -0.0277777777777778 -1.38777878078145e-17	0.723805016719017
+64	0 3 3 0	36 36 36	0.888888888888889	-1.38777878078145e-17 -5.55111512312578e-17 -1.38777878078145e-17	0.723733474251298
+64	0 3 3 0	36 37 36	0.888888888888889	-1.38777878078145e-17 0.0277777777777777 -1.38777878078145e-17	0.723805016719017
+64	0 3 3 0	36 38 36	0.888888888888889	-1.38777878078145e-17 0.0555555555555555 -1.38777878078145e-17	0.72395273727097
+64	0 3 3 0	36 39 36	0.888888888888889	-1.38777878078145e-17 0.0833333333333333 -1.38777878078145e-17	0.724151545837958
+64	0 3 3 0	36 40 36	0.888888888888889	-1.38777878078145e-17 0.111111111111111 -1.38777878078145e-17	0.724342898925177
+64	0 3 3 0	36 41 36	0.888888888888889	-1.38777878078145e-17 0.138888888888889 -1.38777878078145e-17	0.724409709543024
+64	0 3 3 0	36 42 36	0.888888888888889	-1.38777878078145e-17 0.166666666666667 -1.38777878078145e-17	0.724502518106703
+64	0 3 3 0	36 43 36	0.888888888888889	-1.38777878078145e-17 0.194444444444444 -1.38777878078145e-17	0.724720854915558
+
+
+# iteration 96
+# refinement level 0   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 0 0 0	40 0 40	1.33333333333333	0.111111111111111 -1 0.111111111111111	0.877480603359996
+96	0 0 0 0	40 8 40	1.33333333333333	0.111111111111111 -0.777777777777778 0.111111111111111	0.798008089475601
+96	0 0 0 0	40 16 40	1.33333333333333	0.111111111111111 -0.555555555555556 0.111111111111111	0.735090597558599
+96	0 0 0 0	40 24 40	1.33333333333333	0.111111111111111 -0.333333333333333 0.111111111111111	0.677899959110667
+96	0 0 0 0	40 32 40	1.33333333333333	0.111111111111111 -0.111111111111111 0.111111111111111	0.632828813666694
+96	0 0 0 0	40 40 40	1.33333333333333	0.111111111111111 0.111111111111111 0.111111111111111	0.632828813666694
+96	0 0 0 0	40 48 40	1.33333333333333	0.111111111111111 0.333333333333333 0.111111111111111	0.677899959110667
+96	0 0 0 0	40 56 40	1.33333333333333	0.111111111111111 0.555555555555556 0.111111111111111	0.735090597558599
+96	0 0 0 0	40 64 40	1.33333333333333	0.111111111111111 0.777777777777778 0.111111111111111	0.798008089475601
+96	0 0 0 0	40 72 40	1.33333333333333	0.111111111111111 1 0.111111111111111	0.877480603359996
+
+# iteration 96
+# refinement level 0   multigrid level 0   map 0   component 1   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 0 1 0	40 0 40	1.33333333333333	0.111111111111111 -1 0.111111111111111	0.877480603359996
+96	0 0 1 0	40 8 40	1.33333333333333	0.111111111111111 -0.777777777777778 0.111111111111111	0.798008089475601
+96	0 0 1 0	40 16 40	1.33333333333333	0.111111111111111 -0.555555555555556 0.111111111111111	0.735090597558599
+96	0 0 1 0	40 24 40	1.33333333333333	0.111111111111111 -0.333333333333333 0.111111111111111	0.677899959110667
+96	0 0 1 0	40 32 40	1.33333333333333	0.111111111111111 -0.111111111111111 0.111111111111111	0.632828813666694
+96	0 0 1 0	40 40 40	1.33333333333333	0.111111111111111 0.111111111111111 0.111111111111111	0.632828813666694
+96	0 0 1 0	40 48 40	1.33333333333333	0.111111111111111 0.333333333333333 0.111111111111111	0.677899959110667
+96	0 0 1 0	40 56 40	1.33333333333333	0.111111111111111 0.555555555555556 0.111111111111111	0.735090597558599
+96	0 0 1 0	40 64 40	1.33333333333333	0.111111111111111 0.777777777777778 0.111111111111111	0.798008089475601
+96	0 0 1 0	40 72 40	1.33333333333333	0.111111111111111 1 0.111111111111111	0.877480603359996
+
+# iteration 96
+# refinement level 0   multigrid level 0   map 0   component 2   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 0 2 0	40 0 40	1.33333333333333	0.111111111111111 -1 0.111111111111111	0.877480603359996
+96	0 0 2 0	40 8 40	1.33333333333333	0.111111111111111 -0.777777777777778 0.111111111111111	0.798008089475601
+96	0 0 2 0	40 16 40	1.33333333333333	0.111111111111111 -0.555555555555556 0.111111111111111	0.735090597558599
+96	0 0 2 0	40 24 40	1.33333333333333	0.111111111111111 -0.333333333333333 0.111111111111111	0.677899959110667
+96	0 0 2 0	40 32 40	1.33333333333333	0.111111111111111 -0.111111111111111 0.111111111111111	0.632828813666694
+96	0 0 2 0	40 40 40	1.33333333333333	0.111111111111111 0.111111111111111 0.111111111111111	0.632828813666694
+96	0 0 2 0	40 48 40	1.33333333333333	0.111111111111111 0.333333333333333 0.111111111111111	0.677899959110667
+96	0 0 2 0	40 56 40	1.33333333333333	0.111111111111111 0.555555555555556 0.111111111111111	0.735090597558599
+96	0 0 2 0	40 64 40	1.33333333333333	0.111111111111111 0.777777777777778 0.111111111111111	0.798008089475601
+96	0 0 2 0	40 72 40	1.33333333333333	0.111111111111111 1 0.111111111111111	0.877480603359996
+
+# iteration 96
+# refinement level 0   multigrid level 0   map 0   component 3   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 0 3 0	40 0 40	1.33333333333333	0.111111111111111 -1 0.111111111111111	0.877480603359996
+96	0 0 3 0	40 8 40	1.33333333333333	0.111111111111111 -0.777777777777778 0.111111111111111	0.798008089475601
+96	0 0 3 0	40 16 40	1.33333333333333	0.111111111111111 -0.555555555555556 0.111111111111111	0.735090597558599
+96	0 0 3 0	40 24 40	1.33333333333333	0.111111111111111 -0.333333333333333 0.111111111111111	0.677899959110667
+96	0 0 3 0	40 32 40	1.33333333333333	0.111111111111111 -0.111111111111111 0.111111111111111	0.632828813666694
+96	0 0 3 0	40 40 40	1.33333333333333	0.111111111111111 0.111111111111111 0.111111111111111	0.632828813666694
+96	0 0 3 0	40 48 40	1.33333333333333	0.111111111111111 0.333333333333333 0.111111111111111	0.677899959110667
+96	0 0 3 0	40 56 40	1.33333333333333	0.111111111111111 0.555555555555556 0.111111111111111	0.735090597558599
+96	0 0 3 0	40 64 40	1.33333333333333	0.111111111111111 0.777777777777778 0.111111111111111	0.798008089475601
+96	0 0 3 0	40 72 40	1.33333333333333	0.111111111111111 1 0.111111111111111	0.877480603359996
+
+
+# iteration 96
+# refinement level 1   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 1 0 0	36 8 36	1.33333333333333	-9.86623976961809e-17 -0.777777777777778 -9.86623976961809e-17	0.796688784792057
+96	0 1 0 0	36 12 36	1.33333333333333	-9.86623976961809e-17 -0.666666666666667 -9.86623976961809e-17	0.76190136189151
+96	0 1 0 0	36 16 36	1.33333333333333	-9.86623976961809e-17 -0.555555555555556 -9.86623976961809e-17	0.729928937348657
+96	0 1 0 0	36 20 36	1.33333333333333	-9.86623976961809e-17 -0.444444444444444 -9.86623976961809e-17	0.698215010923196
+96	0 1 0 0	36 24 36	1.33333333333333	-9.86623976961809e-17 -0.333333333333333 -9.86623976961809e-17	0.668908746807157
+96	0 1 0 0	36 28 36	1.33333333333333	-9.86623976961809e-17 -0.222222222222222 -9.86623976961809e-17	0.641345710414296
+96	0 1 0 0	36 32 36	1.33333333333333	-9.86623976961809e-17 -0.111111111111111 -9.86623976961809e-17	0.621410197839124
+96	0 1 0 0	36 36 36	1.33333333333333	-9.86623976961809e-17 -5.55111512312578e-17 -9.86623976961809e-17	0.61547287921812
+96	0 1 0 0	36 40 36	1.33333333333333	-9.86623976961809e-17 0.111111111111111 -9.86623976961809e-17	0.621410197839125
+96	0 1 0 0	36 44 36	1.33333333333333	-9.86623976961809e-17 0.222222222222222 -9.86623976961809e-17	0.641345710414297
+96	0 1 0 0	36 48 36	1.33333333333333	-9.86623976961809e-17 0.333333333333333 -9.86623976961809e-17	0.668908746807157
+96	0 1 0 0	36 52 36	1.33333333333333	-9.86623976961809e-17 0.444444444444444 -9.86623976961809e-17	0.698215010923196
+96	0 1 0 0	36 56 36	1.33333333333333	-9.86623976961809e-17 0.555555555555555 -9.86623976961809e-17	0.729928937348657
+96	0 1 0 0	36 60 36	1.33333333333333	-9.86623976961809e-17 0.666666666666667 -9.86623976961809e-17	0.76190136189151
+96	0 1 0 0	36 64 36	1.33333333333333	-9.86623976961809e-17 0.777777777777778 -9.86623976961809e-17	0.796688784792057
+
+# iteration 96
+# refinement level 1   multigrid level 0   map 0   component 1   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 1 1 0	36 8 36	1.33333333333333	-5.55111512312578e-17 -0.777777777777778 -9.86623976961809e-17	0.796688784792057
+96	0 1 1 0	36 12 36	1.33333333333333	-5.55111512312578e-17 -0.666666666666667 -9.86623976961809e-17	0.76190136189151
+96	0 1 1 0	36 16 36	1.33333333333333	-5.55111512312578e-17 -0.555555555555556 -9.86623976961809e-17	0.729928937348657
+96	0 1 1 0	36 20 36	1.33333333333333	-5.55111512312578e-17 -0.444444444444444 -9.86623976961809e-17	0.698215010923196
+96	0 1 1 0	36 24 36	1.33333333333333	-5.55111512312578e-17 -0.333333333333333 -9.86623976961809e-17	0.668908746807157
+96	0 1 1 0	36 28 36	1.33333333333333	-5.55111512312578e-17 -0.222222222222222 -9.86623976961809e-17	0.641345710414296
+96	0 1 1 0	36 32 36	1.33333333333333	-5.55111512312578e-17 -0.111111111111111 -9.86623976961809e-17	0.621410197839124
+96	0 1 1 0	36 36 36	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -9.86623976961809e-17	0.61547287921812
+96	0 1 1 0	36 40 36	1.33333333333333	-5.55111512312578e-17 0.111111111111111 -9.86623976961809e-17	0.621410197839125
+96	0 1 1 0	36 44 36	1.33333333333333	-5.55111512312578e-17 0.222222222222222 -9.86623976961809e-17	0.641345710414297
+96	0 1 1 0	36 48 36	1.33333333333333	-5.55111512312578e-17 0.333333333333333 -9.86623976961809e-17	0.668908746807157
+96	0 1 1 0	36 52 36	1.33333333333333	-5.55111512312578e-17 0.444444444444444 -9.86623976961809e-17	0.698215010923196
+96	0 1 1 0	36 56 36	1.33333333333333	-5.55111512312578e-17 0.555555555555555 -9.86623976961809e-17	0.729928937348657
+96	0 1 1 0	36 60 36	1.33333333333333	-5.55111512312578e-17 0.666666666666667 -9.86623976961809e-17	0.76190136189151
+96	0 1 1 0	36 64 36	1.33333333333333	-5.55111512312578e-17 0.777777777777778 -9.86623976961809e-17	0.796688784792057
+
+# iteration 96
+# refinement level 1   multigrid level 0   map 0   component 2   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 1 2 0	36 8 36	1.33333333333333	-9.86623976961809e-17 -0.777777777777778 -5.55111512312578e-17	0.796688784792057
+96	0 1 2 0	36 12 36	1.33333333333333	-9.86623976961809e-17 -0.666666666666667 -5.55111512312578e-17	0.76190136189151
+96	0 1 2 0	36 16 36	1.33333333333333	-9.86623976961809e-17 -0.555555555555556 -5.55111512312578e-17	0.729928937348657
+96	0 1 2 0	36 20 36	1.33333333333333	-9.86623976961809e-17 -0.444444444444444 -5.55111512312578e-17	0.698215010923196
+96	0 1 2 0	36 24 36	1.33333333333333	-9.86623976961809e-17 -0.333333333333333 -5.55111512312578e-17	0.668908746807157
+96	0 1 2 0	36 28 36	1.33333333333333	-9.86623976961809e-17 -0.222222222222222 -5.55111512312578e-17	0.641345710414296
+96	0 1 2 0	36 32 36	1.33333333333333	-9.86623976961809e-17 -0.111111111111111 -5.55111512312578e-17	0.621410197839124
+96	0 1 2 0	36 36 36	1.33333333333333	-9.86623976961809e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.61547287921812
+96	0 1 2 0	36 40 36	1.33333333333333	-9.86623976961809e-17 0.111111111111111 -5.55111512312578e-17	0.621410197839125
+96	0 1 2 0	36 44 36	1.33333333333333	-9.86623976961809e-17 0.222222222222222 -5.55111512312578e-17	0.641345710414297
+96	0 1 2 0	36 48 36	1.33333333333333	-9.86623976961809e-17 0.333333333333333 -5.55111512312578e-17	0.668908746807157
+96	0 1 2 0	36 52 36	1.33333333333333	-9.86623976961809e-17 0.444444444444444 -5.55111512312578e-17	0.698215010923196
+96	0 1 2 0	36 56 36	1.33333333333333	-9.86623976961809e-17 0.555555555555555 -5.55111512312578e-17	0.729928937348657
+96	0 1 2 0	36 60 36	1.33333333333333	-9.86623976961809e-17 0.666666666666667 -5.55111512312578e-17	0.76190136189151
+96	0 1 2 0	36 64 36	1.33333333333333	-9.86623976961809e-17 0.777777777777778 -5.55111512312578e-17	0.796688784792057
+
+# iteration 96
+# refinement level 1   multigrid level 0   map 0   component 3   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 1 3 0	36 8 36	1.33333333333333	-5.55111512312578e-17 -0.777777777777778 -5.55111512312578e-17	0.796688784792057
+96	0 1 3 0	36 12 36	1.33333333333333	-5.55111512312578e-17 -0.666666666666667 -5.55111512312578e-17	0.76190136189151
+96	0 1 3 0	36 16 36	1.33333333333333	-5.55111512312578e-17 -0.555555555555556 -5.55111512312578e-17	0.729928937348657
+96	0 1 3 0	36 20 36	1.33333333333333	-5.55111512312578e-17 -0.444444444444444 -5.55111512312578e-17	0.698215010923196
+96	0 1 3 0	36 24 36	1.33333333333333	-5.55111512312578e-17 -0.333333333333333 -5.55111512312578e-17	0.668908746807157
+96	0 1 3 0	36 28 36	1.33333333333333	-5.55111512312578e-17 -0.222222222222222 -5.55111512312578e-17	0.641345710414296
+96	0 1 3 0	36 32 36	1.33333333333333	-5.55111512312578e-17 -0.111111111111111 -5.55111512312578e-17	0.621410197839124
+96	0 1 3 0	36 36 36	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.61547287921812
+96	0 1 3 0	36 40 36	1.33333333333333	-5.55111512312578e-17 0.111111111111111 -5.55111512312578e-17	0.621410197839125
+96	0 1 3 0	36 44 36	1.33333333333333	-5.55111512312578e-17 0.222222222222222 -5.55111512312578e-17	0.641345710414297
+96	0 1 3 0	36 48 36	1.33333333333333	-5.55111512312578e-17 0.333333333333333 -5.55111512312578e-17	0.668908746807157
+96	0 1 3 0	36 52 36	1.33333333333333	-5.55111512312578e-17 0.444444444444444 -5.55111512312578e-17	0.698215010923196
+96	0 1 3 0	36 56 36	1.33333333333333	-5.55111512312578e-17 0.555555555555555 -5.55111512312578e-17	0.729928937348657
+96	0 1 3 0	36 60 36	1.33333333333333	-5.55111512312578e-17 0.666666666666667 -5.55111512312578e-17	0.76190136189151
+96	0 1 3 0	36 64 36	1.33333333333333	-5.55111512312578e-17 0.777777777777778 -5.55111512312578e-17	0.796688784792057
+
+
+# iteration 96
+# refinement level 2   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 2 0 0	36 22 36	1.33333333333333	2.46655994240452e-17 -0.388888888888889 2.46655994240452e-17	0.683302448238139
+96	0 2 0 0	36 24 36	1.33333333333333	2.46655994240452e-17 -0.333333333333333 2.46655994240452e-17	0.668908746807157
+96	0 2 0 0	36 26 36	1.33333333333333	2.46655994240452e-17 -0.277777777777778 2.46655994240452e-17	0.654541556639422
+96	0 2 0 0	36 28 36	1.33333333333333	2.46655994240452e-17 -0.222222222222222 2.46655994240452e-17	0.641345710414296
+96	0 2 0 0	36 30 36	1.33333333333333	2.46655994240452e-17 -0.166666666666667 2.46655994240452e-17	0.630026346765969
+96	0 2 0 0	36 32 36	1.33333333333333	2.46655994240452e-17 -0.111111111111111 2.46655994240452e-17	0.621410197839124
+96	0 2 0 0	36 34 36	1.33333333333333	2.46655994240452e-17 -0.0555555555555556 2.46655994240452e-17	0.616824486578861
+96	0 2 0 0	36 36 36	1.33333333333333	2.46655994240452e-17 -5.55111512312578e-17 2.46655994240452e-17	0.61547287921812
+96	0 2 0 0	36 38 36	1.33333333333333	2.46655994240452e-17 0.0555555555555555 2.46655994240452e-17	0.616824486578861
+96	0 2 0 0	36 40 36	1.33333333333333	2.46655994240452e-17 0.111111111111111 2.46655994240452e-17	0.621410197839125
+96	0 2 0 0	36 42 36	1.33333333333333	2.46655994240452e-17 0.166666666666667 2.46655994240452e-17	0.63002634676597
+96	0 2 0 0	36 44 36	1.33333333333333	2.46655994240452e-17 0.222222222222222 2.46655994240452e-17	0.641345710414297
+96	0 2 0 0	36 46 36	1.33333333333333	2.46655994240452e-17 0.277777777777778 2.46655994240452e-17	0.654541556639422
+96	0 2 0 0	36 48 36	1.33333333333333	2.46655994240452e-17 0.333333333333333 2.46655994240452e-17	0.668908746807157
+96	0 2 0 0	36 50 36	1.33333333333333	2.46655994240452e-17 0.388888888888889 2.46655994240452e-17	0.683302448238139
+
+# iteration 96
+# refinement level 2   multigrid level 0   map 0   component 1   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 2 1 0	36 22 36	1.33333333333333	-2.77555756156289e-17 -0.388888888888889 2.46655994240452e-17	0.683302448238139
+96	0 2 1 0	36 24 36	1.33333333333333	-2.77555756156289e-17 -0.333333333333333 2.46655994240452e-17	0.668908746807157
+96	0 2 1 0	36 26 36	1.33333333333333	-2.77555756156289e-17 -0.277777777777778 2.46655994240452e-17	0.654541556639422
+96	0 2 1 0	36 28 36	1.33333333333333	-2.77555756156289e-17 -0.222222222222222 2.46655994240452e-17	0.641345710414296
+96	0 2 1 0	36 30 36	1.33333333333333	-2.77555756156289e-17 -0.166666666666667 2.46655994240452e-17	0.630026346765969
+96	0 2 1 0	36 32 36	1.33333333333333	-2.77555756156289e-17 -0.111111111111111 2.46655994240452e-17	0.621410197839124
+96	0 2 1 0	36 34 36	1.33333333333333	-2.77555756156289e-17 -0.0555555555555556 2.46655994240452e-17	0.616824486578861
+96	0 2 1 0	36 36 36	1.33333333333333	-2.77555756156289e-17 -5.55111512312578e-17 2.46655994240452e-17	0.61547287921812
+96	0 2 1 0	36 38 36	1.33333333333333	-2.77555756156289e-17 0.0555555555555555 2.46655994240452e-17	0.616824486578861
+96	0 2 1 0	36 40 36	1.33333333333333	-2.77555756156289e-17 0.111111111111111 2.46655994240452e-17	0.621410197839125
+96	0 2 1 0	36 42 36	1.33333333333333	-2.77555756156289e-17 0.166666666666667 2.46655994240452e-17	0.63002634676597
+96	0 2 1 0	36 44 36	1.33333333333333	-2.77555756156289e-17 0.222222222222222 2.46655994240452e-17	0.641345710414297
+96	0 2 1 0	36 46 36	1.33333333333333	-2.77555756156289e-17 0.277777777777778 2.46655994240452e-17	0.654541556639422
+96	0 2 1 0	36 48 36	1.33333333333333	-2.77555756156289e-17 0.333333333333333 2.46655994240452e-17	0.668908746807157
+96	0 2 1 0	36 50 36	1.33333333333333	-2.77555756156289e-17 0.388888888888889 2.46655994240452e-17	0.683302448238139
+
+# iteration 96
+# refinement level 2   multigrid level 0   map 0   component 2   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 2 2 0	36 22 36	1.33333333333333	2.46655994240452e-17 -0.388888888888889 -2.77555756156289e-17	0.683302448238139
+96	0 2 2 0	36 24 36	1.33333333333333	2.46655994240452e-17 -0.333333333333333 -2.77555756156289e-17	0.668908746807157
+96	0 2 2 0	36 26 36	1.33333333333333	2.46655994240452e-17 -0.277777777777778 -2.77555756156289e-17	0.654541556639422
+96	0 2 2 0	36 28 36	1.33333333333333	2.46655994240452e-17 -0.222222222222222 -2.77555756156289e-17	0.641345710414296
+96	0 2 2 0	36 30 36	1.33333333333333	2.46655994240452e-17 -0.166666666666667 -2.77555756156289e-17	0.630026346765969
+96	0 2 2 0	36 32 36	1.33333333333333	2.46655994240452e-17 -0.111111111111111 -2.77555756156289e-17	0.621410197839124
+96	0 2 2 0	36 34 36	1.33333333333333	2.46655994240452e-17 -0.0555555555555556 -2.77555756156289e-17	0.616824486578861
+96	0 2 2 0	36 36 36	1.33333333333333	2.46655994240452e-17 -5.55111512312578e-17 -2.77555756156289e-17	0.61547287921812
+96	0 2 2 0	36 38 36	1.33333333333333	2.46655994240452e-17 0.0555555555555555 -2.77555756156289e-17	0.616824486578861
+96	0 2 2 0	36 40 36	1.33333333333333	2.46655994240452e-17 0.111111111111111 -2.77555756156289e-17	0.621410197839125
+96	0 2 2 0	36 42 36	1.33333333333333	2.46655994240452e-17 0.166666666666667 -2.77555756156289e-17	0.63002634676597
+96	0 2 2 0	36 44 36	1.33333333333333	2.46655994240452e-17 0.222222222222222 -2.77555756156289e-17	0.641345710414297
+96	0 2 2 0	36 46 36	1.33333333333333	2.46655994240452e-17 0.277777777777778 -2.77555756156289e-17	0.654541556639422
+96	0 2 2 0	36 48 36	1.33333333333333	2.46655994240452e-17 0.333333333333333 -2.77555756156289e-17	0.668908746807157
+96	0 2 2 0	36 50 36	1.33333333333333	2.46655994240452e-17 0.388888888888889 -2.77555756156289e-17	0.683302448238139
+
+# iteration 96
+# refinement level 2   multigrid level 0   map 0   component 3   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 2 3 0	36 22 36	1.33333333333333	-2.77555756156289e-17 -0.388888888888889 -2.77555756156289e-17	0.683302448238139
+96	0 2 3 0	36 24 36	1.33333333333333	-2.77555756156289e-17 -0.333333333333333 -2.77555756156289e-17	0.668908746807157
+96	0 2 3 0	36 26 36	1.33333333333333	-2.77555756156289e-17 -0.277777777777778 -2.77555756156289e-17	0.654541556639422
+96	0 2 3 0	36 28 36	1.33333333333333	-2.77555756156289e-17 -0.222222222222222 -2.77555756156289e-17	0.641345710414296
+96	0 2 3 0	36 30 36	1.33333333333333	-2.77555756156289e-17 -0.166666666666667 -2.77555756156289e-17	0.630026346765969
+96	0 2 3 0	36 32 36	1.33333333333333	-2.77555756156289e-17 -0.111111111111111 -2.77555756156289e-17	0.621410197839124
+96	0 2 3 0	36 34 36	1.33333333333333	-2.77555756156289e-17 -0.0555555555555556 -2.77555756156289e-17	0.616824486578861
+96	0 2 3 0	36 36 36	1.33333333333333	-2.77555756156289e-17 -5.55111512312578e-17 -2.77555756156289e-17	0.61547287921812
+96	0 2 3 0	36 38 36	1.33333333333333	-2.77555756156289e-17 0.0555555555555555 -2.77555756156289e-17	0.616824486578861
+96	0 2 3 0	36 40 36	1.33333333333333	-2.77555756156289e-17 0.111111111111111 -2.77555756156289e-17	0.621410197839125
+96	0 2 3 0	36 42 36	1.33333333333333	-2.77555756156289e-17 0.166666666666667 -2.77555756156289e-17	0.63002634676597
+96	0 2 3 0	36 44 36	1.33333333333333	-2.77555756156289e-17 0.222222222222222 -2.77555756156289e-17	0.641345710414297
+96	0 2 3 0	36 46 36	1.33333333333333	-2.77555756156289e-17 0.277777777777778 -2.77555756156289e-17	0.654541556639422
+96	0 2 3 0	36 48 36	1.33333333333333	-2.77555756156289e-17 0.333333333333333 -2.77555756156289e-17	0.668908746807157
+96	0 2 3 0	36 50 36	1.33333333333333	-2.77555756156289e-17 0.388888888888889 -2.77555756156289e-17	0.683302448238139
+
+
+# iteration 96
+# refinement level 3   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 3 0 0	36 29 36	1.33333333333333	0 -0.194444444444445 0	0.63539979750899
+96	0 3 0 0	36 30 36	1.33333333333333	0 -0.166666666666667 0	0.630026346765969
+96	0 3 0 0	36 31 36	1.33333333333333	0 -0.138888888888889 0	0.625297419028293
+96	0 3 0 0	36 32 36	1.33333333333333	0 -0.111111111111111 0	0.621410197839124
+96	0 3 0 0	36 33 36	1.33333333333333	0 -0.0833333333333334 0	0.618663308361111
+96	0 3 0 0	36 34 36	1.33333333333333	0 -0.0555555555555556 0	0.616824486578861
+96	0 3 0 0	36 35 36	1.33333333333333	0 -0.0277777777777778 0	0.615777600484678
+96	0 3 0 0	36 36 36	1.33333333333333	0 -5.55111512312578e-17 0	0.61547287921812
+96	0 3 0 0	36 37 36	1.33333333333333	0 0.0277777777777777 0	0.615777600484678
+96	0 3 0 0	36 38 36	1.33333333333333	0 0.0555555555555555 0	0.616824486578861
+96	0 3 0 0	36 39 36	1.33333333333333	0 0.0833333333333333 0	0.618663308361111
+96	0 3 0 0	36 40 36	1.33333333333333	0 0.111111111111111 0	0.621410197839125
+96	0 3 0 0	36 41 36	1.33333333333333	0 0.138888888888889 0	0.625297419028293
+96	0 3 0 0	36 42 36	1.33333333333333	0 0.166666666666667 0	0.63002634676597
+96	0 3 0 0	36 43 36	1.33333333333333	0 0.194444444444444 0	0.63539979750899
+
+# iteration 96
+# refinement level 3   multigrid level 0   map 0   component 1   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 3 1 0	36 29 36	1.33333333333333	-1.38777878078145e-17 -0.194444444444445 0	0.63539979750899
+96	0 3 1 0	36 30 36	1.33333333333333	-1.38777878078145e-17 -0.166666666666667 0	0.630026346765969
+96	0 3 1 0	36 31 36	1.33333333333333	-1.38777878078145e-17 -0.138888888888889 0	0.625297419028293
+96	0 3 1 0	36 32 36	1.33333333333333	-1.38777878078145e-17 -0.111111111111111 0	0.621410197839124
+96	0 3 1 0	36 33 36	1.33333333333333	-1.38777878078145e-17 -0.0833333333333334 0	0.618663308361111
+96	0 3 1 0	36 34 36	1.33333333333333	-1.38777878078145e-17 -0.0555555555555556 0	0.616824486578861
+96	0 3 1 0	36 35 36	1.33333333333333	-1.38777878078145e-17 -0.0277777777777778 0	0.615777600484678
+96	0 3 1 0	36 36 36	1.33333333333333	-1.38777878078145e-17 -5.55111512312578e-17 0	0.61547287921812
+96	0 3 1 0	36 37 36	1.33333333333333	-1.38777878078145e-17 0.0277777777777777 0	0.615777600484678
+96	0 3 1 0	36 38 36	1.33333333333333	-1.38777878078145e-17 0.0555555555555555 0	0.616824486578861
+96	0 3 1 0	36 39 36	1.33333333333333	-1.38777878078145e-17 0.0833333333333333 0	0.618663308361111
+96	0 3 1 0	36 40 36	1.33333333333333	-1.38777878078145e-17 0.111111111111111 0	0.621410197839125
+96	0 3 1 0	36 41 36	1.33333333333333	-1.38777878078145e-17 0.138888888888889 0	0.625297419028293
+96	0 3 1 0	36 42 36	1.33333333333333	-1.38777878078145e-17 0.166666666666667 0	0.63002634676597
+96	0 3 1 0	36 43 36	1.33333333333333	-1.38777878078145e-17 0.194444444444444 0	0.63539979750899
+
+# iteration 96
+# refinement level 3   multigrid level 0   map 0   component 2   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 3 2 0	36 29 36	1.33333333333333	0 -0.194444444444445 -1.38777878078145e-17	0.63539979750899
+96	0 3 2 0	36 30 36	1.33333333333333	0 -0.166666666666667 -1.38777878078145e-17	0.630026346765969
+96	0 3 2 0	36 31 36	1.33333333333333	0 -0.138888888888889 -1.38777878078145e-17	0.625297419028293
+96	0 3 2 0	36 32 36	1.33333333333333	0 -0.111111111111111 -1.38777878078145e-17	0.621410197839124
+96	0 3 2 0	36 33 36	1.33333333333333	0 -0.0833333333333334 -1.38777878078145e-17	0.618663308361111
+96	0 3 2 0	36 34 36	1.33333333333333	0 -0.0555555555555556 -1.38777878078145e-17	0.616824486578861
+96	0 3 2 0	36 35 36	1.33333333333333	0 -0.0277777777777778 -1.38777878078145e-17	0.615777600484678
+96	0 3 2 0	36 36 36	1.33333333333333	0 -5.55111512312578e-17 -1.38777878078145e-17	0.61547287921812
+96	0 3 2 0	36 37 36	1.33333333333333	0 0.0277777777777777 -1.38777878078145e-17	0.615777600484678
+96	0 3 2 0	36 38 36	1.33333333333333	0 0.0555555555555555 -1.38777878078145e-17	0.616824486578861
+96	0 3 2 0	36 39 36	1.33333333333333	0 0.0833333333333333 -1.38777878078145e-17	0.618663308361111
+96	0 3 2 0	36 40 36	1.33333333333333	0 0.111111111111111 -1.38777878078145e-17	0.621410197839125
+96	0 3 2 0	36 41 36	1.33333333333333	0 0.138888888888889 -1.38777878078145e-17	0.625297419028293
+96	0 3 2 0	36 42 36	1.33333333333333	0 0.166666666666667 -1.38777878078145e-17	0.63002634676597
+96	0 3 2 0	36 43 36	1.33333333333333	0 0.194444444444444 -1.38777878078145e-17	0.63539979750899
+
+# iteration 96
+# refinement level 3   multigrid level 0   map 0   component 3   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 3 3 0	36 29 36	1.33333333333333	-1.38777878078145e-17 -0.194444444444445 -1.38777878078145e-17	0.63539979750899
+96	0 3 3 0	36 30 36	1.33333333333333	-1.38777878078145e-17 -0.166666666666667 -1.38777878078145e-17	0.630026346765969
+96	0 3 3 0	36 31 36	1.33333333333333	-1.38777878078145e-17 -0.138888888888889 -1.38777878078145e-17	0.625297419028293
+96	0 3 3 0	36 32 36	1.33333333333333	-1.38777878078145e-17 -0.111111111111111 -1.38777878078145e-17	0.621410197839124
+96	0 3 3 0	36 33 36	1.33333333333333	-1.38777878078145e-17 -0.0833333333333334 -1.38777878078145e-17	0.618663308361111
+96	0 3 3 0	36 34 36	1.33333333333333	-1.38777878078145e-17 -0.0555555555555556 -1.38777878078145e-17	0.616824486578861
+96	0 3 3 0	36 35 36	1.33333333333333	-1.38777878078145e-17 -0.0277777777777778 -1.38777878078145e-17	0.615777600484678
+96	0 3 3 0	36 36 36	1.33333333333333	-1.38777878078145e-17 -5.55111512312578e-17 -1.38777878078145e-17	0.61547287921812
+96	0 3 3 0	36 37 36	1.33333333333333	-1.38777878078145e-17 0.0277777777777777 -1.38777878078145e-17	0.615777600484678
+96	0 3 3 0	36 38 36	1.33333333333333	-1.38777878078145e-17 0.0555555555555555 -1.38777878078145e-17	0.616824486578861
+96	0 3 3 0	36 39 36	1.33333333333333	-1.38777878078145e-17 0.0833333333333333 -1.38777878078145e-17	0.618663308361111
+96	0 3 3 0	36 40 36	1.33333333333333	-1.38777878078145e-17 0.111111111111111 -1.38777878078145e-17	0.621410197839125
+96	0 3 3 0	36 41 36	1.33333333333333	-1.38777878078145e-17 0.138888888888889 -1.38777878078145e-17	0.625297419028293
+96	0 3 3 0	36 42 36	1.33333333333333	-1.38777878078145e-17 0.166666666666667 -1.38777878078145e-17	0.63002634676597
+96	0 3 3 0	36 43 36	1.33333333333333	-1.38777878078145e-17 0.194444444444444 -1.38777878078145e-17	0.63539979750899
+
+
+# iteration 128
+# refinement level 0   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 0 0 0	40 0 40	1.77777777777778	0.111111111111111 -1 0.111111111111111	0.873700996777018
+128	0 0 0 0	40 8 40	1.77777777777778	0.111111111111111 -0.777777777777778 0.111111111111111	0.859471717622308
+128	0 0 0 0	40 16 40	1.77777777777778	0.111111111111111 -0.555555555555556 0.111111111111111	0.863004424640887
+128	0 0 0 0	40 24 40	1.77777777777778	0.111111111111111 -0.333333333333333 0.111111111111111	0.883486952153519
+128	0 0 0 0	40 32 40	1.77777777777778	0.111111111111111 -0.111111111111111 0.111111111111111	0.915799135771738
+128	0 0 0 0	40 40 40	1.77777777777778	0.111111111111111 0.111111111111111 0.111111111111111	0.915799135771738
+128	0 0 0 0	40 48 40	1.77777777777778	0.111111111111111 0.333333333333333 0.111111111111111	0.883486952153519
+128	0 0 0 0	40 56 40	1.77777777777778	0.111111111111111 0.555555555555556 0.111111111111111	0.863004424640887
+128	0 0 0 0	40 64 40	1.77777777777778	0.111111111111111 0.777777777777778 0.111111111111111	0.859471717622308
+128	0 0 0 0	40 72 40	1.77777777777778	0.111111111111111 1 0.111111111111111	0.873700996777018
+
+# iteration 128
+# refinement level 0   multigrid level 0   map 0   component 1   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 0 1 0	40 0 40	1.77777777777778	0.111111111111111 -1 0.111111111111111	0.873700996777018
+128	0 0 1 0	40 8 40	1.77777777777778	0.111111111111111 -0.777777777777778 0.111111111111111	0.859471717622308
+128	0 0 1 0	40 16 40	1.77777777777778	0.111111111111111 -0.555555555555556 0.111111111111111	0.863004424640887
+128	0 0 1 0	40 24 40	1.77777777777778	0.111111111111111 -0.333333333333333 0.111111111111111	0.883486952153519
+128	0 0 1 0	40 32 40	1.77777777777778	0.111111111111111 -0.111111111111111 0.111111111111111	0.915799135771738
+128	0 0 1 0	40 40 40	1.77777777777778	0.111111111111111 0.111111111111111 0.111111111111111	0.915799135771738
+128	0 0 1 0	40 48 40	1.77777777777778	0.111111111111111 0.333333333333333 0.111111111111111	0.883486952153519
+128	0 0 1 0	40 56 40	1.77777777777778	0.111111111111111 0.555555555555556 0.111111111111111	0.863004424640887
+128	0 0 1 0	40 64 40	1.77777777777778	0.111111111111111 0.777777777777778 0.111111111111111	0.859471717622308
+128	0 0 1 0	40 72 40	1.77777777777778	0.111111111111111 1 0.111111111111111	0.873700996777018
+
+# iteration 128
+# refinement level 0   multigrid level 0   map 0   component 2   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 0 2 0	40 0 40	1.77777777777778	0.111111111111111 -1 0.111111111111111	0.873700996777018
+128	0 0 2 0	40 8 40	1.77777777777778	0.111111111111111 -0.777777777777778 0.111111111111111	0.859471717622308
+128	0 0 2 0	40 16 40	1.77777777777778	0.111111111111111 -0.555555555555556 0.111111111111111	0.863004424640887
+128	0 0 2 0	40 24 40	1.77777777777778	0.111111111111111 -0.333333333333333 0.111111111111111	0.883486952153519
+128	0 0 2 0	40 32 40	1.77777777777778	0.111111111111111 -0.111111111111111 0.111111111111111	0.915799135771738
+128	0 0 2 0	40 40 40	1.77777777777778	0.111111111111111 0.111111111111111 0.111111111111111	0.915799135771738
+128	0 0 2 0	40 48 40	1.77777777777778	0.111111111111111 0.333333333333333 0.111111111111111	0.883486952153519
+128	0 0 2 0	40 56 40	1.77777777777778	0.111111111111111 0.555555555555556 0.111111111111111	0.863004424640887
+128	0 0 2 0	40 64 40	1.77777777777778	0.111111111111111 0.777777777777778 0.111111111111111	0.859471717622308
+128	0 0 2 0	40 72 40	1.77777777777778	0.111111111111111 1 0.111111111111111	0.873700996777018
+
+# iteration 128
+# refinement level 0   multigrid level 0   map 0   component 3   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 0 3 0	40 0 40	1.77777777777778	0.111111111111111 -1 0.111111111111111	0.873700996777018
+128	0 0 3 0	40 8 40	1.77777777777778	0.111111111111111 -0.777777777777778 0.111111111111111	0.859471717622308
+128	0 0 3 0	40 16 40	1.77777777777778	0.111111111111111 -0.555555555555556 0.111111111111111	0.863004424640887
+128	0 0 3 0	40 24 40	1.77777777777778	0.111111111111111 -0.333333333333333 0.111111111111111	0.883486952153519
+128	0 0 3 0	40 32 40	1.77777777777778	0.111111111111111 -0.111111111111111 0.111111111111111	0.915799135771738
+128	0 0 3 0	40 40 40	1.77777777777778	0.111111111111111 0.111111111111111 0.111111111111111	0.915799135771738
+128	0 0 3 0	40 48 40	1.77777777777778	0.111111111111111 0.333333333333333 0.111111111111111	0.883486952153519
+128	0 0 3 0	40 56 40	1.77777777777778	0.111111111111111 0.555555555555556 0.111111111111111	0.863004424640887
+128	0 0 3 0	40 64 40	1.77777777777778	0.111111111111111 0.777777777777778 0.111111111111111	0.859471717622308
+128	0 0 3 0	40 72 40	1.77777777777778	0.111111111111111 1 0.111111111111111	0.873700996777018
+
+
+# iteration 128
+# refinement level 1   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 1 0 0	36 8 36	1.77777777777778	-9.86623976961809e-17 -0.777777777777778 -9.86623976961809e-17	0.862653663591007
+128	0 1 0 0	36 12 36	1.77777777777778	-9.86623976961809e-17 -0.666666666666667 -9.86623976961809e-17	0.862560028606905
+128	0 1 0 0	36 16 36	1.77777777777778	-9.86623976961809e-17 -0.555555555555556 -9.86623976961809e-17	0.866965759785718
+128	0 1 0 0	36 20 36	1.77777777777778	-9.86623976961809e-17 -0.444444444444444 -9.86623976961809e-17	0.876167092595071
+128	0 1 0 0	36 24 36	1.77777777777778	-9.86623976961809e-17 -0.333333333333333 -9.86623976961809e-17	0.889185421864397
+128	0 1 0 0	36 28 36	1.77777777777778	-9.86623976961809e-17 -0.222222222222222 -9.86623976961809e-17	0.907998184681559
+128	0 1 0 0	36 32 36	1.77777777777778	-9.86623976961809e-17 -0.111111111111111 -9.86623976961809e-17	0.924033489738888
+128	0 1 0 0	36 36 36	1.77777777777778	-9.86623976961809e-17 -5.55111512312578e-17 -9.86623976961809e-17	0.928389498223199
+128	0 1 0 0	36 40 36	1.77777777777778	-9.86623976961809e-17 0.111111111111111 -9.86623976961809e-17	0.924033489738888
+128	0 1 0 0	36 44 36	1.77777777777778	-9.86623976961809e-17 0.222222222222222 -9.86623976961809e-17	0.907998184681559
+128	0 1 0 0	36 48 36	1.77777777777778	-9.86623976961809e-17 0.333333333333333 -9.86623976961809e-17	0.889185421864397
+128	0 1 0 0	36 52 36	1.77777777777778	-9.86623976961809e-17 0.444444444444444 -9.86623976961809e-17	0.876167092595071
+128	0 1 0 0	36 56 36	1.77777777777778	-9.86623976961809e-17 0.555555555555555 -9.86623976961809e-17	0.866965759785718
+128	0 1 0 0	36 60 36	1.77777777777778	-9.86623976961809e-17 0.666666666666667 -9.86623976961809e-17	0.862560028606905
+128	0 1 0 0	36 64 36	1.77777777777778	-9.86623976961809e-17 0.777777777777778 -9.86623976961809e-17	0.862653663591007
+
+# iteration 128
+# refinement level 1   multigrid level 0   map 0   component 1   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 1 1 0	36 8 36	1.77777777777778	-5.55111512312578e-17 -0.777777777777778 -9.86623976961809e-17	0.862653663591007
+128	0 1 1 0	36 12 36	1.77777777777778	-5.55111512312578e-17 -0.666666666666667 -9.86623976961809e-17	0.862560028606905
+128	0 1 1 0	36 16 36	1.77777777777778	-5.55111512312578e-17 -0.555555555555556 -9.86623976961809e-17	0.866965759785718
+128	0 1 1 0	36 20 36	1.77777777777778	-5.55111512312578e-17 -0.444444444444444 -9.86623976961809e-17	0.876167092595071
+128	0 1 1 0	36 24 36	1.77777777777778	-5.55111512312578e-17 -0.333333333333333 -9.86623976961809e-17	0.889185421864397
+128	0 1 1 0	36 28 36	1.77777777777778	-5.55111512312578e-17 -0.222222222222222 -9.86623976961809e-17	0.907998184681559
+128	0 1 1 0	36 32 36	1.77777777777778	-5.55111512312578e-17 -0.111111111111111 -9.86623976961809e-17	0.924033489738888
+128	0 1 1 0	36 36 36	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -9.86623976961809e-17	0.928389498223199
+128	0 1 1 0	36 40 36	1.77777777777778	-5.55111512312578e-17 0.111111111111111 -9.86623976961809e-17	0.924033489738888
+128	0 1 1 0	36 44 36	1.77777777777778	-5.55111512312578e-17 0.222222222222222 -9.86623976961809e-17	0.907998184681559
+128	0 1 1 0	36 48 36	1.77777777777778	-5.55111512312578e-17 0.333333333333333 -9.86623976961809e-17	0.889185421864397
+128	0 1 1 0	36 52 36	1.77777777777778	-5.55111512312578e-17 0.444444444444444 -9.86623976961809e-17	0.876167092595071
+128	0 1 1 0	36 56 36	1.77777777777778	-5.55111512312578e-17 0.555555555555555 -9.86623976961809e-17	0.866965759785718
+128	0 1 1 0	36 60 36	1.77777777777778	-5.55111512312578e-17 0.666666666666667 -9.86623976961809e-17	0.862560028606905
+128	0 1 1 0	36 64 36	1.77777777777778	-5.55111512312578e-17 0.777777777777778 -9.86623976961809e-17	0.862653663591007
+
+# iteration 128
+# refinement level 1   multigrid level 0   map 0   component 2   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 1 2 0	36 8 36	1.77777777777778	-9.86623976961809e-17 -0.777777777777778 -5.55111512312578e-17	0.862653663591007
+128	0 1 2 0	36 12 36	1.77777777777778	-9.86623976961809e-17 -0.666666666666667 -5.55111512312578e-17	0.862560028606905
+128	0 1 2 0	36 16 36	1.77777777777778	-9.86623976961809e-17 -0.555555555555556 -5.55111512312578e-17	0.866965759785718
+128	0 1 2 0	36 20 36	1.77777777777778	-9.86623976961809e-17 -0.444444444444444 -5.55111512312578e-17	0.876167092595071
+128	0 1 2 0	36 24 36	1.77777777777778	-9.86623976961809e-17 -0.333333333333333 -5.55111512312578e-17	0.889185421864397
+128	0 1 2 0	36 28 36	1.77777777777778	-9.86623976961809e-17 -0.222222222222222 -5.55111512312578e-17	0.907998184681559
+128	0 1 2 0	36 32 36	1.77777777777778	-9.86623976961809e-17 -0.111111111111111 -5.55111512312578e-17	0.924033489738888
+128	0 1 2 0	36 36 36	1.77777777777778	-9.86623976961809e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.928389498223199
+128	0 1 2 0	36 40 36	1.77777777777778	-9.86623976961809e-17 0.111111111111111 -5.55111512312578e-17	0.924033489738888
+128	0 1 2 0	36 44 36	1.77777777777778	-9.86623976961809e-17 0.222222222222222 -5.55111512312578e-17	0.907998184681559
+128	0 1 2 0	36 48 36	1.77777777777778	-9.86623976961809e-17 0.333333333333333 -5.55111512312578e-17	0.889185421864397
+128	0 1 2 0	36 52 36	1.77777777777778	-9.86623976961809e-17 0.444444444444444 -5.55111512312578e-17	0.876167092595071
+128	0 1 2 0	36 56 36	1.77777777777778	-9.86623976961809e-17 0.555555555555555 -5.55111512312578e-17	0.866965759785718
+128	0 1 2 0	36 60 36	1.77777777777778	-9.86623976961809e-17 0.666666666666667 -5.55111512312578e-17	0.862560028606905
+128	0 1 2 0	36 64 36	1.77777777777778	-9.86623976961809e-17 0.777777777777778 -5.55111512312578e-17	0.862653663591007
+
+# iteration 128
+# refinement level 1   multigrid level 0   map 0   component 3   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 1 3 0	36 8 36	1.77777777777778	-5.55111512312578e-17 -0.777777777777778 -5.55111512312578e-17	0.862653663591007
+128	0 1 3 0	36 12 36	1.77777777777778	-5.55111512312578e-17 -0.666666666666667 -5.55111512312578e-17	0.862560028606905
+128	0 1 3 0	36 16 36	1.77777777777778	-5.55111512312578e-17 -0.555555555555556 -5.55111512312578e-17	0.866965759785718
+128	0 1 3 0	36 20 36	1.77777777777778	-5.55111512312578e-17 -0.444444444444444 -5.55111512312578e-17	0.876167092595071
+128	0 1 3 0	36 24 36	1.77777777777778	-5.55111512312578e-17 -0.333333333333333 -5.55111512312578e-17	0.889185421864397
+128	0 1 3 0	36 28 36	1.77777777777778	-5.55111512312578e-17 -0.222222222222222 -5.55111512312578e-17	0.907998184681559
+128	0 1 3 0	36 32 36	1.77777777777778	-5.55111512312578e-17 -0.111111111111111 -5.55111512312578e-17	0.924033489738888
+128	0 1 3 0	36 36 36	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.928389498223199
+128	0 1 3 0	36 40 36	1.77777777777778	-5.55111512312578e-17 0.111111111111111 -5.55111512312578e-17	0.924033489738888
+128	0 1 3 0	36 44 36	1.77777777777778	-5.55111512312578e-17 0.222222222222222 -5.55111512312578e-17	0.907998184681559
+128	0 1 3 0	36 48 36	1.77777777777778	-5.55111512312578e-17 0.333333333333333 -5.55111512312578e-17	0.889185421864397
+128	0 1 3 0	36 52 36	1.77777777777778	-5.55111512312578e-17 0.444444444444444 -5.55111512312578e-17	0.876167092595071
+128	0 1 3 0	36 56 36	1.77777777777778	-5.55111512312578e-17 0.555555555555555 -5.55111512312578e-17	0.866965759785718
+128	0 1 3 0	36 60 36	1.77777777777778	-5.55111512312578e-17 0.666666666666667 -5.55111512312578e-17	0.862560028606905
+128	0 1 3 0	36 64 36	1.77777777777778	-5.55111512312578e-17 0.777777777777778 -5.55111512312578e-17	0.862653663591007
+
+
+# iteration 128
+# refinement level 2   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 2 0 0	36 22 36	1.77777777777778	2.46655994240452e-17 -0.388888888888889 2.46655994240452e-17	0.882075542854246
+128	0 2 0 0	36 24 36	1.77777777777778	2.46655994240452e-17 -0.333333333333333 2.46655994240452e-17	0.889185421864397
+128	0 2 0 0	36 26 36	1.77777777777778	2.46655994240452e-17 -0.277777777777778 2.46655994240452e-17	0.898403242286228
+128	0 2 0 0	36 28 36	1.77777777777778	2.46655994240452e-17 -0.222222222222222 2.46655994240452e-17	0.907998184681559
+128	0 2 0 0	36 30 36	1.77777777777778	2.46655994240452e-17 -0.166666666666667 2.46655994240452e-17	0.916919384356027
+128	0 2 0 0	36 32 36	1.77777777777778	2.46655994240452e-17 -0.111111111111111 2.46655994240452e-17	0.924033489738888
+128	0 2 0 0	36 34 36	1.77777777777778	2.46655994240452e-17 -0.0555555555555556 2.46655994240452e-17	0.927485951077396
+128	0 2 0 0	36 36 36	1.77777777777778	2.46655994240452e-17 -5.55111512312578e-17 2.46655994240452e-17	0.928389498223199
+128	0 2 0 0	36 38 36	1.77777777777778	2.46655994240452e-17 0.0555555555555555 2.46655994240452e-17	0.927485951077396
+128	0 2 0 0	36 40 36	1.77777777777778	2.46655994240452e-17 0.111111111111111 2.46655994240452e-17	0.924033489738888
+128	0 2 0 0	36 42 36	1.77777777777778	2.46655994240452e-17 0.166666666666667 2.46655994240452e-17	0.916919384356027
+128	0 2 0 0	36 44 36	1.77777777777778	2.46655994240452e-17 0.222222222222222 2.46655994240452e-17	0.907998184681559
+128	0 2 0 0	36 46 36	1.77777777777778	2.46655994240452e-17 0.277777777777778 2.46655994240452e-17	0.898403242286228
+128	0 2 0 0	36 48 36	1.77777777777778	2.46655994240452e-17 0.333333333333333 2.46655994240452e-17	0.889185421864397
+128	0 2 0 0	36 50 36	1.77777777777778	2.46655994240452e-17 0.388888888888889 2.46655994240452e-17	0.882075542854246
+
+# iteration 128
+# refinement level 2   multigrid level 0   map 0   component 1   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 2 1 0	36 22 36	1.77777777777778	-2.77555756156289e-17 -0.388888888888889 2.46655994240452e-17	0.882075542854246
+128	0 2 1 0	36 24 36	1.77777777777778	-2.77555756156289e-17 -0.333333333333333 2.46655994240452e-17	0.889185421864397
+128	0 2 1 0	36 26 36	1.77777777777778	-2.77555756156289e-17 -0.277777777777778 2.46655994240452e-17	0.898403242286228
+128	0 2 1 0	36 28 36	1.77777777777778	-2.77555756156289e-17 -0.222222222222222 2.46655994240452e-17	0.907998184681559
+128	0 2 1 0	36 30 36	1.77777777777778	-2.77555756156289e-17 -0.166666666666667 2.46655994240452e-17	0.916919384356027
+128	0 2 1 0	36 32 36	1.77777777777778	-2.77555756156289e-17 -0.111111111111111 2.46655994240452e-17	0.924033489738888
+128	0 2 1 0	36 34 36	1.77777777777778	-2.77555756156289e-17 -0.0555555555555556 2.46655994240452e-17	0.927485951077396
+128	0 2 1 0	36 36 36	1.77777777777778	-2.77555756156289e-17 -5.55111512312578e-17 2.46655994240452e-17	0.928389498223199
+128	0 2 1 0	36 38 36	1.77777777777778	-2.77555756156289e-17 0.0555555555555555 2.46655994240452e-17	0.927485951077396
+128	0 2 1 0	36 40 36	1.77777777777778	-2.77555756156289e-17 0.111111111111111 2.46655994240452e-17	0.924033489738888
+128	0 2 1 0	36 42 36	1.77777777777778	-2.77555756156289e-17 0.166666666666667 2.46655994240452e-17	0.916919384356027
+128	0 2 1 0	36 44 36	1.77777777777778	-2.77555756156289e-17 0.222222222222222 2.46655994240452e-17	0.907998184681559
+128	0 2 1 0	36 46 36	1.77777777777778	-2.77555756156289e-17 0.277777777777778 2.46655994240452e-17	0.898403242286228
+128	0 2 1 0	36 48 36	1.77777777777778	-2.77555756156289e-17 0.333333333333333 2.46655994240452e-17	0.889185421864397
+128	0 2 1 0	36 50 36	1.77777777777778	-2.77555756156289e-17 0.388888888888889 2.46655994240452e-17	0.882075542854246
+
+# iteration 128
+# refinement level 2   multigrid level 0   map 0   component 2   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 2 2 0	36 22 36	1.77777777777778	2.46655994240452e-17 -0.388888888888889 -2.77555756156289e-17	0.882075542854246
+128	0 2 2 0	36 24 36	1.77777777777778	2.46655994240452e-17 -0.333333333333333 -2.77555756156289e-17	0.889185421864397
+128	0 2 2 0	36 26 36	1.77777777777778	2.46655994240452e-17 -0.277777777777778 -2.77555756156289e-17	0.898403242286228
+128	0 2 2 0	36 28 36	1.77777777777778	2.46655994240452e-17 -0.222222222222222 -2.77555756156289e-17	0.907998184681559
+128	0 2 2 0	36 30 36	1.77777777777778	2.46655994240452e-17 -0.166666666666667 -2.77555756156289e-17	0.916919384356027
+128	0 2 2 0	36 32 36	1.77777777777778	2.46655994240452e-17 -0.111111111111111 -2.77555756156289e-17	0.924033489738888
+128	0 2 2 0	36 34 36	1.77777777777778	2.46655994240452e-17 -0.0555555555555556 -2.77555756156289e-17	0.927485951077396
+128	0 2 2 0	36 36 36	1.77777777777778	2.46655994240452e-17 -5.55111512312578e-17 -2.77555756156289e-17	0.928389498223199
+128	0 2 2 0	36 38 36	1.77777777777778	2.46655994240452e-17 0.0555555555555555 -2.77555756156289e-17	0.927485951077396
+128	0 2 2 0	36 40 36	1.77777777777778	2.46655994240452e-17 0.111111111111111 -2.77555756156289e-17	0.924033489738888
+128	0 2 2 0	36 42 36	1.77777777777778	2.46655994240452e-17 0.166666666666667 -2.77555756156289e-17	0.916919384356027
+128	0 2 2 0	36 44 36	1.77777777777778	2.46655994240452e-17 0.222222222222222 -2.77555756156289e-17	0.907998184681559
+128	0 2 2 0	36 46 36	1.77777777777778	2.46655994240452e-17 0.277777777777778 -2.77555756156289e-17	0.898403242286228
+128	0 2 2 0	36 48 36	1.77777777777778	2.46655994240452e-17 0.333333333333333 -2.77555756156289e-17	0.889185421864397
+128	0 2 2 0	36 50 36	1.77777777777778	2.46655994240452e-17 0.388888888888889 -2.77555756156289e-17	0.882075542854246
+
+# iteration 128
+# refinement level 2   multigrid level 0   map 0   component 3   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 2 3 0	36 22 36	1.77777777777778	-2.77555756156289e-17 -0.388888888888889 -2.77555756156289e-17	0.882075542854246
+128	0 2 3 0	36 24 36	1.77777777777778	-2.77555756156289e-17 -0.333333333333333 -2.77555756156289e-17	0.889185421864397
+128	0 2 3 0	36 26 36	1.77777777777778	-2.77555756156289e-17 -0.277777777777778 -2.77555756156289e-17	0.898403242286228
+128	0 2 3 0	36 28 36	1.77777777777778	-2.77555756156289e-17 -0.222222222222222 -2.77555756156289e-17	0.907998184681559
+128	0 2 3 0	36 30 36	1.77777777777778	-2.77555756156289e-17 -0.166666666666667 -2.77555756156289e-17	0.916919384356027
+128	0 2 3 0	36 32 36	1.77777777777778	-2.77555756156289e-17 -0.111111111111111 -2.77555756156289e-17	0.924033489738888
+128	0 2 3 0	36 34 36	1.77777777777778	-2.77555756156289e-17 -0.0555555555555556 -2.77555756156289e-17	0.927485951077396
+128	0 2 3 0	36 36 36	1.77777777777778	-2.77555756156289e-17 -5.55111512312578e-17 -2.77555756156289e-17	0.928389498223199
+128	0 2 3 0	36 38 36	1.77777777777778	-2.77555756156289e-17 0.0555555555555555 -2.77555756156289e-17	0.927485951077396
+128	0 2 3 0	36 40 36	1.77777777777778	-2.77555756156289e-17 0.111111111111111 -2.77555756156289e-17	0.924033489738888
+128	0 2 3 0	36 42 36	1.77777777777778	-2.77555756156289e-17 0.166666666666667 -2.77555756156289e-17	0.916919384356027
+128	0 2 3 0	36 44 36	1.77777777777778	-2.77555756156289e-17 0.222222222222222 -2.77555756156289e-17	0.907998184681559
+128	0 2 3 0	36 46 36	1.77777777777778	-2.77555756156289e-17 0.277777777777778 -2.77555756156289e-17	0.898403242286228
+128	0 2 3 0	36 48 36	1.77777777777778	-2.77555756156289e-17 0.333333333333333 -2.77555756156289e-17	0.889185421864397
+128	0 2 3 0	36 50 36	1.77777777777778	-2.77555756156289e-17 0.388888888888889 -2.77555756156289e-17	0.882075542854246
+
+
+# iteration 128
+# refinement level 3   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 3 0 0	36 29 36	1.77777777777778	0 -0.194444444444445 0	0.912613836832073
+128	0 3 0 0	36 30 36	1.77777777777778	0 -0.166666666666667 0	0.916919384356027
+128	0 3 0 0	36 31 36	1.77777777777778	0 -0.138888888888889 0	0.920818233193455
+128	0 3 0 0	36 32 36	1.77777777777778	0 -0.111111111111111 0	0.924033489738888
+128	0 3 0 0	36 33 36	1.77777777777778	0 -0.0833333333333334 0	0.926147880297958
+128	0 3 0 0	36 34 36	1.77777777777778	0 -0.0555555555555556 0	0.927485951077396
+128	0 3 0 0	36 35 36	1.77777777777778	0 -0.0277777777777778 0	0.928209975180567
+128	0 3 0 0	36 36 36	1.77777777777778	0 -5.55111512312578e-17 0	0.928389498223199
+128	0 3 0 0	36 37 36	1.77777777777778	0 0.0277777777777777 0	0.928209975180567
+128	0 3 0 0	36 38 36	1.77777777777778	0 0.0555555555555555 0	0.927485951077396
+128	0 3 0 0	36 39 36	1.77777777777778	0 0.0833333333333333 0	0.926147880297958
+128	0 3 0 0	36 40 36	1.77777777777778	0 0.111111111111111 0	0.924033489738888
+128	0 3 0 0	36 41 36	1.77777777777778	0 0.138888888888889 0	0.920818233193455
+128	0 3 0 0	36 42 36	1.77777777777778	0 0.166666666666667 0	0.916919384356027
+128	0 3 0 0	36 43 36	1.77777777777778	0 0.194444444444444 0	0.912613836832073
+
+# iteration 128
+# refinement level 3   multigrid level 0   map 0   component 1   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 3 1 0	36 29 36	1.77777777777778	-1.38777878078145e-17 -0.194444444444445 0	0.912613836832073
+128	0 3 1 0	36 30 36	1.77777777777778	-1.38777878078145e-17 -0.166666666666667 0	0.916919384356027
+128	0 3 1 0	36 31 36	1.77777777777778	-1.38777878078145e-17 -0.138888888888889 0	0.920818233193455
+128	0 3 1 0	36 32 36	1.77777777777778	-1.38777878078145e-17 -0.111111111111111 0	0.924033489738888
+128	0 3 1 0	36 33 36	1.77777777777778	-1.38777878078145e-17 -0.0833333333333334 0	0.926147880297958
+128	0 3 1 0	36 34 36	1.77777777777778	-1.38777878078145e-17 -0.0555555555555556 0	0.927485951077396
+128	0 3 1 0	36 35 36	1.77777777777778	-1.38777878078145e-17 -0.0277777777777778 0	0.928209975180567
+128	0 3 1 0	36 36 36	1.77777777777778	-1.38777878078145e-17 -5.55111512312578e-17 0	0.928389498223199
+128	0 3 1 0	36 37 36	1.77777777777778	-1.38777878078145e-17 0.0277777777777777 0	0.928209975180567
+128	0 3 1 0	36 38 36	1.77777777777778	-1.38777878078145e-17 0.0555555555555555 0	0.927485951077396
+128	0 3 1 0	36 39 36	1.77777777777778	-1.38777878078145e-17 0.0833333333333333 0	0.926147880297958
+128	0 3 1 0	36 40 36	1.77777777777778	-1.38777878078145e-17 0.111111111111111 0	0.924033489738888
+128	0 3 1 0	36 41 36	1.77777777777778	-1.38777878078145e-17 0.138888888888889 0	0.920818233193455
+128	0 3 1 0	36 42 36	1.77777777777778	-1.38777878078145e-17 0.166666666666667 0	0.916919384356027
+128	0 3 1 0	36 43 36	1.77777777777778	-1.38777878078145e-17 0.194444444444444 0	0.912613836832073
+
+# iteration 128
+# refinement level 3   multigrid level 0   map 0   component 2   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 3 2 0	36 29 36	1.77777777777778	0 -0.194444444444445 -1.38777878078145e-17	0.912613836832073
+128	0 3 2 0	36 30 36	1.77777777777778	0 -0.166666666666667 -1.38777878078145e-17	0.916919384356027
+128	0 3 2 0	36 31 36	1.77777777777778	0 -0.138888888888889 -1.38777878078145e-17	0.920818233193455
+128	0 3 2 0	36 32 36	1.77777777777778	0 -0.111111111111111 -1.38777878078145e-17	0.924033489738888
+128	0 3 2 0	36 33 36	1.77777777777778	0 -0.0833333333333334 -1.38777878078145e-17	0.926147880297958
+128	0 3 2 0	36 34 36	1.77777777777778	0 -0.0555555555555556 -1.38777878078145e-17	0.927485951077396
+128	0 3 2 0	36 35 36	1.77777777777778	0 -0.0277777777777778 -1.38777878078145e-17	0.928209975180567
+128	0 3 2 0	36 36 36	1.77777777777778	0 -5.55111512312578e-17 -1.38777878078145e-17	0.928389498223199
+128	0 3 2 0	36 37 36	1.77777777777778	0 0.0277777777777777 -1.38777878078145e-17	0.928209975180567
+128	0 3 2 0	36 38 36	1.77777777777778	0 0.0555555555555555 -1.38777878078145e-17	0.927485951077396
+128	0 3 2 0	36 39 36	1.77777777777778	0 0.0833333333333333 -1.38777878078145e-17	0.926147880297958
+128	0 3 2 0	36 40 36	1.77777777777778	0 0.111111111111111 -1.38777878078145e-17	0.924033489738888
+128	0 3 2 0	36 41 36	1.77777777777778	0 0.138888888888889 -1.38777878078145e-17	0.920818233193455
+128	0 3 2 0	36 42 36	1.77777777777778	0 0.166666666666667 -1.38777878078145e-17	0.916919384356027
+128	0 3 2 0	36 43 36	1.77777777777778	0 0.194444444444444 -1.38777878078145e-17	0.912613836832073
+
+# iteration 128
+# refinement level 3   multigrid level 0   map 0   component 3   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 3 3 0	36 29 36	1.77777777777778	-1.38777878078145e-17 -0.194444444444445 -1.38777878078145e-17	0.912613836832073
+128	0 3 3 0	36 30 36	1.77777777777778	-1.38777878078145e-17 -0.166666666666667 -1.38777878078145e-17	0.916919384356027
+128	0 3 3 0	36 31 36	1.77777777777778	-1.38777878078145e-17 -0.138888888888889 -1.38777878078145e-17	0.920818233193455
+128	0 3 3 0	36 32 36	1.77777777777778	-1.38777878078145e-17 -0.111111111111111 -1.38777878078145e-17	0.924033489738888
+128	0 3 3 0	36 33 36	1.77777777777778	-1.38777878078145e-17 -0.0833333333333334 -1.38777878078145e-17	0.926147880297958
+128	0 3 3 0	36 34 36	1.77777777777778	-1.38777878078145e-17 -0.0555555555555556 -1.38777878078145e-17	0.927485951077396
+128	0 3 3 0	36 35 36	1.77777777777778	-1.38777878078145e-17 -0.0277777777777778 -1.38777878078145e-17	0.928209975180567
+128	0 3 3 0	36 36 36	1.77777777777778	-1.38777878078145e-17 -5.55111512312578e-17 -1.38777878078145e-17	0.928389498223199
+128	0 3 3 0	36 37 36	1.77777777777778	-1.38777878078145e-17 0.0277777777777777 -1.38777878078145e-17	0.928209975180567
+128	0 3 3 0	36 38 36	1.77777777777778	-1.38777878078145e-17 0.0555555555555555 -1.38777878078145e-17	0.927485951077396
+128	0 3 3 0	36 39 36	1.77777777777778	-1.38777878078145e-17 0.0833333333333333 -1.38777878078145e-17	0.926147880297958
+128	0 3 3 0	36 40 36	1.77777777777778	-1.38777878078145e-17 0.111111111111111 -1.38777878078145e-17	0.924033489738888
+128	0 3 3 0	36 41 36	1.77777777777778	-1.38777878078145e-17 0.138888888888889 -1.38777878078145e-17	0.920818233193455
+128	0 3 3 0	36 42 36	1.77777777777778	-1.38777878078145e-17 0.166666666666667 -1.38777878078145e-17	0.916919384356027
+128	0 3 3 0	36 43 36	1.77777777777778	-1.38777878078145e-17 0.194444444444444 -1.38777878078145e-17	0.912613836832073
+
+
diff --git a/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_4proc/phi.z.asc b/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_4proc/phi.z.asc
new file mode 100644
index 000000000..343b8054b
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_4proc/phi.z.asc
@@ -0,0 +1,632 @@
+# WAVETOY::phi z (phi)
+#
+# iteration 64
+# refinement level 0   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 0 0 0	40 40 0	0.888888888888889	0.111111111111111 0.111111111111111 -1	0.873700996777018
+64	0 0 0 0	40 40 8	0.888888888888889	0.111111111111111 0.111111111111111 -0.777777777777778	0.807676282684182
+64	0 0 0 0	40 40 16	0.888888888888889	0.111111111111111 0.111111111111111 -0.555555555555556	0.756699260944861
+64	0 0 0 0	40 40 24	0.888888888888889	0.111111111111111 0.111111111111111 -0.333333333333333	0.730294733882952
+64	0 0 0 0	40 40 32	0.888888888888889	0.111111111111111 0.111111111111111 -0.111111111111111	0.725539249473295
+64	0 0 0 0	40 40 40	0.888888888888889	0.111111111111111 0.111111111111111 0.111111111111111	0.725539249473295
+64	0 0 0 0	40 40 48	0.888888888888889	0.111111111111111 0.111111111111111 0.333333333333333	0.730294733882952
+
+# iteration 64
+# refinement level 0   multigrid level 0   map 0   component 1   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 0 1 0	40 40 0	0.888888888888889	0.111111111111111 0.111111111111111 -1	0.873700996777018
+64	0 0 1 0	40 40 8	0.888888888888889	0.111111111111111 0.111111111111111 -0.777777777777778	0.807676282684182
+64	0 0 1 0	40 40 16	0.888888888888889	0.111111111111111 0.111111111111111 -0.555555555555556	0.756699260944861
+64	0 0 1 0	40 40 24	0.888888888888889	0.111111111111111 0.111111111111111 -0.333333333333333	0.730294733882952
+64	0 0 1 0	40 40 32	0.888888888888889	0.111111111111111 0.111111111111111 -0.111111111111111	0.725539249473295
+64	0 0 1 0	40 40 40	0.888888888888889	0.111111111111111 0.111111111111111 0.111111111111111	0.725539249473295
+64	0 0 1 0	40 40 48	0.888888888888889	0.111111111111111 0.111111111111111 0.333333333333333	0.730294733882952
+
+# iteration 64
+# refinement level 0   multigrid level 0   map 0   component 2   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 0 2 0	40 40 24	0.888888888888889	0.111111111111111 0.111111111111111 -0.333333333333333	0.730294733882952
+64	0 0 2 0	40 40 32	0.888888888888889	0.111111111111111 0.111111111111111 -0.111111111111111	0.725539249473295
+64	0 0 2 0	40 40 40	0.888888888888889	0.111111111111111 0.111111111111111 0.111111111111111	0.725539249473295
+64	0 0 2 0	40 40 48	0.888888888888889	0.111111111111111 0.111111111111111 0.333333333333333	0.730294733882952
+64	0 0 2 0	40 40 56	0.888888888888889	0.111111111111111 0.111111111111111 0.555555555555556	0.756699260944861
+64	0 0 2 0	40 40 64	0.888888888888889	0.111111111111111 0.111111111111111 0.777777777777778	0.807676282684182
+64	0 0 2 0	40 40 72	0.888888888888889	0.111111111111111 0.111111111111111 1	0.873700996777018
+
+# iteration 64
+# refinement level 0   multigrid level 0   map 0   component 3   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 0 3 0	40 40 24	0.888888888888889	0.111111111111111 0.111111111111111 -0.333333333333333	0.730294733882952
+64	0 0 3 0	40 40 32	0.888888888888889	0.111111111111111 0.111111111111111 -0.111111111111111	0.725539249473295
+64	0 0 3 0	40 40 40	0.888888888888889	0.111111111111111 0.111111111111111 0.111111111111111	0.725539249473295
+64	0 0 3 0	40 40 48	0.888888888888889	0.111111111111111 0.111111111111111 0.333333333333333	0.730294733882952
+64	0 0 3 0	40 40 56	0.888888888888889	0.111111111111111 0.111111111111111 0.555555555555556	0.756699260944861
+64	0 0 3 0	40 40 64	0.888888888888889	0.111111111111111 0.111111111111111 0.777777777777778	0.807676282684182
+64	0 0 3 0	40 40 72	0.888888888888889	0.111111111111111 0.111111111111111 1	0.873700996777018
+
+
+# iteration 64
+# refinement level 1   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 1 0 0	36 36 8	0.888888888888889	-9.86623976961809e-17 -5.55111512312578e-17 -0.777777777777778	0.808379332677938
+64	0 1 0 0	36 36 12	0.888888888888889	-9.86623976961809e-17 -5.55111512312578e-17 -0.666666666666667	0.779665123251562
+64	0 1 0 0	36 36 16	0.888888888888889	-9.86623976961809e-17 -5.55111512312578e-17 -0.555555555555556	0.756097245006156
+64	0 1 0 0	36 36 20	0.888888888888889	-9.86623976961809e-17 -5.55111512312578e-17 -0.444444444444444	0.739694852518638
+64	0 1 0 0	36 36 24	0.888888888888889	-9.86623976961809e-17 -5.55111512312578e-17 -0.333333333333333	0.729218296316213
+64	0 1 0 0	36 36 28	0.888888888888889	-9.86623976961809e-17 -5.55111512312578e-17 -0.222222222222222	0.725100663327574
+64	0 1 0 0	36 36 32	0.888888888888889	-9.86623976961809e-17 -5.55111512312578e-17 -0.111111111111111	0.724342898925177
+64	0 1 0 0	36 36 36	0.888888888888889	-9.86623976961809e-17 -5.55111512312578e-17 -9.86623976961809e-17	0.723733474251298
+64	0 1 0 0	36 36 40	0.888888888888889	-9.86623976961809e-17 -5.55111512312578e-17 0.111111111111111	0.724342898925177
+64	0 1 0 0	36 36 44	0.888888888888889	-9.86623976961809e-17 -5.55111512312578e-17 0.222222222222222	0.725100663327574
+
+# iteration 64
+# refinement level 1   multigrid level 0   map 0   component 1   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 1 1 0	36 36 8	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -0.777777777777778	0.808379332677938
+64	0 1 1 0	36 36 12	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -0.666666666666667	0.779665123251562
+64	0 1 1 0	36 36 16	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -0.555555555555556	0.756097245006156
+64	0 1 1 0	36 36 20	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -0.444444444444444	0.739694852518638
+64	0 1 1 0	36 36 24	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -0.333333333333333	0.729218296316213
+64	0 1 1 0	36 36 28	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -0.222222222222222	0.725100663327574
+64	0 1 1 0	36 36 32	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -0.111111111111111	0.724342898925177
+64	0 1 1 0	36 36 36	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -9.86623976961809e-17	0.723733474251298
+64	0 1 1 0	36 36 40	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 0.111111111111111	0.724342898925177
+64	0 1 1 0	36 36 44	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 0.222222222222222	0.725100663327574
+
+# iteration 64
+# refinement level 1   multigrid level 0   map 0   component 2   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 1 2 0	36 36 32	0.888888888888889	-9.86623976961809e-17 -5.55111512312578e-17 -0.111111111111111	0.724342898925177
+64	0 1 2 0	36 36 36	0.888888888888889	-9.86623976961809e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.723733474251298
+64	0 1 2 0	36 36 40	0.888888888888889	-9.86623976961809e-17 -5.55111512312578e-17 0.111111111111111	0.724342898925177
+64	0 1 2 0	36 36 44	0.888888888888889	-9.86623976961809e-17 -5.55111512312578e-17 0.222222222222222	0.725100663327574
+64	0 1 2 0	36 36 48	0.888888888888889	-9.86623976961809e-17 -5.55111512312578e-17 0.333333333333333	0.729218296316213
+64	0 1 2 0	36 36 52	0.888888888888889	-9.86623976961809e-17 -5.55111512312578e-17 0.444444444444444	0.739694852518638
+64	0 1 2 0	36 36 56	0.888888888888889	-9.86623976961809e-17 -5.55111512312578e-17 0.555555555555555	0.756097245006155
+64	0 1 2 0	36 36 60	0.888888888888889	-9.86623976961809e-17 -5.55111512312578e-17 0.666666666666667	0.779665123251561
+64	0 1 2 0	36 36 64	0.888888888888889	-9.86623976961809e-17 -5.55111512312578e-17 0.777777777777778	0.808379332677938
+
+# iteration 64
+# refinement level 1   multigrid level 0   map 0   component 3   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 1 3 0	36 36 32	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -0.111111111111111	0.724342898925177
+64	0 1 3 0	36 36 36	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.723733474251298
+64	0 1 3 0	36 36 40	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 0.111111111111111	0.724342898925177
+64	0 1 3 0	36 36 44	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 0.222222222222222	0.725100663327574
+64	0 1 3 0	36 36 48	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 0.333333333333333	0.729218296316213
+64	0 1 3 0	36 36 52	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 0.444444444444444	0.739694852518638
+64	0 1 3 0	36 36 56	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 0.555555555555555	0.756097245006155
+64	0 1 3 0	36 36 60	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 0.666666666666667	0.779665123251561
+64	0 1 3 0	36 36 64	0.888888888888889	-5.55111512312578e-17 -5.55111512312578e-17 0.777777777777778	0.808379332677938
+
+
+# iteration 64
+# refinement level 2   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 2 0 0	36 36 22	0.888888888888889	2.46655994240452e-17 -5.55111512312578e-17 -0.388888888888889	0.733688776948746
+64	0 2 0 0	36 36 24	0.888888888888889	2.46655994240452e-17 -5.55111512312578e-17 -0.333333333333333	0.729218296316213
+64	0 2 0 0	36 36 26	0.888888888888889	2.46655994240452e-17 -5.55111512312578e-17 -0.277777777777778	0.726552055334392
+64	0 2 0 0	36 36 28	0.888888888888889	2.46655994240452e-17 -5.55111512312578e-17 -0.222222222222222	0.725100663327574
+64	0 2 0 0	36 36 30	0.888888888888889	2.46655994240452e-17 -5.55111512312578e-17 -0.166666666666667	0.724502518106703
+64	0 2 0 0	36 36 32	0.888888888888889	2.46655994240452e-17 -5.55111512312578e-17 -0.111111111111111	0.724342898925177
+64	0 2 0 0	36 36 34	0.888888888888889	2.46655994240452e-17 -5.55111512312578e-17 -0.0555555555555555	0.72395273727097
+64	0 2 0 0	36 36 36	0.888888888888889	2.46655994240452e-17 -5.55111512312578e-17 2.46655994240452e-17	0.723733474251298
+64	0 2 0 0	36 36 38	0.888888888888889	2.46655994240452e-17 -5.55111512312578e-17 0.0555555555555556	0.72395273727097
+64	0 2 0 0	36 36 40	0.888888888888889	2.46655994240452e-17 -5.55111512312578e-17 0.111111111111111	0.724342898925177
+
+# iteration 64
+# refinement level 2   multigrid level 0   map 0   component 1   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 2 1 0	36 36 22	0.888888888888889	-2.77555756156289e-17 -5.55111512312578e-17 -0.388888888888889	0.733688776948746
+64	0 2 1 0	36 36 24	0.888888888888889	-2.77555756156289e-17 -5.55111512312578e-17 -0.333333333333333	0.729218296316213
+64	0 2 1 0	36 36 26	0.888888888888889	-2.77555756156289e-17 -5.55111512312578e-17 -0.277777777777778	0.726552055334392
+64	0 2 1 0	36 36 28	0.888888888888889	-2.77555756156289e-17 -5.55111512312578e-17 -0.222222222222222	0.725100663327574
+64	0 2 1 0	36 36 30	0.888888888888889	-2.77555756156289e-17 -5.55111512312578e-17 -0.166666666666667	0.724502518106703
+64	0 2 1 0	36 36 32	0.888888888888889	-2.77555756156289e-17 -5.55111512312578e-17 -0.111111111111111	0.724342898925177
+64	0 2 1 0	36 36 34	0.888888888888889	-2.77555756156289e-17 -5.55111512312578e-17 -0.0555555555555555	0.72395273727097
+64	0 2 1 0	36 36 36	0.888888888888889	-2.77555756156289e-17 -5.55111512312578e-17 2.46655994240452e-17	0.723733474251298
+64	0 2 1 0	36 36 38	0.888888888888889	-2.77555756156289e-17 -5.55111512312578e-17 0.0555555555555556	0.72395273727097
+64	0 2 1 0	36 36 40	0.888888888888889	-2.77555756156289e-17 -5.55111512312578e-17 0.111111111111111	0.724342898925177
+
+# iteration 64
+# refinement level 2   multigrid level 0   map 0   component 2   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 2 2 0	36 36 34	0.888888888888889	2.46655994240452e-17 -5.55111512312578e-17 -0.0555555555555556	0.72395273727097
+64	0 2 2 0	36 36 36	0.888888888888889	2.46655994240452e-17 -5.55111512312578e-17 -2.77555756156289e-17	0.723733474251298
+64	0 2 2 0	36 36 38	0.888888888888889	2.46655994240452e-17 -5.55111512312578e-17 0.0555555555555555	0.72395273727097
+64	0 2 2 0	36 36 40	0.888888888888889	2.46655994240452e-17 -5.55111512312578e-17 0.111111111111111	0.724342898925177
+64	0 2 2 0	36 36 42	0.888888888888889	2.46655994240452e-17 -5.55111512312578e-17 0.166666666666667	0.724502518106703
+64	0 2 2 0	36 36 44	0.888888888888889	2.46655994240452e-17 -5.55111512312578e-17 0.222222222222222	0.725100663327574
+64	0 2 2 0	36 36 46	0.888888888888889	2.46655994240452e-17 -5.55111512312578e-17 0.277777777777778	0.726552055334391
+64	0 2 2 0	36 36 48	0.888888888888889	2.46655994240452e-17 -5.55111512312578e-17 0.333333333333333	0.729218296316213
+64	0 2 2 0	36 36 50	0.888888888888889	2.46655994240452e-17 -5.55111512312578e-17 0.388888888888889	0.733688776948745
+
+# iteration 64
+# refinement level 2   multigrid level 0   map 0   component 3   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 2 3 0	36 36 34	0.888888888888889	-2.77555756156289e-17 -5.55111512312578e-17 -0.0555555555555556	0.72395273727097
+64	0 2 3 0	36 36 36	0.888888888888889	-2.77555756156289e-17 -5.55111512312578e-17 -2.77555756156289e-17	0.723733474251298
+64	0 2 3 0	36 36 38	0.888888888888889	-2.77555756156289e-17 -5.55111512312578e-17 0.0555555555555555	0.72395273727097
+64	0 2 3 0	36 36 40	0.888888888888889	-2.77555756156289e-17 -5.55111512312578e-17 0.111111111111111	0.724342898925177
+64	0 2 3 0	36 36 42	0.888888888888889	-2.77555756156289e-17 -5.55111512312578e-17 0.166666666666667	0.724502518106703
+64	0 2 3 0	36 36 44	0.888888888888889	-2.77555756156289e-17 -5.55111512312578e-17 0.222222222222222	0.725100663327574
+64	0 2 3 0	36 36 46	0.888888888888889	-2.77555756156289e-17 -5.55111512312578e-17 0.277777777777778	0.726552055334391
+64	0 2 3 0	36 36 48	0.888888888888889	-2.77555756156289e-17 -5.55111512312578e-17 0.333333333333333	0.729218296316213
+64	0 2 3 0	36 36 50	0.888888888888889	-2.77555756156289e-17 -5.55111512312578e-17 0.388888888888889	0.733688776948745
+
+
+# iteration 64
+# refinement level 3   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 3 0 0	36 36 29	0.888888888888889	0 -5.55111512312578e-17 -0.194444444444445	0.724720854915558
+64	0 3 0 0	36 36 30	0.888888888888889	0 -5.55111512312578e-17 -0.166666666666667	0.724502518106703
+64	0 3 0 0	36 36 31	0.888888888888889	0 -5.55111512312578e-17 -0.138888888888889	0.724409709543024
+64	0 3 0 0	36 36 32	0.888888888888889	0 -5.55111512312578e-17 -0.111111111111111	0.724342898925177
+64	0 3 0 0	36 36 33	0.888888888888889	0 -5.55111512312578e-17 -0.0833333333333334	0.724151545837958
+64	0 3 0 0	36 36 34	0.888888888888889	0 -5.55111512312578e-17 -0.0555555555555556	0.72395273727097
+64	0 3 0 0	36 36 35	0.888888888888889	0 -5.55111512312578e-17 -0.0277777777777778	0.723805016719017
+64	0 3 0 0	36 36 36	0.888888888888889	0 -5.55111512312578e-17 0	0.723733474251298
+64	0 3 0 0	36 36 37	0.888888888888889	0 -5.55111512312578e-17 0.0277777777777778	0.723805016719017
+64	0 3 0 0	36 36 38	0.888888888888889	0 -5.55111512312578e-17 0.0555555555555556	0.72395273727097
+
+# iteration 64
+# refinement level 3   multigrid level 0   map 0   component 1   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 3 1 0	36 36 29	0.888888888888889	-1.38777878078145e-17 -5.55111512312578e-17 -0.194444444444445	0.724720854915558
+64	0 3 1 0	36 36 30	0.888888888888889	-1.38777878078145e-17 -5.55111512312578e-17 -0.166666666666667	0.724502518106703
+64	0 3 1 0	36 36 31	0.888888888888889	-1.38777878078145e-17 -5.55111512312578e-17 -0.138888888888889	0.724409709543024
+64	0 3 1 0	36 36 32	0.888888888888889	-1.38777878078145e-17 -5.55111512312578e-17 -0.111111111111111	0.724342898925177
+64	0 3 1 0	36 36 33	0.888888888888889	-1.38777878078145e-17 -5.55111512312578e-17 -0.0833333333333334	0.724151545837958
+64	0 3 1 0	36 36 34	0.888888888888889	-1.38777878078145e-17 -5.55111512312578e-17 -0.0555555555555556	0.72395273727097
+64	0 3 1 0	36 36 35	0.888888888888889	-1.38777878078145e-17 -5.55111512312578e-17 -0.0277777777777778	0.723805016719017
+64	0 3 1 0	36 36 36	0.888888888888889	-1.38777878078145e-17 -5.55111512312578e-17 0	0.723733474251298
+64	0 3 1 0	36 36 37	0.888888888888889	-1.38777878078145e-17 -5.55111512312578e-17 0.0277777777777778	0.723805016719017
+64	0 3 1 0	36 36 38	0.888888888888889	-1.38777878078145e-17 -5.55111512312578e-17 0.0555555555555556	0.72395273727097
+
+# iteration 64
+# refinement level 3   multigrid level 0   map 0   component 2   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 3 2 0	36 36 35	0.888888888888889	0 -5.55111512312578e-17 -0.0277777777777778	0.723805016719017
+64	0 3 2 0	36 36 36	0.888888888888889	0 -5.55111512312578e-17 -1.38777878078145e-17	0.723733474251298
+64	0 3 2 0	36 36 37	0.888888888888889	0 -5.55111512312578e-17 0.0277777777777778	0.723805016719017
+64	0 3 2 0	36 36 38	0.888888888888889	0 -5.55111512312578e-17 0.0555555555555555	0.72395273727097
+64	0 3 2 0	36 36 39	0.888888888888889	0 -5.55111512312578e-17 0.0833333333333333	0.724151545837958
+64	0 3 2 0	36 36 40	0.888888888888889	0 -5.55111512312578e-17 0.111111111111111	0.724342898925177
+64	0 3 2 0	36 36 41	0.888888888888889	0 -5.55111512312578e-17 0.138888888888889	0.724409709543024
+64	0 3 2 0	36 36 42	0.888888888888889	0 -5.55111512312578e-17 0.166666666666667	0.724502518106703
+64	0 3 2 0	36 36 43	0.888888888888889	0 -5.55111512312578e-17 0.194444444444444	0.724720854915557
+
+# iteration 64
+# refinement level 3   multigrid level 0   map 0   component 3   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+64	0 3 3 0	36 36 35	0.888888888888889	-1.38777878078145e-17 -5.55111512312578e-17 -0.0277777777777778	0.723805016719017
+64	0 3 3 0	36 36 36	0.888888888888889	-1.38777878078145e-17 -5.55111512312578e-17 -1.38777878078145e-17	0.723733474251298
+64	0 3 3 0	36 36 37	0.888888888888889	-1.38777878078145e-17 -5.55111512312578e-17 0.0277777777777778	0.723805016719017
+64	0 3 3 0	36 36 38	0.888888888888889	-1.38777878078145e-17 -5.55111512312578e-17 0.0555555555555555	0.72395273727097
+64	0 3 3 0	36 36 39	0.888888888888889	-1.38777878078145e-17 -5.55111512312578e-17 0.0833333333333333	0.724151545837958
+64	0 3 3 0	36 36 40	0.888888888888889	-1.38777878078145e-17 -5.55111512312578e-17 0.111111111111111	0.724342898925177
+64	0 3 3 0	36 36 41	0.888888888888889	-1.38777878078145e-17 -5.55111512312578e-17 0.138888888888889	0.724409709543024
+64	0 3 3 0	36 36 42	0.888888888888889	-1.38777878078145e-17 -5.55111512312578e-17 0.166666666666667	0.724502518106703
+64	0 3 3 0	36 36 43	0.888888888888889	-1.38777878078145e-17 -5.55111512312578e-17 0.194444444444444	0.724720854915557
+
+
+# iteration 96
+# refinement level 0   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 0 0 0	40 40 0	1.33333333333333	0.111111111111111 0.111111111111111 -1	0.877480603359996
+96	0 0 0 0	40 40 8	1.33333333333333	0.111111111111111 0.111111111111111 -0.777777777777778	0.798008089475601
+96	0 0 0 0	40 40 16	1.33333333333333	0.111111111111111 0.111111111111111 -0.555555555555556	0.735090597558599
+96	0 0 0 0	40 40 24	1.33333333333333	0.111111111111111 0.111111111111111 -0.333333333333333	0.677899959110667
+96	0 0 0 0	40 40 32	1.33333333333333	0.111111111111111 0.111111111111111 -0.111111111111111	0.632828813666694
+96	0 0 0 0	40 40 40	1.33333333333333	0.111111111111111 0.111111111111111 0.111111111111111	0.632828813666694
+96	0 0 0 0	40 40 48	1.33333333333333	0.111111111111111 0.111111111111111 0.333333333333333	0.677899959110667
+
+# iteration 96
+# refinement level 0   multigrid level 0   map 0   component 1   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 0 1 0	40 40 0	1.33333333333333	0.111111111111111 0.111111111111111 -1	0.877480603359996
+96	0 0 1 0	40 40 8	1.33333333333333	0.111111111111111 0.111111111111111 -0.777777777777778	0.798008089475601
+96	0 0 1 0	40 40 16	1.33333333333333	0.111111111111111 0.111111111111111 -0.555555555555556	0.735090597558599
+96	0 0 1 0	40 40 24	1.33333333333333	0.111111111111111 0.111111111111111 -0.333333333333333	0.677899959110667
+96	0 0 1 0	40 40 32	1.33333333333333	0.111111111111111 0.111111111111111 -0.111111111111111	0.632828813666694
+96	0 0 1 0	40 40 40	1.33333333333333	0.111111111111111 0.111111111111111 0.111111111111111	0.632828813666694
+96	0 0 1 0	40 40 48	1.33333333333333	0.111111111111111 0.111111111111111 0.333333333333333	0.677899959110667
+
+# iteration 96
+# refinement level 0   multigrid level 0   map 0   component 2   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 0 2 0	40 40 24	1.33333333333333	0.111111111111111 0.111111111111111 -0.333333333333333	0.677899959110667
+96	0 0 2 0	40 40 32	1.33333333333333	0.111111111111111 0.111111111111111 -0.111111111111111	0.632828813666694
+96	0 0 2 0	40 40 40	1.33333333333333	0.111111111111111 0.111111111111111 0.111111111111111	0.632828813666694
+96	0 0 2 0	40 40 48	1.33333333333333	0.111111111111111 0.111111111111111 0.333333333333333	0.677899959110667
+96	0 0 2 0	40 40 56	1.33333333333333	0.111111111111111 0.111111111111111 0.555555555555556	0.735090597558599
+96	0 0 2 0	40 40 64	1.33333333333333	0.111111111111111 0.111111111111111 0.777777777777778	0.798008089475601
+96	0 0 2 0	40 40 72	1.33333333333333	0.111111111111111 0.111111111111111 1	0.877480603359996
+
+# iteration 96
+# refinement level 0   multigrid level 0   map 0   component 3   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 0 3 0	40 40 24	1.33333333333333	0.111111111111111 0.111111111111111 -0.333333333333333	0.677899959110667
+96	0 0 3 0	40 40 32	1.33333333333333	0.111111111111111 0.111111111111111 -0.111111111111111	0.632828813666694
+96	0 0 3 0	40 40 40	1.33333333333333	0.111111111111111 0.111111111111111 0.111111111111111	0.632828813666694
+96	0 0 3 0	40 40 48	1.33333333333333	0.111111111111111 0.111111111111111 0.333333333333333	0.677899959110667
+96	0 0 3 0	40 40 56	1.33333333333333	0.111111111111111 0.111111111111111 0.555555555555556	0.735090597558599
+96	0 0 3 0	40 40 64	1.33333333333333	0.111111111111111 0.111111111111111 0.777777777777778	0.798008089475601
+96	0 0 3 0	40 40 72	1.33333333333333	0.111111111111111 0.111111111111111 1	0.877480603359996
+
+
+# iteration 96
+# refinement level 1   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 1 0 0	36 36 8	1.33333333333333	-9.86623976961809e-17 -5.55111512312578e-17 -0.777777777777778	0.796688784792057
+96	0 1 0 0	36 36 12	1.33333333333333	-9.86623976961809e-17 -5.55111512312578e-17 -0.666666666666667	0.76190136189151
+96	0 1 0 0	36 36 16	1.33333333333333	-9.86623976961809e-17 -5.55111512312578e-17 -0.555555555555556	0.729928937348657
+96	0 1 0 0	36 36 20	1.33333333333333	-9.86623976961809e-17 -5.55111512312578e-17 -0.444444444444444	0.698215010923196
+96	0 1 0 0	36 36 24	1.33333333333333	-9.86623976961809e-17 -5.55111512312578e-17 -0.333333333333333	0.668908746807157
+96	0 1 0 0	36 36 28	1.33333333333333	-9.86623976961809e-17 -5.55111512312578e-17 -0.222222222222222	0.641345710414296
+96	0 1 0 0	36 36 32	1.33333333333333	-9.86623976961809e-17 -5.55111512312578e-17 -0.111111111111111	0.621410197839124
+96	0 1 0 0	36 36 36	1.33333333333333	-9.86623976961809e-17 -5.55111512312578e-17 -9.86623976961809e-17	0.61547287921812
+96	0 1 0 0	36 36 40	1.33333333333333	-9.86623976961809e-17 -5.55111512312578e-17 0.111111111111111	0.621410197839124
+96	0 1 0 0	36 36 44	1.33333333333333	-9.86623976961809e-17 -5.55111512312578e-17 0.222222222222222	0.641345710414297
+
+# iteration 96
+# refinement level 1   multigrid level 0   map 0   component 1   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 1 1 0	36 36 8	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -0.777777777777778	0.796688784792057
+96	0 1 1 0	36 36 12	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -0.666666666666667	0.76190136189151
+96	0 1 1 0	36 36 16	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -0.555555555555556	0.729928937348657
+96	0 1 1 0	36 36 20	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -0.444444444444444	0.698215010923196
+96	0 1 1 0	36 36 24	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -0.333333333333333	0.668908746807157
+96	0 1 1 0	36 36 28	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -0.222222222222222	0.641345710414296
+96	0 1 1 0	36 36 32	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -0.111111111111111	0.621410197839124
+96	0 1 1 0	36 36 36	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -9.86623976961809e-17	0.61547287921812
+96	0 1 1 0	36 36 40	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 0.111111111111111	0.621410197839124
+96	0 1 1 0	36 36 44	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 0.222222222222222	0.641345710414297
+
+# iteration 96
+# refinement level 1   multigrid level 0   map 0   component 2   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 1 2 0	36 36 32	1.33333333333333	-9.86623976961809e-17 -5.55111512312578e-17 -0.111111111111111	0.621410197839124
+96	0 1 2 0	36 36 36	1.33333333333333	-9.86623976961809e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.61547287921812
+96	0 1 2 0	36 36 40	1.33333333333333	-9.86623976961809e-17 -5.55111512312578e-17 0.111111111111111	0.621410197839124
+96	0 1 2 0	36 36 44	1.33333333333333	-9.86623976961809e-17 -5.55111512312578e-17 0.222222222222222	0.641345710414297
+96	0 1 2 0	36 36 48	1.33333333333333	-9.86623976961809e-17 -5.55111512312578e-17 0.333333333333333	0.668908746807157
+96	0 1 2 0	36 36 52	1.33333333333333	-9.86623976961809e-17 -5.55111512312578e-17 0.444444444444444	0.698215010923196
+96	0 1 2 0	36 36 56	1.33333333333333	-9.86623976961809e-17 -5.55111512312578e-17 0.555555555555555	0.729928937348657
+96	0 1 2 0	36 36 60	1.33333333333333	-9.86623976961809e-17 -5.55111512312578e-17 0.666666666666667	0.76190136189151
+96	0 1 2 0	36 36 64	1.33333333333333	-9.86623976961809e-17 -5.55111512312578e-17 0.777777777777778	0.796688784792057
+
+# iteration 96
+# refinement level 1   multigrid level 0   map 0   component 3   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 1 3 0	36 36 32	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -0.111111111111111	0.621410197839124
+96	0 1 3 0	36 36 36	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.61547287921812
+96	0 1 3 0	36 36 40	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 0.111111111111111	0.621410197839124
+96	0 1 3 0	36 36 44	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 0.222222222222222	0.641345710414297
+96	0 1 3 0	36 36 48	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 0.333333333333333	0.668908746807157
+96	0 1 3 0	36 36 52	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 0.444444444444444	0.698215010923196
+96	0 1 3 0	36 36 56	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 0.555555555555555	0.729928937348657
+96	0 1 3 0	36 36 60	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 0.666666666666667	0.76190136189151
+96	0 1 3 0	36 36 64	1.33333333333333	-5.55111512312578e-17 -5.55111512312578e-17 0.777777777777778	0.796688784792057
+
+
+# iteration 96
+# refinement level 2   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 2 0 0	36 36 22	1.33333333333333	2.46655994240452e-17 -5.55111512312578e-17 -0.388888888888889	0.683302448238139
+96	0 2 0 0	36 36 24	1.33333333333333	2.46655994240452e-17 -5.55111512312578e-17 -0.333333333333333	0.668908746807157
+96	0 2 0 0	36 36 26	1.33333333333333	2.46655994240452e-17 -5.55111512312578e-17 -0.277777777777778	0.654541556639422
+96	0 2 0 0	36 36 28	1.33333333333333	2.46655994240452e-17 -5.55111512312578e-17 -0.222222222222222	0.641345710414296
+96	0 2 0 0	36 36 30	1.33333333333333	2.46655994240452e-17 -5.55111512312578e-17 -0.166666666666667	0.630026346765969
+96	0 2 0 0	36 36 32	1.33333333333333	2.46655994240452e-17 -5.55111512312578e-17 -0.111111111111111	0.621410197839124
+96	0 2 0 0	36 36 34	1.33333333333333	2.46655994240452e-17 -5.55111512312578e-17 -0.0555555555555555	0.616824486578861
+96	0 2 0 0	36 36 36	1.33333333333333	2.46655994240452e-17 -5.55111512312578e-17 2.46655994240452e-17	0.61547287921812
+96	0 2 0 0	36 36 38	1.33333333333333	2.46655994240452e-17 -5.55111512312578e-17 0.0555555555555556	0.616824486578861
+96	0 2 0 0	36 36 40	1.33333333333333	2.46655994240452e-17 -5.55111512312578e-17 0.111111111111111	0.621410197839124
+
+# iteration 96
+# refinement level 2   multigrid level 0   map 0   component 1   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 2 1 0	36 36 22	1.33333333333333	-2.77555756156289e-17 -5.55111512312578e-17 -0.388888888888889	0.683302448238139
+96	0 2 1 0	36 36 24	1.33333333333333	-2.77555756156289e-17 -5.55111512312578e-17 -0.333333333333333	0.668908746807157
+96	0 2 1 0	36 36 26	1.33333333333333	-2.77555756156289e-17 -5.55111512312578e-17 -0.277777777777778	0.654541556639422
+96	0 2 1 0	36 36 28	1.33333333333333	-2.77555756156289e-17 -5.55111512312578e-17 -0.222222222222222	0.641345710414296
+96	0 2 1 0	36 36 30	1.33333333333333	-2.77555756156289e-17 -5.55111512312578e-17 -0.166666666666667	0.630026346765969
+96	0 2 1 0	36 36 32	1.33333333333333	-2.77555756156289e-17 -5.55111512312578e-17 -0.111111111111111	0.621410197839124
+96	0 2 1 0	36 36 34	1.33333333333333	-2.77555756156289e-17 -5.55111512312578e-17 -0.0555555555555555	0.616824486578861
+96	0 2 1 0	36 36 36	1.33333333333333	-2.77555756156289e-17 -5.55111512312578e-17 2.46655994240452e-17	0.61547287921812
+96	0 2 1 0	36 36 38	1.33333333333333	-2.77555756156289e-17 -5.55111512312578e-17 0.0555555555555556	0.616824486578861
+96	0 2 1 0	36 36 40	1.33333333333333	-2.77555756156289e-17 -5.55111512312578e-17 0.111111111111111	0.621410197839124
+
+# iteration 96
+# refinement level 2   multigrid level 0   map 0   component 2   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 2 2 0	36 36 34	1.33333333333333	2.46655994240452e-17 -5.55111512312578e-17 -0.0555555555555556	0.616824486578861
+96	0 2 2 0	36 36 36	1.33333333333333	2.46655994240452e-17 -5.55111512312578e-17 -2.77555756156289e-17	0.61547287921812
+96	0 2 2 0	36 36 38	1.33333333333333	2.46655994240452e-17 -5.55111512312578e-17 0.0555555555555555	0.616824486578861
+96	0 2 2 0	36 36 40	1.33333333333333	2.46655994240452e-17 -5.55111512312578e-17 0.111111111111111	0.621410197839124
+96	0 2 2 0	36 36 42	1.33333333333333	2.46655994240452e-17 -5.55111512312578e-17 0.166666666666667	0.63002634676597
+96	0 2 2 0	36 36 44	1.33333333333333	2.46655994240452e-17 -5.55111512312578e-17 0.222222222222222	0.641345710414297
+96	0 2 2 0	36 36 46	1.33333333333333	2.46655994240452e-17 -5.55111512312578e-17 0.277777777777778	0.654541556639422
+96	0 2 2 0	36 36 48	1.33333333333333	2.46655994240452e-17 -5.55111512312578e-17 0.333333333333333	0.668908746807157
+96	0 2 2 0	36 36 50	1.33333333333333	2.46655994240452e-17 -5.55111512312578e-17 0.388888888888889	0.683302448238139
+
+# iteration 96
+# refinement level 2   multigrid level 0   map 0   component 3   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 2 3 0	36 36 34	1.33333333333333	-2.77555756156289e-17 -5.55111512312578e-17 -0.0555555555555556	0.616824486578861
+96	0 2 3 0	36 36 36	1.33333333333333	-2.77555756156289e-17 -5.55111512312578e-17 -2.77555756156289e-17	0.61547287921812
+96	0 2 3 0	36 36 38	1.33333333333333	-2.77555756156289e-17 -5.55111512312578e-17 0.0555555555555555	0.616824486578861
+96	0 2 3 0	36 36 40	1.33333333333333	-2.77555756156289e-17 -5.55111512312578e-17 0.111111111111111	0.621410197839124
+96	0 2 3 0	36 36 42	1.33333333333333	-2.77555756156289e-17 -5.55111512312578e-17 0.166666666666667	0.63002634676597
+96	0 2 3 0	36 36 44	1.33333333333333	-2.77555756156289e-17 -5.55111512312578e-17 0.222222222222222	0.641345710414297
+96	0 2 3 0	36 36 46	1.33333333333333	-2.77555756156289e-17 -5.55111512312578e-17 0.277777777777778	0.654541556639422
+96	0 2 3 0	36 36 48	1.33333333333333	-2.77555756156289e-17 -5.55111512312578e-17 0.333333333333333	0.668908746807157
+96	0 2 3 0	36 36 50	1.33333333333333	-2.77555756156289e-17 -5.55111512312578e-17 0.388888888888889	0.683302448238139
+
+
+# iteration 96
+# refinement level 3   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 3 0 0	36 36 29	1.33333333333333	0 -5.55111512312578e-17 -0.194444444444445	0.63539979750899
+96	0 3 0 0	36 36 30	1.33333333333333	0 -5.55111512312578e-17 -0.166666666666667	0.630026346765969
+96	0 3 0 0	36 36 31	1.33333333333333	0 -5.55111512312578e-17 -0.138888888888889	0.625297419028293
+96	0 3 0 0	36 36 32	1.33333333333333	0 -5.55111512312578e-17 -0.111111111111111	0.621410197839124
+96	0 3 0 0	36 36 33	1.33333333333333	0 -5.55111512312578e-17 -0.0833333333333334	0.618663308361111
+96	0 3 0 0	36 36 34	1.33333333333333	0 -5.55111512312578e-17 -0.0555555555555556	0.616824486578861
+96	0 3 0 0	36 36 35	1.33333333333333	0 -5.55111512312578e-17 -0.0277777777777778	0.615777600484678
+96	0 3 0 0	36 36 36	1.33333333333333	0 -5.55111512312578e-17 0	0.61547287921812
+96	0 3 0 0	36 36 37	1.33333333333333	0 -5.55111512312578e-17 0.0277777777777778	0.615777600484678
+96	0 3 0 0	36 36 38	1.33333333333333	0 -5.55111512312578e-17 0.0555555555555556	0.616824486578861
+
+# iteration 96
+# refinement level 3   multigrid level 0   map 0   component 1   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 3 1 0	36 36 29	1.33333333333333	-1.38777878078145e-17 -5.55111512312578e-17 -0.194444444444445	0.63539979750899
+96	0 3 1 0	36 36 30	1.33333333333333	-1.38777878078145e-17 -5.55111512312578e-17 -0.166666666666667	0.630026346765969
+96	0 3 1 0	36 36 31	1.33333333333333	-1.38777878078145e-17 -5.55111512312578e-17 -0.138888888888889	0.625297419028293
+96	0 3 1 0	36 36 32	1.33333333333333	-1.38777878078145e-17 -5.55111512312578e-17 -0.111111111111111	0.621410197839124
+96	0 3 1 0	36 36 33	1.33333333333333	-1.38777878078145e-17 -5.55111512312578e-17 -0.0833333333333334	0.618663308361111
+96	0 3 1 0	36 36 34	1.33333333333333	-1.38777878078145e-17 -5.55111512312578e-17 -0.0555555555555556	0.616824486578861
+96	0 3 1 0	36 36 35	1.33333333333333	-1.38777878078145e-17 -5.55111512312578e-17 -0.0277777777777778	0.615777600484678
+96	0 3 1 0	36 36 36	1.33333333333333	-1.38777878078145e-17 -5.55111512312578e-17 0	0.61547287921812
+96	0 3 1 0	36 36 37	1.33333333333333	-1.38777878078145e-17 -5.55111512312578e-17 0.0277777777777778	0.615777600484678
+96	0 3 1 0	36 36 38	1.33333333333333	-1.38777878078145e-17 -5.55111512312578e-17 0.0555555555555556	0.616824486578861
+
+# iteration 96
+# refinement level 3   multigrid level 0   map 0   component 2   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 3 2 0	36 36 35	1.33333333333333	0 -5.55111512312578e-17 -0.0277777777777778	0.615777600484678
+96	0 3 2 0	36 36 36	1.33333333333333	0 -5.55111512312578e-17 -1.38777878078145e-17	0.61547287921812
+96	0 3 2 0	36 36 37	1.33333333333333	0 -5.55111512312578e-17 0.0277777777777778	0.615777600484678
+96	0 3 2 0	36 36 38	1.33333333333333	0 -5.55111512312578e-17 0.0555555555555555	0.616824486578861
+96	0 3 2 0	36 36 39	1.33333333333333	0 -5.55111512312578e-17 0.0833333333333333	0.618663308361111
+96	0 3 2 0	36 36 40	1.33333333333333	0 -5.55111512312578e-17 0.111111111111111	0.621410197839124
+96	0 3 2 0	36 36 41	1.33333333333333	0 -5.55111512312578e-17 0.138888888888889	0.625297419028293
+96	0 3 2 0	36 36 42	1.33333333333333	0 -5.55111512312578e-17 0.166666666666667	0.63002634676597
+96	0 3 2 0	36 36 43	1.33333333333333	0 -5.55111512312578e-17 0.194444444444444	0.63539979750899
+
+# iteration 96
+# refinement level 3   multigrid level 0   map 0   component 3   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+96	0 3 3 0	36 36 35	1.33333333333333	-1.38777878078145e-17 -5.55111512312578e-17 -0.0277777777777778	0.615777600484678
+96	0 3 3 0	36 36 36	1.33333333333333	-1.38777878078145e-17 -5.55111512312578e-17 -1.38777878078145e-17	0.61547287921812
+96	0 3 3 0	36 36 37	1.33333333333333	-1.38777878078145e-17 -5.55111512312578e-17 0.0277777777777778	0.615777600484678
+96	0 3 3 0	36 36 38	1.33333333333333	-1.38777878078145e-17 -5.55111512312578e-17 0.0555555555555555	0.616824486578861
+96	0 3 3 0	36 36 39	1.33333333333333	-1.38777878078145e-17 -5.55111512312578e-17 0.0833333333333333	0.618663308361111
+96	0 3 3 0	36 36 40	1.33333333333333	-1.38777878078145e-17 -5.55111512312578e-17 0.111111111111111	0.621410197839124
+96	0 3 3 0	36 36 41	1.33333333333333	-1.38777878078145e-17 -5.55111512312578e-17 0.138888888888889	0.625297419028293
+96	0 3 3 0	36 36 42	1.33333333333333	-1.38777878078145e-17 -5.55111512312578e-17 0.166666666666667	0.63002634676597
+96	0 3 3 0	36 36 43	1.33333333333333	-1.38777878078145e-17 -5.55111512312578e-17 0.194444444444444	0.63539979750899
+
+
+# iteration 128
+# refinement level 0   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 0 0 0	40 40 0	1.77777777777778	0.111111111111111 0.111111111111111 -1	0.873700996777018
+128	0 0 0 0	40 40 8	1.77777777777778	0.111111111111111 0.111111111111111 -0.777777777777778	0.859471717622308
+128	0 0 0 0	40 40 16	1.77777777777778	0.111111111111111 0.111111111111111 -0.555555555555556	0.863004424640887
+128	0 0 0 0	40 40 24	1.77777777777778	0.111111111111111 0.111111111111111 -0.333333333333333	0.883486952153519
+128	0 0 0 0	40 40 32	1.77777777777778	0.111111111111111 0.111111111111111 -0.111111111111111	0.915799135771738
+128	0 0 0 0	40 40 40	1.77777777777778	0.111111111111111 0.111111111111111 0.111111111111111	0.915799135771738
+128	0 0 0 0	40 40 48	1.77777777777778	0.111111111111111 0.111111111111111 0.333333333333333	0.883486952153519
+
+# iteration 128
+# refinement level 0   multigrid level 0   map 0   component 1   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 0 1 0	40 40 0	1.77777777777778	0.111111111111111 0.111111111111111 -1	0.873700996777018
+128	0 0 1 0	40 40 8	1.77777777777778	0.111111111111111 0.111111111111111 -0.777777777777778	0.859471717622308
+128	0 0 1 0	40 40 16	1.77777777777778	0.111111111111111 0.111111111111111 -0.555555555555556	0.863004424640887
+128	0 0 1 0	40 40 24	1.77777777777778	0.111111111111111 0.111111111111111 -0.333333333333333	0.883486952153519
+128	0 0 1 0	40 40 32	1.77777777777778	0.111111111111111 0.111111111111111 -0.111111111111111	0.915799135771738
+128	0 0 1 0	40 40 40	1.77777777777778	0.111111111111111 0.111111111111111 0.111111111111111	0.915799135771738
+128	0 0 1 0	40 40 48	1.77777777777778	0.111111111111111 0.111111111111111 0.333333333333333	0.883486952153519
+
+# iteration 128
+# refinement level 0   multigrid level 0   map 0   component 2   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 0 2 0	40 40 24	1.77777777777778	0.111111111111111 0.111111111111111 -0.333333333333333	0.883486952153519
+128	0 0 2 0	40 40 32	1.77777777777778	0.111111111111111 0.111111111111111 -0.111111111111111	0.915799135771738
+128	0 0 2 0	40 40 40	1.77777777777778	0.111111111111111 0.111111111111111 0.111111111111111	0.915799135771738
+128	0 0 2 0	40 40 48	1.77777777777778	0.111111111111111 0.111111111111111 0.333333333333333	0.883486952153519
+128	0 0 2 0	40 40 56	1.77777777777778	0.111111111111111 0.111111111111111 0.555555555555556	0.863004424640887
+128	0 0 2 0	40 40 64	1.77777777777778	0.111111111111111 0.111111111111111 0.777777777777778	0.859471717622308
+128	0 0 2 0	40 40 72	1.77777777777778	0.111111111111111 0.111111111111111 1	0.873700996777018
+
+# iteration 128
+# refinement level 0   multigrid level 0   map 0   component 3   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 0 3 0	40 40 24	1.77777777777778	0.111111111111111 0.111111111111111 -0.333333333333333	0.883486952153519
+128	0 0 3 0	40 40 32	1.77777777777778	0.111111111111111 0.111111111111111 -0.111111111111111	0.915799135771738
+128	0 0 3 0	40 40 40	1.77777777777778	0.111111111111111 0.111111111111111 0.111111111111111	0.915799135771738
+128	0 0 3 0	40 40 48	1.77777777777778	0.111111111111111 0.111111111111111 0.333333333333333	0.883486952153519
+128	0 0 3 0	40 40 56	1.77777777777778	0.111111111111111 0.111111111111111 0.555555555555556	0.863004424640887
+128	0 0 3 0	40 40 64	1.77777777777778	0.111111111111111 0.111111111111111 0.777777777777778	0.859471717622308
+128	0 0 3 0	40 40 72	1.77777777777778	0.111111111111111 0.111111111111111 1	0.873700996777018
+
+
+# iteration 128
+# refinement level 1   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 1 0 0	36 36 8	1.77777777777778	-9.86623976961809e-17 -5.55111512312578e-17 -0.777777777777778	0.862653663591007
+128	0 1 0 0	36 36 12	1.77777777777778	-9.86623976961809e-17 -5.55111512312578e-17 -0.666666666666667	0.862560028606905
+128	0 1 0 0	36 36 16	1.77777777777778	-9.86623976961809e-17 -5.55111512312578e-17 -0.555555555555556	0.866965759785718
+128	0 1 0 0	36 36 20	1.77777777777778	-9.86623976961809e-17 -5.55111512312578e-17 -0.444444444444444	0.876167092595071
+128	0 1 0 0	36 36 24	1.77777777777778	-9.86623976961809e-17 -5.55111512312578e-17 -0.333333333333333	0.889185421864397
+128	0 1 0 0	36 36 28	1.77777777777778	-9.86623976961809e-17 -5.55111512312578e-17 -0.222222222222222	0.90799818468156
+128	0 1 0 0	36 36 32	1.77777777777778	-9.86623976961809e-17 -5.55111512312578e-17 -0.111111111111111	0.924033489738888
+128	0 1 0 0	36 36 36	1.77777777777778	-9.86623976961809e-17 -5.55111512312578e-17 -9.86623976961809e-17	0.928389498223199
+128	0 1 0 0	36 36 40	1.77777777777778	-9.86623976961809e-17 -5.55111512312578e-17 0.111111111111111	0.924033489738888
+128	0 1 0 0	36 36 44	1.77777777777778	-9.86623976961809e-17 -5.55111512312578e-17 0.222222222222222	0.90799818468156
+
+# iteration 128
+# refinement level 1   multigrid level 0   map 0   component 1   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 1 1 0	36 36 8	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -0.777777777777778	0.862653663591007
+128	0 1 1 0	36 36 12	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -0.666666666666667	0.862560028606905
+128	0 1 1 0	36 36 16	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -0.555555555555556	0.866965759785718
+128	0 1 1 0	36 36 20	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -0.444444444444444	0.876167092595071
+128	0 1 1 0	36 36 24	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -0.333333333333333	0.889185421864397
+128	0 1 1 0	36 36 28	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -0.222222222222222	0.90799818468156
+128	0 1 1 0	36 36 32	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -0.111111111111111	0.924033489738888
+128	0 1 1 0	36 36 36	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -9.86623976961809e-17	0.928389498223199
+128	0 1 1 0	36 36 40	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 0.111111111111111	0.924033489738888
+128	0 1 1 0	36 36 44	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 0.222222222222222	0.90799818468156
+
+# iteration 128
+# refinement level 1   multigrid level 0   map 0   component 2   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 1 2 0	36 36 32	1.77777777777778	-9.86623976961809e-17 -5.55111512312578e-17 -0.111111111111111	0.924033489738888
+128	0 1 2 0	36 36 36	1.77777777777778	-9.86623976961809e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.928389498223199
+128	0 1 2 0	36 36 40	1.77777777777778	-9.86623976961809e-17 -5.55111512312578e-17 0.111111111111111	0.924033489738888
+128	0 1 2 0	36 36 44	1.77777777777778	-9.86623976961809e-17 -5.55111512312578e-17 0.222222222222222	0.90799818468156
+128	0 1 2 0	36 36 48	1.77777777777778	-9.86623976961809e-17 -5.55111512312578e-17 0.333333333333333	0.889185421864397
+128	0 1 2 0	36 36 52	1.77777777777778	-9.86623976961809e-17 -5.55111512312578e-17 0.444444444444444	0.876167092595071
+128	0 1 2 0	36 36 56	1.77777777777778	-9.86623976961809e-17 -5.55111512312578e-17 0.555555555555555	0.866965759785718
+128	0 1 2 0	36 36 60	1.77777777777778	-9.86623976961809e-17 -5.55111512312578e-17 0.666666666666667	0.862560028606905
+128	0 1 2 0	36 36 64	1.77777777777778	-9.86623976961809e-17 -5.55111512312578e-17 0.777777777777778	0.862653663591007
+
+# iteration 128
+# refinement level 1   multigrid level 0   map 0   component 3   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 1 3 0	36 36 32	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -0.111111111111111	0.924033489738888
+128	0 1 3 0	36 36 36	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 -5.55111512312578e-17	0.928389498223199
+128	0 1 3 0	36 36 40	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 0.111111111111111	0.924033489738888
+128	0 1 3 0	36 36 44	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 0.222222222222222	0.90799818468156
+128	0 1 3 0	36 36 48	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 0.333333333333333	0.889185421864397
+128	0 1 3 0	36 36 52	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 0.444444444444444	0.876167092595071
+128	0 1 3 0	36 36 56	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 0.555555555555555	0.866965759785718
+128	0 1 3 0	36 36 60	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 0.666666666666667	0.862560028606905
+128	0 1 3 0	36 36 64	1.77777777777778	-5.55111512312578e-17 -5.55111512312578e-17 0.777777777777778	0.862653663591007
+
+
+# iteration 128
+# refinement level 2   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 2 0 0	36 36 22	1.77777777777778	2.46655994240452e-17 -5.55111512312578e-17 -0.388888888888889	0.882075542854246
+128	0 2 0 0	36 36 24	1.77777777777778	2.46655994240452e-17 -5.55111512312578e-17 -0.333333333333333	0.889185421864397
+128	0 2 0 0	36 36 26	1.77777777777778	2.46655994240452e-17 -5.55111512312578e-17 -0.277777777777778	0.898403242286228
+128	0 2 0 0	36 36 28	1.77777777777778	2.46655994240452e-17 -5.55111512312578e-17 -0.222222222222222	0.90799818468156
+128	0 2 0 0	36 36 30	1.77777777777778	2.46655994240452e-17 -5.55111512312578e-17 -0.166666666666667	0.916919384356027
+128	0 2 0 0	36 36 32	1.77777777777778	2.46655994240452e-17 -5.55111512312578e-17 -0.111111111111111	0.924033489738888
+128	0 2 0 0	36 36 34	1.77777777777778	2.46655994240452e-17 -5.55111512312578e-17 -0.0555555555555555	0.927485951077396
+128	0 2 0 0	36 36 36	1.77777777777778	2.46655994240452e-17 -5.55111512312578e-17 2.46655994240452e-17	0.928389498223199
+128	0 2 0 0	36 36 38	1.77777777777778	2.46655994240452e-17 -5.55111512312578e-17 0.0555555555555556	0.927485951077396
+128	0 2 0 0	36 36 40	1.77777777777778	2.46655994240452e-17 -5.55111512312578e-17 0.111111111111111	0.924033489738888
+
+# iteration 128
+# refinement level 2   multigrid level 0   map 0   component 1   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 2 1 0	36 36 22	1.77777777777778	-2.77555756156289e-17 -5.55111512312578e-17 -0.388888888888889	0.882075542854246
+128	0 2 1 0	36 36 24	1.77777777777778	-2.77555756156289e-17 -5.55111512312578e-17 -0.333333333333333	0.889185421864397
+128	0 2 1 0	36 36 26	1.77777777777778	-2.77555756156289e-17 -5.55111512312578e-17 -0.277777777777778	0.898403242286228
+128	0 2 1 0	36 36 28	1.77777777777778	-2.77555756156289e-17 -5.55111512312578e-17 -0.222222222222222	0.90799818468156
+128	0 2 1 0	36 36 30	1.77777777777778	-2.77555756156289e-17 -5.55111512312578e-17 -0.166666666666667	0.916919384356027
+128	0 2 1 0	36 36 32	1.77777777777778	-2.77555756156289e-17 -5.55111512312578e-17 -0.111111111111111	0.924033489738888
+128	0 2 1 0	36 36 34	1.77777777777778	-2.77555756156289e-17 -5.55111512312578e-17 -0.0555555555555555	0.927485951077396
+128	0 2 1 0	36 36 36	1.77777777777778	-2.77555756156289e-17 -5.55111512312578e-17 2.46655994240452e-17	0.928389498223199
+128	0 2 1 0	36 36 38	1.77777777777778	-2.77555756156289e-17 -5.55111512312578e-17 0.0555555555555556	0.927485951077396
+128	0 2 1 0	36 36 40	1.77777777777778	-2.77555756156289e-17 -5.55111512312578e-17 0.111111111111111	0.924033489738888
+
+# iteration 128
+# refinement level 2   multigrid level 0   map 0   component 2   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 2 2 0	36 36 34	1.77777777777778	2.46655994240452e-17 -5.55111512312578e-17 -0.0555555555555556	0.927485951077396
+128	0 2 2 0	36 36 36	1.77777777777778	2.46655994240452e-17 -5.55111512312578e-17 -2.77555756156289e-17	0.928389498223199
+128	0 2 2 0	36 36 38	1.77777777777778	2.46655994240452e-17 -5.55111512312578e-17 0.0555555555555555	0.927485951077396
+128	0 2 2 0	36 36 40	1.77777777777778	2.46655994240452e-17 -5.55111512312578e-17 0.111111111111111	0.924033489738888
+128	0 2 2 0	36 36 42	1.77777777777778	2.46655994240452e-17 -5.55111512312578e-17 0.166666666666667	0.916919384356027
+128	0 2 2 0	36 36 44	1.77777777777778	2.46655994240452e-17 -5.55111512312578e-17 0.222222222222222	0.90799818468156
+128	0 2 2 0	36 36 46	1.77777777777778	2.46655994240452e-17 -5.55111512312578e-17 0.277777777777778	0.898403242286228
+128	0 2 2 0	36 36 48	1.77777777777778	2.46655994240452e-17 -5.55111512312578e-17 0.333333333333333	0.889185421864397
+128	0 2 2 0	36 36 50	1.77777777777778	2.46655994240452e-17 -5.55111512312578e-17 0.388888888888889	0.882075542854246
+
+# iteration 128
+# refinement level 2   multigrid level 0   map 0   component 3   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 2 3 0	36 36 34	1.77777777777778	-2.77555756156289e-17 -5.55111512312578e-17 -0.0555555555555556	0.927485951077396
+128	0 2 3 0	36 36 36	1.77777777777778	-2.77555756156289e-17 -5.55111512312578e-17 -2.77555756156289e-17	0.928389498223199
+128	0 2 3 0	36 36 38	1.77777777777778	-2.77555756156289e-17 -5.55111512312578e-17 0.0555555555555555	0.927485951077396
+128	0 2 3 0	36 36 40	1.77777777777778	-2.77555756156289e-17 -5.55111512312578e-17 0.111111111111111	0.924033489738888
+128	0 2 3 0	36 36 42	1.77777777777778	-2.77555756156289e-17 -5.55111512312578e-17 0.166666666666667	0.916919384356027
+128	0 2 3 0	36 36 44	1.77777777777778	-2.77555756156289e-17 -5.55111512312578e-17 0.222222222222222	0.90799818468156
+128	0 2 3 0	36 36 46	1.77777777777778	-2.77555756156289e-17 -5.55111512312578e-17 0.277777777777778	0.898403242286228
+128	0 2 3 0	36 36 48	1.77777777777778	-2.77555756156289e-17 -5.55111512312578e-17 0.333333333333333	0.889185421864397
+128	0 2 3 0	36 36 50	1.77777777777778	-2.77555756156289e-17 -5.55111512312578e-17 0.388888888888889	0.882075542854246
+
+
+# iteration 128
+# refinement level 3   multigrid level 0   map 0   component 0   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 3 0 0	36 36 29	1.77777777777778	0 -5.55111512312578e-17 -0.194444444444445	0.912613836832073
+128	0 3 0 0	36 36 30	1.77777777777778	0 -5.55111512312578e-17 -0.166666666666667	0.916919384356027
+128	0 3 0 0	36 36 31	1.77777777777778	0 -5.55111512312578e-17 -0.138888888888889	0.920818233193455
+128	0 3 0 0	36 36 32	1.77777777777778	0 -5.55111512312578e-17 -0.111111111111111	0.924033489738888
+128	0 3 0 0	36 36 33	1.77777777777778	0 -5.55111512312578e-17 -0.0833333333333334	0.926147880297958
+128	0 3 0 0	36 36 34	1.77777777777778	0 -5.55111512312578e-17 -0.0555555555555556	0.927485951077396
+128	0 3 0 0	36 36 35	1.77777777777778	0 -5.55111512312578e-17 -0.0277777777777778	0.928209975180567
+128	0 3 0 0	36 36 36	1.77777777777778	0 -5.55111512312578e-17 0	0.928389498223199
+128	0 3 0 0	36 36 37	1.77777777777778	0 -5.55111512312578e-17 0.0277777777777778	0.928209975180567
+128	0 3 0 0	36 36 38	1.77777777777778	0 -5.55111512312578e-17 0.0555555555555556	0.927485951077396
+
+# iteration 128
+# refinement level 3   multigrid level 0   map 0   component 1   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 3 1 0	36 36 29	1.77777777777778	-1.38777878078145e-17 -5.55111512312578e-17 -0.194444444444445	0.912613836832073
+128	0 3 1 0	36 36 30	1.77777777777778	-1.38777878078145e-17 -5.55111512312578e-17 -0.166666666666667	0.916919384356027
+128	0 3 1 0	36 36 31	1.77777777777778	-1.38777878078145e-17 -5.55111512312578e-17 -0.138888888888889	0.920818233193455
+128	0 3 1 0	36 36 32	1.77777777777778	-1.38777878078145e-17 -5.55111512312578e-17 -0.111111111111111	0.924033489738888
+128	0 3 1 0	36 36 33	1.77777777777778	-1.38777878078145e-17 -5.55111512312578e-17 -0.0833333333333334	0.926147880297958
+128	0 3 1 0	36 36 34	1.77777777777778	-1.38777878078145e-17 -5.55111512312578e-17 -0.0555555555555556	0.927485951077396
+128	0 3 1 0	36 36 35	1.77777777777778	-1.38777878078145e-17 -5.55111512312578e-17 -0.0277777777777778	0.928209975180567
+128	0 3 1 0	36 36 36	1.77777777777778	-1.38777878078145e-17 -5.55111512312578e-17 0	0.928389498223199
+128	0 3 1 0	36 36 37	1.77777777777778	-1.38777878078145e-17 -5.55111512312578e-17 0.0277777777777778	0.928209975180567
+128	0 3 1 0	36 36 38	1.77777777777778	-1.38777878078145e-17 -5.55111512312578e-17 0.0555555555555556	0.927485951077396
+
+# iteration 128
+# refinement level 3   multigrid level 0   map 0   component 2   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 3 2 0	36 36 35	1.77777777777778	0 -5.55111512312578e-17 -0.0277777777777778	0.928209975180567
+128	0 3 2 0	36 36 36	1.77777777777778	0 -5.55111512312578e-17 -1.38777878078145e-17	0.928389498223199
+128	0 3 2 0	36 36 37	1.77777777777778	0 -5.55111512312578e-17 0.0277777777777778	0.928209975180567
+128	0 3 2 0	36 36 38	1.77777777777778	0 -5.55111512312578e-17 0.0555555555555555	0.927485951077396
+128	0 3 2 0	36 36 39	1.77777777777778	0 -5.55111512312578e-17 0.0833333333333333	0.926147880297958
+128	0 3 2 0	36 36 40	1.77777777777778	0 -5.55111512312578e-17 0.111111111111111	0.924033489738888
+128	0 3 2 0	36 36 41	1.77777777777778	0 -5.55111512312578e-17 0.138888888888889	0.920818233193455
+128	0 3 2 0	36 36 42	1.77777777777778	0 -5.55111512312578e-17 0.166666666666667	0.916919384356027
+128	0 3 2 0	36 36 43	1.77777777777778	0 -5.55111512312578e-17 0.194444444444444	0.912613836832073
+
+# iteration 128
+# refinement level 3   multigrid level 0   map 0   component 3   time level 0
+# column format: it	tl rl c ml	ix iy iz	time	x y z	data
+128	0 3 3 0	36 36 35	1.77777777777778	-1.38777878078145e-17 -5.55111512312578e-17 -0.0277777777777778	0.928209975180567
+128	0 3 3 0	36 36 36	1.77777777777778	-1.38777878078145e-17 -5.55111512312578e-17 -1.38777878078145e-17	0.928389498223199
+128	0 3 3 0	36 36 37	1.77777777777778	-1.38777878078145e-17 -5.55111512312578e-17 0.0277777777777778	0.928209975180567
+128	0 3 3 0	36 36 38	1.77777777777778	-1.38777878078145e-17 -5.55111512312578e-17 0.0555555555555555	0.927485951077396
+128	0 3 3 0	36 36 39	1.77777777777778	-1.38777878078145e-17 -5.55111512312578e-17 0.0833333333333333	0.926147880297958
+128	0 3 3 0	36 36 40	1.77777777777778	-1.38777878078145e-17 -5.55111512312578e-17 0.111111111111111	0.924033489738888
+128	0 3 3 0	36 36 41	1.77777777777778	-1.38777878078145e-17 -5.55111512312578e-17 0.138888888888889	0.920818233193455
+128	0 3 3 0	36 36 42	1.77777777777778	-1.38777878078145e-17 -5.55111512312578e-17 0.166666666666667	0.916919384356027
+128	0 3 3 0	36 36 43	1.77777777777778	-1.38777878078145e-17 -5.55111512312578e-17 0.194444444444444	0.912613836832073
+
+
diff --git a/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_4proc/phi_maximum.xg b/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_4proc/phi_maximum.xg
new file mode 100644
index 000000000..e25d17d43
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_4proc/phi_maximum.xg
@@ -0,0 +1,4 @@
+"phi v time
+0.8888888888889	0.8737009967770
+1.3333333333333	0.8774806033600
+1.7777777777778	0.9157991357717
diff --git a/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_4proc/phi_minimum.xg b/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_4proc/phi_minimum.xg
new file mode 100644
index 000000000..973172b6d
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_4proc/phi_minimum.xg
@@ -0,0 +1,4 @@
+"phi v time
+0.8888888888889	0.6796516032334
+1.3333333333333	0.6328288136667
+1.7777777777778	0.6796516032334
diff --git a/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_4proc/phi_norm1.xg b/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_4proc/phi_norm1.xg
new file mode 100644
index 000000000..6e8792bca
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_4proc/phi_norm1.xg
@@ -0,0 +1,4 @@
+"phi v time
+0.8888888888889	0.7927895664365
+1.3333333333333	0.7914772512610
+1.7777777777778	0.8208628581415
diff --git a/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_4proc/phi_norm2.xg b/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_4proc/phi_norm2.xg
new file mode 100644
index 000000000..124c22eda
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/test/CarpetWaveToyRecover_test_4proc/phi_norm2.xg
@@ -0,0 +1,4 @@
+"phi v time
+0.8888888888889	0.7937177688028
+1.3333333333333	0.7927587735574
+1.7777777777778	0.8218200985038
diff --git a/Carpet/CarpetIOHDF5/test/input_initial_data.par b/Carpet/CarpetIOHDF5/test/input_initial_data.par
new file mode 100644
index 000000000..a16e312c6
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/test/input_initial_data.par
@@ -0,0 +1,57 @@
+# $Header: /home/eschnett/C/carpet/Carpet/Carpet/CarpetIOHDF5/test/input_initial_data.par,v 1.3 2004/08/19 11:47:31 tradke Exp $
+
+# deactivate the IDScalarWaveMoL thorn
+# initial data is read from files using the filereader
+ActiveThorns = "Carpet CarpetLib CarpetRegrid CarpetInterp CarpetReduce CarpetSlab Boundary CoordBase SymBase CartGrid3D Periodic LocalInterp NaNChecker Slab Time MoL WaveToyMoL IOUtil IOBasic CarpetIOHDF5"
+#ActiveThorns = "Carpet CarpetLib CarpetRegrid CarpetInterp CarpetReduce CarpetSlab Boundary CoordBase SymBase CartGrid3D Periodic LocalInterp NaNChecker Slab Time MoL IDScalarWaveMoL WaveToyMoL IOUtil IOBasic CarpetIOHDF5"
+
+
+Cactus::cctk_itlast = 128
+
+Driver::global_nx  = 24
+Driver::global_ny  = 5
+Driver::global_nz  = 5
+Driver::ghost_size = 2
+
+Carpet::max_refinement_levels    = 3
+Carpet::prolongation_order_space = 3
+Carpet::prolongation_order_time  = 2
+Carpet::init_each_timelevel      = "yes"
+
+CarpetRegrid::refinement_levels = 1
+CarpetRegrid::refined_regions   = "manual-gridpoint-list"
+CarpetRegrid::gridpoints        = "[[ ([28,4,4]:[68,12,12]:[2,2,2]) ]]"
+CarpetRegrid::outerbounds       = "[[ [[0,0],[1,1],[1,1]] ]]"
+
+NaNChecker::check_every     = 16
+NaNChecker::check_vars      = "all"
+NaNChecker::action_if_found = "terminate"
+
+Grid::domain = full
+Grid::type   = byrange
+Grid::xmin   = -0.1
+Grid::ymin   = -0.15
+Grid::zmin   = -0.15
+Grid::xmax   = +1.05
+Grid::ymax   = +0.15
+Grid::zmax   = +0.15
+
+Periodic::periodic = "yes"
+
+Time::dtfac = 0.25
+
+MoL::ODE_Method = "ICN"
+
+IO::out_dir       = $parfilename
+IO::out_fileinfo  = "none"
+IO::parfile_write = "no"
+IO::out_every     = 1
+IO::out_yline_x   = 0.5
+IO::out_zline_x   = 0.5
+ 
+IOBasic::outInfo_vars   = "WaveToyMoL::scalarevolve"
+IOBasic::outScalar_vars = "WaveToyMoL::scalarevolve"
+
+# filereader input files and directory
+IO::filereader_ID_files = "psi phi"
+IO::recover_dir  = "../../../arrangements/Carpet/CarpetIOHDF5/test/input_initial_data"
diff --git a/Carpet/CarpetIOHDF5/test/input_initial_data/phi.h5 b/Carpet/CarpetIOHDF5/test/input_initial_data/phi.h5
new file mode 100644
index 000000000..86ad6542c
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/test/input_initial_data/phi.h5
diff --git a/Carpet/CarpetIOHDF5/test/input_initial_data/phi_maximum.xg b/Carpet/CarpetIOHDF5/test/input_initial_data/phi_maximum.xg
new file mode 100644
index 000000000..78ce3fbaa
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/test/input_initial_data/phi_maximum.xg
@@ -0,0 +1,34 @@
+"phi v time
+0.0000000000000	1.0000000000000
+0.0125000000000	0.9986404587860
+0.0250000000000	0.9945673789444
+0.0375000000000	1.0119432126264
+0.0500000000000	1.0268055861612
+0.0625000000000	1.0388778598256
+0.0750000000000	1.0481272355649
+0.0875000000000	1.0716903046796
+0.1000000000000	1.0982829974947
+0.1125000000000	1.1218913478250
+0.1250000000000	1.1424512117393
+0.1375000000000	1.1599067288796
+0.1500000000000	1.1900017850006
+0.1625000000000	1.2219191687257
+0.1750000000000	1.2505162515579
+0.1875000000000	1.2757153346263
+0.2000000000000	1.2974479523686
+0.2125000000000	1.3156550585508
+0.2250000000000	1.3380762519805
+0.2375000000000	1.3673894350252
+0.2500000000000	1.3929870450400
+0.2625000000000	1.4147995341575
+0.2750000000000	1.4327676397030
+0.2875000000000	1.4468425452064
+0.3000000000000	1.4569860130280
+0.3125000000000	1.4631704882406
+0.3250000000000	1.4713646259286
+0.3375000000000	1.4861345916159
+0.3500000000000	1.4968663526009
+0.3625000000000	1.5035307556225
+0.3750000000000	1.5061096994421
+0.3875000000000	1.5045961840087
+0.4000000000000	1.4989943294590
diff --git a/Carpet/CarpetIOHDF5/test/input_initial_data/phi_minimum.xg b/Carpet/CarpetIOHDF5/test/input_initial_data/phi_minimum.xg
new file mode 100644
index 000000000..d62cc5b13
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/test/input_initial_data/phi_minimum.xg
@@ -0,0 +1,34 @@
+"phi v time
+0.0000000000000	-1.0000000000000
+0.0125000000000	-0.9986404587860
+0.0250000000000	-0.9945673789444
+0.0375000000000	-1.0119432126264
+0.0500000000000	-1.0268055861612
+0.0625000000000	-1.0388778598256
+0.0750000000000	-1.0481272355649
+0.0875000000000	-1.0716903046796
+0.1000000000000	-1.0982829974947
+0.1125000000000	-1.1218913478250
+0.1250000000000	-1.1424512117393
+0.1375000000000	-1.1599067288796
+0.1500000000000	-1.1900017850006
+0.1625000000000	-1.2219191687257
+0.1750000000000	-1.2505162515579
+0.1875000000000	-1.2757153346263
+0.2000000000000	-1.2974479523686
+0.2125000000000	-1.3156550585508
+0.2250000000000	-1.3380762519805
+0.2375000000000	-1.3673894350252
+0.2500000000000	-1.3929870450400
+0.2625000000000	-1.4147995341575
+0.2750000000000	-1.4327676397030
+0.2875000000000	-1.4468425452064
+0.3000000000000	-1.4569860130280
+0.3125000000000	-1.4631704882406
+0.3250000000000	-1.4713646259286
+0.3375000000000	-1.4861345916159
+0.3500000000000	-1.4968663526009
+0.3625000000000	-1.5035307556225
+0.3750000000000	-1.5061096994421
+0.3875000000000	-1.5045961840087
+0.4000000000000	-1.4989943294590
diff --git a/Carpet/CarpetIOHDF5/test/input_initial_data/phi_norm1.xg b/Carpet/CarpetIOHDF5/test/input_initial_data/phi_norm1.xg
new file mode 100644
index 000000000..c3618d28a
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/test/input_initial_data/phi_norm1.xg
@@ -0,0 +1,34 @@
+"phi v time
+0.0000000000000	0.6807763773465
+0.0125000000000	0.6883135845966
+0.0250000000000	0.6939804679330
+0.0375000000000	0.6977616325549
+0.0500000000000	0.6996468076312
+0.0625000000000	0.7113978965384
+0.0750000000000	0.7221770888255
+0.0875000000000	0.7309939361006
+0.1000000000000	0.7378244845393
+0.1125000000000	0.7471033499543
+0.1250000000000	0.7616523319997
+0.1375000000000	0.7741316985445
+0.1500000000000	0.7845075440356
+0.1625000000000	0.7927516787444
+0.1750000000000	0.7988417053545
+0.1875000000000	0.8110731791327
+0.2000000000000	0.8232064841264
+0.2125000000000	0.8331029209643
+0.2250000000000	0.8407356028304
+0.2375000000000	0.8460837941132
+0.2500000000000	0.8491329667380
+0.2625000000000	0.8498748396317
+0.2750000000000	0.8565790617590
+0.2875000000000	0.8664821742987
+0.3000000000000	0.8740308325676
+0.3125000000000	0.8792045294337
+0.3250000000000	0.8819892111320
+0.3375000000000	0.8823773154395
+0.3500000000000	0.8803677922124
+0.3625000000000	0.8759661062267
+0.3750000000000	0.8691842223177
+0.3875000000000	0.8713683085882
+0.4000000000000	0.8728859333298
diff --git a/Carpet/CarpetIOHDF5/test/input_initial_data/phi_norm2.xg b/Carpet/CarpetIOHDF5/test/input_initial_data/phi_norm2.xg
new file mode 100644
index 000000000..bd291cc98
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/test/input_initial_data/phi_norm2.xg
@@ -0,0 +1,34 @@
+"phi v time
+0.0000000000000	0.7489861339271
+0.0125000000000	0.7518419444399
+0.0250000000000	0.7562195954773
+0.0375000000000	0.7620457996251
+0.0500000000000	0.7692251813885
+0.0625000000000	0.7776434804038
+0.0750000000000	0.7871711042100
+0.0875000000000	0.7976667955515
+0.1000000000000	0.8089812100628
+0.1125000000000	0.8209602464387
+0.1250000000000	0.8334480229821
+0.1375000000000	0.8462894436775
+0.1500000000000	0.8593323383447
+0.1625000000000	0.8724291925272
+0.1750000000000	0.8854385033832
+0.1875000000000	0.8982258092504
+0.2000000000000	0.9106644447590
+0.2125000000000	0.9226360724714
+0.2250000000000	0.9340310379450
+0.2375000000000	0.9447485893398
+0.2500000000000	0.9546969963259
+0.2625000000000	0.9637935967973
+0.2750000000000	0.9719647941895
+0.2875000000000	0.9791460232367
+0.3000000000000	0.9852816978433
+0.3125000000000	0.9903251513560
+0.3250000000000	0.9942385768297
+0.3375000000000	0.9969929727762
+0.3500000000000	0.9985680982614
+0.3625000000000	0.9989524399743
+0.3750000000000	0.9981431929225
+0.3875000000000	0.9961462556366
+0.4000000000000	0.9929762400874
diff --git a/Carpet/CarpetIOHDF5/test/input_initial_data/psi.h5 b/Carpet/CarpetIOHDF5/test/input_initial_data/psi.h5
new file mode 100644
index 000000000..fbb911bff
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/test/input_initial_data/psi.h5
diff --git a/Carpet/CarpetIOHDF5/test/input_initial_data/psi_maximum.xg b/Carpet/CarpetIOHDF5/test/input_initial_data/psi_maximum.xg
new file mode 100644
index 000000000..27be32935
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/test/input_initial_data/psi_maximum.xg
@@ -0,0 +1,34 @@
+"psi v time
+0.0000000000000	6.2831853071796
+0.0125000000000	6.2746430577991
+0.0250000000000	6.2490511423835
+0.0375000000000	6.2064791316818
+0.0500000000000	6.1470427352297
+0.0625000000000	6.1059995615621
+0.0750000000000	6.0823508015521
+0.0875000000000	6.0421749067230
+0.1000000000000	5.9855810749628
+0.1125000000000	5.9127231153562
+0.1250000000000	5.8237990301668
+0.1375000000000	5.7190504767363
+0.1500000000000	5.5987621107645
+0.1625000000000	5.4972069587517
+0.1750000000000	5.4231800426911
+0.1875000000000	5.3344172069405
+0.2000000000000	5.2311596682320
+0.2125000000000	5.1136880283712
+0.2250000000000	4.9823215117033
+0.2375000000000	4.8471541168578
+0.2500000000000	4.7760344394342
+0.2625000000000	4.6919372838490
+0.2750000000000	4.5950911858351
+0.2875000000000	4.4857593225805
+0.3000000000000	4.3764423117654
+0.3125000000000	4.3393959614856
+0.3250000000000	4.2905585196070
+0.3375000000000	4.2300627107315
+0.3500000000000	4.1580729377426
+0.3625000000000	4.1642308303415
+0.3750000000000	4.1713735624227
+0.3875000000000	4.1671816776830
+0.4000000000000	4.1516665873958
diff --git a/Carpet/CarpetIOHDF5/test/input_initial_data/psi_minimum.xg b/Carpet/CarpetIOHDF5/test/input_initial_data/psi_minimum.xg
new file mode 100644
index 000000000..fb6cf4909
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/test/input_initial_data/psi_minimum.xg
@@ -0,0 +1,34 @@
+"psi v time
+0.0000000000000	-6.2831853071796
+0.0125000000000	-6.2746430577991
+0.0250000000000	-6.2490511423835
+0.0375000000000	-6.2064791316818
+0.0500000000000	-6.1470427352296
+0.0625000000000	-6.1059995615621
+0.0750000000000	-6.0823508015522
+0.0875000000000	-6.0421749067230
+0.1000000000000	-5.9855810749628
+0.1125000000000	-5.9127231153562
+0.1250000000000	-5.8237990301668
+0.1375000000000	-5.7190504767363
+0.1500000000000	-5.5987621107645
+0.1625000000000	-5.4972069587517
+0.1750000000000	-5.4231800426911
+0.1875000000000	-5.3344172069405
+0.2000000000000	-5.2311596682320
+0.2125000000000	-5.1136880283712
+0.2250000000000	-4.9823215117033
+0.2375000000000	-4.8471541168578
+0.2500000000000	-4.7760344394342
+0.2625000000000	-4.6919372838490
+0.2750000000000	-4.5950911858351
+0.2875000000000	-4.4857593225805
+0.3000000000000	-4.3764423117654
+0.3125000000000	-4.3393959614856
+0.3250000000000	-4.2905585196070
+0.3375000000000	-4.2300627107315
+0.3500000000000	-4.1580729377426
+0.3625000000000	-4.1642308303415
+0.3750000000000	-4.1713735624227
+0.3875000000000	-4.1671816776830
+0.4000000000000	-4.1516665873958
diff --git a/Carpet/CarpetIOHDF5/test/input_initial_data/psi_norm1.xg b/Carpet/CarpetIOHDF5/test/input_initial_data/psi_norm1.xg
new file mode 100644
index 000000000..8fd7f80b2
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/test/input_initial_data/psi_norm1.xg
@@ -0,0 +1,34 @@
+"psi v time
+0.0000000000000	3.6215553016171
+0.0125000000000	3.6364763601998
+0.0250000000000	3.6415162291713
+0.0375000000000	3.6366612322730
+0.0500000000000	3.6219245799508
+0.0625000000000	3.5973463334095
+0.0750000000000	3.5629932957073
+0.0875000000000	3.5189588301869
+0.1000000000000	3.4653626067366
+0.1125000000000	3.4178909850438
+0.1250000000000	3.4031487535687
+0.1375000000000	3.3791593932900
+0.1500000000000	3.3459881060254
+0.1625000000000	3.3037250429285
+0.1750000000000	3.2524850594811
+0.1875000000000	3.1924074033578
+0.2000000000000	3.1283204712833
+0.2125000000000	3.1162479409102
+0.2250000000000	3.0957078538273
+0.2375000000000	3.0667560379254
+0.2500000000000	3.0294711776388
+0.2625000000000	2.9839546000991
+0.2750000000000	2.9543847137139
+0.2875000000000	2.9417629539386
+0.3000000000000	2.9211477552395
+0.3125000000000	2.8925951487277
+0.3250000000000	2.8561827332153
+0.3375000000000	2.8621799018707
+0.3500000000000	2.8633237516454
+0.3625000000000	2.8566872748419
+0.3750000000000	2.8422885187004
+0.3875000000000	2.8676161308430
+0.4000000000000	2.8922779097055
diff --git a/Carpet/CarpetIOHDF5/test/input_initial_data/psi_norm2.xg b/Carpet/CarpetIOHDF5/test/input_initial_data/psi_norm2.xg
new file mode 100644
index 000000000..f6e6af93d
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/test/input_initial_data/psi_norm2.xg
@@ -0,0 +1,34 @@
+"psi v time
+0.0000000000000	4.1631485516862
+0.0125000000000	4.1541748755646
+0.0250000000000	4.1403181106482
+0.0375000000000	4.1216804330909
+0.0500000000000	4.0984006495679
+0.0625000000000	4.0706544048325
+0.0750000000000	4.0386544211818
+0.0875000000000	4.0026507506396
+0.1000000000000	3.9629310129817
+0.1125000000000	3.9198205828109
+0.1250000000000	3.8736826762988
+0.1375000000000	3.8249182725450
+0.1500000000000	3.7739657855260
+0.1625000000000	3.7213003803309
+0.1750000000000	3.6674328023249
+0.1875000000000	3.6129075612872
+0.2000000000000	3.5583002867509
+0.2125000000000	3.5042140494355
+0.2250000000000	3.4512744321941
+0.2375000000000	3.4001231394120
+0.2500000000000	3.3514099647860
+0.2625000000000	3.3057830027058
+0.2750000000000	3.2638770952932
+0.2875000000000	3.2263006583662
+0.3000000000000	3.1936212201833
+0.3125000000000	3.1663502209452
+0.3250000000000	3.1449278306735
+0.3375000000000	3.1297087101756
+0.3500000000000	3.1209497227563
+0.3625000000000	3.1188005684516
+0.3750000000000	3.1232981418444
+0.3875000000000	3.1343651203794
+0.4000000000000	3.1518129128318
diff --git a/Carpet/CarpetIOHDF5/test/output_initial_data.parfile b/Carpet/CarpetIOHDF5/test/output_initial_data.parfile
new file mode 100644
index 000000000..2b69c050c
--- /dev/null
+++ b/Carpet/CarpetIOHDF5/test/output_initial_data.parfile
@@ -0,0 +1,57 @@
+# $Header: /home/eschnett/C/carpet/Carpet/Carpet/CarpetIOHDF5/test/output_initial_data.parfile,v 1.1 2004/08/23 13:44:41 tradke Exp $
+
+ActiveThorns = "Carpet CarpetLib CarpetRegrid CarpetInterp CarpetReduce CarpetSlab Boundary CoordBase SymBase CartGrid3D Periodic LocalInterp NaNChecker Slab Time MoL IDScalarWaveMoL WaveToyMoL IOUtil IOBasic CarpetIOHDF5"
+
+
+Cactus::cctk_itlast = 128
+
+Driver::global_nx  = 24
+Driver::global_ny  = 5
+Driver::global_nz  = 5
+Driver::ghost_size = 2
+
+Carpet::max_refinement_levels    = 3
+Carpet::prolongation_order_space = 3
+Carpet::prolongation_order_time  = 2
+Carpet::init_each_timelevel      = "yes"
+
+CarpetRegrid::refinement_levels = 1
+CarpetRegrid::refined_regions   = "manual-gridpoint-list"
+CarpetRegrid::gridpoints        = "[[ ([28,4,4]:[68,12,12]:[2,2,2]) ]]"
+CarpetRegrid::outerbounds       = "[[ [[0,0],[1,1],[1,1]] ]]"
+
+NaNChecker::check_every     = 16
+NaNChecker::check_vars      = "all"
+NaNChecker::action_if_found = "terminate"
+
+Grid::domain = full
+Grid::type   = byrange
+Grid::xmin   = -0.1
+Grid::ymin   = -0.15
+Grid::zmin   = -0.15
+Grid::xmax   = +1.05
+Grid::ymax   = +0.15
+Grid::zmax   = +0.15
+
+Periodic::periodic = "yes"
+
+Time::dtfac = 0.25
+
+MoL::ODE_Method = "ICN"
+
+IDScalarWaveMoL::wave_number[0] = 1.0
+
+IO::out_dir       = "input_initial_data"
+IO::out_fileinfo  = "none"
+IO::parfile_write = "no"
+IO::out_every     = 1
+IO::out_yline_x   = 0.5
+IO::out_zline_x   = 0.5 
+ 
+IOBasic::outInfo_vars   = "WaveToyMoL::scalarevolve"
+IOBasic::outScalar_vars = "WaveToyMoL::scalarevolve"
+
+# HDF5 output only for initial data
+IOHDF5::out3D_every = 513
+IOHDF5::out3D_vars  = "WaveToyMoL::scalarevolve"
+IOHDF5::out3D_dir   = "input_initial_data"