Version 4 will be the next release of Carpet, which is still under development. This page describes the current state of the development version and lists the changes from Version 3.
Main contributors to this version were, so far, Erik Schnetter (LSU), Thomas Radke (AEI), and Christian D. Ott (UA). Special thanks go to Luca Baiotti, Denis Pollney, Christian Reisswig, Jian Tao, and also to the CCT numerical relativity group, the AEI numerical relativity group, and the XiRel collaboration.
The data structures and algorithms storing and handling the communication schedule are much more efficient on large numbers (several hundred or more) processors. This makes Carpet scale to more than 8,000 cores.
The interface for defining and making dynamic changes to grid hierarchies is simpler, and buffer zones are handled in a cleaner manner. This makes it easier to write user code which defines or updates the grid hierarchy, and reduces the chance of inconsistencies therein.
During checkpointing and recovery, the grid structure is saved and restored by default. This avoids accidental changes upon recovery.
The efficiency of I/O has been increased, especially for HDF5 based binary I/O. It is possible to combine several variables into one file to reduce the number of output files.
A new thorn LoopControl offers iterators over grid points, implemented as C-style macros. These iterators allow additional important loop-level optimisations, such as loop tiling or OpenMP parallelisation. Efficient cache handling and hybrid communication models have a large potential for performance improvements on current and future architectures.
The parameter Carpet::enable_all_storabe
is
off by default, reducing memory usage. This can uncover hidden
errors in schedule.ccl files. You can use poisoning
(Carpet::poison_new_timelevels = yes
) to find these
problems.
The parameter Carpet::regrid_in_level_mode
is on by default. This balances the load more efficiently
between processors, especially for multi-patch
simulations.
Carpet requires thorn InitBase. This allows Carpet to
check whether its handling of initial data is consistent with
other thorns, assuming that the other thorns also use InitBase.
A new parameter Carpet::init_fill_timelevels
instructs Carpet to initialise all past time levels by making
copies of the current time level, very similar to
what MoL::initial_data_is_crap
does.
Buffer zones are specified differently. Buffer zones are
enabled by setting use_buffer_zones
, and outer
buffer zones are now the default. (Outer buffer zones are added
to the outside of the refined levels, instead of taking away
grid points from the interior of the refined levels.) The
necessary number of buffer zones is calculated automatically if
thorn MoL is used for time evolution.
There are new schedule
bins CCTK_PREREGRIDINITIAL
and CCTK_POSTREGRIDINITIAL
, called before and after
regridding during initial data setup, respectively. As a rule
of thumb, routines scheduled in CCTK_BASEGRID
must
also be scheduled in both CCTK_POSTREGRID
and CCTK_POSTREGRIDINITIAL
. You can use poisoning
(see above) to catch errors in your schedule.ccl files.
Thorn CarpetRegrid2 stores its state in grid variables instead of in parameters. This makes it easier to modify the grid hierarchy from other thorns, and is automatically consistent upon checkpointing and recovery. It also simplifies outputting the grid structure to files. These state variables can be modified at any time, but their values are only significant during regridding.
There is a new
parameter CarpetRegrid2::symmetry_rotating180
,
which forces the generated grid structure to have the correct
symmetry for a 180 degree rotational symmetry about
the z
axis. (It is unfortunate that errors in the
symmetry of the grid structure do not lead to errors but are
currently silently ignored. You can use poisoning -- see above
-- to catch errors in the grid structure.) A
parameter CarpetRegrid2::symmetry_rotating90
is
planned, but not yet implemented.
Checkpointing saves and restores the grid structure by default.
All Carpet I/O thorns support the new
parameter one_file_per_group
, which combines all
variables from a variable group into a single file. This
reduces the number of output files.
The algorithm determining the communication schedule is new and much more fussy. It will detect all (one hopes) inconsistencies in the grid structure. (Note that this does unfortunately not include errors regarding the outer or symmetry boundaries.)
The communication schedule is stored in a compressed (sparse) form. This is necessary since the communication schedule between N processors has N2 entries, which cannot be handled efficiently in full.
The prolongation operators have been rewritten. Time interpolation is now handled separately, reducing the number of operators.
Some unused CarpetLib parameters have been removed, some new CarpetLib parameters have been introduced. The default settings should be reasonable.
Carpet uses the new
thorn CactusBase/InitBase
. Update from the Cactus
CVS repository to obtain this thorn.
A new thorn Carpet/CarpetMask
can be used to
exclude certain regions, such as e.g. the interior of black
holes, from global reduction operations. This allows meaningful
norms of constraints
when Turduckened
initial data are used.
Carpet offers a new thorn Carpet/LoopControl
providing iterators over grid points. These iterators are
implemented as C-style macros which can replace for
or do
loops over grid functions. LoopControl
provides loop tiling for increased cache efficiency, and OpenMP
parallelisation to reduce parallelisation overhead on multi-core
machines.
Go back to the Carpet home page.