diff options
| author | Thomas Radke <tradke@aei.mpg.de> | 2005-03-30 15:28:00 +0000 |
|---|---|---|
| committer | Thomas Radke <tradke@aei.mpg.de> | 2005-03-30 15:28:00 +0000 |
| commit | 94daef6e66284e4bd9cd44cfe9f554932c7773c7 (patch) | |
| tree | df80da062422e5363d871063b9154b3589d7f71b /Carpet/CarpetLib/interface.ccl | |
| parent | b6ed81fe31a0a5571ea7433f1e2e2e93a630de16 (diff) | |
CarpetLib, Carpet: implement and use collective communication buffers
Collective buffers are used to gather all components' data on a processor
before it gets send off to other processors in one go. This minimizes the
number of outstanding MPI communications down to O(N-1) and thus improves
overall efficiency as benchmarks show.
Each processor allocates a pair of single send/recv buffers to communicate
with all other processors. For this the class (actually, the struct) comm_state
was extended by 3 more states:
state_get_buffer_sizes: accumulates the sizes for the send/recv buffers
state_fill_send_buffers: gathers all the data into the send buffers
state_empty_recv_buffers: copies the data from the recv buffer back into
the processor's components
Send/recv buffers are exchanged during state_fill_send_buffers and
state_empty_recv_buffers. The constructor for a comm_state struct now takes
an argument <datatype> which denotes the CCTK datatype to use for the
attached collective buffers. If a negative value is passed here then it falls
back to using the old send/recv/wait communication scheme. The datatype
argument has a default value of -1 to maintain backwards compatibility to
existing code (which therefore will keep using the old scheme).
The new communication scheme is chosen by setting the parameter
CarpetLib::use_collective_communication_buffers to "yes". It defaults to "no"
meaning that the old send/recv/wait scheme is still used.
So far all the comm_state objects in the higher-level routines in thorn Carpet
(restriction/prolongation, regridding, synchronization) have been enabled to
use collective communication buffers.
Other thorns (CarpetInterp, CarpetIO*, CarpetSlab) will follow in separate
commits.
darcs-hash:20050330152811-776a0-51f426887fea099d1a67b42bd79e4f786979ba91.gz
Diffstat (limited to 'Carpet/CarpetLib/interface.ccl')
| -rw-r--r-- | Carpet/CarpetLib/interface.ccl | 2 |
1 files changed, 2 insertions, 0 deletions
diff --git a/Carpet/CarpetLib/interface.ccl b/Carpet/CarpetLib/interface.ccl index 80b18e909..dcd518f16 100644 --- a/Carpet/CarpetLib/interface.ccl +++ b/Carpet/CarpetLib/interface.ccl @@ -19,3 +19,5 @@ includes header: gh.hh in gh.hh includes header: th.hh in th.hh includes header: operators.hh in operators.hh + +uses include header: carpet_typecase.hh |