| Commit message (Collapse) | Author | Age |
|---|
| | |
|
| | |
|
| |
|
|
| |
Thorn LSUThorns/NoMPI provides a fake MPI implementation that runs only on a single processor if no real MPI implementation is found.
|
| |
|
|
| |
Ignore-this: 309b4dd613f4af2b84aa5d6743fdb6b3
|
| | |
|
| |
|
|
|
|
| |
Correct a memory leak and simplify the code in the commstate class by using
C++ datatypes instead of new and delete.
Add many assert statements to catch potential problems.
|
| |
|
|
|
|
|
|
|
|
| |
Use vector<char> instead of new char[] in the commstate class. This
corrects a memory management error.
Use .AT() instead of [] to access vector elements to catch indexing
errors.
darcs-hash:20080219044221-dae7b-ecd72b45833617920a33311953d5c2f00c42568c.gz
|
| |
|
|
| |
darcs-hash:20070601132426-dae7b-eccfbfc01ba3b102ff0b889b927ffb8f4d026797.gz
|
| |
|
|
|
|
|
|
|
| |
Always use collective communication buffers in commstate class.
Add functions to reserve space in a commbuf, to get a pointer into the
space, and to commit space. This encapsulates using commbufs.
darcs-hash:20070419013946-dae7b-fce3d05b5e90fb37588939d1b11dce6d48ea2ead.gz
|
| |
|
|
| |
darcs-hash:20051119220215-dae7b-93b99ff6e2924297870075037a03959eb8c5fd75.gz
|
| |
|
|
| |
darcs-hash:20051119203028-dae7b-833af6bcab1804e0e6bd9858c703308652edfd57.gz
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
constructor to match declaration order
Long comment for this patch
===========================
C++ guarantees that an object's members will be destroyed in the reverse
order from the order in which they were constructed. (This is A Good
Thing; if nothing else, it's essential for exception-safety, since it's
possible for an exception to be thrown part-way through the construction
of an object.)
Since there might be multiple constructors, and it would be expensive
(slow!) to keep a run-time record of the actual order in which various
constructors constructed things, C++ fixes a standard order, namely
the order in which the members are declared, and always uses that.
(Hence C++ destructors, including ones automagically generated by
the compiler, always destroy members in reverse-declaration order.)
That is, if you declare a class (or struct)
class foo
{
int A, B, C;
foo();
};
then the 3 members will ALWAYS be initialized in the order A,B,C,
and destroyed in the order C,B,A, whether you write the constructor
like this
foo::foo: A(69), B(105), C(42) { } // version 1
or like this
foo::foo: B(105), C(42), A(69) { } // version 2
So far all is well. But look what happens if you use one member in
the calculation of another member's initializer, eg
foo::foo: A(69), B(A+36), C(42) { } // version 3
This is still ok... but if we now use a different order for the
member initializers,
foo::foo: B(105), C(42), A(B-36) { } // version 4
then disaster strikes: As explained above, even though they were
written in the order B,C,A, these initializers will actuallly be
executed in the order the members were delcared, i.e. A,B,C. This
means that the A initializer will use the not-yet-initialized
(and hence garbage) value of B , and hence will initialize A
to a garbage value.
Alternatively, starting with the version 3 constructor, if someone
now permutes the declarations of A, B, and C such that B is declared
before A, then the same disaster will strike again (B will be
"initialized" to garbage).
There are two standard rules of C++ style which exist precisely
to avoid this problem:
1. ALWAYS write member initializers in the same order as the members
are declared. Many C++ compilers will in fact give a warning if you
violate this rule.
2. Prefer to not use the value of any member in the calculation of
the initializer for any other member. If you need to violate
this rule, there should be a comment on both the declaration
of the members, and on the initializers, that the code relies
on a particular ordering.
For more details see, for example, item 13 in Meyeres "Effective C++".
Ok, given all this explanation... the purpose of *this* patch is to
fix the comm_state::procbufdesc() constructor to comply with rule #1
(in this case the code is still correct even when violating that rule,
but it's cleaner style to always comply.)
Previously g++ had warned of the violation:
In file included from /home/jonathan/cactus/Cactus/arrangements/Carpet/CarpetLib/src/gdata.hh:13,
from /home/jonathan/cactus/Cactus/arrangements/Carpet/CarpetLib/src/data.hh:14,
from /home/jonathan/cactus/Cactus/configs/gzmultipatch-tofu-mp/bindings/include/data.hh:4,
from /home/jonathan/cactus/Cactus/arrangements/Carpet/Carpet/src/variables.hh:21,
from /home/jonathan/cactus/Cactus/arrangements/Carpet/Carpet/src/carpet_public.hh:11,
from /home/jonathan/cactus/Cactus/configs/gzmultipatch-tofu-mp/bindings/include/carpet.hh:4,
from /home/jonathan/cactus/Cactus/configs/gzmultipatch-tofu-mp/build/GZPatchSystem/patch/angular_interpatch_ghost_zone.cc:45:
/home/jonathan/cactus/Cactus/arrangements/Carpet/CarpetLib/src/commstate.hh: In
constructor `comm_state::procbufdesc::procbufdesc()':
/home/jonathan/cactus/Cactus/arrangements/Carpet/CarpetLib/src/commstate.hh:93: warning: `
comm_state::procbufdesc::recvbuf' will be initialized after
/home/jonathan/cactus/Cactus/arrangements/Carpet/CarpetLib/src/commstate.hh:83: warning:
`char*comm_state::procbufdesc::sendbufbase'
darcs-hash:20050921145044-b0a3f-b59f991ea43729fd06fca6a5ea5d7397cefebbc3.gz
|
| |
|
|
|
|
|
|
|
| |
CarpetLib's comm_state class (actually, it's still just a struct) has been
extended to handle collective buffer communications for all possible C datatypes
at the same time. This makes it unnecessary for the higher-level communication
routines to loop over each individual datatype separately.
darcs-hash:20050815150023-776a0-dddc1aca7ccaebae872f9f451b2c3595cd951fed.gz
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
single components
The default communication scheme in Carpet (which does an individual send/recv
operation for each component) comes with two parameters for fine tuning:
CarpetLib::use_lightweight_buffers
CarpetLib::combine_recv_send
the defaults of which are set to use a well-tested but also slower
communication pattern (as turned out during benchmark runs).
This patch cleans up the implementation of this communication scheme so that the
fastest communication pattern (combined posting of send/recv; use of lightweight
buffers) is now always used. The above parameters therefore became obsolete
and shouldn't be used anymore in parfiles.
darcs-hash:20050526114253-776a0-780933a1539a260d74da8b92522fa2f48c714964.gz
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Receive operations are posted earlier now (don't wait until send buffers
are filled).
* A send operation is posted as soon as its send buffer is full (don't wait
until all send buffers have been filled).
* MPI_Irsend() is used instead of MPI_Isend()
This probably doesn't make a difference with most MPI implementations.
* Use MPI_Waitsome() to allow for overlapping of communication and computation
to some extent: data from already finished receive operations can be
copied back while active receive operations are still going on.
MPI_Waitsome() is now called (instead of MPI_Waitall()) to wait for
(one or more) posted receive operations to finish. The receive buffers
for those operations are then flagged as ready for data copying.
The drawback of this overlapping communication/computation scheme is
that the comm_state loop may be iterated more often now. My benchmarks on
up to 16 processors showed no performance win compared to using MPI_Waitall()
(in fact, the performance decreased). Maybe it performs better on larger
numbers of processors when there is more potential for network congestion.
The feature can be turned on/off by setting CarpetLib::use_waitall to yes/no.
For now I recommend using CarpetLib::use_waitall = "yes" (which is not the
default setting).
darcs-hash:20050411122235-776a0-e4f4179f46fce120572231b19cacb69c940f7b82.gz
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Collective buffers were accidentally used (eg. by CarpetIOHDF5 or CarpetIOASCII)
even if CarpetLib::use_collective_communication_buffers was set to "no".
Now this parameter is evaluated in the comm_state constructor (together with
the variable type given) and the result stored in a flag
comm_state::uses__collective_communication_buffers. This flag is then used
later in comm_state::step() to decide about communication paths.
darcs-hash:20050411100916-776a0-aef034c4a23dac96f515cf831d15c8b7e2ce2f9d.gz
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Collective buffers are used to gather all components' data on a processor
before it gets send off to other processors in one go. This minimizes the
number of outstanding MPI communications down to O(N-1) and thus improves
overall efficiency as benchmarks show.
Each processor allocates a pair of single send/recv buffers to communicate
with all other processors. For this the class (actually, the struct) comm_state
was extended by 3 more states:
state_get_buffer_sizes: accumulates the sizes for the send/recv buffers
state_fill_send_buffers: gathers all the data into the send buffers
state_empty_recv_buffers: copies the data from the recv buffer back into
the processor's components
Send/recv buffers are exchanged during state_fill_send_buffers and
state_empty_recv_buffers. The constructor for a comm_state struct now takes
an argument <datatype> which denotes the CCTK datatype to use for the
attached collective buffers. If a negative value is passed here then it falls
back to using the old send/recv/wait communication scheme. The datatype
argument has a default value of -1 to maintain backwards compatibility to
existing code (which therefore will keep using the old scheme).
The new communication scheme is chosen by setting the parameter
CarpetLib::use_collective_communication_buffers to "yes". It defaults to "no"
meaning that the old send/recv/wait scheme is still used.
So far all the comm_state objects in the higher-level routines in thorn Carpet
(restriction/prolongation, regridding, synchronization) have been enabled to
use collective communication buffers.
Other thorns (CarpetInterp, CarpetIO*, CarpetSlab) will follow in separate
commits.
darcs-hash:20050330152811-776a0-51f426887fea099d1a67b42bd79e4f786979ba91.gz
|
| |
|
|
|
|
|
| |
Restructure the lightweight communication buffers.
Use lightweight communication buffers for interpolation as well.
darcs-hash:20050103200712-891bb-7e42816d3b8d667916084e3f32527c8f35327d7f.gz
|
| |
|
|
|
|
|
|
|
|
|
| |
Lightweight communication buffers use essentially only a vector<T>
instead of a data<T> to transfer data between processors. This should
reduce the computational overhead.
Set the parameter "use_lightweight_buffers" to use this feature. This
feature is completely untested.
darcs-hash:20050102173524-891bb-6a3999cbd63e367c8520c175c8078374d294eaa8.gz
|
|
|
darcs-hash:20050101193846-891bb-7bb505d29a25b04c0d23e792eea7ff404d1f4200.gz
|
