From 11c4d98017cbb86d08e15fd1b549180184b58a26 Mon Sep 17 00:00:00 2001 From: Erik Schnetter Date: Thu, 3 Sep 2009 16:19:15 -0500 Subject: Import Carpet Ignore-this: 309b4dd613f4af2b84aa5d6743fdb6b3 --- Carpet/CarpetWeb/index.html | 352 +++++++++++++++----------------------------- 1 file changed, 118 insertions(+), 234 deletions(-) (limited to 'Carpet/CarpetWeb/index.html') diff --git a/Carpet/CarpetWeb/index.html b/Carpet/CarpetWeb/index.html index 44f987855..236defe56 100644 --- a/Carpet/CarpetWeb/index.html +++ b/Carpet/CarpetWeb/index.html @@ -5,17 +5,20 @@ - CarpetCode + Carpet — AMR for Cactus -

CarpetCode

Carpet — Adaptive Mesh Refinement for the + Cactus Framework

CarpetCode
home page

@@ -47,7 +50,7 @@ Mailing List

Results
-Publications

+Publications

Related
Cactus
@@ -56,13 +59,14 @@ numrel@aei
Whisky
Taka
-parca

+ParCa

Carpet Users
AEI Potsdam
Caltech
+ Georgia Tech
Jena
KISTI
@@ -80,7 +84,8 @@ UIUC
UNAM
-WashU

+WashU
+YITP

Feedback
Send email

@@ -88,7 +93,7 @@

Carpet is an adaptive mesh refinement driver for +

Carpet is an adaptive mesh refinement and multi-patch driver for the Cactus Framework. Cactus is a software framework for solving time-dependent partial differential equations on block-structured grids, and Carpet acts @@ -114,248 +119,127 @@

News

March 29, 2008: We have benchmarked McLachlan, a new - BSSN-type vacuum Einstein code, using Carpet for unigrid and AMR - calculations. We compare several current large machines: - Franklin (NERSC), Queen Bee (LONI), and Ranger (TACC). -

March 30, 2009: We have ported Carpet to + the BlueGene/P + architecture, using + the Surveyor + system at the ALCF. The + graph to the right shows preliminary performance and scaling + results, comparing different compilers and options + (gcc, IBM's + XL compilers without OpenMP, and IBM's XL compilers + with OpenMP, which required + reducing the optimisation level). For these benchmarks, the + problem size was reduced to about one eighth of the standard + size, using 13³ grid points per core. The results + show that Carpet scales fine up to the size of the total machine + (4k cores), but further work on compiler options is + required.

March 1, 2008: Carpet has a logo! This logo is - a Sierpiński - carpet, which is a fractal pattern with - a Hausdorff - dimension of 1.89279.

March 20, 2009: Carpet can now perform performance + experiments by artificially increasing the size or the + number of MPI messages exchanged between processes. This can + help determine whether either the communication bandwidth or the + communication latency are a bottleneck of a particular + simulation. The figure to the right shows results for the + standard McLachlan + AMR benchmark run on + the Cray XT4 + Kraken, using 25³ grid points per core. These + results indicate that the additional latency from increasing the + number of messages has no significant effect, and hence the + benchmark is bandwidth limited for this problem size.

- -

March 1, 2008: We have improved the development version - of Carpet significantly:

The data structures and algorithms storing and handling - the communication schedule are much more efficient on large - numbers (several hundred or more) processors. This makes Carpet - scale to more than 8,000 cores.
The interface for defining and making dynamic changes to - grid hierarchies is simpler, and buffer zones are handled in a - cleaner manner. This makes it easier to write user code which - defines or updates the grid hierarchy, and reduces the chance of - inconsistencies therein.
During checkpointing and recovery, the grid structure is - saved and restored by default. This avoids accidental changes - upon recovery.
The efficiency of I/O has been increased, especially for - HDF5 based binary I/O. It is possible to combine several - variables into one file to reduce the number of output - files.
A new thorn LoopControl offers iterators over grid - points, implemented as C-style macros. These iterators allow - additional important loop-level optimisations, such - as loop - tiling or - OpenMP parallelisation. - Efficient cache handling and hybrid communication models have a - large potential for performance improvements on current and - future architectures.

More details can be found here. - These improvements are largely due - to Erik Schnetter - (LSU), - Thomas Radke (AEI), and - Christian D. Ott - (UA). Special thanks go to Christian Reisswig and Luca - Baiotti.

- -

March 1, 2008: The development version of Carpet is now - maintained using git instead - of darcs. Git offers a very - similar set of features to darcs, most importantly supporting - decentralised development. Git has a much larger user community - than darcs, and we hope that this makes it easier to use. - The download instructions contain - details on using git to obtain Carpet, and point to further - information. (The darcs repository for the development version - will not see any further changes.)

- -

March 1, 2008: The repository for the development - version of Carpet moved today to - a new server. The - stable versions of Carpet continue to be served from the old - server for the time being. We plan to move all of carpetcode.org - to this new server in the future. The new server is a courtesy - of Christian - D. Ott.

- +

January 14, 2008: Carpet's communication - infrastructure has been improved significantly, making Carpet - scale to at least 4,000 processors, including mesh refinement. - Using "friendly user time" - on Ranger, - the new 60,000 - core TeraGrid - supercomputer - at TACC, we measured - the benchmark results below for a numerical relativity kernel - solving the BSSN equations. These benchmarks emply a hybrid - communication scheme - combining MPI - and - OpenMP, using the shared - memory capabilities of Ranger's nodes to reduce the memory - overhead of parallelisation. We are grateful for the help we - received from Ranger's support team.

The graph below shows weak scaling tests for both unigrid and - mesh refinement benchmarks. The problem size per core was - kept fixed, and there were 4 OpenMP threads per MPI process, - with 1 MPI process per socket. The benchmark was also run - with the PUGH driver for comparison for certain core counts. - As the graphs show, this benchmark scales near perfectly for - unigrid, and has only small variations in run time for nine - levels of mesh refinement.

March 16, 2009: Erik Schnetter and Steve Brandt + published a white + paper Relativistic + Astrophysics on the SiCortex Architecture. This paper + expands on a + webinar + by Erik and Steve that was hosted + by SiCortex.

The graph at the right shows Carpet's parallel scalability + using + the McLachlan + code with nine levels of AMR for a set of current HPC systems. + The results have been rescaled to the architectures' theoretical + single-core peak performance. This makes it possible to compare + Carpet's scalability on different architectures. (It is not + possible to compare the systems' absolute performance in this + figure.)

+ +

November 9, 2008: In the context of + the XiRel project, + we re-designed Carpet's communication layer to avoid many + operations that had a cost of O(N), growing linearly + with the number of MPI processes. Such costs are generally not + acceptable when running on several thousand cores, and have to + be reduced e.g. to O(log N). Carpet now stores the + communication schedule (mostly) in a distributed manner, + increasing performance and reducing its memory requirement. + These improvements are currently being tested; preliminary + scaling results are shown in the figure to the right.

- - -

October 3, 2007: Carpet's timing infrastructure has been - extended to automatically measure both time spent computing and - time spent in I/O. The performance of large simulations depends - not only on the computational efficiency and communication - latency, but also on the throughput to file servers. These new - statistics give a real-time overview and can point out - performance problems. The statistics are collected in the - existing Carpet::timing variables.

- -

August 30, 2007: So far this year, ten of the - publications from three research groups examining the dynamics - of binary black hole systems are based on simulations performed - with Cactus and Carpet:
-           - Astrophys. J. 661, 430-436 (2007) - (arXiv:gr-qc/0701143)
-           - Phys. Rev. Lett. 99, 041102 (2007) - (arXiv:gr-qc/0701163)
-           - Astrophys. J. 659, L5-L8 (2007) - (arXiv:gr-qc/0701164)
-           - Phys. Rev. Lett. 98, 231102 (2007) - (arXiv:gr-qc/0702133)
-           - Class. Quantum Grav. 24, 3911-3918 (2007) - (arXiv:gr-qc/0701038)
-           - arXiv:0705.3829 [gr-qc]
-           - arXiv:0706.2541 [gr-qc]
-           - arXiv:0707.2559 [gr-qc]
-           - arXiv:0708.3999 [gr-qc]
-           - arXiv:0708.4048 [gr-qc]
- These publications mainly examine the spin dynamics and the - gravitational wave recoil in BBH systems. Since not all - research groups use Cactus and Carpet, this represents only part - of the published work on this subject.

June 25, 2008: We are happy to announce + the Simulation + Factory, a tool to help access remote HPC systems, + manage source trees, and submit and control simulations. The + Simulation Factory contains a set of abstractions of the tasks + which are necessary to set up and successfully finish numerical + simulations using the Cactus framework. These abstractions hide + tedious low-level management tasks, they capture "best + practices" of experienced users, and they create a log trail + ensuring repeatable and well-documented scientific results. + Using these abstractions, many types of potentially disastrous + user errors are avoided, and different supercomputers can be + used in a uniform manner.

August 26, 2007: In experiments with hybrid - communication schemes - combining MPI - and - OpenMP, we found a 20% - speed improvement when using a single node - of Abe - at NCSA, and a - substantial scaling improvement when using 1024 and more CPUs. - (Abe has 8 CPUs per node.) These experiments included cache - optimisations when traversing the 3D arrays. The tests were - performed with a modified version of - the Cactus WaveToy - example application without using I/O or analysis methods.

March 29, 2008: We have benchmarked McLachlan, a new + BSSN-type vacuum Einstein code, using Carpet for unigrid and AMR + calculations. We compare several current large machines: + Franklin + (NERSC), Queen + Bee (LONI), + and Ranger + (TACC). +

+ href="scaling-amr/results-carpet-1lev.pdf"> Unigrid benchmark results

August 15, 2007: We are happy to hear that our - proposal ALPACA: Cactus tools for Application Level Profiling - And Correctness Analysis will be funded by - NSF's SDCI - programme for three years. - The ALPACA - project is aiming at developing complex, collaborative - scientific applications, appropriate for highly scalable - hardware architectures, providing fault tolerance, advanced - debugging, and transparency against new developments in - communication, programming, and execution models. Such tools - are especially rare at the application level, where they are - most critically needed.

- -

July 31, 2007: We are happy to hear that our - proposal XiRel: Cyberinfrastructure for Numerical - Relativity will be funded by - NSF's PIF - programme for three - years. XiRel is - collaborative proposal - by LSU, PSU, - and UTB - (now RIT). The central goal of - XiRel is the development of a highly scalable, efficient, and - accurate adaptive mesh refinement layer based on the current - Carpet driver, which will be fully integrated and supported in - Cactus and optimised for numerical relativity.

Old News...

@@ -436,7 +320,7 @@

+ width="80" alt="lapse height field" />

@@ -606,7 +490,7 @@

-Last modified: Sat Mar 01 2008 +Last modified: Mon Mar 30 2009

-- cgit v1.2.3