diff options
| author | Erik Schnetter <schnetter@cct.lsu.edu> | 2009-09-03 16:19:15 -0500 |
|---|---|---|
| committer | Barry Wardell <barry.wardell@gmail.com> | 2011-12-14 16:42:31 +0000 |
| commit | 11c4d98017cbb86d08e15fd1b549180184b58a26 (patch) | |
| tree | 2546a154c6f7bc0bec87de7316125ae7d1453569 /Carpet/CarpetWeb/index.html | |
| parent | f520477b1c14e02f1495cfa8d3e09f4e21ab34d0 (diff) | |
Import Carpet
Ignore-this: 309b4dd613f4af2b84aa5d6743fdb6b3
Diffstat (limited to 'Carpet/CarpetWeb/index.html')
| -rw-r--r-- | Carpet/CarpetWeb/index.html | 352 |
1 files changed, 118 insertions, 234 deletions
diff --git a/Carpet/CarpetWeb/index.html b/Carpet/CarpetWeb/index.html index 44f987855..236defe56 100644 --- a/Carpet/CarpetWeb/index.html +++ b/Carpet/CarpetWeb/index.html @@ -5,17 +5,20 @@ <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> - <title>CarpetCode</title> + <title>Carpet — AMR for Cactus</title> </head> <body> - <h1 align="center">CarpetCode</h1> + <h1 align="center">Carpet — Adaptive Mesh Refinement for the + Cactus Framework</h1> <table> <tr> <td valign="top" bgcolor="#ffcc77" width="1%"> +<a href="logo/Sierpinski.pdf"><img src="logo/Sierpinski.png" width="150" alt="Carpet logo (a Sierpiński carpet)" /></a> + <p><b>CarpetCode</b><br /> <a href="http://www.carpetcode.org/">home page</a></p> @@ -47,7 +50,7 @@ <a href="https://mailserv.aei.mpg.de/mailman/listinfo/visualization/">Mailing List</a></p> <p><b>Results</b><br /> -<a href="publications.html">Publications</a></p> +<a href="publications/publications.html">Publications</a></p> <p><b>Related</b><br /> <a href="http://www.cactuscode.org/">Cactus</a><br /> @@ -56,13 +59,14 @@ <a href="http://numrel.aei.mpg.de/">numrel@aei</a><br /> <a href="http://www.whiskycode.org/">Whisky</a><br /> <a href="http://relativity.phys.lsu.edu/postdocs/matt/taka.php">Taka</a><br /> -<a href="http://sbir.nasa.gov/SBIR/abstracts/05/sttr/phase1/STTR-05-1-T4.02-9864.html?solicitationId=STTR_05_P1">parca</a></p> +<a href="http://sbir.nasa.gov/SBIR/abstracts/05/sttr/phase1/STTR-05-1-T4.02-9864.html?solicitationId=STTR_05_P1">ParCa</a></p> <p><b>Carpet Users</b><br /> <a href="http://numrel.aei.mpg.de/">AEI Potsdam</a><br /> <!-- <a href="http://www.as.arizona.edu/">University of Arizona</a><br /> --> <!-- <a href="http://www.astro.auth.gr/Science-Subjects/Gravity.html">AUTH</a><br /> --> <a href="http://www.tapir.caltech.edu/">Caltech</a><br /> +<!-- <a href="http://astrosun2.astro.cornell.edu/research/projects/blackholes/">Cornell</a><br /> --> <a href="http://www.cra.gatech.edu/">Georgia Tech</a><br /> <a href="http://ww2.tpi.uni-jena.de/gravity/">Jena</a><br /> <a href="http://www.kisti.re.kr/english/">KISTI</a><br /> @@ -80,7 +84,8 @@ <a href="http://research.physics.uiuc.edu/CTA/IRG/">UIUC</a><br /> <a href="http://www.nuclecu.unam.mx/~gravit/Gravit/">UNAM</a><br /> <!-- <a href="http://cgwa.phys.utb.edu/">UTB</a><br /> --> -<a href="http://wugrav.wustl.edu/">WashU</a></p> +<a href="http://wugrav.wustl.edu/">WashU</a><br /> +<a href="http://www.yukawa.kyoto-u.ac.jp/english/contents/labs/astro.html">YITP</a></p> <p><b>Feedback</b><br /> <a href="mailto:schnetter@carpetcode.org">Send email</a></p> @@ -88,7 +93,7 @@ </td> <td valign="top"> - <p>Carpet is an adaptive mesh refinement driver for + <p>Carpet is an adaptive mesh refinement and multi-patch driver for the <a href="http://www.cactuscode.org/">Cactus Framework</a>. Cactus is a software framework for solving time-dependent partial differential equations on block-structured grids, and Carpet acts @@ -114,248 +119,127 @@ <h2>News</h2> <table><tr><td valign="top"> - <p><b>March 29, 2008:</b> We have benchmarked McLachlan, a new - BSSN-type vacuum Einstein code, using Carpet for unigrid and AMR - calculations. We compare several current large machines: - Franklin (NERSC), Queen Bee (LONI), and Ranger (TACC). - <!-- These machines have different architectures and - interconnects.--></p> - </td><td valign="top"> - <p><a - href="scaling-amr/results-carpet-1lev.pdf"><img - src="scaling-amr/results-carpet-1lev.png" width="180" - alt="Unigrid benchmark results"/></a></p> + <p><b>March 30, 2009:</b> We have ported Carpet to + the <a href="http://www-03.ibm.com/systems/deepcomputing/bluegene/">BlueGene/P</a> + architecture, using + the <a href="http://www.alcf.anl.gov/resources/storage.php">Surveyor</a> + system at the <a href="http://www.alcf.anl.gov/">ALCF</a>. The + graph to the right shows preliminary performance and scaling + results, comparing different compilers and options + (<a href="http://gcc.gnu.org/">gcc</a>, <a href="http://www.ibm.com/software/awdtools/xlcpp/">IBM's + XL compilers</a> without OpenMP, and IBM's XL compilers + with <a href="http://www.openmp.org/">OpenMP</a>, which required + reducing the optimisation level). For these benchmarks, the + problem size was reduced to about one eighth of the standard + size, using 13<sup>3</sup> grid points per core. The results + show that Carpet scales fine up to the size of the total machine + (4k cores), but further work on compiler options is + required.</p> </td><td valign="top"> - <p><a - href="scaling-amr/results-carpet-9lev.pdf"><img - src="scaling-amr/results-carpet-9lev.png" width="180" - alt="AMR benchmark results"/></a></p> + <p><a href="scaling-surveyor/results-surveyor.pdf"><img + src="scaling-surveyor/results-surveyor.png" + width="180" alt="AMR benchmark results" /></a></p> </td></tr></table> <table><tr><td valign="top"> - <p><b>March 1, 2008:</b> Carpet has a logo! This logo is - a <a href="http://en.wikipedia.org/wiki/Sierpinski_carpet">Sierpiński - carpet</a>, which is a fractal pattern with - a <a href="http://en.wikipedia.org/wiki/Hausdorff_dimension">Hausdorff - dimension</a> of 1.89279.</p> + <p><b>March 20, 2009:</b> Carpet can now perform <i>performance + experiments</i> by artificially increasing the size or the + number of MPI messages exchanged between processes. This can + help determine whether either the communication bandwidth or the + communication latency are a bottleneck of a particular + simulation. The figure to the right shows results for the + standard <a href="http://www.cct.lsu.edu/~eschnett/McLachlan/">McLachlan</a> + AMR benchmark run on + the <a href="http://en.wikipedia.org/wiki/Cray_XT4">Cray XT4</a> + <a href="http://www.nics.tennessee.edu/computing-resources/kraken">Kraken</a>, using 25<sup>3</sup> grid points per core. These + results indicate that the additional latency from increasing the + number of messages has no significant effect, and hence the + benchmark is bandwidth limited for this problem size.</p> </td><td valign="top"> - <p><a href="logo/Sierpinski.pdf"><img src="logo/Sierpinski.png" - width="100" alt="Carpet logo (a Sierpiński - carpet)"/></a></p> + <p><a href="scaling-whatif/results-whatif.pdf"><img + src="scaling-whatif/results-whatif.png" + width="180" alt="AMR benchmark results" /></a></p> </td></tr></table> - - <p><b>March 1, 2008:</b> We have improved the development version - of Carpet significantly:<br /></p> - <ul> - <li><p>The data structures and algorithms storing and handling - the communication schedule are much more efficient on large - numbers (several hundred or more) processors. This makes Carpet - scale to more than 8,000 cores.</p></li> - - <li><p>The interface for defining and making dynamic changes to - grid hierarchies is simpler, and buffer zones are handled in a - cleaner manner. This makes it easier to write user code which - defines or updates the grid hierarchy, and reduces the chance of - inconsistencies therein.</p></li> - - <li><p>During checkpointing and recovery, the grid structure is - saved and restored by default. This avoids accidental changes - upon recovery.</p></li> - - <li><p>The efficiency of I/O has been increased, especially for - HDF5 based binary I/O. It is possible to combine several - variables into one file to reduce the number of output - files.</p></li> - - <li><p>A new thorn LoopControl offers iterators over grid - points, implemented as C-style macros. These iterators allow - additional important loop-level optimisations, such - as <a href="http://en.wikipedia.org/wiki/Loop_tiling">loop - tiling</a> or - <a href="http://www.openmp.org/">OpenMP</a> parallelisation. - Efficient cache handling and hybrid communication models have a - large potential for performance improvements on current and - future architectures.</p></li> - </ul> - <p>More details can be found <a href="version-4.html">here</a>. - These improvements are largely due - to <a href="http://www.cct.lsu.edu/~eschnett/">Erik Schnetter</a> - (LSU), - <a href="http://www.aei.mpg.de/~tradke/">Thomas Radke</a> (AEI), and - <a href="http://www.tapir.caltech.edu/~cott/">Christian D. Ott</a> - (UA). Special thanks go to Christian Reisswig and Luca - Baiotti.</p> - - <p><b>March 1, 2008:</b> The development version of Carpet is now - maintained using <a href="http://git.or.cz/">git</a> instead - of <a href="http://www.darcs.net/">darcs</a>. Git offers a very - similar set of features to darcs, most importantly supporting - decentralised development. Git has a much larger user community - than darcs, and we hope that this makes it easier to use. - The <a href="get-carpet.html">download instructions</a> contain - details on using git to obtain Carpet, and point to further - information. (The darcs repository for the development version - will not see any further changes.)</p> - - <p><b>March 1, 2008:</b> The repository for the development - version of Carpet moved today to - a <a href="http://carpetcode.dyndns.org/">new server</a>. The - stable versions of Carpet continue to be served from the old - server for the time being. We plan to move all of carpetcode.org - to this new server in the future. The new server is a courtesy - of <a href="http://www.tapir.caltech.edu/~cott/">Christian - D. Ott</a>.</p> - + <table><tr><td valign="top"> - <p><b>January 14, 2008:</b> Carpet's communication - infrastructure has been improved significantly, making Carpet - scale to at least 4,000 processors, including mesh refinement. - Using "friendly user time" - on <a - href="http://www.tacc.utexas.edu/services/userguides/ranger/">Ranger</a>, - the new 60,000 - core <a href="http://www.teragrid.org/">TeraGrid</a> - supercomputer - at <a href="http://www.tacc.utexas.edu/">TACC</a>, we measured - the benchmark results below for a numerical relativity kernel - solving the BSSN equations. These benchmarks emply a hybrid - communication scheme - combining <a href="http://www-unix.mcs.anl.gov/mpi/">MPI</a> - and - <a href="http://www.openmp.org/">OpenMP</a>, using the shared - memory capabilities of Ranger's nodes to reduce the memory - overhead of parallelisation. We are grateful for the help we - received from Ranger's support team.</p> - <p>The graph below shows weak scaling tests for both unigrid and - mesh refinement benchmarks. The problem size per core was - kept fixed, and there were 4 OpenMP threads per MPI process, - with 1 MPI process per socket. The benchmark was also run - with the PUGH driver for comparison for certain core counts. - As the graphs show, this benchmark scales near perfectly for - unigrid, and has only small variations in run time for nine - levels of mesh refinement.</p> + <p><b>March 16, 2009:</b> Erik Schnetter and Steve Brandt + published a white + paper <a href="http://www.cct.lsu.edu/CCT-TR/CCT-TR-2009-4"><i>Relativistic + Astrophysics on the SiCortex Architecture</i></a>. This paper + expands on a + <a href="http://www.sicortex.com/news_events/campaigns/lsu_webinar">webinar</a> + by Erik and Steve that was hosted + by <a href="http://www.sicortex.com/">SiCortex</a>.</p> + <p>The graph at the right shows Carpet's parallel scalability + using + the <a href="http://www.cct.lsu.edu/~eschnett/McLachlan/">McLachlan</a> + code with nine levels of AMR for a set of current HPC systems. + The results have been rescaled to the architectures' theoretical + single-core peak performance. This makes it possible to compare + Carpet's scalability on different architectures. (It is not + possible to compare the systems' absolute performance in this + figure.)</p> </td><td valign="top"> - <p><a - href="scaling-ranger/results-ranger.pdf"><img - src="scaling-ranger/results-ranger.png" width="234" - alt="Scaling graph for Ranger"/></a></p> + <p><a href="sicortex/results-scaled.pdf"><img + src="sicortex/results-scaled.png" + width="180" alt="AMR benchmark results" /></a></p> + </td></tr></table> + + <table><tr><td valign="top"> + <p><b>November 9, 2008:</b> In the context of + the <a href="http://www.cct.lsu.edu/xirel/">XiRel project</a>, + we re-designed Carpet's communication layer to avoid many + operations that had a cost of O(<var>N</var>), growing linearly + with the number of MPI processes. Such costs are generally not + acceptable when running on several thousand cores, and have to + be reduced e.g. to O(log <var>N</var>). Carpet now stores the + communication schedule (mostly) in a distributed manner, + increasing performance and reducing its memory requirement. + These improvements are currently being tested; preliminary + scaling results are shown in the figure to the right.</p> + </td><td valign="top"> + <p><a href="scaling-improved/results-best.pdf"><img + src="scaling-improved/results-best.png" + width="180" alt="AMR benchmark results" /></a></p> </td></tr></table> -<!-- - <p><b>October 4, 2007:</b> Announcement: <i>The BBH factory</i><br /> - <b>Physicist version:</b> The BBH factory is a glorified set of shell - scripts which help maintaining source code on different machines - and building Cactus executables there. They also simplify the - task of managing many simulations on many machines and try to - prevent stupid errors.<br /> - <b>Computer scientist version:</b> The BBH factory is a set of - abstractions over the various low-level interface that current - supercomputers offer for maintaining source code and performing - simulations. As middleware, they encompass policies and best - practices that prevent common human errors. They would also be - well suited for implementing graphical user interfaces.</p> ---> - - <p><b>October 3, 2007:</b> Carpet's timing infrastructure has been - extended to automatically measure both time spent computing and - time spent in I/O. The performance of large simulations depends - not only on the computational efficiency and communication - latency, but also on the throughput to file servers. These new - statistics give a real-time overview and can point out - performance problems. The statistics are collected in the - existing <tt>Carpet::timing</tt> variables.</p> - - <p><b>August 30, 2007:</b> So far this year, ten of the - publications from three research groups examining the dynamics - of binary black hole systems are based on simulations performed - with Cactus and Carpet:<br /> - - <a href="http://www.journals.uchicago.edu/ApJ/journal/issues/ApJ/v661n1/71342/71342.html">Astrophys. J. <b>661</b>, 430-436 (2007)</a> - (<a href="http://arxiv.org/abs/gr-qc/0701143">arXiv:gr-qc/0701143</a>)<br /> - - <a href="http://link.aps.org/abstract/PRL/v99/e041102">Phys. Rev. Lett. <b>99</b>, 041102 (2007)</a> - (<a href="http://arxiv.org/abs/gr-qc/0701163">arXiv:gr-qc/0701163</a>)<br /> - - <a href="http://www.journals.uchicago.edu/ApJ/journal/issues/ApJL/v659n1/21515/brief/21515.abstract.html">Astrophys. J. <b>659</b>, L5-L8 (2007)</a> - (<a href="http://arxiv.org/abs/gr-qc/0701164">arXiv:gr-qc/0701164</a>)<br /> - - <a href="http://link.aps.org/abstract/PRL/v98/e231102">Phys. Rev. Lett. <b>98</b>, 231102 (2007)</a> - (<a href="http://arxiv.org/abs/gr-qc/0702133">arXiv:gr-qc/0702133</a>)<br /> - - <a href="http://www.iop.org/EJ/abstract/0264-9381/24/15/009/">Class. Quantum Grav. <b>24</b>, 3911-3918 (2007)</a> - (<a href="http://arxiv.org/abs/gr-qc/0701038">arXiv:gr-qc/0701038</a>)<br /> - - <a href="http://arxiv.org/abs/0705.3829">arXiv:0705.3829 [gr-qc]</a><br /> - - <a href="http://arxiv.org/abs/0706.2541">arXiv:0706.2541 [gr-qc]</a><br /> - - <a href="http://arxiv.org/abs/0707.2559">arXiv:0707.2559 [gr-qc]</a><br /> - - <a href="http://arxiv.org/abs/0708.3999">arXiv:0708.3999 [gr-qc]</a><br /> - - <a href="http://arxiv.org/abs/0708.4048">arXiv:0708.4048 [gr-qc]</a><br /> - These publications mainly examine the spin dynamics and the - gravitational wave recoil in BBH systems. Since not all - research groups use Cactus and Carpet, this represents only part - of the published work on this subject.</p> + <p><b>June 25, 2008:</b> We are happy to announce + the <a href="http://www.cct.lsu.edu/~eschnett/SimFactory"><i>Simulation + Factory</i></a>, a tool to help access remote HPC systems, + manage source trees, and submit and control simulations. The + Simulation Factory contains a set of abstractions of the tasks + which are necessary to set up and successfully finish numerical + simulations using the Cactus framework. These abstractions hide + tedious low-level management tasks, they capture "best + practices" of experienced users, and they create a log trail + ensuring repeatable and well-documented scientific results. + Using these abstractions, many types of potentially disastrous + user errors are avoided, and different supercomputers can be + used in a uniform manner.</p> <table><tr><td valign="top"> - <p><b>August 26, 2007:</b> In experiments with hybrid - communication schemes - combining <a href="http://www-unix.mcs.anl.gov/mpi/">MPI</a> - and - <a href="http://www.openmp.org/">OpenMP</a>, we found a 20% - speed improvement when using a single node - of <a - href="http://www.ncsa.uiuc.edu/UserInfo/Resources/Hardware/Intel64Cluster/">Abe</a> - at <a href="http://www.ncsa.uiuc.edu">NCSA</a>, and a - substantial scaling improvement when using 1024 and more CPUs. - (Abe has 8 CPUs per node.) These experiments included cache - optimisations when traversing the 3D arrays. The tests were - performed with a modified version of - the <a - href="http://www.cactuscode.org/">Cactus</a> <a - href="http://www.cactuscode.org/WaveToyDemo/">WaveToy</a> - example application without using I/O or analysis methods.</p> + <p><b>March 29, 2008:</b> We have benchmarked McLachlan, a new + BSSN-type vacuum Einstein code, using Carpet for unigrid and AMR + calculations. We compare several current large machines: + <a href="http://www.nersc.gov/nusers/systems/franklin/">Franklin</a> + (NERSC), <a href="http://www.loni.org/systems/system.php?system=QueenBee">Queen + Bee</a> (LONI), + and <a href="http://www.tacc.utexas.edu/services/userguides/ranger/">Ranger</a> + (TACC). + <!-- These machines have different architectures and + interconnects.--></p> </td><td valign="top"> <p><a - href="hybrid-scaling/results-wavetoy-abe.pdf"><img - src="hybrid-scaling/results-wavetoy-abe.png" width="200" - alt="Scaling graph for Abe"/></a></p> + href="scaling-amr/results-carpet-1lev.pdf"><img + src="scaling-amr/results-carpet-1lev.png" width="180" + alt="Unigrid benchmark results" /></a></p> + </td><td valign="top"> + <p><a + href="scaling-amr/results-carpet-9lev.pdf"><img + src="scaling-amr/results-carpet-9lev.png" width="180" + alt="AMR benchmark results" /></a></p> </td></tr></table> - <p><b>August 15, 2007:</b> We are happy to hear that our - proposal <i>ALPACA: Cactus tools for Application Level Profiling - And Correctness Analysis</i> will be funded by - <a - href="http://www.nsf.gov/">NSF's</a> <a - href="http://www.nsf.gov/publications/pub_summ.jsp?ods_key=nsf07503">SDCI</a> - programme for three years. - The <a - href="http://www.cactuscode.org/Development/alpaca">ALPACA</a> - project is aiming at developing complex, collaborative - scientific applications, appropriate for highly scalable - hardware architectures, providing fault tolerance, advanced - debugging, and transparency against new developments in - communication, programming, and execution models. Such tools - are especially rare at the application level, where they are - most critically needed.</p> - - <p><b>July 31, 2007:</b> We are happy to hear that our - proposal <i>XiRel: Cyberinfrastructure for Numerical - Relativity</i> will be funded by - <a href="http://www.nsf.gov/">NSF's</a> <a href="http://www.nsf.gov/funding/pgm_summ.jsp?pims_id=6681">PIF</a> - programme for three - years. <a href="http://www.cct.lsu.edu/xirel/">XiRel</a> is - collaborative proposal - by <a href="http://www.cct.lsu.edu/">LSU</a>, <a href="http://gravity.psu.edu/numrel/">PSU</a>, - and <a href="http://www.phys.utb.edu/numrel/">UTB</a> - (now <a href="http://ccrg.rit.edu/">RIT</a>). The central goal of - XiRel is the development of a highly scalable, efficient, and - accurate adaptive mesh refinement layer based on the current - Carpet driver, which will be fully integrated and supported in - Cactus and optimised for numerical relativity.</p> - <p><a href="olds.html"><b>Old News...</b></a></p> <hr /> @@ -436,7 +320,7 @@ <td valign="top" width="10%"> <p><a href="pictures/meudon-lapse-height.png"><img src="pictures/thumbnail-meudon-lapse-height.png" height="80" - width="80" alt="lapse height field"/></a></p> + width="80" alt="lapse height field" /></a></p> </td> <td valign="top" width="38%"> @@ -606,7 +490,7 @@ <p> <!-- Created: Tue Aug 12 12:12:08 CEST 2003 --> <!-- hhmts start --> -Last modified: Sat Mar 01 2008 +Last modified: Mon Mar 30 2009 <!-- hhmts end --> </p> |