Earth Simulator

The Earth system extends from the uppermost ionosphere to the innermost solid core of the planet and exhibits a wide variety of phenomena, such as climate change, ocean circulation, earthquakes, and geomagnetism. A vast number of parameters and precise models are needed to understand these complicated phenomena, which can significantly affect human society. Until recently, even the state-of-the-art supercomputers were not sufficiently powerful to run realistic numerical models of these phenomena. The Earth Simulator has been designed to provide a powerful tool for investigating these phenomena numerically with unprecedented resolution. It was built in 2002 as a collaborative project of the National Space Development Agency of Japan (NASDA), Japan Atomic Energy Research Institute (JAERI), and Japan Agency for Marine-Earth Science and Technology Center (JAMSTEC).

The Earth Simulator is a highly parallel vector supercomputer and consists of 640 processor nodes, which are connected by a high-speed network (Fig. 1). Each processor node has 8 vector arithmetic processors and 16 billion bytes of memory. The peak performance of each arithmetic processor is 8 gigaflops. (A gigaflop is a unit of computer speed equal to one billion floating-point arithmetic operations per second.) Because the Earth Simulator has 5120 (640 × 8) arithmetic processors, its entire memory amounts to 10 trillion bytes and its theoretical peak performance is 40 teraflops, enabling 40 trillion arithmetic calculations per second. Immediately after it started operation in March 2002, the Earth Simulator recorded the highest performance on record of 35.86 teraflops with the Linpack benchmark, which solves a dense system of linear equations. In June 2002 it was ranked first among the 500 most powerful supercomputers and presently is the world's fastest and largest supercomputer.

 

 

Fig. 1  Earth Simulator. It has 320 boxes, each equipped with 16 arithmetic processors, housed in the Earth Simulator Building at the JAMSTEC Yokohama Institute, in Japan. (Earth Simulation Center)

 

 

 

 

Projects

 

Thirty-four collaborative research projects have been selected by the Mission Definition Committee of the Earth Simulator Center for large-scale numerical simulation in the fields of (1) ocean and atmospheric science, (2) solid-earth science, (3) computer science, and (4) epoch-making simulation. These projects include research at Japanese institutions and universities, as well as international collaborative research among the Earth Simulator Center and institutions outside Japan.

 

Ocean and atmospheric science

 

In 2003 there were 12 research projects categorized as ocean and atmospheric science. These projects covered subjects such as climate change prediction using a high-resolution coupled ocean–atmosphere climate model, development of an integrated Earth system model for predicting global environmental change, and an atmospheric composition change study using regional chemical transport models. The main target of the ocean and atmospheric research is to provide reliable estimates for possible climate change associated with global warming. Typical climate models used on the Earth Simulator divide the atmosphere into 56 layers and the ocean into 46 layers and take into account the coupling of the ocean and the atmosphere. The Earth Simulator enables the use of climate models with unprecedented spatial resolution and provides precise estimates of future climate change.

 

Computer science and epoch-making simulations

 

There are 13 research projects categorized as computer science and epoch-making simulations. These projects include geospace (Sun–Earth) environment simulations, biosimulations, nuclear reactor simulations, and large-scale simulation of the properties of carbon nanotubes. For example, it has become possible to perform large-scale simulations of carbon nano-tubes to study various physical properties such as their thermal conductivity.

 

Solid-earth science

 

There are nine research projects categorized as solid-earth science. These projects include simulations of seismic-wave propagation, the Earth's magnetic field, mantle convection, and the earthquake generation process. The Earth Simulator has shown its powerful potential in simulating seismic waves expected to be generated by powerful earthquakes along the Nankai Trough off the south coast of Japan. This kind of simulation will help in hazard mitigation planning against the strong ground shaking from future earthquakes. Modeling of the Earth's magnetic field using dynamo theory, which is driven by thermal convection within the Earth's fluid core, is also expected to be advanced by large-scale numerical simulation on the Earth Simulator.

 

 

Large-scale numerical simulations in seismic studies

 

The use of seismic waves has been a unique way to probe the Earth, which we cannot directly study inside. Therefore, accurate modeling of seismic-wave propagation in three-dimensional (3D) Earth models is of considerable importance in studies for determining both the 3D seismic-wave velocity structure of the Earth and the rupture process during a large earthquake. Numerical modeling of seismic-wave propagation in 3D structures has made significant progress in the last few years due to the introduction of the spectral element method (SEM), a version of the finite element method. Although SEM can be implemented on a parallel computer, lack of available computer resources has limited its application. For instance, on a PC cluster with 150 processors, it was shown that seismic waves calculated with the SEM were accurate at periods of 18 s and longer. However, these periods are not short enough to capture the important effects on wave propagation due to smaller 3D heterogeneities of the Earth. To examine the shortest period attainable, SEM software was implemented on the Earth Simulator.

In the SEM, the Earth is divided into grid points, where a finer grid interval results in higher resolution in seismic-wave calculation. A typical computation of seismic propagation on the Earth Simulator with 5.467 billion grid points uses 243 nodes (1944 processors). This translates into an approximate grid spacing of 2.9 km (1.8 mi) along the Earth's surface. This number of grid points comprises all the known 3D structure inside the Earth, including the 3D seismic velocity structure inside the mantle, the 3D structure of the crust, and the topography and bathymetry at the Earth's surface. Using 243 nodes, a simulation of 60 min of seismic-wave propagation through the Earth (accurate at periods of 5 s and longer) requires about 15 h of computational time.

The computation of seismic waves on a global scale has been done using a normal mode summation technique (traditional simulation). This technique has computational difficulties in obtaining accurate normal modes at shorter periods and is accurate up to 6 s for seismic waves that propagate in a spherically symmetric Earth model. Figure 2 compares theoretical seismic waves calculated using the traditional normal mode summation technique for a spherically symmetric Earth model and those calculated using the SEM for a fully 3D Earth model. Because the seismic waves calculated by normal mode summation are accurate up to 8 s and longer, 48 nodes of the Earth Simulator were used to calculate the synthetic seismograms using the SEM. The results clearly demonstrate that the agreement between synthetics and actual seismograms (obtained at seismic observatories) is significantly improved by including the 3D Earth structure in the SEM synthetics. It is remarkable that the Earth Simulator allows us to simulate global seismic-wave propagation in a fully 3D Earth model at shorter periods than a traditional quasi-analytical technique for a spherically symmetric Earth model.

Figure 3 compares the theoretical seismic waves calculated using the SEM on the Earth Simulator and the observed seismic waves for the magnitude 7.9 Denali earthquake that occurred on November 3, 2002, in Alaska. Because hundreds of seismic observatories now record seismic waves generated by earthquakes, it is possible to directly compare theoretical seismic waves with real data from recorded earthquakes. The 3D models of the seismic-wave velocity structure of the Earth are traditionally built based on a combination of travel-time anomalies of short-period body waves and long-period surface waves. However, independent validation of such existing 3D Earth models has never been attempted before, owing to the lack of an independent numerical way of computing the seismic response in such models. The agreement between the theoretical seismic waves and observed records is excellent for these stations, which means that the 3D seismic velocity model used in this simulation is accurate. Therefore, this simulation demonstrates that the 3D model represents the general picture of the Earth's interior fairly well. The results demonstrate that the combination of the SEM with the Earth Simulator makes it possible to model seismic waves generated by large earthquakes accurately. This waveform-modeling tool should allow us to further investigate and improve Earth models.

 

 

 Fig. 2  Seismic waves calculated by two different means. (a) Broadband data and synthetic displacement seismograms for the September 2, 1997, Colombia earthquake bandpass-filtered between periods of 8 and 150 seconds. Vertical component data (black) and synthetic (color) displacement seismograms aligned on the arrival time of the surface wave (Rayleigh wave) are shown. Synthetics were calculated using a traditional normal mode summation technique. For each set of seismograms the azimuth is printed above the records to the left, and the station name and epicenter distance are printed to the right. (b) The same stations are as shown in a, but the synthetics were calculated with the SEM using 48 nodes of the Earth Simulator. (Courtesy of Jeroen Tromp, California Institute of Technology)

 

بزرگنمایی تصویر

 

 

Fig. 3  Theoretical seismic waves and data for the 2002 Denali, Alaska, earthquake, bandpass-filtered between periods of 5 and 150 seconds. Horizontal (transverse) component data (black) and theoretical seismic waves (color) of ground displacement aligned on the arrival time of the secondary wave (S) wave are shown. For each set of traces the azimuth is plotted above the records to the left, and the station name and epicenter distance are plotted to the right. (From S. Tsuboi et al, 2003)

 

 

 

 See also: Earth interior; Earthquake; Finite element method; Geodynamo; Geophysics; Microprocessor; Seismology; Simulation; Supercomputer

Seiji Tsuboi

 

Bibliography

 

 

  • F. A. Dahlen and J. Tromp, Theoretical Global Seismology, Princeton University Press, 1998
  • D. Komatitsch, J. Ritsema, and J. Tromp, The spectral-element method, Beowulf computing, and global seismology, Science, 298:1737–1742, 2002
  • D. Komatitsch and J. Tromp, spectral-element simulations of global seismic wave propagation, I. Validation, Geophys. J. Int., 149:390–412, 2002
  • S. Tsuboi et al., Broadband modeling of the 2003 Denali fault earthquake on the Earth Simulator, Phys. Earth Planet. Int., 139:305–312, 2003
  • Alifazeli=egeology.blogfa.com

 

Additional Readings

 

 

  • The Earth Simulator Center
  • Top 500 Supercomputer Sites
  • The results of research projects completed in 2002 are summarized in the Annual Report of the Earth Simulator Center, April 2002–March 2003
  • SC2002 Conference News
  • SC2003 Conference Gordon Bell Awards
  • Alifazeli=egeology.blogfa.com