Chapter 1

Nek5000 is designed to simulate laminar, transitional, and turbulent incompressible or low Mach-number flows with heat transfer and species transport. It is also suitable for incompressible magnetohydrodynamics (MHD). Nek5000 is written in f77 and C. It uses MPI for message passing (but can be compiled without MPI for serial applications) and some LAPACK routines for eigenvalue computations (depending on the particular solver employed). In addition, it can be optionally coupled with MOAB, which provides an interface to meshes generated with CUBIT. Nek5000 output formats can be read by either postx or the parallel visualization package VisIt developed by Hank Childs and colleagues at LLNL/LBNL. VisIt is mandatory for large problems (e.g., more than 100,000 spectral elements).

Computational approach

The spatial discretization is based on the spectral element method (SEM) [1], which is a high-order weighted residual technique similar to the finite element method. In the SEM, the solution and data are represented in terms of N  th-order tensor-product polynomials within each of E  deformable hexahedral (brick) elements. Typical discretizations involve E  =100–10,000 elements of order N  =8–16 (corresponding to 512–4096 points per element). Vectorization and cache efficiency derive from the local lexicographical ordering within each macro-element and from the fact that the action of discrete operators, which nominally have       6
O(EN  )  nonzeros, can be evaluated in only       4
O (EN  )  work and       3
O (EN  )  storage through the use of tensor-product-sum factorization [2]. The SEM exhibits very little numerical dispersion and dissipation, which can be important, for example, in stability calculations, for long time integrations, and for high Reynolds number flows. We refer to [3] for more details. The code Nek5000 is based on the following design principles

Nek5000 solves the unsteady incompressible two-dimensional, axisymmetric, or three-dimensional Stokes or Navier-Stokes equations with forced or natural convection heat transfer in both stationary (fixed) or time-dependent geometry. It also solves the compressible Navier-Stokes in the Low Mach regime, the magnetohydrodynamic equation (MHD). The solution variables are the fluid velocity u = (ux,uy,uz)  , the pressure p  , the temperature T  . All of the above field variables are functions of space x = (x,y,z)  and time t  in domains Ωf  and/or Ωs  defined in Fig. 1.1. Additionally Nek5000 can handle conjugate heat transfer problems.


Figure 1.1: Computational domain showing respective fluid and solid subdomains, Ωf  and Ωs  . The shared boundaries are denoted ∂Ωf = ∂Ωs  and the solid boundary which is not shared by fluid is ----
∂Ωs  , while the fluid boundary not shared by solid ----
∂Ωf  .

1.1 Incompressible Navier–Stokes equations

The governing equations of flow motion in dimensional form are

ρ(∂tu+ u ⋅∇u ) = - ∇p + ∇ ⋅τ + ρf,in Ωf,   (Momentum  )               (1.1)
where τ = μ[∇u  + ∇uT ]  .
∇ ⋅u = 0,in Ωf ,  (Continuity)                            (1.2)

If the fluid viscosity is constant in the entire domain the viscous stress tensor can be contracted ∇ ⋅τ = μ Δu  , therefore one may solve the Navier–Stokes equations in either the stress formulation, or no stress

1.2 Non-dimensional Navier-Stokes

Let us introduce the following non-dimensional variables x*  = xL-  , u* =  uU-  , t* =  -t--
      L∕U  . For the pressure scale we have two options

For highly convective flows we choose the first scaling of the pressure and obtain the non-dimensional Navier-Stokes:

∂u *    *    *        *   1      *   1  f
∂t*-+ u  ⋅∇u   = - ∇p  +  Re∇  ⋅τ + F-rg-.

where τ* = [∇u * + ∇u *T]  . The two non-dimensional numbers here are the Reynolds number Re = UνL-  F r  and the Froude number, defined as F r = U2-
      gL  .

1.3 Energy equation

In addition to the fluid flow, Nek5000 computes automatically the energy equation

ρcp(∂tT + u ⋅∇T ) = ∇ ⋅(k∇T ) + qvol,in Ωf ∪ Ωs (Energy)                (1.4)

1.4 Non-dimensional energy/passive scalar equation

A similar non-dimensionalization as for the flow equations using the non-dimensional variables x * = xL-  , u * = u-
      U  , t* = --t-
     L∕U  , T =  T*-T0
      δT  leads to

∂t*T * + u* ⋅∇T * =-1-∇ ⋅∇T * + qvol,in Ωf ∪ Ωs (Energy)                (1.5)
                   P e
where P e = LU ∕α  , with α = k∕ρcp  .

1.5 Passive scalars

We can additionally solve a convection-diffusion equation for each passive scalar ϕi  , i  =1,2,...  in Ωf ∪ Ωs

(ρcp)i(∂tϕi + u ⋅∇ ϕi) = ∇ ⋅(ki∇ ϕi)+ (qvol)i.                    (1.6)

The terminology and restrictions of the temperature equations are retained for the passive scalars, so that it is the responsibility of the user to convert the notation of the passive scalar parameters to their thermal analogues. For example, in the context of mass transfer, the user should recognize that the values specified for temperature and heat flux will represent concentration and mass flux, respectively. Any combination of these equation characteristics is permissible with the following restrictions. First, the equation must be set to unsteady if it is time-dependent or if there is any type of advection. For these cases, the steady-state (if it exists) is found as stable evolution of the initial-value-problem. Secondly, the stress formulation must be selected if the geometry is time-dependent. In addition, stress formulation must be employed if there are traction boundary conditions applied on any fluid boundary, or if any mixed velocity/traction boundaries, such as symmetry and outflow/n, are not aligned with either one of the Cartesian x, y  or z  axes. Other capabilities of Nek5000 are the linearized Navier-Stokes for flow stability, magnetohydrodynamic flows etc.

1.6 Unsteady Stokes

In the case of flows dominated by viscous effects Nek5000 can solve the reduced Stokes equations

ρ(∂tu ) = - ∇p + ∇ ⋅τ + ρf,in Ωf (Momentum )                     (1.7)
where                      T
∇ ⋅τ = ∇  ⋅μ[∇u  + ∇u  ]  and
∇ ⋅u =  0,in Ωf (Continuity)                             (1.8)
Also here we can distinguish between the stress and non-stress formulation according to whether the viscosity is variable or not. The non-dimensional form of these equations can be obtained using the viscous scaling of the pressure.

1.7 Steady Stokes

If there is no time-dependence, then Nek5000 can further reduce to

- ∇p + ∇  ⋅τ + ρf = 0,in Ωf (Momentum )                       (1.9)
where ∇ ⋅τ = ∇  ⋅μ[∇u  + ∇uT ]  and
∇ ⋅u =  0,in Ωf (Continuity)                            (1.10)

1.8 Linearized Equations

In addition to the basic evolution equations described above, Nek5000 provides support for the evolution of small perturbations about a base state by solving the linearized equations

ρ(∂ u ′ + u ⋅∇u ′ + u ′⋅∇u ) = - ∇p ′+ μ∇2u ′, ∇ ⋅u ′= 0,
   t i         i    i            i       i          i

for multiple perturbation fields i = 1,2,...  subject to different initial conditions and (typically) homogeneous boundary conditions. These solutions can be evolved concurrently with the base fields (u,p,T )  . There is also support for computing perturbation solutions to the Boussinesq equations for natural convection. Calculations such as these can be used to estimate Lyapunov exponents of chaotic flows, etc.

1.9 Steady conduction

The energy Eq. 1.4 in which the advection term u ⋅∇T  and the transient term ∂tT  are zero. In essence this represents a Poisson equation.

1.10 Low-Mach Navier-Stokes

The compressible Navier-Stokes differ mathematically from the incompressible ones mainly in the divergence constraint ∇ ⋅u ⁄= 0  . In this case the system of equations is not closed and an additional equation of state (EOS) is required to connect the state variables, e.g. p = f(ρ,T )  . However Nek5000 can only solve the Low Mach approximation of the compressible Navier-Stokes. The Low-Mach approximation decouples the pressure from the velocity leading to a system of equations which can be solved numerically in a similar fashion as the incompressible Navier-Stokes.

The Low Mach equations in non-dimensional form are

 (            )
ρ  du-+ u ⋅∇u   = - ∇p + ∇ ⋅τ + ρf                     (1.12)
dt-+ u ⋅∇ ρ = - ρ∇ ⋅ u
 (            )
ρ  dT-+ u ⋅∇T   =  - ∇ ⋅k ∇T
where                T   2
τ = μ [∇u  + ∇u   - 3∇ ⋅uI]  .

The implementation of the equation if state for the Low Mach formulation is for the moment hard-coded to be the ideal gas equation of state p = ρRT  . This allows for both variable density and variable viscosity. The system is solved by substituting ρ = f (p,T )  into the continuity equation and obtaining a so-called thermal divergence (the term ∇ ⋅u  is given as a function of the temperature). A more detailed description on how these equations connect is given in section 1.10 as well as in the developer’s manual.

1.11 Incompressible MHD equations

Magnetohydrodynamics is based on the idea that magnetic fields can induce currents in a moving conductive fluid, which in turn creates forces on the fluid and changing the magnetic field itself. The set of equations which describe MHD are a combination of the Navier-Stokes equations of fluid dynamics and Maxwell’s equations of electromagnetism. These differential equations have to be solved simultaneously, and Nek5000 has an implementation for the incompressible MHD.

Consider a fluid of velocity u  subject to a magnetic field B  then the incompressible MHD equations are

ρ(∂tu + u ⋅∇u )  =  - ∇p + μ Δu + B ⋅∇B   ,                  (1.13)
         ∇  ⋅u  =  0                                        (1.14)
  ∂B  + u⋅ ∇B   =  - ∇q + ηΔB  + B  ⋅∇u ,
         ∇ ⋅B   =  0
where ρ  is the density μ  the viscosity, η  resistivity, and pressure p  .

The total magnetic field can be split into two parts: B =  B0 + b  (mean + fluctuations). The above equations become in terms of Elsässer variables (z± = u ± b  )

   ±                (     )
∂z-- ∓ (B0 ⋅∇) z± +  z∓ ⋅ ∇ z± = - ∇p + ν+∇2z ± + ν- ∇2z ∓            (1.15)
where ν± = ν ± η  .

The important non-dimensional parameters for MHD are Re = U L ∕ν  and the magnetic Re ReM  = U L∕ η  .

1.12 Adaptive Lagrangian-Eulerian (ALE)

We consider unsteady incompressible flow in a domain with moving boundaries:

  ∂u              1
  ---  =  - ∇p +  --∇  ⋅(∇  + ∇T )u+ N L,                    (1.16)
   ∂t             Re
∇  ⋅u  =  0                                                 (1.17)
Here, N L  represents the quadratic nonlinearities from the convective term.

Our free-surface hydrodynamic formulation is based upon the arbitrary Lagrangian-Eulerian (ALE) formulation described in [4]. Here, the domain Ω(t)  is also an unknown. As with the velocity, the geometry x  is represented by high-order polynomials. For viscous free-surface flows, the rapid convergence of the high-order surface approximation to the physically smooth solution minimizes surface-tension-induced stresses arising from non-physical cusps at the element interfaces, where only   0
C   continuity is enforced. The geometric deformation is specified by a mesh velocity w  := x˙  that is essentially arbitrary, provided that w  satisfies the kinematic condition w  ⋅ ˆn|Γ = u⋅nˆ|Γ   , where ˆn  is the unit normal at the free surface Γ (x,y,t)  . The ALE formulation provides a very accurate description of the free surface and is appropriate in situations where wave-breaking does not occur.

To highlight the key aspects of the ALE formulation, we introduce the weighted residual formulation of (1.16): Find          N     N
(u,p) ∈ X  × Y  such that:

d-(v,u) = (∇ ⋅v,p) - -2-(∇v, S )+ (v,N L )+ c(v,w, u),    (∇ ⋅u,q) = 0,      (1.18)
dt                   Re
for all test functions (v,q) ∈ XN  × YN  . Here (XN ,Y N )  are the compatible velocity-pressure approximation spaces introduced in [5], (.,.)  denotes the inner-product (f,g) := ∫   f ⋅gdV
         Ω(t)  , and S  is the stress tensor       1 ∂ui   ∂uj
Sij := 2(∂xj + ∂xi)  . For simplicity, we have neglected the surface tension term. A new term in (1.18) is the trilinear form involving the mesh velocity
             ∫    3   3
c(v,w, u) :=      ∑   ∑  v ∂wjuidV,                         (1.19)
              Ω(t)        i ∂xj
                 i=1 j=1
which derives from the Reynolds transport theorem when the time derivative is moved outside the bilinear form (v,ut)  . The advantage of (1.18) is that it greatly simplifies the time differencing and avoids grid-to-grid interpolation as the domain evolves in time. With the time derivative outside of the integral, each bilinear or trilinear form involves functions at a specific time, tn-q  , integrated over Ω (tn- q)  . For example, with a second-order backward-difference/extrapolation scheme, the discrete form of (1.18) is
--1- [3 (vn, un)n - 4 (vn -1,un-1)n-1 + (vn -2,un-2)n-2] = Ln(u )+ 2^N Ln- 1 - ^N Ln- 2. (1.20)
2 Δt
Here, Ln(u)  accounts for all linear terms in (1.18), including the pressure and divergence-free constraint, which are evaluated implicitly (i.e., at time level tn  , on Ω (tn)  ), and ^N Ln- q  accounts for all nonlinear terms, including the mesh motion term (1.19), at time-level  n-q
t  . The superscript on the inner-products     n-q
(.,.)  indicates integration over    n- q
Ω (t  )  . The overall time advancement is as follows. The mesh position xn ∈ Ω(tn)  is computed explicitly using wn -1   and wn -2   ; the new mass, stiffness, and gradient operators involving integrals and derivatives on Ω(tn)  are computed; the extrapolated right-hand-side terms are evaluated; and the implicit linear system is solved for un  . Note that it is only the operators that are updated, not the matrices. Matrices are never formed in Nek5000 and because of this, the overhead for the moving domain formulation is very low.