4.5. Notes for Developers and Code Maintainers
This section documents how a few large sections of code with LAMMPS work at a conceptual level. Comments on code in source files typically document what a variable stores, what a small section of code does, or what a function does or its input/outputs. The topics on this page are intended to document code at a higher level.
4.5.1. KSpace PPPM FFT grids
The various KSpace PPPM styles in LAMMPS use FFTs to solve Poisson’s equation. This subsection describes:
how FFT grids are defined
how they are decomposed across processors
how they are indexed by each processor
how particle charge and electric field values are mapped to/from the grid
An FFT grid cell is a 3d volume; grid points are corners of a grid cell and the code stores values assigned to grid points in vectors or 3d arrays. A global 3d FFT grid has points indexed 0 to N-1 inclusive in each dimension.
Each processor owns two subsets of the grid, each subset is brick-shaped. Depending on how it is used, these subsets are allocated as a 1d vector or 3d array. Either way, the ordering of values within contiguous memory x fastest, then y, z slowest.
3d decomposition of the grid, the global grid is
partitioned into bricks that correspond to the sub-domains of the
simulation box that each processor owns. Often, this is a regular 3d
array (Px by Py by Pz) of bricks, where P = number of processors =
Px * Py * Pz. More generally it can be a tiled decomposition, where
each processor owns a brick and the union of all the bricks is the
global grid. Tiled decompositions are produced by load balancing with
the RCB algorithm; see the balance rcb command.
FFT decompostion of the grid, each processor owns a brick
that spans the entire x dimension of the grid while the y and z
dimensions are partitioned as a regular 2d array (P1 by P2), where P =
P1 * P2.
The following indices store the inclusive bounds of the brick a processor owns, within the global grid:
nxlo_in,nxhi_in,nylo_in,nyhi_in,nzlo_in,nzhi_in = 3d decomposition brick nxlo_fft,nxhi_fft,nylo_fft,nyhi_fft,nzlo_fft,nzhi_fft = FFT decomposition brick nxlo_out,nxhi_out,nylo_out,nyhi_out,nzlo_out,nzhi_out = 3d decomposition brick + ghost cells
fft indices are from 0 to N-1 inclusive in each
dimension, where N is the grid size.
out indices index an array which stores the
in subset of
the grid plus ghost cells that surround it. These indices can thus be
< 0 or >= N.
The number of ghost cells a processor owns in each of the 6 directions is a function of:
neighbor skin distance (since atoms can move outside a proc subdomain) qdist = offset or charge from atom due to TIP4P fictitious charge order = mapping stencil size shift = factor used when order is an even number (see below)
Here is an explanation of how the PPPM variables
OFFSET work. They are the
relevant variables that determine how atom charge is mapped to grid
points and how field values are mapped from grid points to atoms:
order = # of nearby grid points in each dim that atom charge/field are mapped to/from nlower,nupper = extent of stencil around the grid point an atom is assigned to OFFSET = large integer added/subtracted when mapping to avoid int(-0.75) = 0 when -1 is the desired result
The particle_map() method assigns each atom to a grid point.
If order is even, say 4:
atom is assigned to grid point to its left (in each dim) shift = OFFSET nlower = -1, nupper = 2, which are offsets from assigned grid point window of mapping grid pts is thus 2 grid points to left of atom, 2 to right
If order is odd, say 5:
atom is assigned to left/right grid pt it is closest to (in each dim) shift = OFFSET + 0.5 nlower = 2, nupper = 2 if point is in left half of cell, then window of affected grid pts is 3 grid points to left of atom, 2 to right if point is in right half of cell, then window of affected grid pts is 2 grid points to left of atom, 3 to right
These settings apply to each dimension, so that if order = 5, an atom’s charge is mapped to 125 grid points that surround the atom.