Skip to content

Eliminate the need for a ghost force reduction #4556

@RudolfWeeber

Description

@RudolfWeeber

Currently, Es calculates forces/energies in the RegularDecomposition as follows:
\

  • Iterate over all local (not ghost) cells
  • For each of them consider all upper/right/back neighbor cells (i.e. some 14 of the 27), including ghost cells, and calculate forces. Forces are added to the particles on both sides. I.e. a force contribution is added to particles in local cells AND to particles in ghost cells
  • At the end, a reduction is carried out across mpi node boundaries: the forces of physical particels on one node and the forces on the corresponding ghost particles on other nodes are accumulated, so that the total force from all interactiosn (including thos frome other mpi nodes) is available on the physical particle

The drawback of this scheme is the need for an additional communication step, i.e., the ghost force reduction. This is assumed to be one of the main obstacles to Espresso's scaleability, as all nodes have to wait for each other in the communicaiot step of the reduction.
The aim of this project is to gradually remove the need for this ghost force reducion.

[ ] Phase 1: Short-range non-bonded forces

At the end of this step, an LJ fluid should run without ghost force reduciotn. Other featuers such as bonds and vitual sites will break.

  1. Add a flag to the constructor of the RegularDecomposition without_ghost_force_reduction
  2. Support it in the Python interface. See the additional parameters of the HybridDecomposition for inspiration.
  3. If the flag is true, the local cells neighboring ghost cells ON ALL SIDES have to be added to the ghost cells red_neighbors. This is done in RegularDecomposition::init_cell_interactions()
  4. If the flag is true, omit the ghost force reduction by returning an emtyp GhostCommunicator from RegularDecomopsition::collect_ghost_forces_comm()
  5. Expose the flag via an interface function in the ParticleDecomposition class hierarchy (via a a virtual function returning false in the ParticleDecomposition base class). For the RegularDecomposition override it, returning the actual value of the flag.
  6. In CellStructure::linked_cell(), query the without_ghost_force_reudciont flag of the decomposition stored in CellStructure::m_decomposition via the interface function added in the previous step. If true, run the algorithm::linked_cell() on the m_ghost_cells in addition to the current call on m_local_cells. You'll have to replicate the construciotn of fisrt/last the way it's done for m_local_cells.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions