2D Space Charge in Particle Tracking #909

ax3l · 2025-03-19T22:38:54Z

Add 2D Space Charge in Particle Tracking. First part of #401

To Do

For this to compile, you need to set:
cmake --fresh -S . -B build -DImpactX_FFT=ON -DImpactX_ablastr_branch=a76eecf1553f5d0e72ec948c9a67db4111a1544f (WarpX commit of the merged PR BLAST-WarpX/warpx#6201 )

Follow-up

MR support (fix segfaults in OpenBC - backward_doit - ParallelCopy)
Add current weighting and longitudinal kick for 2.5D space charge #1121
- calculate 1D current along t (for a fixed s) -- take model from CSR? expose resolution as separate option?

ax3l · 2025-03-19T23:37:54Z

src/particles/spacecharge/GatherAndPush.cpp

+                        // push momentum
+                        px += field_interp[0] * push_consts;
+                        py += field_interp[1] * push_consts;
+                        pz += field_interp[2] * push_consts;  // TODO: is this always += 0.0?


Non-zero, because we will model a variation of the current density along t.

Might in practice often be neglected. We can start without long. kick and add it later.

src/particles/spacecharge/GatherAndPush.cpp

src/particles/spacecharge/ForceFromSelfFields.cpp

src/particles/spacecharge/GatherAndPush.cpp

cemitch99 · 2025-09-06T00:10:06Z

The test here currently fails with the simple error:

domain size in direction 2 is 1
blocking_factor is 8
amrex::Error::0::domain size not divisible by blocking_factor !!!

We should make this error conditional on the space charge solver. This solver requires a grid size of 1 in z.

src/particles/spacecharge/GatherAndPush.cpp

cemitch99

Need to touch base on the scale/normalization of the fields returned by the 2D solve, in order to correctly set the value of push_constants in GatherAndPush.cpp.

Enable interpolating fields to particles in 1D or 2D even when running a 3D simulation. Needed for 2D and 2.5D space charge in ImpactX: BLAST-ImpactX/impactx#909

src/particles/spacecharge/ForceFromSelfFields.cpp

cemitch99 · 2025-09-12T22:38:35Z

Here is a more interesting observation. I ran with an axisymmetric KV beam, expanding in free space, with 2D space charge for an identical number of grid points in x and y, but the envelopes show significantly different evolution. This is evidence of x/y asymmetry in the underlying 2D Poisson solve. The push constants need to be updated, but they are identical in x and y.
Expanding_beam_2D_envelopes.pdf

# Particle Beam(s)
beam.npart = 100000  # outside tests, use 1e5 or more
beam.units = static
beam.kin_energy = 250.0
beam.current = 1.0e-3
beam.particle = proton
beam.distribution = kvdist
beam.lambdaX = 4.472135955e-4
beam.lambdaY = beam.lambdaX
beam.lambdaT = 9.12241869e-7
beam.lambdaPx = 0.0
beam.lambdaPy = 0.0 
beam.lambdaPt = 0.0

# Beamline: lattice elements and segments
lattice.elements = monitor drift1 monitor
lattice.nslice = 40

drift1.type = drift
drift1.ds = 6.0

monitor.type = beam_monitor
monitor.backend = h5

# Algorithms
algo.particle_shape = 2
algo.space_charge = 2D
algo.poisson_solver = "fft"

amr.n_cell = 16 16 1
amr.blocking_factor_x = 8
amr.blocking_factor_y = 8
amr.blocking_factor_z = 1

geometry.prob_relative = 1.01

cemitch99 · 2025-09-16T00:24:02Z

src/particles/spacecharge/GatherAndPush.cpp

+                        px += field_interp[0] * push_consts * beta_gamma * dr[2] / (c0_SI);
+                        py += field_interp[1] * push_consts * beta_gamma * dr[2] / (c0_SI);  //this should be field_interp[1]


The constants appearing in px and py are currently in draft form only, and will continue to be updated.

tests/python/test_charge_deposition_2D.py

Keep simple for now.

Phi is then blown up again to 3D with indentical values in z. Co-authored-by: Weiqun Zhang <[email protected]>

ax3l · 2025-10-16T20:38:11Z

examples/CMakeLists.txt

+)
+add_impactx_test(expanding-fft-2d.py
+    examples/expanding_beam/run_expanding_fft_2D.py
+    OFF   # ImpactX MPI-parallel


If we want 100% the same AMReX blocks ("grids output"):

Suggested change

OFF # ImpactX MPI-parallel

ON # ImpactX MPI-parallel

examples/CMakeLists.txt

Co-authored-by: Axel Huebl <[email protected]>

examples/fodo_space_charge/analysis_fodo_2d_sc.py

cemitch99 · 2025-10-17T00:56:41Z

FYI: I had to relax the tolerance a great deal for the Python tests, especially the FODO test, while the C++/app tests passed with no problem. I'm still concerned that there may be something else going on specific to the Python test implementation or bindings.

* [pre-commit.ci] pre-commit autoupdate (BLAST-ImpactX#1161) updates: - [github.com/astral-sh/ruff-pre-commit: v0.13.1 → v0.13.2](astral-sh/ruff-pre-commit@v0.13.1...v0.13.2) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [pre-commit.ci] pre-commit autoupdate (BLAST-ImpactX#1165) updates: - [github.com/astral-sh/ruff-pre-commit: v0.13.2 → v0.13.3](astral-sh/ruff-pre-commit@v0.13.2...v0.13.3) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Gauss 3D: Fix Push Constants (BLAST-ImpactX#1168) This fixes the push constants for non-equal x,y,z and adds an end-point correction for the integration. It also changes the number of integration points by default to 101 instead of 401. Co-authored-by: Ji Qiang <[email protected]> * [pre-commit.ci] pre-commit autoupdate (BLAST-ImpactX#1174) updates: - [github.com/astral-sh/ruff-pre-commit: v0.13.3 → v0.14.0](astral-sh/ruff-pre-commit@v0.13.3...v0.14.0) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Bump stefanzweifel/git-auto-commit-action from 6 to 7 (BLAST-ImpactX#1175) Bumps [stefanzweifel/git-auto-commit-action](https://github.com/stefanzweifel/git-auto-commit-action) from 6 to 7. - [Release notes](https://github.com/stefanzweifel/git-auto-commit-action/releases) - [Changelog](https://github.com/stefanzweifel/git-auto-commit-action/blob/master/CHANGELOG.md) - [Commits](stefanzweifel/git-auto-commit-action@v6...v7) --- updated-dependencies: - dependency-name: stefanzweifel/git-auto-commit-action dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump github/codeql-action from 3 to 4 (BLAST-ImpactX#1176) Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3 to 4. - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](github/codeql-action@v3...v4) --- updated-dependencies: - dependency-name: github/codeql-action dependency-version: '4' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Fix `BeamMonitor`: Delay Open (BLAST-ImpactX#1178) `BeamMonitor` opens the openPMD series now on first output. This simplifies usage dramatically, now allowing to: * create the lattice (monitor elements) before the `sim` is even initialized * creating a monitor element and not using it in a lattice * CMake `pip_install`: Package Examples (BLAST-ImpactX#1179) Ensure the `examples/` directory is part of the `pip` wheels we build, e.g., on `cmake --build build -j 6 --target pip_install` * Envelope: Silence Warning w/o Space-Charge (BLAST-ImpactX#1173) Consistent with the other two tracking modes, we only print options upfront and do not warn every simulation step if space charge is not used. * Release 25.10 (BLAST-ImpactX#1180) * Update Stub Files * AMReX/ABLASTR/WarpX: development (BLAST-ImpactX#1177) * ABLASTR/WarpX: development Update the ABLASTR dependency to the latest development branch. * pyAMReX: `development` * Python: Improve `KnownElementsList` (BLAST-ImpactX#1181) Add per-element by-reference access for manipulation. * Update Stub Files * Element Selection Syntax (BLAST-ImpactX#1182) * Implementation 5hr vibe pair coded in Cursor. Maybe should have just done it xD * Doc * Fix Type Hints * `select`: Fix All-Selected Path Avoid implicit return of `None` * Cleanup * Update Stub Files * Fix Python import errors (BLAST-ImpactX#1186) * Update Stub Files * Fix RFCavity edge case (BLAST-ImpactX#1185) * Fix RFCavity edge case. * Remove extra semicolon. * Fix el typing (BLAST-ImpactX#1190) * Fix `KnownElementsList` Stubs Avoid importing `typing` for type hints, because pybind11-stubgen forgets the import when generating `pyi` files... * Update Stubs * Update PALS to 0.2.0 (BLAST-ImpactX#1184) * Update PALS version * Fix missing imports in KnownElementsList.pyi * Fix FODO example script * Revert changes to stub file * fix some math issues in examle documentation (BLAST-ImpactX#1191) * "Starting step" Print: Spaces (BLAST-ImpactX#1187) - newline before new step - no space before `++++ Starting step` * Element Names: All, `None` (BLAST-ImpactX#1189) It is easier for user-facing workflows if `.name` exists for all elements, even `Empty`. In Python, instead of throwing an exception we return the `None` value for elements without a user-provided name, which simplifies user-logic. In this case `.has_name` is `False`. * Update Stub Files --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Axel Huebl <[email protected]> Co-authored-by: Ji Qiang <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: ax3l <[email protected]> Co-authored-by: Edoardo Zoni <[email protected]> Co-authored-by: Richard Pausch <[email protected]>

cemitch99 · 2025-10-17T22:12:37Z

Note: Something is still not quite right.

cemitch99 · 2025-10-20T19:13:48Z

@WeiqunZhang and @ax3l The test mentioned earlier today for this PR is located in: examples/expanding_beam/

C++ input file: input_expanding_fft_2D.in
Python input file: run_expanding_fft_2D.py

I have added a script for plotting the x-y beam size evolution called plot_expanding_fft_2D.py.

If the test works, the beam size at the end (at the maximum value of $s$ shown in the plot) should be 1 mm.

To see the convergence issue, I suggest running the Python test with npart = 1M for a few values of sim.blocking_factor_x and sim.blocking_factor_y. You can see the effect this has on the beam moments by running the plot script in each case.

for more information, see https://pre-commit.ci

cemitch99 · 2025-10-20T21:18:53Z

@WeiqunZhang and @ax3l Note: I tried the same test using 3D space charge (using examples/expanding_beam/run_expanding_fft.py after turning off mesh refinement), and this issue does not occur.

cemitch99 · 2025-10-30T22:22:41Z

@WeiqunZhang and @ax3l Here is the latest update: If I run the app example "input_expanding_fft_2D.in", then I always see that 1 grid is used, independently of the choice of blocking factor ("1 grids 1024 cells"), and the results are consistent:

However, if I run the corresponding Python example "run_expanding_fft_2D.py", then as I change the blocking factor, the number of grids varies (bf = 32, grids = 1; bf = 16, grids = 4; bf = 8, grids = 16; bf = 4, grids = 64; bf = 2, grids = 256). The results are consistent for bf >= 8, but disagree for smaller bf. Here, the number of cells is [32,32,1] in each case.

This issue has now been fixed. Thanks @WeiqunZhang !

Co-authored-by: Axel Huebl <[email protected]>

src/particles/spacecharge/PoissonSolve.cpp

cemitch99 · 2025-11-05T19:16:18Z

For this PR, we need to resolve the following error resulting in failing tests:

amrex::Abort::1::Particle container needs to have at least one grid. !!!
SIGABRT

Grids Summary:
Level 0 1 grids 1024 cells 100 % of domain

ax3l added the component: space charge Space charge & potential solver label Mar 19, 2025

ax3l added this to the Advanced Methods (SciDAC-5) milestone Mar 19, 2025

ax3l requested a review from WeiqunZhang March 19, 2025 22:38

ax3l added the tracking: particles label Mar 19, 2025

ax3l commented Mar 19, 2025

View reviewed changes

ax3l force-pushed the topic-space-charge-particles-2D branch 2 times, most recently from de299e6 to 7b53d47 Compare July 15, 2025 18:53

ax3l requested a review from cemitch99 July 15, 2025 18:56

github-advanced-security bot found potential problems Jul 15, 2025

View reviewed changes

src/particles/spacecharge/GatherAndPush.cpp Fixed Show fixed Hide fixed

src/particles/spacecharge/GatherAndPush.cpp Fixed Show fixed Hide fixed

ax3l force-pushed the topic-space-charge-particles-2D branch from 7b53d47 to 79b5d1c Compare August 28, 2025 21:19

github-advanced-security bot found potential problems Aug 28, 2025

View reviewed changes

src/particles/spacecharge/ForceFromSelfFields.cpp Fixed Show fixed Hide fixed

src/particles/spacecharge/ForceFromSelfFields.cpp Fixed Show fixed Hide fixed

src/particles/spacecharge/ForceFromSelfFields.cpp Fixed Show fixed Hide fixed

ax3l mentioned this pull request Aug 28, 2025

FieldGather: Enable Lower D BLAST-WarpX/warpx#6127

Merged

ax3l changed the title ~~[WIP] 2.5D Space Charge in Particle Tracking~~ [WIP] 2D Space Charge in Particle Tracking Aug 28, 2025

ax3l commented Aug 28, 2025

View reviewed changes

src/particles/spacecharge/GatherAndPush.cpp Show resolved Hide resolved

cemitch99 changed the title ~~[WIP] 2D Space Charge in Particle Tracking~~ 2D Space Charge in Particle Tracking Aug 28, 2025

cemitch99 reviewed Sep 9, 2025

View reviewed changes

src/particles/spacecharge/GatherAndPush.cpp Outdated Show resolved Hide resolved

cemitch99 reviewed Sep 9, 2025

View reviewed changes

cemitch99 reviewed Sep 12, 2025

View reviewed changes

src/particles/spacecharge/ForceFromSelfFields.cpp Show resolved Hide resolved

cemitch99 reviewed Sep 12, 2025

View reviewed changes

src/particles/spacecharge/ForceFromSelfFields.cpp Show resolved Hide resolved

ax3l mentioned this pull request Sep 15, 2025

ABLASTR/WarpX: development #1143

Merged

1 task

cemitch99 reviewed Sep 16, 2025

View reviewed changes

github-advanced-security bot found potential problems Sep 17, 2025

View reviewed changes

tests/python/test_charge_deposition_2D.py Fixed Show fixed Hide fixed

ax3l and others added 5 commits September 23, 2025 15:28

2D: ForceFromSelfFields

ca47387

[Draft] GatherAndPush

b02e732

3D Force & Push

273c812

Keep simple for now.

2D Poisson Solve: Flattened Rho

ee8485d

Phi is then blown up again to 3D with indentical values in z. Co-authored-by: Weiqun Zhang <[email protected]>

2D Force & Push

5478a5b

ax3l commented Oct 16, 2025

View reviewed changes

examples/CMakeLists.txt Outdated Show resolved Hide resolved

cemitch99 and others added 5 commits October 16, 2025 13:41

Update examples/CMakeLists.txt

8da14c6

Co-authored-by: Axel Huebl <[email protected]>

Relax tolerance in analysis_expanding_fft_2D.py

ba7764e

Relax tolerance in analysis_fodo_2d_sc.py

376ebee

Relax tolerance for analysis_expanding_fft_2D.py

1926720

Relax tolerance in analysis_fodo_2d_sc.py

7db9d76

cemitch99 reviewed Oct 16, 2025

View reviewed changes

examples/fodo_space_charge/analysis_fodo_2d_sc.py Outdated Show resolved Hide resolved

cemitch99 added 3 commits October 16, 2025 15:20

Relax analysis_fodo_2d_sc.py tolerance again.

5e0bce6

Relax tolerance in analysis_fodo_2d_sc.py

20ff802

Relax tolerance in analysis_fodo_2d_sc.py

bbbbb8d

ax3l and others added 3 commits October 17, 2025 19:29

Same Grid, Same Mass

6859b3d

Same grid, same mass for FODO.

954251d

Add plotting script.

8d77c8e

pre-commit-ci bot and others added 2 commits October 20, 2025 20:24

[pre-commit.ci] auto fixes from pre-commit.com hooks

9d2b3fc

for more information, see https://pre-commit.ci

Add Weiqun's fix for duplicate values on shared nodes.

e8d4e5b

Merge branch 'development' into topic-space-charge-particles-2D

f7b6e14

Doc: NAPAC25 Preprint Link Text (#10)

0b4c71f

Co-authored-by: Axel Huebl <[email protected]>

WeiqunZhang reviewed Oct 31, 2025

View reviewed changes

src/particles/spacecharge/PoissonSolve.cpp Outdated Show resolved Hide resolved

WeiqunZhang reviewed Oct 31, 2025

View reviewed changes

src/particles/spacecharge/PoissonSolve.cpp Show resolved Hide resolved

WeiqunZhang added 3 commits November 3, 2025 09:50

Build phi and force with flatten boxes in True_2D case

864b8fa

Update PoissonSolve for the new layout of phi and rho

033dadb

Fix index type

d95f987

		px += field_interp[0] * push_consts * beta_gamma * dr[2] / (c0_SI);
		py += field_interp[1] * push_consts * beta_gamma * dr[2] / (c0_SI); //this should be field_interp[1]

2D Space Charge in Particle Tracking #909

Are you sure you want to change the base?

2D Space Charge in Particle Tracking #909

Uh oh!

Conversation

ax3l commented Mar 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

To Do

Follow-up

Uh oh!

ax3l Mar 19, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cemitch99 commented Sep 6, 2025

Uh oh!

Uh oh!

cemitch99 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

cemitch99 commented Sep 12, 2025 • edited by ax3l Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cemitch99 Sep 16, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ax3l Oct 16, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

cemitch99 commented Oct 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cemitch99 commented Oct 17, 2025

Uh oh!

cemitch99 commented Oct 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cemitch99 commented Oct 20, 2025

Uh oh!

cemitch99 commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cemitch99 commented Nov 5, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ax3l commented Mar 19, 2025 •

edited

Loading

cemitch99 commented Sep 12, 2025 •

edited by ax3l

Loading

cemitch99 commented Oct 17, 2025 •

edited

Loading

cemitch99 commented Oct 20, 2025 •

edited

Loading

cemitch99 commented Oct 30, 2025 •

edited

Loading