Conduit Device Support by JustinPrivitera · Pull Request #1358 · llnl/conduit

JustinPrivitera · 2025-01-24T00:30:36Z

Resolves #1151

See #1614 for ongoing development feature-tracking

JustinPrivitera · 2026-08-27T17:48:38Z

+#if defined(_WIN32) && defined(CONDUIT_WINDOWS_DLL_EXPORTS) && \
+    !defined(CONDUIT_EXPORTS_DEFINED) && \
+    !defined(CONDUIT_TU_IS_CUDA) && !defined(CONDUIT_TU_IS_HIP)
+


we should use special CONDUIT WINDOWS macro that exists somewhere

This is a gaping wound for @JustinPrivitera to figure out. Can we move things back to the cpp?

https://github.com/llnl/conduit/blob/develop/src/libs/conduit/conduit_exports.h.in

This is related to the discussion about RDC

JustinPrivitera · 2026-08-27T17:56:30Z

TODO update changelog

JustinPrivitera · 2026-08-28T00:34:03Z

TODO all transforms should provide a node/accessor to global execution options to get the right exec policy/output space.

JustinPrivitera · 2026-09-02T04:36:41Z

TODO address this concern: #1673 (comment)

* Initial attempt at porting the new bm infra to device support branch * Avoid a segfault by restricting generate_* to host only * Initial commit * Update plots * Cleanup * Revert sync change, make tests better * More optimization * Differentiate backends when plotting * Make host_only a per-benchmark option * Don't include deallocation in timing * Rename take_data_ptr to assume_data_ptr * Fix timing leak * Don't copy into output buffers * Initial commit * Mirror changes in the pre-DE version of this PR * Address feedback * Make many improvements * Caliper tutorial tweaks * Initial commit * Greatly simplify the API * Add TODOs * Add another TODO * Make comments better * Cleanup * Add better comment * Exploratory notes * Remove resolved TODOs * Remove a case that didn't make sense * Finish porting * Support DataArrays * Polishing, better tests * Update src/libs/conduit/conduit_execution_dispatch.hpp Co-authored-by: Justin Privitera <35237779+JustinPrivitera@users.noreply.github.com> * Address feedback * Improve verbiage * More polishing * Use dispatches, some cleanup * Always dispatch the non-polyhedral case * Reorder things in anticipation of more overloads being added * Redo unique_mask * Polishing * Add info to docs * Remove TODOs * Cleanup duplicated TODOs * Remove LUTs * Shorten header * Update src/libs/blueprint/conduit_blueprint_mesh.cpp Co-authored-by: Justin Privitera <35237779+JustinPrivitera@users.noreply.github.com> * Remove maps * Exercise other meshes and shape types * Use correct policy querying * Remove what I can * Polish * Make centroids test stricter --------- Co-authored-by: Justin Privitera <35237779+JustinPrivitera@users.noreply.github.com>

siramok · 2026-09-02T18:07:49Z

MSVC was throwing this compilation error for C++17:

D:\a\conduit\conduit\src\libs\blueprint\conduit_blueprint_mesh.cpp(1876,36): error C2131: expression did not evaluate to a constant

The code in question:

const index_t max_stack_npts = 32;
conduit::execution::forall(policy, 0, topo_num_elems, [=] CONDUIT_EXEC(index_t ei)
{
    ...
    index_t elem_coord_indices[max_stack_npts];

It seems that declaring max_stack_npts as const was not sufficient for MSVC, because the forall captures it by value [=]. Initializing stack arrays with a non-const expression is not allowed, but the Linux compilers are smart enough to know that our usage is valid, hence the error on Windows only.

The simple (smallest diff) way to fix this was to make max_stack_npts a compile-time constant using constexpr:

constexpr index_t max_stack_npts = 32;

It should also work to just move max_stack_npts into the forall body itself (which I will try next if this didn't fix CI).

siramok · 2026-09-02T18:14:53Z

constexpr did not save us, so I have moved max_stack_npts into the forall body. Hopefully that does the trick.

JustinPrivitera · 2026-09-02T21:01:03Z

constexpr did not save us, so I have moved max_stack_npts into the forall body. Hopefully that does the trick.

Thanks for fixing this. We also don't have to lambda capture by value (I think???). Maybe we could try that as well.

siramok · 2026-09-03T01:04:10Z

We also don't have to lambda capture by value

I looked into it very briefly at one point, and my takeaway was that foralls would need to capture by value if they would ever be executed on device (host-only kernels didn't have to worry). I don't actually know if that's true.

But now that I have unified memory on the brain, I do wonder if 1) our kernels could capture by reference in unified memory and 2) if there's a kernel anywhere that would benefit from that.

* Initial attempt at porting the new bm infra to device support branch * Avoid a segfault by restricting generate_* to host only * Initial commit * Update plots * Cleanup * Revert sync change, make tests better * More optimization * Differentiate backends when plotting * Make host_only a per-benchmark option * Don't include deallocation in timing * Rename take_data_ptr to assume_data_ptr * Fix timing leak * Don't copy into output buffers * Initial commit * Mirror changes in the pre-DE version of this PR * Address feedback * Make many improvements * Caliper tutorial tweaks * Greatly simplify the API * Add TODOs * Add another TODO * Make comments better * Cleanup * Add better comment * Initial commit * Better naming consistency * Remove resolved TODOs * Remove a case that didn't make sense * Support DataArrays * Polishing, better tests * Port DataArray and DataAccessor to execution model * Update src/libs/conduit/conduit_execution_dispatch.hpp Co-authored-by: Justin Privitera <35237779+JustinPrivitera@users.noreply.github.com> * Address feedback * Improve verbiage * More polishing * Reorder things in anticipation of more overloads being added * Polishing * Address feedback * Add info to docs * Lazily cache active space * Make set_value_helper private again * Cache the execution policy instead of the memory space * Support magic sets * Add test comments * Address feedback * Add temp buffer comment --------- Co-authored-by: Justin Privitera <35237779+JustinPrivitera@users.noreply.github.com>

siramok · 2026-09-03T01:52:24Z

Speaking of unified memory and whether we're handling it correctly, here's a link to my initial investigation on it: #1614 (comment)

siramok · 2026-09-03T17:27:51Z

This job was abandoned. We have detected that logs from the agent may have not finished uploading. We have included our in-memory record of all log lines uploaded before we lost contact with the agent

CI failed after 90 minutes exactly, I wonder if we exceeding a time limit?

siramok · 2026-09-03T17:30:18Z

For context, the PR before this most recent one (which does increase compile times) finished in 1h 0m 2s.

….com:LLNL/conduit into task/JustinPrivitera/05_01_24/device_support

siramok · 2026-09-03T17:38:16Z

Oh jeez, our in progress check was running for 14 hours...

I have restarted CI. If it fails again at 90 min then I think this issue may be above my pay grade.

JustinPrivitera · 2026-09-03T20:09:25Z

This is really strange.

siramok · 2026-09-03T20:11:32Z

Do you have a button to manually stop CI? I suspect that this in progress check will continue running in perpetuity.

siramok · 2026-09-03T20:15:10Z

[error] We stopped hearing from agent Azure Pipelines 5. Verify the agent machine is running and has a healthy network connection. Anything that terminates an agent process, starves it for CPU, or blocks its network access can cause this error. For more information, see: https://go.microsoft.com/fwlink/?linkid=846610

Since it happened almost exactly at 90 minutes again, this does feel like a timeout.

siramok · 2026-09-04T00:54:39Z

Cyrus says we should be able to run >90 minutes without an issue. The running out of space theory is looking plausible:

[warning] Free disk space on / is lower than 5%; Currently used: 98.11%

#1691 should alleviate this problem. Alternatively, I don't know if it's possible to increase the amount of space for the static build containers specifically (without adding more GH actions cost)?

Another thought: maybe rm -rf on the build folders after we install all of the tpls/deps would claw back enough space to enable conduit to finish installing? Although, if this is just executing ./build_conduit.sh then maybe that's not a viable solution.

siramok · 2026-09-04T22:49:52Z

~~If we don't want to merge #1692 today, that's okay, but would someone with power mind stopping this endless CI run?~~

Merged!

…1692) * Initial commit * Whoops, forgot build_conduit.sh

JustinPrivitera added 30 commits May 1, 2024 16:54

I'm really thinking about GPUs this time

864e52f

oops

ab961e3

start

cede461

I know what I'm doing

b3b5246

adasdas

8343fdc

thanks cyrus

1950d8a

trying to get it to build

28bed36

COMPILE!

c2c37c5

maybe we allow non raja open mp

921695e

these changes are from 2 weeks ago

2a6c51b

have more mystery changes

56e72a7

Merge branch 'develop' into task/JustinPrivitera/05_01_24/device_support

c86066a

myth busted

ca41133

help what do i do next

ea27d0d

sigh

08eb646

new source files

a0c64f7

exec array src

a7b2191

I did it I made GPUs work

e6d04c0

It works, it just needs to compile

df1e13d

correction to sync and replace

d8fd62c

im really dong it

02d74a1

Merge branch 'develop' into task/JustinPrivitera/05_01_24/device_support

44572a3

starting to add the screte sauce

50b591d

thx cyrus

3325854

replace goes away

4757552

new world of node linked accessor

1b00fd3

working through use with cases

73f3283

use with

0c5aef2

im doing data array

1190e41

have a header

2c81fd1

Remove the pointless newline added

5091fae

This comment was marked as outdated.

Sign in to view

JustinPrivitera commented Aug 27, 2026

View reviewed changes

This comment was marked as outdated.

Sign in to view

Simplify the Windows CI fix (#1688)

0e0e2e0

This comment was marked as outdated.

Sign in to view

siramok and others added 2 commits September 2, 2026 09:27

Make max_stack_npts constexpr

a96c0a4

Move max_stack_npts into forall body

c3bb2e7

siramok added 2 commits September 3, 2026 10:36

Restart CI

fbf94b1

Merge branch 'task/JustinPrivitera/05_01_24/device_support' of github…

845452a

….com:LLNL/conduit into task/JustinPrivitera/05_01_24/device_support

Add a build option to control compilation of typed dispatch kernels (#…

39bb138

…1692) * Initial commit * Whoops, forgot build_conduit.sh

Uh oh!

Conversation

JustinPrivitera commented Jan 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as outdated.

This comment was marked as outdated.

JustinPrivitera Aug 27, 2026

Choose a reason for hiding this comment

Uh oh!

JustinPrivitera Aug 27, 2026

Choose a reason for hiding this comment

Uh oh!

JustinPrivitera Sep 1, 2026

Choose a reason for hiding this comment

Uh oh!

JustinPrivitera commented Aug 27, 2026

Uh oh!

JustinPrivitera commented Aug 28, 2026

Uh oh!

This comment was marked as outdated.

This comment was marked as outdated.

JustinPrivitera commented Sep 2, 2026

Uh oh!

siramok commented Sep 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

siramok commented Sep 2, 2026

Uh oh!

JustinPrivitera commented Sep 2, 2026

Uh oh!

siramok commented Sep 3, 2026

Uh oh!

siramok commented Sep 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

siramok commented Sep 3, 2026

Uh oh!

siramok commented Sep 3, 2026

Uh oh!

siramok commented Sep 3, 2026

Uh oh!

JustinPrivitera commented Sep 3, 2026

Uh oh!

siramok commented Sep 3, 2026

Uh oh!

siramok commented Sep 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

siramok commented Sep 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

siramok commented Sep 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

JustinPrivitera commented Jan 24, 2025 •

edited

Loading

siramok commented Sep 2, 2026 •

edited

Loading

siramok commented Sep 3, 2026 •

edited

Loading

siramok commented Sep 3, 2026 •

edited

Loading

siramok commented Sep 4, 2026 •

edited

Loading

siramok commented Sep 4, 2026 •

edited

Loading