How can I find where Thrust sets the parameters for threads_per_block, the number of blocks, and the grid size when calling Thrust functions?

[Wilson1211]

Many open-source projects use Thrust as their low-level implementation. How can I locate the part of the code that determines threads_per_block, as well as the number of blocks and grids? I know that ncu can show the value of threads_per_block at runtime, but I would like to understand how Thrust sets these parameters in the code.

[pauleonix]

Thrust is generally using CUB for its CUDA backend (that does not mean that every Thrust algorithm has a corresponding CUB device primitive since some Thrust algorithms are implemented in terms of other Thrust algorithms). CUB in turn has so-called policies that determine the block size and other tuning parameters based on architecture that it is being compiled for (therefore it is important to compile for the right architecture instead of relying on forward compatibility via PTX). These values come from a tuning process where we try to find optimal parameters for each architecture by running benchmarks. You can find the tuned policies in cub/cub/device/dispatch/tuning.

This is all somewhat involved and the point of the Thrust and CUB abstractions is that you don't need to worry about these parameters (your code should not depend on these implementation details). If you have a use-case that currently is underperforming you can let us know via an issue so that we can consider incorporating it in our tuning process. Since it is impossible for us to tune for every use-case there could be, afaik there is also a long-term goal to make the tuning and policy infrastructure available to the users (for CUB device primitives, not the higher level Thrust algorithms) at some point.

[bernhardmgruber]

As @pauleonix already explained, most Thrust algorithms use CUB. CUB has its host side setup in the dispatch layer, which are files inside cub/device/dispatch/*.cuh. Concretely, look for calls like:

 THRUST_NS_QUALIFIER::cuda_cub::detail::triple_chevron(grid_size, block_size, 0, stream) .doit(kernel_func, arguments...);

or

launcher_factory(...).doit(...);

with basically the same arguments.