[FEA]: cuda.parallel - forbid (or handle) non-contiguous arrays as inputs to algorithms

[shwina]

Is this a duplicate?

• I confirmed there appear to be no duplicate issues for this request and that I agree to the Code of Conduct

Area

cuda.parallel (Python)

Is your feature request related to a problem? Please describe.

See #3216 (comment).

Describe the solution you'd like

See #3216 (comment).

Describe alternatives you've considered

No response

Additional context

No response

[shwina]

Here are some potential solutions:

1. We raise when the input array has a non-zero stride.
   • It can be the user's responsibility to construct an iterator to describe strided input.
   • In this case, we should consider whether to provide a special-purpose iterator StridedIterator (alternately, the user would have to construct a TransformIterator themselves)
2. We don't raise - instead we translate the array with non-zero stride into an iterator on the user's behalf.
3. We change the API so as to not accept arrays directly at all, instead accepting RawPointer which is necessarily a pointer to contiguous device memory.

My personal preference is for (1) or (3) with a slight preference for (1).

[leofang]

We should raise for now since we clearly can't handle it today. A strided iterator would be cool to have in the future, but it could have performance concerns. By the time we start working on CuPy integration, we should revisit this subject and see if/what we could do better. A naive copy to a contiguous buffer could also be considered in certain cases.