You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
The self-profiler in #10870 needs automated testing to ensure it keeps working. However this cannot be done as a simple integration test since it requires special startup-only configs.
Describe the solution you'd like
Add an explicit section to the spark-tests script that will launch sample self-profiling sessions. The tests should minimally startup the Spark session with the profiling path prefix config pointing to a filesystem (distributed would be best, but local could be used if no distributed filesystem is available/configured in the test environment). A simple, sample query can be run in the Spark session, and the test script can verify a profile is created at the expected destination. It can then further verify the profile data by using the profile converter in NVIDIA/spark-rapids-jni#2066 to convert the data to JSON and grep the results for lkey profiling landmarks, e.g.: kernel names, Acquire GPU or other common NVTX ranges, etc.
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
The self-profiler in #10870 needs automated testing to ensure it keeps working. However this cannot be done as a simple integration test since it requires special startup-only configs.
Describe the solution you'd like
Add an explicit section to the spark-tests script that will launch sample self-profiling sessions. The tests should minimally startup the Spark session with the profiling path prefix config pointing to a filesystem (distributed would be best, but local could be used if no distributed filesystem is available/configured in the test environment). A simple, sample query can be run in the Spark session, and the test script can verify a profile is created at the expected destination. It can then further verify the profile data by using the profile converter in NVIDIA/spark-rapids-jni#2066 to convert the data to JSON and grep the results for lkey profiling landmarks, e.g.: kernel names, Acquire GPU or other common NVTX ranges, etc.
The text was updated successfully, but these errors were encountered: