Improve AutoTuner cluster configuration recommendations for GPU runs

[parthosa]

Fixes #1121
Fixes #1334

Summary

Introduces a new strategy-based approach for recommending GPU cluster configurations (num executors, core count and num worker nodes (CSPs only)).

Key Changes

• Standardized executor core count to 16 cores per executor for GPU runs.
• Shifted from multi-GPU instances to smaller, single-GPU instances
• Implemented two configuration strategies:
  1. Cluster Property Strategy: Generates recommendations based on user-specified cluster properties
  2. Event Log Strategy: Generates recommendations based on event logs

Reasoning

• Analysis of NDS performance metrics shows 16 cores/executor provides optimal performance
• Configurations with 4 or 64 cores/executor showed reduced performance and cost efficiency
• Smaller instances with distributed GPUs improve:
  • System fault tolerance
  • I/O performance (both disk and network)
  • Resource utilization

Code Changes

Cluster configuration strategy:

• core/src/main/scala/com/nvidia/spark/rapids/tool/ClusterConfigurationStrategy.scala: Added new classes ClusterConfigurationStrategy, ClusterPropertyBasedStrategy, and EventLogBasedStrategy to encapsulate different cluster configuration strategies.
• core/src/main/scala/com/nvidia/spark/rapids/tool/Platform.scala: Refactored the Platform class to use the new strategy classes for recommending cluster configurations and removed redundant methods. [1] [2] [3]

Enhancements to instance information handling:

• core/src/main/scala/com/nvidia/spark/rapids/tool/Platform.scala: Added getMemoryPerExec method to InstanceInfo and created a companion object for default instance creation.
• core/src/main/scala/com/nvidia/spark/rapids/tool/Platform.scala: Updated DatabricksAwsPlatform to use getInstanceByResourcesMap for instance type mapping.

Default recommendations and properties:

• core/src/main/scala/com/nvidia/spark/rapids/tool/Platform.scala: Added default recommendations for cores per executor and GPUs per node.

[parthosa]

WIP: AutoTuner tests need to be updated with the new logic.

[amahussein]

LGTME
Thanks @parthosa

[amahussein]

Thanks @parthosa
LGTME