Improve dev/Dockerfile

[kevinjqliu]

Feature Request / Improvement

We had issues with downloading spark for the integration test docker image.

• Using archive.apache.org may be unstable sometimes (docker: The archive seems unstable #1425)
• Using dlcdn.apache.org breaks the script and force us to upgrade whenever there's a new release ([infra] download Spark from archive.apache.org #1523)

The issue comes from the need to download spark from an URL.

Since dev/Dockerfile is building a spark docker image with some extra configs.
I was wondering if we can use the dockerhub spark image as the base and configure it.

[kevinjqliu]

possibly reuse for rust as well apache/iceberg-rust#894 (comment)