[Cache Offload] Remove device sync overhead

[Edenzzzz]

Motivation

Device synchronization is expensive. In most cases we can use event or stream sync. https://developer.download.nvidia.com/compute/DevZone/docs/html/C/doc/html/group__CUDART__EVENT_g14c387cc57ce2e328f6669854e6020a5.html

Modifications

1. Replace device sync with event sync, which only blocks a single stream.
2. Removed device sync after tensor.to(). It should block the current stream until completion. I ran the benchmark and didn't encounter any illegal memory access issues. If you want to use another stream to write through in the background, we will also need pin_memory()?
   cc @xiezhq-hermann , would appreciate if you could share a bug reproducer for that.

Checklist

• Format your code according to the Contributor Guide.
• Add unit tests as outlined in the Contributor Guide.
• Update documentation as needed, including docstrings or example tutorials.