[WIP] Compare and understand TorchSharp Memory Management through PyTorch's Approach

[GeorgeS2019]

PyTorch Memory Management

• A comprehensive guide to memory usage in PyTorch

• SIMPLIFYING PYTORCH MEMORY MANAGEMENT WITH TENSORDICT

• How to avoid "CUDA out of memory" in PyTorch

• Why do I get CUDA out of memory when running PyTorch model [with enough GPU memory]?

TorchSharp Memory Management

Memory Management

Discussions

• How can I limit the usage of cuda memory to avoid CUDA out of memory

Then, surround your train and test logic with a dispose scope, preferably so that temporaries are disposed of once every batch, but definitely once every epoch. See CIFAR10 Train and CIFAR10 Test for examples on how to do this.