Best practices for using intel optimizations for tensorflow

[myoung3]

It would be great if we could get some documentation for using the intel optimizations for tensorflow in R.

The first step in getting this working is to get reticulate to point to a conda environment that has the intel ai toolkit (and corresponding versions of tensorflow installed), but I believe that's not sufficient. How might we change, from R, the settings described (here, here, here, and here)?

[myoung3]

to get the ball rolling here, I've done some testing and found a good configuration to get speedups on a high-core machine. I did testing on my 64-virtual core/32-physical core machine using a singularity image hosted here and found that the following settings were a major (~10x) improvement over the defaults (not setting anything related to core counts).

Note this image contains the tensorflow package build for avx512. I don't know how much an improvement using this special version of tensorflow makes because I didn't test the standard version of tensorflow.

If you have singularity installed and compatable intel hardware you can test this yourself. If you get prompted to install miniconda just say no. all the packages are pre-installed and the image isn't writable in any case.

singularity shell oras://ghcr.io/kaufman-lab/geospatial_plus_ml:4.1.0
R
library(tensorflow)
library(keras)

#count physical cores:
  #this count of physical cores assumes hyperthreading is enabled:
n_physical <-  as.integer(parallel::detectCores()/2L )
n <- as.integer(.5*n_physical) #in my testing this was faster than n=n_physical

reticulate::py_run_string('import os')
reticulate::py_run_string(paste0('os.environ["OMP_NUM_THREADS"] = str(',n,')'))
tf$config.threading$set_inter_op_parallelism_threads(2L)
tf$config.threading$set_intra_op_parallelism_threads(n_physical)
k_set_image_data_format('channels_first') #this doesn't do anything here because there's no image data.


mnist <- dataset_mnist()
mnist$train$x <- mnist$train$x/255
mnist$test$x <- mnist$test$x/255

model <- keras_model_sequential() %>% 
  layer_flatten(input_shape = c(28, 28)) %>% 
  layer_dense(units = 128, activation = "relu") %>% 
  layer_dropout(0.2) %>% 
  layer_dense(10, activation = "softmax")
summary(model)

model %>% 
  compile(
    loss = "sparse_categorical_crossentropy",
    optimizer = "adam",
    metrics = "accuracy"
  )

model %>% 
  fit(
    x = mnist$train$x, y = mnist$train$y,
    epochs = 5,
    validation_split = 0.3,
    verbose = 2
  )

[myoung3]

I'm biased here but I think prioritizing improving your documentation on optimizing for CPUs and CPU-basedHPC environments is a worthwhile endeavor for this project . The subset of people interested in tensorflow who use R leans more towards academia than the normal python tensorflow community, and there are typically a ton of CPU-based HPC resources on campuses.

Just as important as figuring out how to optimize speed on a single node is figuring out and documenting using distributed training in a multi-node context. Hopefully I can make some progress on that front and I'll update as I learn more. Any help is appreciated.