forked from NOAA-GFDL/MOM6
-
Notifications
You must be signed in to change notification settings - Fork 0
Re-factor of MOM_ANN #3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 5 commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
5b8212a
Moved MOM_ANN.F90 to src/framework/
adcroft f87c65a
Minor refactor of MOM_ANN
adcroft c4984c4
Adds unit tests and timing test to MOM_ANN
adcroft ea1cc31
Adding multiple forms of inference
adcroft 4e98437
Renamed ANN variants and added some module documentation
adcroft da17218
Removed alternative variants of ANN in favor of optimized
adcroft File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,232 @@ | ||
| program time_MOM_ANN | ||
|
|
||
| ! This file is part of MOM6. See LICENSE.md for the license. | ||
|
|
||
| use MOM_ANN, only : ANN_CS | ||
| use MOM_ANN, only : ANN_allocate, ANN_apply, ANN_end | ||
| use MOM_ANN, only : ANN_apply_vector_orig, ANN_apply_vector_oi, ANN_apply_vector_io | ||
| use MOM_ANN, only : ANN_apply_vector_oit | ||
| use MOM_ANN, only : ANN_apply_array_soi, ANN_apply_array_sio, ANN_apply_array_ois | ||
| use MOM_ANN, only : ANN_random | ||
|
|
||
| implicit none | ||
|
|
||
| ! Command line options | ||
| integer :: nargs ! Number of command line arguments | ||
| character(len=12) :: cmd_ln_arg !< Command line argument (if any) | ||
|
|
||
| ! ANN parameters | ||
| integer :: nlayers ! Number of layers | ||
| integer :: nin ! Number of inputs | ||
| integer :: layer_width ! Width of hidden layers | ||
| integer :: nout ! Number of outputs | ||
| ! Timing parameters | ||
| integer :: nsamp ! Number of measurements | ||
| integer :: nits ! Number of calls to time | ||
| integer :: nxy ! Spatial dimension | ||
|
|
||
| nlayers = 7; nin = 4; layer_width = 16; nout = 1 ! Deep network | ||
| !nlayers = 4; nin = 4; layer_width = 48; nout = 1 ! Shallow-wide network | ||
| !nlayers = 3; nin = 4; layer_width = 20; nout = 1 ! Small network | ||
|
|
||
| nsamp = 100 | ||
| nits = 20000 | ||
| !nits = 300000 ! Needed for robust measurements on small networks | ||
| nxy = 100 ! larger array | ||
| !nxy = 10 ! small array | ||
|
|
||
| ! Optionally grab ANN and timing parameters from the command line | ||
| nargs = command_argument_count() | ||
| if (nargs==7) then | ||
| call get_command_argument(1, cmd_ln_arg) | ||
| read(cmd_ln_arg,*) nlayers | ||
| call get_command_argument(2, cmd_ln_arg) | ||
| read(cmd_ln_arg,*) nin | ||
| call get_command_argument(3, cmd_ln_arg) | ||
| read(cmd_ln_arg,*) layer_width | ||
| call get_command_argument(4, cmd_ln_arg) | ||
| read(cmd_ln_arg,*) nout | ||
| call get_command_argument(5, cmd_ln_arg) | ||
| read(cmd_ln_arg,*) nsamp | ||
| call get_command_argument(6, cmd_ln_arg) | ||
| read(cmd_ln_arg,*) nits | ||
| call get_command_argument(7, cmd_ln_arg) | ||
| read(cmd_ln_arg,*) nxy | ||
| endif | ||
|
|
||
| ! Fastest variants on Intel Xeon W-2223 CPU @ 3.60GHz (gfortran-13.2 -O3) | ||
| ! | vector(nxy=1) | nxy = 10 | nxy = 100 | ||
| ! ---------------------------------------------------------------------------- | ||
| ! Small ANN | vector_oi | array_soi | array_sio | ||
| ! Shallow-wide ANN | vector_oi | array_ois | array_sio | ||
| ! Deep ANN | vector_oi | array_ois | array_sio | ||
|
|
||
| write(*,'(a)') "{" | ||
|
|
||
| call time_ANN(nlayers, nin, layer_width, nout, nsamp, nits, nxy, & | ||
| 0, "MOM_ANN:ANN_apply(vector)") | ||
| write(*,"(',')") | ||
| call time_ANN(nlayers, nin, layer_width, nout, nsamp, nits, nxy, & | ||
| 1, "MOM_ANN:ANN_apply_vector_orig(array)") | ||
| write(*,"(',')") | ||
| call time_ANN(nlayers, nin, layer_width, nout, nsamp, nits, nxy, & | ||
| 2, "MOM_ANN:ANN_apply_vector_oi(array)") | ||
| write(*,"(',')") | ||
| call time_ANN(nlayers, nin, layer_width, nout, nsamp, nits, nxy, & | ||
| 3, "MOM_ANN:ANN_apply_vector_io(array)") | ||
| write(*,"(',')") | ||
| call time_ANN(nlayers, nin, layer_width, nout, nsamp, nits, nxy, & | ||
| 4, "MOM_ANN:ANN_apply_vector_oit(array)") | ||
| write(*,"(',')") | ||
| call time_ANN(nlayers, nin, layer_width, nout, nsamp, nits, nxy, & | ||
| 11, "MOM_ANN:ANN_apply_array_soi(array)") | ||
| write(*,"(',')") | ||
| call time_ANN(nlayers, nin, layer_width, nout, nsamp, nits, nxy, & | ||
| 12, "MOM_ANN:ANN_apply_array_sio(array)") | ||
| write(*,"(',')") | ||
| call time_ANN(nlayers, nin, layer_width, nout, nsamp, nits, nxy, & | ||
| 13, "MOM_ANN:ANN_apply_array_ois(array)") | ||
| write(*,"()") | ||
|
|
||
| write(*,'(a)') "}" | ||
|
|
||
| contains | ||
|
|
||
| !> Time ANN inference. | ||
| !! | ||
| !! Times are measured over the "nits effective calls" and appropriately scaled to the | ||
| !! time per call per single vector of input features. For array inputs, the number of | ||
| !! actual calls is reduced by the size of the array. The timing measurement is repeated | ||
| !! "nsamp" times, to check the statistics of the timing measurement. | ||
| subroutine time_ANN(nlayers, nin, width, nout, nsamp, nits, nxy, impl, label) | ||
| integer, intent(in) :: nlayers !< Number of layers | ||
| integer, intent(in) :: nin !< Number of inputs | ||
| integer, intent(in) :: width !< Width of hidden layers | ||
| integer, intent(in) :: nout !< Number of outputs | ||
| integer, intent(in) :: nsamp !< Number of measurements | ||
| integer, intent(in) :: nits !< Number of calls to time | ||
| integer, intent(in) :: nxy !< Spatial dimension | ||
| integer, intent(in) :: impl !< Implementation to time | ||
| character(len=*), intent(in) :: label !< Label for YAML output | ||
| ! Local variables | ||
| type(ANN_CS) :: ANN ! ANN | ||
| integer :: widths(nlayers) ! Width of each layer | ||
| real :: x_s(nin) ! Inputs (just features) [nondim] | ||
| real :: y_s(nin) ! Outputs (just features) [nondim] | ||
| real :: x_fs(nin,nxy) ! Inputs (feature, space) [nondim] | ||
| real :: y_fs(nin,nxy) ! Outputs (feature, space) [nondim] | ||
| real :: x_sf(nin,nxy) ! Inputs (space, feature) [nondim] | ||
| real :: y_sf(nin,nxy) ! Outputs (space, feature) [nondim] | ||
| integer :: iter, samp ! Loop counters | ||
| integer :: ij ! Horizontal loop index | ||
| real :: start, finish, timing ! CPU times [s] | ||
| real :: tmin, tmax, tmean, tstd ! Min, max, mean, and standard deviation, of CPU times [s] | ||
| integer :: asamp ! Actual samples of timings | ||
| integer :: aits ! Actual iterations | ||
| real :: words_per_sec ! Operations per sec estimated from parameters [# s-1] | ||
|
|
||
| widths(:) = width | ||
| widths(1) = nin | ||
| widths(nlayers) = nout | ||
|
|
||
| call ANN_random(ANN, nlayers, widths) | ||
| call random_number(x_fs) | ||
| call random_number(x_sf) | ||
|
|
||
|
|
||
| tmin = 1e9 | ||
| tmax = 0. | ||
| tmean = 0. | ||
| tstd = 0. | ||
| asamp = nits ! Most cases below use this | ||
| aits = nits / nxy ! Most cases below use this | ||
|
|
||
| do samp = 1, nsamp | ||
| select case (impl) | ||
| case (0) | ||
| aits = nits | ||
| call cpu_time(start) | ||
| do iter = 1, nits ! Make many passes to reduce sampling error | ||
| call ANN_apply(x_s, y_s, ANN) | ||
| enddo | ||
| call cpu_time(finish) | ||
| case (1) | ||
| call cpu_time(start) | ||
| do iter = 1, aits ! Make many passes to reduce sampling error | ||
| do ij = 1, nxy | ||
| call ANN_apply_vector_orig(x_fs(:,ij), y_fs(:,ij), ANN) | ||
| enddo | ||
| enddo | ||
| call cpu_time(finish) | ||
| case (2) | ||
| call cpu_time(start) | ||
| do iter = 1, aits ! Make many passes to reduce sampling error | ||
| do ij = 1, nxy | ||
| call ANN_apply_vector_oi(x_fs(:,ij), y_fs(:,ij), ANN) | ||
| enddo | ||
| enddo | ||
| call cpu_time(finish) | ||
| case (3) | ||
| call cpu_time(start) | ||
| do iter = 1, aits ! Make many passes to reduce sampling error | ||
| do ij = 1, nxy | ||
| call ANN_apply_vector_io(x_fs(:,ij), y_fs(:,ij), ANN) | ||
| enddo | ||
| enddo | ||
| call cpu_time(finish) | ||
| case (4) | ||
| call cpu_time(start) | ||
| do iter = 1, aits ! Make many passes to reduce sampling error | ||
| do ij = 1, nxy | ||
| call ANN_apply_vector_oit(x_fs(:,ij), y_fs(:,ij), ANN) | ||
| enddo | ||
| enddo | ||
| call cpu_time(finish) | ||
| case (11) | ||
| call cpu_time(start) | ||
| do iter = 1, aits ! Make many passes to reduce sampling error | ||
| call ANN_apply_array_soi(nxy, x_sf(:,:), y_sf(:,:), ANN) | ||
| enddo | ||
| call cpu_time(finish) | ||
| asamp = nsamp * aits ! Account for working on whole arrays | ||
| case (12) | ||
| call cpu_time(start) | ||
| do iter = 1, aits ! Make many passes to reduce sampling error | ||
| call ANN_apply_array_sio(nxy, x_sf(:,:), y_sf(:,:), ANN) | ||
| enddo | ||
| call cpu_time(finish) | ||
| asamp = nsamp * aits ! Account for working on whole arrays | ||
| case (13) | ||
| call cpu_time(start) | ||
| do iter = 1, aits ! Make many passes to reduce sampling error | ||
| call ANN_apply_array_ois(nxy, x_fs(:,:), y_fs(:,:), ANN) | ||
| enddo | ||
| call cpu_time(finish) | ||
| asamp = nsamp * aits ! Account for working on whole arrays | ||
| end select | ||
|
|
||
| timing = ( finish - start ) / real(nits) ! Average time per call | ||
|
|
||
| tmin = min( tmin, timing ) | ||
| tmax = max( tmax, timing ) | ||
| tmean = tmean + timing | ||
| tstd = tstd + timing**2 | ||
| enddo | ||
|
|
||
| tmean = tmean / real(nsamp) | ||
| tstd = tstd / real(nsamp) ! convert to mean of squares | ||
| tstd = tstd - tmean**2 ! convert to variance | ||
| tstd = sqrt( tstd * real(nsamp) / real(nsamp-1) ) ! convert to standard deviation | ||
| words_per_sec = ANN%parameters / ( tmean * 1024 * 1024 ) | ||
|
|
||
| write(*,"(2x,3a)") '"', trim(label), '": {' | ||
| write(*,"(4x,a,1pe11.4,',')") '"min": ', tmin | ||
| write(*,"(4x,a,1pe11.4,',')") '"mean":', tmean | ||
| write(*,"(4x,a,1pe11.4,',')") '"std": ', tstd | ||
| write(*,"(4x,a,i0,',')") '"n_samples": ', asamp | ||
| write(*,"(4x,a,1pe11.4,',')") '"max": ', tmax | ||
| write(*,"(4x,a,1pe11.4,'}')", advance="no") '"MBps": ', words_per_sec | ||
|
|
||
| end subroutine time_ANN | ||
|
|
||
| end program time_MOM_ANN | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| program test_MOM_ANN | ||
|
|
||
| use MOM_ANN, only : ANN_unit_tests | ||
| use MOM_error_handler, only : set_skip_mpi | ||
|
|
||
| call set_skip_mpi(.true.) ! This unit tests is not expecting MPI to be used | ||
|
|
||
| if ( ANN_unit_tests(.true.) ) stop 1 | ||
|
|
||
| end program test_MOM_ANN |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems that this metric changed its meaning. Before it was measuring number of operators per second (flops) and now it is the memory throughput. I am not sure which metric would be the most relevant as it is unclear a priori if this code will be compute-bounded or memory-bounded. Is this words_per_sec metric supposed to be compared to L1/L2/L3 cache throughput?
I somewhat find Gflops to be more relevant metric as its range on single CPU core is clearly defined from approximately 0.5Gflops for scalar operators (typically, in scalar code per one floating point operation there are ~5 service operations, matmul benchmark) up to approximately 50-100Gflops for FMA instructions in longest vector registers assuming no memory transfer between registers and cache. Typical ocean model has 3Gflops performance on average which is much better than scalar code but still far away from the compute bound. I would say a metric of success for ANN module is to be more efficient than ocean model on average, i.e. be in a range of
3Gflops-100Gflops
May be, if we want Gflops, we may need to estimate the number of floating point operations, which is for matmul not the number of parameters in matrix, but approximately twice (one add and one multiply per element).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As @marshallward pointed out, what I'm calculating is the number of words for storage. When reporting Gflops, I had made the assumption that this was likely the number of multiply-adds, but as you say there is ambiguity in whether an FMA should count as one or two ops. Switching to memory processed avoided the ambiguity but in truth, it's probably better to just return the times (as we do in the other tests).