Foundation is giving OOM errors with 120 routes, 45 tables and around 250 I18N messages

[ibarrae]

Hey folks, more than an issue I would like to receive some help (if possible) because I have the following project overview:

• Around 250 I18N messages
• 120 routes
• 45 tables ~ persist models

With this set-up and by using Circle CI, the build is giving OOM errors when building with stack build at it hangs at Foundation.hs as you can see below:

[312 of 445] Compiling Foundation       ( src/Foundation.hs, .stack-work/dist/x86_64-linux/Cabal-2.4.0.1/build/Foundation.o )

--  While building package cerebro-0.0.0 using:
      /root/.stack/setup-exe-cache/x86_64-linux/Cabal-simple_mPHDZzAJ_2.4.0.1_ghc-8.6.3 --builddir=.stack-work/dist/x86_64-linux/Cabal-2.4.0.1 build lib:cerebro exe:cerebro exe:rank exe:read-translation-csv exe:seed exe:sync-p2s exe:write-translations --ghc-options " -ddump-hi -ddump-to-file -fdiagnostics-color=always"
    Process exited with code: ExitFailure (-9) (THIS MAY INDICATE OUT OF MEMORY)

As you can see, we know have 445 modules and I have done the following attempts to fix this issue:

• I have splitted the big "Model" containing all the tables to smaller files, each one with its table. At the moment, Model.hs looks like this:

module Model 
  ( module Model.Table1
  , module Model.Table2
  ) where

import Model.Table1
import Model.Table2

• I have removed almost 300 no longer necessary I18N translations (since I checked the following issue)
• I have done the following workaround for forms with more than 10 fields

Some extra information:

• The current LTS in the CI is 13.11 ~ GHC 8.6.3
• The docker image used to build in the CI is fpco/stack-build:lts-13.11
• On my local machine, the outcome from the command /usr/bin/time -f "mem=%K RSS=%M elapsed=%E cpu.sys=%S .user=%U" stack build is:

mem=0 RSS=4402548 elapsed=6:54.16 cpu.sys=12.01 .user=401.93

I almost have run out of ideas 😞, maybe any of you folks would have another idea in order to make it build on Circle CI? Thanks for your help and excuse me if this is kind of weird, English is not my main language 😄

[eahlberg]

I remember getting OOM errors on CircleCI a good while ago and using the jobs flag fixed it for me, e.g. stack build -j1. However, it was a much smaller project and it slows down the build, but it could be worth a try. 🤷‍♂️

[ibarrae]

I tried so, but it still uses more than 4GB to build the project 😞 thanks for your help @eahlberg

[MaxGabriel]

If Foundation is specifically the problem, maybe try splitting it up? Something like yesodweb/yesod-scaffold#198 maybe

At work we use giant AWS instances to build our project so haven’t run into this.

That said, is this a commercial project? If so can you just pay for more memory from Circle CI? https://circleci.com/docs/2.0/configuration-reference/#resource_class

[MaxGabriel]

Also, my guess on the OOM would be a significant amount of generated code is in Foundation. You can compile with -ddump-splices -ddump-to-file (see https://github.com/yesodweb/persistent/blob/master/persistent-template/README.md#development-tips) to see what is generated. This might help give you an idea of what is using all the memory.

[ibarrae]

I'll try splitting foundation! Thanks @MaxGabriel, also using the flags you are suggesting 😄

[MaxGabriel]

Any luck @ibarrae ?

[ibarrae]

Thanks for the suggestions @MaxGabriel @eahlberg, splitting Foundation did the trick 🎉. I really appreciate your help throughout this issue, and sorry I forgot to get back to you with the good news. I'm closing this now 😄

[rudymatela]

For future reference, more specifically, @ibarrae split foundation by making each call to mkMessageFor in a separate file. E.g.:

This:

module Foundation where

...
mkMessage "App" "messages/general" "en"
mkMessageFor "App" "MyDataType" "messages/..."
mkMessageFor "App" "MyOtherDataType" "messages/..."
mkMessageFor "App" "MyOtherOtherDataType" "messages/..."
...

Became this:

• Foundation.hs

  module Foundation where
  
  import Foundation.Messages.General
  import Foundation.Messages.MyDataType
  import Foundation.Messages.MyOtherDataType
  import Foundation.Messages.MyOtherOtherDataType
  

• Foundation/Messages/General.hs

  module Foundation.Messages.General where
  ...
  mkMessage "App" "messages/general" "en"
  

• Foundation/Messages/MyDataType

  module Foundation.Messages.MyDataType where
  ...
  import Foundation.Messages.MyDataType
  

etc...

That did the trick and we are able to compile under our memory constraints.