Increasing RAM use while fscrawling

[ScottCov]

I have indexed about 25,000 PDFs so far and my available RAM continues to decline . When I started fscrawler this process was using only 625MB...and it went up to 3.25GB. Obviously it seem to have something to do with logging. I deleted most of the log files and it is now down to 2.5GB. I am using docker for fscrawler and when I ran it thought I set it up so all log files wouldn't be saved inside the container:

docker run -d --env FS_JAVA_OPTS=-DDOC_LEVEL=debug --name fscrawler -v /home/serveracct/logs/log1:/usr/share/fscrawler/logs -v /home/serveracct/logs/log2:/tmp -v ~/.fscrawler:/root/.fscrawler -v /mnt/cloud/cases:/tmp/es:ro dadoonet/fscrawler fscrawler job_name
Here is a description of the process that is taking up increasing memory:

Any ideas how to stop the excessive use of RAM?

And incidentally it may say 794MB of memory above but its 2.37GB:

Scott

[ScottCov]

OK it looks like its gone down now and is using only 724MB...now can someone explain what is going on?

[dadoonet]

Could it be caused by tesseract? I mean: is this happening the same way if you disable OCR?

[dadoonet]

Another question: if you are using a recent build, could it be caused by semantic text? I just added this recently.
Could you check if anything related to this are in your logs?

[ScottCov]

Hi David-

The problem seems to go away after I reboot so it's been alleviated at least temporarily. If its ok I may want to circle back on this at a later date though. I will go thru my logs though and see if I can spot anything. Previously I couldn't reboot since fscrawler seemed to then want to reindex but I solved this by turning off the option to delete files which is another issue that I know we've discussed in the past and I intend to circle back with you on in the future once I get thru some pdfs that really need to indexed.

Much obliged/merci beaucoup!

Scott

[dadoonet]

Avec plaisir 😉

just let me know what are the very first lines of the logs when you restart. Specifically something about semantic