Memory Usage/Leak

[OptrixAU]

I'm using XLCalculator as part of a back-end service, and I'm finding that memory is leaking.

Using TraceMalloc, I can see ast_nodes.py consuming more and more memory over time.

Any idea what I can do about this? Is there some method of forcing a garbage collection? Something I can do to release all of the memory that ast_nodes is consuming?

Might this be an issue caused by using XLCalculator from more than one thread?

[OptrixAU]

Note that this has nothing to do with multi-threading and will happen in a single-threaded application.

In fact, I can easily replicate it in a very small app.

MemLeak.zip

I'd love to fix it myself, but I have no idea about how to track down python memory leaks, I'm ASSUMING this is something to do with a cyclic reference?

[OptrixAU]

OK - one massive source of the issue is the lru_cache directive found in eval_cell. This is causing an unlimited number of EvaluatorContext objects to be kept in memory.

Setting a maxsize does help to reduce the issue, but doesn't eliminate it. Commenting out the line however has a massive impact on the memory leak. Further testing will find if the problem is completely eliminated.

[OptrixAU]

Removing the lru_cache directive has one unexpected side-effect - the cycle detector in EvaluatorContext.eval_cell starts throwing up unexpected and incorrect cycle exceptions when you've got a formula that references the same cell twice.

Currently to get this working I've simply commented out the cycle processing, but this would need to be resolved before the issue is fixed.

The assumption here is that the seen member of the EvaluatorContext is having cell names added, but they don't then get removed when the execution is complete.