Command line utility

[mightimatti]

Hi,
this is more of a feature/support-request than a bug-report and I apologize if it is not appropriate for this format:

I am currently in the process of developing a Python/Tkinter as a user interface for a custom scanner application that produces substantial amounts large, 4-channel whole-slide images, the fine alignment of which I would like to perform using MIST, as it has yielded excellent results on the manual tests I performed. Unfortunately, opening imageJ(even with macro) is very cumbersome. The output of my application is such, that coarse alignment (theoretical position of the tiles based on reproduction ratio and coordinates of mechanical stage) is already calculated, i.e.TileConfiguration.txt could be trivially output into a folder along with my filed. What I wish to do is run mist to optimize the fine alignment, as is is a perfect fit for my use case, works well and runs fast.

While I understand, that implementing this whole library in python would be a lot of work and I wouldn't dream of requesting this, I feel like formulating a series of commands which allow me to spawn a new process to directly call the underlying Java application(doing away with the need to open imageJ) in a separate process from within my application, might actually be quite simple, I would very much appreciate if you could maybe add a section to the README, detailing what commands would allow an advanced usage to be automated. I saw that some docker files were addest to master that seem to do just this, but I couldn't find any documentation on how to use them.

thank you for this excellent software in any case

[mmajurski]

The docker interface is for https://isg.nist.gov/deepzoomweb/software/wipp
There is a docker image version of MIST in dockerhub, but it would not just perform the refinement of tile locations, it runs the full stitching application.

https://github.com/usnistgov/MIST/blob/master/docker_example.sh should give you an idea of how to launch the MIST container with a set of arguments outside of FIJI.

[mightimatti]

Hi @mmajurski,
thank you for the rapid reply. I was hoping to use this program from the commandline, locally installing the required Java dependencies and running it as a process in my host OS. My experience with Java being limited and dating back to the early days of my CS bachelor's I wasn't able to compile the neccessary bytecode though and even the dockerfile gave me errors I wasn't able to solve.

The fact that the application can be run within docker(I'm guessing without ImageJ running) and a brief look at MISTMain.java seem to indicate that the application is already setup to not require the GUI.

I think basic instructions to use this powerful piece of software in an automated workflow would be beneficial not only to me, but also to future users and I would greatly appreciate if you could add some instructions on how to build this app for that purpose. I already figured out, that I need to install maven and currently I'm getting some errors I am stuck on, I assume related to incorrect maven CLI arguments.

[mmajurski]

The application can be run without the GUI, so long as you compile the whole project with all of the required dependencies, which is the hard part. The docker file should work out of the box. We use that docker image in a downstream system fairly regularly, so I want to make sure its kept operational.

What errors did you get working with the docker file?

[tblattner]

I have some progress with getting things running from command-line directly using java -jar directly.

It isn't pretty though. I need to chat with Mike first on how we want to deploy this version... But I think we can make it happen...

[tblattner]

Compilation and Execution instructions have been created:

https://github.com/USNISTGOV/MIST/wiki/Install-Guide#compilation-from-source
https://github.com/usnistgov/MIST/wiki/User-Guide#command-line-execution-of-mist

Note that if using IntelliJ, we used the following version of the Java JDK: corretto-1.8.0_372\bin\java.exe

This is the version that gets packaged with IntelliJ. IntelliJ is not required, but currently is the way we have gotten it setup.

If you go through the steps to get the java and maven environments setup outside of IntelliJ, please provide your steps and I can add them into the instructions.

[mightimatti]

Thanks a lot for the update. I was in fact running the wrong Maven command and mvn package ran without issues.

After having spent all afternoon playing with combinations of running the app from ImageJ and calling it again for the same files from the command line, I can say that initially there seems to be something about running it from the CLI with the provided parameters(from the example) that caused the program to crash, where an equivalent call from the GUI, constructed manually in the ImageJ plugin, succeeded.

The resulting jar seems to run into some issues surrounding filesystem access. Both if I open the gui through the ..._with_dependencies.jar as well as if I run a test prompt based on the instructions for command line usage. In the gui, I can open the folder I am looking for but I can not see the files it contains. In either case the CLI output gives me this:

java -jar /home/bean/Software/MIST/target/MIST_-2.1-jar-with-dependencies.jar --filenamePattern {rr}_{cc}.tif --filenamePatternType ROWCOL --gridHeight 4 --gridWidth 4 --gridOrigin UR --imageDir /home/bean/Desktop/mvTest/out/batch_1688307275.2059991 --startCol 0 --startRow 0 --programType FFTW --fftwLibraryFilename libfftw3.la --fftwLibraryName libfftw3f --fftwLibraryPath "/home/bean/Software/fftw-3.3.10/"
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by gov.nist.isg.mist.lib.libraryloader.LibraryUtils (file:/home/bean/Software/MIST/target/MIST_-2.1-jar-with-dependencies.jar) to field java.lang.ClassLoader.usr_paths
WARNING: Please consider reporting this to the maintainers of gov.nist.isg.mist.lib.libraryloader.LibraryUtils
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
filenamepattern={rr}_{cc}.tif 
filenamepatterntype=ROWCOL 
gridheight=4 
gridwidth=4 
gridorigin=UR 
imagedir=/home/bean/Desktop/mvTest/out/batch_1688307275.2059991 
startcol=0 
startrow=0 
programtype=FFTW 
fftwlibraryfilename=libfftw3.la 
fftwlibraryname=libfftw3f 
fftwlibrarypath=/home/bean/Software/fftw-3.3.10/ 
OpenJDK 64-Bit Server VM warning: You have loaded library /tmp/BridJExtractedLibraries13881406874826794967/libbridj.so which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
Warning: the following files will be overwritten:
/home/bean/img-global-positions-1.txt
/home/bean/img-log.txt
/home/bean/img-relative-positions-1.txt
/home/bean/img-relative-positions-no-optimization-1.txt
/home/bean/img-statistics.txt
MIST is stitching a Sub Grid.
STITCHING BEGINS!
ImageJ can only open 8 and 16 bit/channel images (32)
Successfully loaded plan from file
Loading FFTW Plan...
Saving plan to file: /home/bean/Software/MIST/lib/fftw/fftPlans/3384x2712MeasurePlan_f.dat
Finished loading/saving FFTW plan. Commencing stitching.
memory pool size: 35
Writing relative positions (no optimization) to: /home/bean/img-relative-positions-no-optimization-1.txt
Computed North overlap: 37.0%
Computed West overlap: 16.0%
Warning: no good translations found for North direction. Estimated translations generated from the overlap.
Warning: no good translations found for North direction. Repeatability has been set to zero.
Please check the statistics file for more details.
Repeatability for North: 0 pixels
Warning: no good translations found for West direction. Estimated translations generated from the overlap.
Warning: no good translations found for West direction. Repeatability has been set to zero.
Please check the statistics file for more details.
Repeatability for West: 0 pixels
Calculated Repeatability: 3 pixels
Completed Stitching in 366ms
Writing global positions to: /home/bean/img-global-positions-1.txt
Writing relative positions to: /home/bean/img-relative-positions-1.txt
Writing relative positions (no optimization) to: /home/bean/img-relative-positions-no-optimization-1.txt
Completed output options for slice 1.
Saving Statistics to "/home/bean/img-statistics.txt"
Saving Log to "/home/bean/img-log.txt"
Done

I have further noticed, that the script running outside of imageJ fails to recognize my machine's available memory, while the information is present when run from ImageJ. I am not sure whether this is only relevant for the statistics output or meaningfully impacts the computation, but it is something I noticed.
Here is an output from running the program through the CLI:

System information:
Java Version: 11.0.19
Operating System: Linux amd64 v5.15.0-60-generic
CPU threads used: 32
Free memory available to JVM (GB): 0
Total memory available to JVM (GB): 0
Max memory for JVM (GB): 15
Execution type: Auto

Execution timing and general information:
Statistics Output Version: 1.0

as opposed to ImageJ:

System information:
Java Version: 1.8.0_322
Operating System: Linux amd64 v5.15.0-60-generic
CPU threads used: 32
Free memory available to JVM (GB): 4
Total memory available to JVM (GB): 6
Max memory for JVM (GB): 43
Execution type: Auto

Execution timing and general information:
Statistics Output Version: 1.0

After many hours of this not working(in hindsight possibly because I was linking to a different FFTW install), I ended up running the program from the GUI with parameters that succeeded, opened the image-statistics.txt and reshaped it's "Stitching Run-time parameters" section into a massive prompt,e.g by going through all lines and transforming gridWidth: 4 into --gridWidth "4". Thereby explicitly defining every single parameter for exectution. For anyone running into similar issues with the CLI tool, but successfully using it from the GUI, I would reccomend doing the same. This resulted in this prompt:

java -jar /home/bean/Software/MIST/target/MIST_-2.1-jar-with-dependencies.jar --gridWidth "4"\
--gridHeight "4"\
--startTileRow "0"\
--startTileCol "0"\
--imageDir "/home/bean/Desktop/mvTest/out/batch_1688307227.5980275"\
--filenamePattern "{rr}_{cc}.tif"\
--filenamePatternType "ROWCOL"\
--gridOrigin "UL"\
--numberingPattern "HORIZONTALCOMBING"\
--assembleFromMetadata "false"\
--assembleNoOverlap "false"\
--startRow "0"\
--startCol "0"\
--extentWidth "4"\
--extentHeight "4"\
--timeSlices "0"\
--isTimeSlicesEnabled "false"\
--outputPath "/home/bean/Desktop/mvTest/out/batch_1688307227.5980275"\
--outputFullImage "false"\
--outputMeta "true"\
--outputImgPyramid "false"\
--blendingMode "LINEAR"\
--blendingAlpha "NaN"\
--compressionMode "UNCOMPRESSED"\
--outFilePrefix "img-"\
--unit "MICROMETER"\
--unitX "1.0"\
--unitY "1.0"\
--programType "AUTO"\
--numCPUThreads "32"\
--loadFFTWPlan "true"\
--saveFFTWPlan "true"\
--fftwPlanType "MEASURE"\
--fftwLibraryName "libfftw3f"\
--fftwLibraryFilename "libfftw3f.dll"\
--planPath "/home/bean/lib/fftw/fftPlans"\
--fftwLibraryPath "/home/bean/lib/fftw"\
--stageRepeatability "20"\
--horizontalOverlap "2.0"\
--verticalOverlap "2.0"\
--numFFTPeaks "0"\
--overlapUncertainty "0.5"\
--isUseDoublePrecision "false"\
--isUseBioFormats "false"\
--isSuppressModelWarningDialog "false"\
--isEnableCudaExceptions "false"\
--translationRefinementMethod "SINGLE_HILL_CLIMB"\
--numTranslationRefinementStartPoints "8"\
--headless "false"\
--logLevel "MANDATORY"\
--debugLevel "NONE"

I now have a working version and will attempt to incorporate it into my program, as I am only interested in the metadata(refined tile locations) I am happy to report that this script runs in approx 12s for 2-300MP stitches. I am very happy with this. Thank you for developing this excellent tool.

On an unrelated note: I have come to realize, that the automatic parameters fail for some of my problem instances which I find odd, because my "tiles" are tack-sharp, low-distortion, flat-field corrected and have fairly consistent(albeit inperfect) spacing. The gui-popup informs me that you are interested in problematic datasets and I would love to contribute one to aid your development efforts, could you let me know how to supply it? Unfortunately as a jpeg, the high resolutions results in 40MB of files.

I apologize for the long winded answer and I will post another update as soon as I have run this for some more examples. Thank you again for the help.

[mmajurski]

Glad that you got the command line working. Had you been about a day slower, I would likely have gotten the python version complete. Enough people have asked about a stand alone simplified python version that it was time to start building it. Right now I have the translation computation and half of the stage model done.

As for problematic datasets, if you are willing to share, email me a google drive link ([email protected]) and I will see whats going on. Likely if you are getting the result you want, your stage is just looser than MIST expects. The warning shows up if the stage repeatability is >10pixels, but that was an arbitrary threshold we chose base on high quality motorized stages. The real test is whether the result image looks correct to you. Also, if you are just using the image positions, you can assemble the stitched image outside of Fiji using https://github.com/usnistgov/MIST/blob/master/assemble_stitched_image.py.

[tblattner]

Happy to hear you've gotten things to work, and for sharing your performance numbers!

I will add another TODO item to my list where we can pass in the image-statistics file at the command-line, while customizing it by specifying other options. My thought right now for instance is to have java - jar MIST.jar --config-file /path/to/config --gridHeight 4 --gridWidth 4 ... etc. This would load all options in the config file and then customize grid height and grid width or whatever other parameters. The CLI was developed many years ago, so its very much in need of a refresh.

For the illegal reflective access issue that is a bummer. We actually borrowed that snippet of code from a Java Sun engineer many years ago. The functionality in this case was to add a directory to the java library path at runtime. So we'll need to find another way to do this in the future, otherwise FFTW will either not work, or we will need to find another way to handle dynamically loading FFTW.

The memory discrepancy could be related to the command-line call to java. There are several options that can be passed to the java virtual machine. Fiddling with these might allow you to increase your virtual machine's RAM: https://docs.oracle.com/cd/E19159-01/819-3681/abeik/index.html
Namely "-Xms and -Xmx" flags.

As for your dataset that may or may not be challenging. Does the resulting stitched image look good? If you are using the same stage for all of your acquisitions, then tuning your advanced parameters will help the algorithm accurately stitch. In particular horizontal/vertical overlaps and repeatability. These can be used to account for the variable spacing your seeing.

[mightimatti]

@mmajurski The 10 pixel threshold being the issue sounds about right. I eventually ended up setting it to 20 and that seemed to solve the issue. TBH I don't quite understand the difference/function of overlapUncertainty and stageRepeatability but setting them to 0.5 and 20 respectively I solved the problem. My stage is actually a Maerzhaeuser manual stage I retrofitted stepper motors, and a belt driven gear reduction to, loosely based on this: . Unfortunately fancy scanning stages weren't in my budget. However MIST perfectly overcomes this shortcoming.

@tblattner thank you for the advice regarding the JVM parameters. I am so out of the Java loop that I hadn't even considered that it might be a matter of changing runtime parameters. The stitched image looks just fine and I ended up setting the parameters you mention based on the grid I generated for the stages coordinates and a generous margin of error on the repeatability. If i could make one suggestions regarding my current understanding of the parameters: "Use image directory as output directory" seems to be a GUI feature that copies the input directory to the output directory which is then explicitly passed to the main program. IMHO it would make sense to make this a feature of the main app itself(toggled by a command line argument) as passing the folder twice is redundant and not passing the output directory results in things getting written to the home directory on my machine.

[mmajurski]

@mightimatti If you are interested in playing with the new python version:
https://github.com/usnistgov/MIST/tree/python
let me know if you find any bugs.

The basic algorithm is there, but it lacks some of the bells and whistles of the Fiji plugin.