Multiprocessing components do not work properly

[andrewstenger]

The functions in Boto-Plus that implement multiprocessing via Python's multiprocessing library do not work properly if the use_multiprocessing argument is set to True. The issue is that boto3's resource/client objects cannot be pickled. But the multiprocessing library tries to pickle all objects that are provided as input to the Pool() function. So when the use_multiprocessing argument is supplied, the functions will fail.

The solution to this will involve a non-trivial refactoring of the code. One idea I've had is that when a function is called with the use_multiprocessing argument, it can instantiate another class-object made specifically for multiprocessing that instantiates the boto3 resource/client entirely within the _mp_ function that gets applied during multiprocessing via a with statement similar to the code below

with mp.Pool() as pool:
    uris = pool.map(self.__copy_object_mp_unpack, payloads)

In this case, we would have to instantiate the boto3 objects within the self.__copy_object_mp_unpack function. Alternative solutions can also be explored if this is too unwieldy.