Skip to content

Conversation

grahamrow
Copy link
Member

Yikes.

Trying to solve a few problems here:

  1. Improved pipeline stability - @martin-gustafsson identified a lot of sketchy behavior of the pipeline. I moved to having all of the filer objects sharing Exit and now Panic events that the filter set upon Exceptions and the experiment periodically checks to see if it should bail. I have no real-world stability improvements to claim.
  2. Move to methods running actual filter code - using class methods allows Processes to be started without any explicit dependence on instance variables. Verbosity increases here since we now have to explicitly pass lots of important constructs, but in principle it is now much easier to write a fast filter in some external language. Currently configuration parameters live in a dictionary, but this could be spit out in ctypes struct later on.
  3. Finally allow things to work using spawn - this is a disaster as references to shared objects such as Queues and RawArrays are not properly synchronized without a multiprocessing Manager entity. Adding this still took a TON of hacks to fix numpy RawArray mappers, return values from IO buffers, etc. etc. etc. Using vanilla multiprocessing is

Still need to convert the following to using class methods:

  • Alazar listener
  • Filters other than averager and buffer

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant