Make get_movie setup eager, acquisition lazy; optimize serval

[Baharis]

This PR is work in progress. It contains irrelevant history from a branch with multiple other features that should be ignored here.

Eager setup – lazy acquisition

Instamatic has two methods to get camera feed, get_image and get_movie. The latter is underutilized to the degree where many cameras implement it as loop over get_image. In my work, I strive to flesh out get_movie as a distinct alternative that allows receives images as they are collected, minimizes dead time, and in general allows collecting short burst-series in contrast to the more general get_image. For this reason, I modified get_movie to return iterator in #121 and updated/supported it since (#137, #154, #160). Here I suggest another improvement to get_movie that will make it more responsive.

In python, every callable can have two exit points, return or yield. When called, methods that return run immediately, but methods that yield do not; instead they return a Generator that executes only once it is iterated using next. This creates lazy iterables, executed only when demanded, which is very useful e.g. for demanding or infinite loops.

The lazy/eager distinction of return/yield can be used to minimize get_movie's dead time. Because get_movie always returns an iterable, it can receive two unique signals to start: one when it is called, and one when it is iterated over. From my experience working with Instamatic I realized that this can be used to create really fine control over timing.

Here I suggest utilizing the double-start mechanism of get_movie in the following way:

• When get_movie is called: videostream is blocked, camera is configured for a movie, and a generator is created.
• When generator is iterated: camera immediately starts collecting images.
• When iteration is over: camera is configured back for get_image and videostream is unblocked.

This behavior allows for a really fine control over timing. When writing multi-threaded code I noticed that I often want to start camera and some TEM change concurrently. With proposed mechanism, this can be done the following way:

n_frames, exposure = initial_setup()
movie = ctrl.get_movie(n_frames, exposure)  # stream is stopped, camera is configured here
following_setup()
start_tem_change()
for image, header in movie:                 # trigger: image acquisition is started here
    modify_header_or_tem()                  # cleanup is run automatically after the loop

In order to get this behavior, two changes are required. Firstly, LiveVideoStream and MediaGrabber responsible for streaming need a separate callable with yield that is called it as late as possible. Secondly, individual implementations of cameras need to do the same. This allows cameras to "prepare" for movie acquisition, and then start it quickly, which makes synchronizing camera and TEM much easier.

Finally, suggested change is non-invasive. If any existing script already uses get_movie, it will behave just as expected. Ultimately, most persons interested in movies will call get_movie immediately before iterating them (for i, h in ctrl.get_movie(n, e)), in which case separating startup and iteration will have little to no effect. No changes to other implementations are also strictly needed, unless someone desired to reap benefits of this eager setup / lazy acquisition mechanism.

Serval optimization

The current implementation of get_movie in serval is nice and easy but falls off at high data rates. Serval client first asks camera for N frames, and then get_requests every one individually. In my tests I noticed that this introduces delays, to the degree where I could not get more than 8 images per second. To counter it, I implemented a custom TCP stream reader that circumvents ASI package. With it in place, all movie data is streamed by the server continuously and read in milliseconds. The mechanism is completely optional and can be switched on by setting STREAM_MOVIES_VIA_TCP = True in camera_serval.py or stream_movies_via_tcp = true in camera.yaml.