Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
128 commits
Select commit Hold shift + click to select a range
5b82b7f
Design a system to identify (and sweep across) rectangular grid windows
Baharis Dec 5, 2025
eb3f986
Bugfixes with chatgpt
Baharis Dec 16, 2025
20997ab
Don't crash if you can't save raw csv files
Baharis Dec 16, 2025
37a8763
Convenience features used when debugging
Baharis Dec 16, 2025
4229ae4
Reworking grit tools into separate instamatic.grid (more to put here)
Baharis Dec 19, 2025
b1ecb2b
Collect movies using tcp stream instead of http request to allow high…
Baharis Dec 19, 2025
7cb5b73
Not faster but does connect (and raises errors, r=None in videoframe)…
Baharis Dec 19, 2025
a566ecc
This can read movie @ 10 fps no problem, but not 100 fps.
Baharis Jan 15, 2026
5a71ef8
First fast design of movie reader/scanner, capable of reading ~2000fp…
Baharis Jan 16, 2026
b6e1a32
New implementation of serval `get_movie` theoretically reading ~1k fr…
Baharis Jan 16, 2026
c47f861
Merge branch 'main' into sped
Baharis Jan 19, 2026
1679d38
First rough implementation of the SPED multiprocessing image logic
Baharis Jan 19, 2026
9026a5b
Mostly finished scaling and formalized multiprocessing logic.
Baharis Jan 20, 2026
306f399
Some fixes to the logic: TODO diffraction algos, entry points, testing,
Baharis Jan 20, 2026
7e6dbfa
Very decent chatgpt algorithm that is WAY too slow, saving for history
Baharis Jan 21, 2026
fdef9af
Decent fast algo (20-40ms), but can we do better?
Baharis Jan 21, 2026
9c9fd1f
Accurate (40-100ms) algo that successfully finds 0 refls if empty, an…
Baharis Jan 21, 2026
4b1cd21
Optimized (20-30 ms) algo that successfully finds 0 refls if empty, a…
Baharis Jan 22, 2026
e568f92
Working implementation of Ulam spiral in hex and rect space for Grid …
Baharis Jan 22, 2026
95cfa1c
Define an initial shape of SPEDState class that will hold experiment …
Baharis Jan 23, 2026
14691a0
Adding Grid implementation
Baharis Jan 23, 2026
47f7bab
Implement (largely GPT) the HexagonalWindow: now both are defined, in…
Baharis Jan 26, 2026
860896e
Remove unused imports
Baharis Jan 26, 2026
e6faefe
Adapt SPEDState not to use grid, to register scans
Baharis Jan 26, 2026
8793404
Improve readability, bind closer SPED Journal and State
Baharis Jan 26, 2026
f4e205c
Add a ProgressListbox Frame that displays all successful experiments
Baharis Jan 27, 2026
9dcc2dc
Attach progress table to state class
Baharis Jan 27, 2026
af071f5
Added GUI. With grid_columnconfigure, it is a bit too wide, but fine …
Baharis Jan 27, 2026
56f221b
Added initial experiment logic and A LOT of TODOs since it sucks!
Baharis Jan 28, 2026
aaad5d1
WIP; rewrite state, progress, experiment using new window and step sy…
Baharis Jan 29, 2026
0d2578c
Its friday evening, God only remembers what fixes are here TBH
Baharis Jan 30, 2026
597a075
Finish implementing `run_scan` w/ new simplified dispatcher structure
Baharis Feb 2, 2026
f7e67d0
Move grid.grid to grid.polygon to make ground for generalization
Baharis Feb 2, 2026
eca8b3d
Reorganize the grid code to allow for more kinds of grid, WIP
Baharis Feb 2, 2026
2a88a46
Reorganize the window code to allow for more kinds of grid, WIP
Baharis Feb 2, 2026
7f4e244
Reorganize the window code to allow for more kinds of grid, WIP
Baharis Feb 2, 2026
9ec2958
Fix window serialization for journaling (converted to dict, type infe…
Baharis Feb 2, 2026
62ea890
Move plotting logic into a separate artist module, rm unused files
Baharis Feb 3, 2026
1b47806
Update experiment logic to the grid & windows changes
Baharis Feb 3, 2026
a30e300
Add GUI options to change alpha with progress and determine windows m…
Baharis Feb 3, 2026
06712d9
Add option to change alpha during scan (0 to max, then -max to 0)
Baharis Feb 3, 2026
841aec0
Add `Experiment.set_stop_event_if_target_met`
Baharis Feb 3, 2026
662845f
Implement a way to add at start new windows manually or disabling add…
Baharis Feb 3, 2026
ad15fe8
These changes made SPED work mostly fine, only minor issues remain
Baharis Feb 4, 2026
df1c5fe
Give proper feedback whenever buttons are pressed, experiment stopped…
Baharis Feb 4, 2026
30607c5
Fix rastering/serpentine behaviour
Baharis Feb 4, 2026
508ac19
When adding a scan, add also an error_margin to both sides, covering …
Baharis Feb 4, 2026
0d65370
Address critical issues found by GPT
Baharis Feb 6, 2026
a478e0d
Address issues with windows/grid found by GPT
Baharis Feb 6, 2026
6f6a418
Auto-save figures of windows whenever they are added
Baharis Feb 6, 2026
263f1c1
Auto-save figures of windows whenever they are added (fix typo)
Baharis Feb 6, 2026
92b61c2
Fix movie: auto-determine what IP serval should connect to (+ minor b…
Baharis Feb 6, 2026
4aa6142
Fix issue where points in windows were treated as in distance 0
Baharis Feb 6, 2026
b098f29
`self.ctrl.stage.set_with_speed(**setter_kwargs, wait=False)` LOL
Baharis Feb 6, 2026
9b4b321
Add buffer name for better tiff names
Baharis Feb 6, 2026
d5c169d
Avoid counting single pixels on dark background by using small Gaussi…
Baharis Feb 7, 2026
50abfd8
Avoid soft bleed from under mask and favouring edge pixel due to gaus…
Baharis Feb 7, 2026
abf4dfb
Replace repr with shorter str when representing Window in ProgressTable
Baharis Feb 9, 2026
c25dc98
Gently unlock GUI if journal does not exist
Baharis Feb 9, 2026
3e54b89
When plotting, make figures square, dont force include 0 axes
Baharis Feb 9, 2026
bb4520c
Add grid plotting, fix adding grid manually soft-locked
Baharis Feb 9, 2026
17c0cc3
Release stop button only if error happens, not always
Baharis Feb 9, 2026
98722f8
0.1 sweeper threshold works fine with diffraction mode
Baharis Feb 9, 2026
6fe2138
TODO: ask Serval why the hell they swapped bytes 1 and 3
Baharis Feb 12, 2026
2c68d3c
Don't overwrite previous images with _edge_xys
Baharis Feb 12, 2026
47267c4
Fix typo in movie byte-shifting mechanism
Baharis Feb 17, 2026
41bddd3
Merge branch 'main' into sped
Baharis Feb 23, 2026
3d1dc1a
WIP: large refactor of the grid logic to allow multi-window fitting
Baharis Mar 13, 2026
d3b13c2
WIP: move registry to geom, auto-find windows in limits, adapt artist…
Baharis Mar 18, 2026
a412cca
Incorporate changes to the logic of new grid geometry into serial exp…
Baharis Mar 19, 2026
7f01f23
Add abstraction logic to the state: region and line for multiple scans
Baharis Mar 20, 2026
ad5e9b7
Fix issues, integrate dispatch and detection, add light handling (no …
Baharis Mar 23, 2026
05dcd2f
Implement, extract defining, fitting to scan profile as a function of…
Baharis Mar 25, 2026
ac8863d
Add a potentially-faulty option to heatmap onto the grid image.
Baharis Mar 26, 2026
a22b5a3
Temporarily, plot heatmaps w alpha=0.5 for debug purposes
Baharis Mar 26, 2026
3c86c05
EXport the region-windows logic to separate region.py
Baharis Mar 26, 2026
c300654
Assert no data is lost when summing light
Baharis Apr 16, 2026
eeac1b4
WIP: fixes needed to get SPED almost back to working, fitting does no…
Baharis Apr 23, 2026
6090214
Fix window locating, refinement
Baharis Apr 23, 2026
5b98b2d
Clamp theta to -90 to 90 after refinement, it shouldn't ever be away.
Baharis Apr 23, 2026
be1934c
Further bugfixes/ideas
Baharis Apr 23, 2026
d3b9491
Fix wrong line being read when processing
Baharis May 29, 2026
40f5365
You cannot `get_data_from_shared_memory` if data is an error report
Baharis May 30, 2026
2fb2616
Merge branch 'instamatic-dev:main' into main
Baharis Jun 3, 2026
9cf1a99
Merge branch 'instamatic-dev:main' into main
Baharis Jun 12, 2026
cdab30e
Serval: do not raise if byte1 is not 0
Baharis Jun 18, 2026
7f6d3a7
Add an option to save all images
Baharis Jun 18, 2026
238bbee
Allow changing diffhunt configuration during experiment
Baharis Jun 30, 2026
fade29a
Add new mode mechanism, explicitly initialize state
Baharis Jun 30, 2026
b3ce20a
Implement option to reprocess old data
Baharis Jun 30, 2026
22545f6
Fix radius min should be 40, not 0.5 by default (px not A-1)
Baharis Jun 30, 2026
9b62979
Fix some issues found by "coworker"
Baharis Jul 1, 2026
bcb8fc9
Clear unused imports
Baharis Jul 1, 2026
b789c38
Simplify, replace dispatch dataclasses with dicts
Baharis Jul 2, 2026
140271d
Fix controller typing bug; TODO better re/processing
Baharis Jul 2, 2026
6d256b7
Implement optimized processing via free worker pool instead of round …
Baharis Jul 7, 2026
1fd4a3e
Further simplify. link state in dispatcher, add faster reprocessing, …
Baharis Jul 7, 2026
5a0560f
Baseline changes needed to get (re)processing to work (quite nicely!)
Baharis Jul 8, 2026
e65f2be
Some fixes, remove debug statements; TODO: check movie encoding again...
Baharis Jul 8, 2026
b5cb749
Temporary change to print/log raw image data
Baharis Jul 9, 2026
5e658e2
Potential fixes to the images to be tested
Baharis Jul 10, 2026
58fe097
Fix the TCP data stream reading
Baharis Jul 14, 2026
3f49f63
Establish a strong bijection between buffer name and region/scan/line…
Baharis Jul 14, 2026
db60b2f
`load_from_journal`: don't apply `fill_encoded_scan` if `self.mode ==…
Baharis Jul 14, 2026
fc47012
These changes allow streaming images at 10 fps (but not much faster...)
Baharis Jul 15, 2026
8c984cd
Revert changes to __main__, fix IGNORE type/instance typing
Baharis Jul 15, 2026
05a55ce
Today's approach to camera rework, continue
Baharis Jul 20, 2026
b366ee0
Temp changes, trying to debug new connection
Baharis Jul 22, 2026
c09ee2b
These changes are needed for fast-lazy movie
Baharis Jul 22, 2026
98b840c
Prevent ScanProfile.sigmoid from diverging when fitting
Baharis Jul 24, 2026
ff5ec8e
Merge remote-tracking branch 'origin/sped' into sped
Baharis Jul 24, 2026
31cf352
Next round of preemptive fixes for artist heatmap plotting
Baharis Jul 24, 2026
db83591
Improve imports
Baharis Jul 24, 2026
601b9d0
THIS PLT IS NECESSARY!
Baharis Jul 24, 2026
8b274a2
Remove comments, fixes post-refactor at microscope
Baharis Jul 24, 2026
8c1bd0b
Serialize lists of integers when creating new windows
Baharis Jul 24, 2026
911019d
Remove debug prints, correctly close on exit
Baharis Aug 7, 2026
bd7f03e
Try adding the http-fallback mechanism
Baharis Aug 7, 2026
25d3e98
Merge remote-tracking branch 'origin/sped' into sped
Baharis Aug 7, 2026
dfe8e22
Try fixing establish_connection before testing movie
Baharis Aug 7, 2026
57921fc
Timeout-based TCP control is impossible: add env use_tcp_for_movie (b…
Baharis Aug 7, 2026
4084602
Give up, define new STREAM_MOVIES_VIA_TCP to control TCP streaming
Baharis Aug 7, 2026
68699ad
Minor docstring improvement
Baharis Aug 7, 2026
06d2ca7
Minor docstring improvement
Baharis Aug 7, 2026
f147e03
Merge branch 'main' into sped
Baharis Aug 7, 2026
ddb566e
Merge branch 'instamatic-dev:main' into main
Baharis Aug 7, 2026
74ffc5f
Merge branch 'main' into sped
Baharis Aug 7, 2026
d1f47fc
Detector changes picked from Baharis/sped branch
Baharis Aug 7, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
320 changes: 199 additions & 121 deletions src/instamatic/camera/camera_serval.py

Large diffs are not rendered by default.

25 changes: 19 additions & 6 deletions src/instamatic/camera/videostream.py
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ def __init__(self, cam: CameraBase, callback, frametime: float = 0.05):

self.stopEvent = threading.Event()
self.acquireInitiateEvent = threading.Event()
self.acquireInitiateEvent2 = threading.Event()
self.continuousCollectionEvent = threading.Event()

def run(self):
Expand All @@ -79,7 +80,9 @@ def run(self):
self.callback(media, request=r)
else: # isinstance(r, MovieRequest):
n = r.n_frames if r.n_frames else 1
for media in self.cam.get_movie(n_frames=n, exposure=e, binsize=b):
m = self.cam.get_movie(n_frames=n, exposure=e, binsize=b)
self.acquireInitiateEvent2.wait()
for media in m:
self.callback(media, request=r)
time.sleep(0) # yields thread priority to VideoStream

Expand Down Expand Up @@ -206,16 +209,26 @@ def get_image(self, exposure=None, binsize=None) -> np.ndarray:
def get_movie(
self, n_frames: int, exposure=None, binsize=None
) -> Generator[np.ndarray, None, None]:
self.blocked().__enter__() # Stop the passive collection during request acquisition
try:
with self.blocked(): # Stop the passive collection during request acquisition
self.grabber.request = MovieRequest(n_frames, exposure, binsize)
self.grabber.acquireInitiateEvent.set()
self.grabber.request = MovieRequest(n_frames, exposure, binsize)
self.grabber.acquireInitiateEvent.set()
except Exception:
self.blocked().__exit__(None, None, None)

def _movie_generator() -> Generator[np.ndarray, None, None]:
try:
self.grabber.acquireInitiateEvent2.set()
for _ in range(n_frames):
while not self.requested:
time.sleep(0) # yields thread priority to MediaGrabber
yield self.requested.popleft()
finally:
self.grabber.request = None
finally:
self.grabber.request = None
self.grabber.acquireInitiateEvent2.clear()
self.blocked().__exit__(None, None, None)

return _movie_generator()

def update_frametime(self, frametime):
self.frametime = frametime
Expand Down
1 change: 1 addition & 0 deletions src/instamatic/config/camera/serval.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -37,3 +37,4 @@ detector_config:
bpc_file_path: '/home/asi/Desktop/Factory_settings/SPM-HGM/config.bpc'
dacs_file_path: '/home/asi/Desktop/Factory_settings/SPM-HGM/config.dacs'
url: 'http://localhost:8080'
stream_movies_via_tcp: false
Loading