1 / 20
rx-controls-suite · Prefect Orchestration
Same Scan.
New Conductor.
The same reactive pipeline the facility already trusts — now with a control-room view: live progress, an automatic pause on beam loss, an automatic resume, and a full audit trail. No rewrite.
Igor Khokhriakov  ·  Principal Software Engineer
github.com/scientific-software-hub/rx-controls-suite
2 / 20
00 · Why Prefect One Pipeline. A New Front End. Reuse first, orchestration second
What changed
  • The layer that starts the scan and watches it run
  • Where progress and status live — a web page, not a terminal you have to keep open
  • How the team learns something went wrong — a red badge and a stated reason, not a support ticket
What didn't
  • The safety gates — pause on beam loss, abort on interlock
  • The cross-system read that ties EPICS and Tango together into one frame
  • The HDF5 file every downstream analysis script already expects
Prefect wraps the same scan the facility already trusts. It does not replace any of its safety logic — it makes that logic visible.
3 / 20
01 · What Is Prefect Five Words You'll See In Every Screenshot A vocabulary primer, not a training course
  • Flow — the procedure. Here: one guarded tomography scan.
  • Task — one step of the procedure. Here: arm the beamline · acquire one sweep · close out and report.
  • Run — one timestamped execution of the flow, with its own page, its own logs, its own outcome.
  • Artifact — a report attached to a run: a progress bar, a data table, a written summary.
  • State — the traffic light: Running · Completed · Paused · Failed.
Without an orchestrator

A terminal window. A log file, if anyone thought to save one. Someone has to be watching when it happens, or it's a mystery reconstructed after the fact.

With Prefect

A web page anyone can open, during the run or a year later. A badge that says what happened. A number the shift report can cite directly.

4 / 20
02 · Architecture One Reused Core. Three Interchangeable Fronts. Prefect is a wrapper, not a rewrite
Before
Plain Script
One terminal, one process
Also In Use
Bluesky
Scientific run engine
This Talk
Prefect
Web dashboard, auto-recovery
Reused Core — identical under all three
Cross-system read — EPICS × Tango, never a half-written frame
Safety gates — beam-loss wait · sustained-loss watchdog · interlock abort
The HDF5 writer every analysis script already reads
Nothing about how the beamline is controlled changed. Only how we watch it — and how it recovers — did.
5 / 20
02 · Architecture Every Run, The Same Three Steps Whether it finishes clean or gets interrupted
Step 1
prepare_beamline
Arms the motor speed and exposure, opens the shutter. Retries automatically if the very first connection is flaky.
Step 2
run_sweep ×3
Acquires one third of the projections. Reports live progress and a table of what it acquired. Where the beam-loss pause happens, if it happens.
Step 3
finalize
Always runs, even after an abort. Closes the data file, writes the one-glance summary and a link to it.
A run is launched the same way an engineer launches any script today — one command. A click-to-launch console is a small addition on top of this, not something this demo needed to build to prove the point.
6 / 20
Scenario 1
Happy Path
Twelve projections · three sweeps · nothing goes wrong
7 / 20
03 · Happy Path · 1 of 4 One Page, The Whole Story Run overview + log
Prefect flow run overview for a clean happy-path scan: green Completed badge, three sweep tasks running back to back with no gaps, log lines below.
  • Completed, top left — the outcome, before reading anything else.
  • Three sweep tasks run back-to-back with no gaps — the visual shape of an uneventful scan, compare it to Scenario 2's shape later.
8 / 20
03 · Happy Path · 2 of 4 What The Run Leaves Behind Artifacts tab
Artifacts tab for the happy-path run: three sweep tables, a progress bar, a scan summary, and a link to the HDF5 file.
  • One data table per sweep, a live progress bar, a written summary, a direct link to the HDF5 file — no terminal, no searching a filesystem.
9 / 20
03 · Happy Path · 3 of 4 The One Number A Shift Report Needs Scan summary artifact
Scan summary artifact for the happy-path run: 12 of 12 frames acquired, 100 percent quality, 0 beam-loss pauses, outcome completed.
  • 12 / 12 frames · 100% quality · 0 pauses · completed — the whole outcome in one glance, no scrolling required.
10 / 20
03 · Happy Path · 4 of 4 Without Reading A Line Of Code What just got established
  • Anyone with the link can confirm the scan completed, exactly how long it took, and how many steps it consisted of.
  • Every projection is individually accounted for — angle, counts, ring current, orbit, pass/fail — not just a final total.
  • The audit trail exists whether or not anyone was watching when it ran.
11 / 20
Scenario 2
Beam Loss
Same 12 projections · beam drops mid-scan · watch what Prefect does on its own
12 / 20
04 · Beam Loss · 1 of 6 The Pause, Visible Before You Read Anything Run overview
Prefect flow run overview for the beam-loss scan: a large gap in the timeline between the first sweep finishing and the next task starting.
  • A wide gap in the timeline where nothing advances — compare its shape to Scenario 1's continuous bar.
  • That gap is the flow deliberately waiting, not a stall or a crash — next two slides show exactly why.
13 / 20
04 · Beam Loss · 2 of 6 Caught Live: The Moment It Paused Captured mid-run, not after the fact
Close-up of the Prefect UI badge showing the flow run state as Paused, with a Resume button, while the run was actually paused.
This screenshot was taken while the run was genuinely in this state — Paused, with a Resume button, right where Completed usually sits.
14 / 20
04 · Beam Loss · 3 of 6 The Whole Story, In Its Own Words Log tab, uninterrupted sequence
Prefect log tab showing the full sequence: beam LOW warning, pausing flow run for operator visibility, Resuming flow run execution, beam recovered resuming sweep.
beam LOW → pausing flow run for operator visibility → Resuming flow run execution! → beam recovered — resuming sweep. No operator typed any of this.
15 / 20
04 · Beam Loss · 4 of 6 Nothing About The Interruption Reaches The Data Sweep table — the interrupted one
Sweep table artifact for the interrupted sweep: four rows, all marked quality OK, no visible seam where the pause occurred.
  • Same table format as the happy path — four rows, all quality OK.
  • Two rows were acquired before the pause, two after; nothing in this table marks where the seam is.
16 / 20
04 · Beam Loss · 5 of 6 One Pause, Still A Clean Result Scan summary artifact
Scan summary artifact for the beam-loss run: 12 of 12 frames acquired, 100 percent quality, 1 beam-loss pause, outcome completed.
  • 12 / 12 frames · 100% quality · 1 pause · completed — the pause is a recorded number, not a hidden one.
17 / 20
04 · Beam Loss · 6 of 6 What Just Got Demonstrated Without a single manual action
  • Self-detected — the system noticed beam current was unsafe on its own, from the same reading it already polls every second.
  • Self-paused — not after acquiring bad data, before the next projection even started.
  • Self-resumed — the instant beam recovered, with no operator click required.
  • Zero data loss, full audit trail — every second of it, including the pause itself, is on the record.
18 / 20
05 · The Comparison Same Shape, One Difference The actual deliverable
Scenario 1 Happy Path
  • 12 / 12 frames acquired
  • 100% quality
  • 0 beam-loss pauses
  • Completed, green
Scenario 2 Beam Loss
  • 12 / 12 frames acquired
  • 100% quality
  • 1 beam-loss pause — recorded, not hidden
  • Completed, green
The system behaved correctly and visibly in both cases. That consistency — not any one screenshot — is what makes it trustworthy enough to run unattended.
19 / 20
06 · Beyond These Two Two Scenarios, Not The Whole Story What else already works
Handled the same way, not walked through here
  • A vacuum-burst interlock aborts the run and ends it Failed, with the reason attached, not silently completed.
  • Orbit drift flags frame quality in the same per-sweep tables shown above, without pausing or aborting anything.
Coming next
  • n8n — a genuinely different problem: no in-process Python option at all, so the same scan gets exposed as a web service and n8n's visual node graph becomes the orchestration.
  • Expect its live feedback to be coarser than Prefect's — that gap is itself one of the findings worth publishing.
20 / 20
Thank You
Same Reactive Pipeline.
Now Under Prefect.
github.com/scientific-software-hub/rx-controls-suite
Go deeper
demo/workflow-engines/README.md — run it yourself
demo/workflow-engines/prefect-walkthrough.html — the engineering deep-dive