This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Experiment framework

The experiment runtime system: task queue/execution engine, loops, sequence editing, timing inspection and run monitoring.

Overview

The experiment framework covers the complete chain from writing an experiment through scheduling to running it:

  • Experiment scripts: subclass Experiment and implement build / run (for an introduction see Writing your first experiment; for device mapping see Configuring instruments);
  • Scheduling and execution: the experiment queue (ExperimentQueue) → the execution engine (ExperimentManager) fetches tasks by priority/reservation time; the preparation station and the looper participate in enqueueing;
  • Editing and monitoring: sequence editing/simulation, FPGA timing inspection, run monitoring, the multithreaded task framework and the base window widgets.

Scheduling and execution

PageModuleDescription
Experiment Queue Managerexperiment/experiment_queueTabular task queue
Experiment Run Managerexperiment/experiment_managerExecution engine (measurement/sequence/run)
Experiment Run Schedulerexperiment/Overview of the scheduling mechanism
Experiment Preparationexperiment/prepstationPreparation station (stage/push)
Loop Managerlooper/Conditional loop execution
Process Managerentries/processmanagerProcess start/stop panel

Editing, timing and monitoring

PageModuleDescription
Sequence Editoreditors/editor_sequencerCode editing + sequence simulation
Timeline Viewerentries/experiment_timelineFPGA timing visualisation
Run Monitorstreammonitor/Data/message monitoring
Experiment Browserbrowser/Main application browser

Widgets and utilities

PageModuleDescription
Data Widgetswidgets/arg_boxesParameter input widgets
Experiment Parameter Managerentries/dashboardDefault parameter system
Collapsible Sectionswidgets/theexpandablesCollapsible sections
Preparation Queue Managerwidgets/modelsQueue/preparation table models
Worker Task Managerwidgets/browser_workersWorker threads
Multithreaded Worker Frameworkwidgets/helper_workersWorker threads
Auto-save Editoreditors/auto_save_editorPeriodic save times
Base Window Widgetswidgets/quatmQtGUI base widgets

1 - Data Widgets

Experiment parameter editing widgets: FloatBox/BoolBox/ComboBox, writing changes straight back to the parameter attribute.

Overview

quatm/GUI/widgets/arg_boxes.py provides data widgets: a family of input boxes that present experiment parameters as editable controls. FloatBox / BoolBox / ComboBox and the other widgets subclass the corresponding manager (FloatManager / BoolManager / ComboManager), carry a unit, range and step, and write a changed value straight back to the attribute (including last_set); they are used by the parameter grid of the experiment window and by the parameter system.

Components

ComponentDescription
FloatBoxFloating-point parameter input box
BoolBoxBoolean parameter checkbox
ComboBoxDrop-down list parameter
IntManager / FloatManager / BoolManager / ComboManager / FrequencyManagerWidget managers for the individual parameter types

Usage

from quatm.GUI.widgets.arg_boxes import FloatBox, BoolBox, ComboBox

# usually the parameter system generates the matching widget from the parameter declaration

For a fuller description see Parameter boxes and Parameter system.

2 - Base Window Widgets

GUI base widgets: BWidget/BFrame/BMainWindow and searchable combo boxes, etc.

Overview

quatm/GUI/widgets/quatmQt.py provides the GUI base widget classes: BWidget / BFrame / BMainWindow are the base classes of the various windows/panels/main windows (with a property namespace _props, saving the geometry to QSettings on close and restoring it on start-up); SearchComboBox and others provide searchable combo boxes with autocompletion and filtering.

Components

ComponentDescription
BWidgetBase QWidget (properties + geometry persistence)
BFrameBase QFrame
BMainWindowBase main window (QMainWindow + title/geometry persistence)
CustomCompleterCompleter supporting contains / begins matching
SearchComboBoxSearchable combo box (with a built-in SearchLineEdit)
SearchLineEditCustom line edit of SearchComboBox (select all on focus, Esc restores)

Usage

from quatm.GUI.widgets.quatmQt import BMainWindow, BWidget, BFrame

win = BMainWindow(name="My window")   # geometry persisted automatically

For a fuller description see GUI base classes.

3 - Sequence Editor

Experiment code editing and sequence simulation: syntax highlighting, parsing and output-channel timing visualisation.

Overview

quatm/GUI/editors/editor_sequencer.py provides the sequence editor (edit and simulate): the CodeEditorParser composite window combines the CodeEditor code editor on the left with the sequence visualisation on the right; after editing an experiment script you can simulate a run with one click and the timing of the individual output channels is drawn as step plots.

Components

ComponentDescription
CodeEditorCode editor: line numbers, Python highlighting, automatic indentation and completion
PythonHighlighter / LineNumberAreaSyntax highlighting / line-number margin
ExperimentParserParses an experiment: buildExperiment / simulateExperiment return the command stack
ExperimentSequencerDraws the sequence over time as a step plot for each output channel
CodeEditorParserComposite window: Save / Simulate / Clear toolbar + dock layout

Usage

from quatm.GUI.editors.editor_sequencer import CodeEditorParser

win = CodeEditorParser(parent=parent, filename="experiments/my_experiment.py")

For a fuller description see Sequence editor.

4 - Experiment Run Manager

The core experiment execution engine: takes tasks from the queue and schedules them by measurement/sequence/run level.

Overview

quatm/GUI/experiment/experiment_manager.py provides the experiment run manager (ExperimentManager): the engine that takes tasks out of the experiment queue and executes them. The queue polling runs in a worker thread, and tasks are scheduled in the hierarchy measurement (measurement) → repetition (repetition) → sequence/scan (sequence) → single run (run).

Key methods

MethodDescription
queue_fnContinuously polls the task dictionary: picks tasks by priority, task number and reservation time
runTask-run wrapper: guarantees teardown however the task ends
pausePauses / resumes the current experiment
terminate_experimentGraceful termination: lets the current run finish naturally before stopping
start_measurement / start_sequence / end_sequence / end_measurementHooks for the individual experiment stages
set_dictWrites a single parameter to an attribute of the experiment object
set_run_nr / set_rep_nrSynchronises the run / repetition number
start_queue / shutdownStarts / stops the resident queue worker

Usage

from quatm.GUI.experiment.experiment_manager import ExperimentManager

manager = ExperimentManager(browser)   # browser supplies the queue and the interface
manager.start_queue()

For a fuller description see Experiment manager.

5 - Experiment Browser

The main application browser: file tree, queue/preparation station/looper and MDI experiment windows.

Overview

quatm/GUI/browser/ provides the experiment browser (Browser): the main application window of Quatm, built on QMainWindow + QMdiArea. The FileSelector on the left browses the experiment scripts, the experiment queue, preparation station and looper are docked at the top, and each experiment has its own ExperimentWindow sub-window in the central MDI area.

Core components

ComponentModuleDescription
Browserbrowser/main_window.pyMain window: MDI + dock widgets + lifecycle
FileSelectorbrowser/file_selector.pyExperiment file tree (double-click to open/simulate)
ExperimentWindowbrowser/experiment_window.pySingle-experiment control panel (parameters/submit/scan)
ExperimentQueue / PrepStationexperiment/Queue and preparation station (docked at the top)
Looperlooper/Conditional looper (docked at the top)

Lifecycle

  • open_experiment: opens an experiment window; if it is already open, it is restored and brought to the front.
  • closeEvent: saves the main-window geometry and the sub-window state, then stops the worker threads.
  • moveEvent / resizeEvent: debounced saving of the geometry for restoration on the next start.

Usage

from quatm.GUI.browser.main_window import Browser

browser = Browser()
browser.show()

For a fuller description see Experiment browser.

6 - Experiment Queue Manager

Tabular experiment task queue: task dictionary, priority sorting, pause/termination and reserved execution.

Overview

quatm/GUI/experiment/experiment_queue.py provides the experiment queue manager (ExperimentQueue ): it shows the task queue as a table (the task dictionary expDict, keyed by task number) and supports priority sorting, state tracking and controls such as pause/terminate/delete. submit() is the only enqueueing entry point on the GUI thread (task numbers are allocated centrally, and the Looper also goes through it via the submitItem signal).

Key methods

MethodDescription
submitThe single enqueueing entry point: allocates task numbers centrally
pausePauses/resumes the currently running experiment
set_sleepingSets a task to Sleeping / Queued (reserved execution)
terminate_clicked / terminate_allGracefully terminates the selected / all tasks (Termination Pending)
delete_item / delete_clickedDeletes a task (graceful termination if it is running)
update_itemRefreshes the row of a task
set_modelReplaces the table model (ScheduleModel)

Task-state enumeration: Queued / Waiting / Sleeping / Running / Paused / Scanning / Terminating / Termination Pending / Done / Failed.

Usage

from quatm.GUI.experiment.experiment_queue import ExperimentQueue

q = ExperimentQueue(browser=parent)   # embedded in the experiment browser
q.submit(task_dict)   # enqueue (called on the GUI thread)

For a fuller description see Experiment queue.

7 - Worker Task Manager

QRunnable multithreaded tasks: callback execution in Worker + WorkerSignals signalling back to the main thread.

Overview

quatm/GUI/widgets/browser_workers.py provides the worker task manager: Worker subclasses QRunnable and executes a callback in the QThreadPool thread pool, then sends finished / error / result / progress signals back to the main thread via WorkerSignals. It is used by the experiment browser (experiment queue, looper, etc.); the isomorphic helper_workers is used by alignment/auxiliary tools.

Components

ComponentDescription
WorkerQRunnable worker thread: setup, execution and teardown (including exception handling)
WorkerSignalsWorker-thread signals: finished / error / result / progress, etc.

Usage

from quatm.GUI.widgets.browser_workers import Worker

worker = Worker(fn=long_running, args=(...), kwargs={...})
worker.signals.result.connect(on_result)
QThreadPool.globalInstance().start(worker)

For a fuller description see Background worker threads.

8 - Auto-save Editor

Editor for the list of periodic HDF5 auto-save points in time: checkbox + date/time save instants.

Overview

quatm/GUI/editors/auto_save_editor.py provides the auto-save editor (AutoSaveEditor): a popup dialog for editing the “list of points in time” of the periodic HDF5 auto-save, where every row is one (tick mark + date/time) pair and a ticked row triggers one auto-save at the corresponding instant. The configuration is stored in a property (such as regular_auto_save_times).

Components

ComponentDescription
AutoSaveEditorAuto-save point-in-time editor (BWidget popup)
DateTimeWidgetSingle-row date/time + tick widget

Key methods

MethodDescription
load_AutoSaveEditor_propsReads the date/times and tick states from the properties (static)
update_propsWrites the date/times and tick states back to the properties (static)
update_datetime / update_tickedUpdates when the date/time / tick state of a row changes
acceptSaves the changes and closes the window

Usage

from quatm.GUI.editors.auto_save_editor import AutoSaveEditor

editor = AutoSaveEditor(props=props, max_items=20, parent=parent)
editor.show()

For a fuller description see Auto-save.

9 - Timeline Viewer

FPGA experiment timing visualisation: a step-plot viewer of the DAC/TTL output channels over time.

Overview

quatm/GUI/entries/experiment_timeline.py provides the timeline viewer (Timeline): it loads an experiment-timing JSON file (experiment.json), plots the variation over time of each DAC/TTL output channel as a step subplot, and the output-channel check panel determines which curves are visible; it is used to debug and verify the experiment timing.

Components

ComponentDescription
TimelineTiming-viewer main window: file loading, plotting and channel control
TimelinePlotterTiming-visualisation widget: draws the state of every output in use over time
SelectorWidgetOutput-channel check panel: select all / select none

Key methods (Timeline)

MethodDescription
load_fileLoads and parses the timing JSON, builds the time series and refreshes the channel selector
plot_experimental_sequenceCreates an individual subplot per output according to the channel selection and draws a step plot
decode_ttl_bitmaskSplits a 32-bit TTL bitmask into set/clear commands for the individual channels
construct_time_sequencesConverts the step-wise timing into times/vals sequences organised per channel
reverse_device_db_transformationReverse device-database transformation: maps internal names back to readable names

Usage

from quatm.GUI.entries.experiment_timeline import Timeline

tl = Timeline()
tl.load_file("experiment.json")   # load the timing file and plot it
tl.show()

For a fuller description see Timing browser.

10 - Multithreaded Worker Framework

QRunnable multithreaded worker framework: callback execution in Worker + WorkerSignals signalling back to the main thread.

Overview

quatm/GUI/widgets/helper_workers.py (and the isomorphic browser_workers.py) provides the multithreaded worker framework: Worker subclasses QRunnable and puts an arbitrary callback function into the QThreadPool thread pool for execution, and returns the result to the GUI main thread through WorkerSignals using finished / error / result / progress signals.

  • browser_workers: used by the experiment browser (experiment queue, looper, etc.);
  • helper_workers: used by alignment/auxiliary and similar tools.

Components

ComponentDescription
WorkerQRunnable worker thread: setup, execution and teardown (including exception handling)
WorkerSignalsSignals a worker thread can emit: finished / error / result / progress, etc.

Usage

from quatm.GUI.widgets.helper_workers import Worker

worker = Worker(fn=long_running, args=(...), kwargs={...})
worker.signals.result.connect(on_result)   # main-thread slot
worker.signals.error.connect(on_error)
QThreadPool.globalInstance().start(worker)

For a fuller description see Background worker threads.

11 - Loop Manager

Conditional loop executor: automatically repeat/jump/terminate a group of tasks based on measurement results.

Overview

quatm/GUI/looper/ provides the loop manager (Looper): task entries are organised as “loop groups (tabs)”, the entries of a group are executed in order, and the jumps or termination are decided from measurement results. LoopManager runs the loop in a background thread; the loop definition is persisted in loopfile.json and shared with the web remote-control platform.

Core logic

LoopManager.run_loop(): start from the first group, loop indefinitely until terminate
  └─→ execute the entries of the group in order
       ├─ Task: submit to the experiment queue, advance once complete (maximum count configurable)
       └─ Conditional: read data-stream fields and compare → jump if satisfied / else branch otherwise

Components

ComponentDescription
LoopGroupLoop group (one tab)
TaskItem / ListItemTask entry / task-list entry
ConditionalItem etc.Condition entries such as data conditions, arrival index and time
BaustellePanel for adding/removing/editing the entries of a group
DataManagerSubscribes to data streams and accumulates the latest measurement values
LoopManagerLoop executor (run_loop() / terminate(), background thread)

Usage

from quatm.GUI.looper import Looper

looper = Looper(parent=browser)   # looper tab of the dock at the top of the browser

For a fuller description see Looper.

12 - Preparation Queue Manager

Table models for prepared tasks (preparation station): the list model PrepModel and the queue ScheduleModel.

Overview

quatm/GUI/widgets/models.py provides base classes that map Python dictionaries / lists onto Qt table models (derived from the ARTIQ models). The preparation queue (preparation station) uses the list-type PrepModel: prepared tasks need no unique ID and their order can be changed freely; the experiment queue uses the dictionary-type ScheduleModel (sorted by priority, then by task number).

Core classes

ClassDescription
_SyncSubstructHelper wrapping the dictionary operations of the underlying container (append/insert/pop callbacks refresh)
DictSyncModelDictionary → table model base class
ListSyncModelList → table model base class
ScheduleModelExperiment queue (ExperimentQueue) model: tasks sorted by priority/task number
PrepModelPreparation queue (PrepStation) model: list type, order adjustable

Concrete business models implement three hooks in a subclass: convert (key/value/column to cell content), sort_key (sorting key) and flags (edit flags).

Usage

from quatm.GUI.widgets.models import PrepModel

model = PrepModel(init=prep_list)   # list of prepared tasks

The models are described in Data models; preparation-queue operations are described in Preparation station.

13 - Experiment Run Scheduler

The run scheduling mechanism: central enqueueing in the queue, workers fetching tasks by priority/reservation time.

Overview

Experiment run scheduling is accomplished by the experiment queue and the execution engine working together: submit() of the ExperimentQueue is the only enqueueing entry point on the GUI thread (task numbers are allocated centrally, and the preparation station and looper also enqueue through it); the resident worker polls the task dictionary in ExperimentManager.queue_fn and selects and executes tasks by priority → task number → reservation time (due_check) (measurement → sequence → run).

ComponentModuleDescription
ExperimentQueueexperiment/experiment_queue.pyTask queue: enqueueing, state tracking, pause/termination
ExperimentManagerexperiment/experiment_manager.pyExecution engine: picks tasks, runs measurement/sequence/run
PrepStationexperiment/prepstation.pyPrepared tasks, pushed into the queue
Looperlooper/Conditional loop, enqueued via submit

Scheduling flow

PrepStation / Looper / experiment window --submit()--> queue task dictionary
        ↓
ExperimentManager.queue_fn (worker thread, by priority/task number/reservation time)
        ↓
start_measurement → start_sequence → several runs → end_sequence → end_measurement

At run time pause / resume is supported (pause) as well as graceful termination (terminate_experiment, Termination Pending).

14 - Experiment Preparation

The preparation station: staging, editing, ordering and pushing tasks before they are sent to the queue (prepfile persistence).

Overview

quatm/GUI/experiment/prepstation.py provides experiment preparation (PrepStation): the preparation station in the dock at the top of the experiment browser, used for staging and editing tasks before they are sent to the experiment queue. The task list is displayed with a PrepModel table (its order can be changed freely) and persisted to prepfile.json.

Key methods

MethodDescription
push / push_row / push_firstPushes the selected/designated/first task to the experiment queue
push_to_looperPushes the selected task to the looper
move_up / move_downChanges the order of tasks
delete_clicked / set_sleepingDeletes / sleeps a task
open_editor / open_viewer / save_paramsEdits parameters, views parameters, saves them as JSON
update_prep_fileSaves the task list to prepfile.json
load_previousRestores the tasks of the previous session from prepfile.json

Usage

from quatm.GUI.experiment.prepstation import PrepStation

ps = PrepStation(browser=parent)
ps.push_first()   # push the first prepared task into the queue

For a fuller description see Preparation station.

15 - Process Manager

Distributed process panel: start/stop background processes grouped as Servers/Driver/GUI/Viewer/Monitor.

Overview

quatm/GUI/entries/processmanager.py provides the process manager: it reads the configuration grouped as Servers / Driver / GUI / Viewer / Monitor and generates a SingleProcess start/stop button panel for every script entry, managing the whole set of experiment-software processes in one place. A process is started with the current interpreter, either as a script or as a module (python -m), and carries the project root and the parent-process watchdog environment.

Components

ComponentDescription
ProcessManagerCategory panel: reads the configuration and generates a start/stop button for every entry
SingleProcessSingle-entry panel: start / asynchronous termination / status polling colouring

Key methods (SingleProcess)

MethodDescription
startProcessStops the old instance, then starts the child process with the current interpreter
stopProcessTerminates the child process asynchronously without blocking the GUI thread
updateStatusPeriodically polls the exit status and colours it (running / exited / crashed)

Usage

from quatm.GUI.entries.processmanager import ProcessManager

pm = ProcessManager(parent=parent)
pm.show()

For a fuller description see Process manager.

16 - Experiment Parameter Manager

The default experiment parameter system: DefaultExp aggregates device parameters, Dashboard edits them grouped by device.

Overview

quatm/GUI/entries/dashboard.py provides the experiment parameter manager: DefaultExp is the experiment base class that aggregates the attributes of all devices (build() walks the device database and groups them by type), and the per-type parameter managers (BasicManager / IntManager / FloatManager / BoolManager / ComboManager / FrequencyManager) generate input widgets with unit/range from the parameter declarations; the Dashboard panel presents the device groups as editable default-parameter sections.

Components

ComponentDescription
DefaultExpDefault experiment: aggregates all device attributes and groups them by device
DashboardDefault-parameter panel: generates parameter widgets grouped by device
BasicManager etc.Per-type parameter-widget managers (Int/Float/Bool/Combo/Frequency)

Usage

from quatm.GUI.entries.dashboard import Dashboard

panel = Dashboard(parent=parent)   # show the default parameters grouped by device

For a fuller description see Parameter system.

17 - Run Monitor

Data-stream monitoring GUI: data/image/command/message multi-tab monitoring and single-stream panels.

Overview

quatm/GUI/streammonitor/ provides the run monitor: MainWindow hosts monitoring tabs for data streams (Data), image streams (Image), commands (Command), the message log (Message) and history (History); each monitoring tab consists of a StreamMonitor that subscribes to a single data stream and displays it in real time.

Components

ComponentDescription
MainWindowMonitoring main window: multi-tab container + menu
StreamMonitorSingle-stream monitoring panel: subscribes to one data stream and displays it in real time
MessageTableModelMessage-log table model (timestamp/source/level/content, coloured by level)
ImageTableModelImage-stream metadata table (fixed time/source columns + dynamic key columns)

Usage

from quatm.GUI.streammonitor import MainWindow, StreamMonitor

win = MainWindow(parent=parent)   # multi-tab monitoring main window
panel = StreamMonitor(name="PMT", stream_type="Data", parent=parent)

For a fuller description see Data stream monitor.

18 - Collapsible Sections

Collapsible-section dialog: SectionExpandButton toggling + CollapsibleDialog defining the sections.

Overview

quatm/GUI/widgets/theexpandables.py provides collapsible section components: CollapsibleDialog (a QDialog subclass) organises several collapsible sections with a QTreeWidget whose header is hidden; each section header is a SectionExpandButton button, and clicking it expands/collapses the corresponding content.

Components

ComponentDescription
SectionExpandButtonButton that expands/collapses a section (QPushButton)
CollapsibleDialogBase class for dialogs to which collapsible sections can be added

Key methods (CollapsibleDialog)

MethodDescription
define_sectionsOverride this method to define all sections as (title, widget) tuples
add_sectionAdd one collapsible section
add_button / add_widgetCreate a top-level entry containing a button / attach a content widget

Usage

from quatm.GUI.widgets.theexpandables import CollapsibleDialog

class MyDialog(CollapsibleDialog):
    def define_sections(self):
        self.add_section("Parameters", some_widget)
        self.add_section("Advanced", advanced_widget)

dlg = MyDialog()
dlg.define_sections()
dlg.show()

For a fuller description see Collapsible components.