Pipeline Configuration

Sense Core pipelines are configured with Pydantic models in s6.app.pipeline and s6.schema.pipeline_settings. PipelineLoader accepts either a config path or an already-built config object, validates the shared base fields first with PipelineConfigBase, then resolves pipeline_name to PipelineT1 and parses the T1-specific model.

Loading

  • s6 track --config <path> accepts JSON or YAML.

  • The CLI default is configs/pipeline.config.yaml.

  • PipelineLoader accepts either an explicit config path or an already-built PipelineConfigBase instance.

  • PipelineLoader and direct BasePipeline construction do not auto-discover repository config files when called with None.

  • CLI commands such as s6 track and s6 monitor may still supply their own default config path before calling the loader.

Shared Base Fields

PipelineConfigBase defines the fields common to every pipeline:

  • pipeline_name: str = "PipelineT1"

  • platform: PlatformConfig = PlatformConfig()

  • run_cpp: bool = False

  • run_level: RunLevel = "normal"

  • calibration_file: str = "configs/calibration.config.json"

run_level accepts performance, normal, dev, or debug. The loader coerces string values to the RunLevel enum.

Platform Settings

PlatformConfig selects the hardware prototype and the matching GStreamer shape:

  • prototype must be one of T1, V1, or V2

  • gstreamer.client or gstreamer.local may be supplied

  • camera_mappings.client and camera_mappings.local are optional built-in device-id to camera-key mappings for live capture

  • gstreamer.local is used by both s6 track --input gst-local and s6 track --input gst-local-v2

  • if both are omitted, the model builds a default remote pygst client config with the camera count expected by the selected prototype

  • the configured device count must match the prototype camera count:

    • T1 -> 2 cameras

    • V1 -> 3 cameras

    • V2 -> 4 cameras

  • each configured camera mapping must be a complete one-to-one assignment:

    • the mapping keys must exactly match the configured device ids for that source block

    • the mapping values must be a permutation of the platform camera keys

    • duplicate, unknown, partial, or cross-source mappings fail validation

  • gst-local-v2 reuses the same gstreamer.local schema but reads one combined local stream through pygst.client.CombinedPipeline; the local-capture multi-source sync knobs (stale_frame_timeout_sec, max_seq_skew) are ignored in that mode

  • when a built-in mapping exists for the active live source, live gst capture uses that mapping directly and bypasses heuristic camera identification

Pipeline Model

PipelineConfigT1 inherits the shared base fields and adds the T1 schema:

  • PipelineConfigT1

    • platform.prototype is fixed to T1

    • solver

    • tracking

    • tip

    • export PipelineConfig is an alias for PipelineConfigT1, and s6.schema.pipeline_config re-exports the same models for compatibility.

Sub-Configs

The shared sub-config fragments live in s6.schema.pipeline_settings:

  • TrackingConfig

    • enable_prediction

    • search_radius_px

    • trajectory_maxlen

    • search_radius_m

    • prediction_jitter_velocity_threshold

    • acceleration_rejection_threshold

    • prediction_displacement_ema_alpha

PipelineT1 extends that shared tracking block with tracking.roi_prediction_use_extrapolation, tracking.roi_prediction_recenter_timeout_sec, and tracking.roi_prediction_recenter_invalid_frames, and tracking.tracking_volume_radius, which default to true, 0.15, null, and 3.0.

  • SolverConfig

    • instrument_length

    • tip_mask_erase_radius

    • tip_line_distance_expected

    • tip_line_distance_tolerance

  • DetectionConfig

    • components_area_thresholds

    • fallback_margin_frac

  • RefineConfig

    • zoom_factor

    • patch_size

  • BoundaryConfig

    • smoothing_window

    • max_radius_change_frac

    • max_center_change_frac

    • default_center

    • default_radius

  • TipConfig

    • boundary_margin_px

    • tracking_box_radius_px

    • tracking_box_next_radius_px

    • refine_area_thresholds

    • suppression_radius_px

    • triplet_tip_source

  • ExportConfig

    • preview_size

    • preview_format

The models coerce common list-or-tuple inputs into typed tuples and validate basic bounds at load time, so malformed values fail early.

Example

calibration_file: configs/calibration.config.json
pipeline_name: PipelineT1
run_level: normal
solver:
  instrument_length: 0.135
  tip_line_distance_expected: 1.4
tracking:
  enable_prediction: true
  roi_prediction_use_extrapolation: true
  roi_prediction_recenter_timeout_sec: 0.15
  roi_prediction_recenter_invalid_frames: null
  tracking_volume_radius: 3.0
  search_radius_px: 150
  trajectory_maxlen: 20
  prediction_jitter_velocity_threshold: 60.0
  prediction_displacement_ema_alpha: 0.7

For PipelineT1, solver.tip_line_distance_expected defaults to 1.4 and solver.tip_line_distance_tolerance defaults to 0.2. When the loaded detector exposes at least three keypoints, T1 automatically skips mask output requests and mask-derived support-line fitting, then triangulates the predicted tip/turn/end triplet directly. tip.triplet_tip_source defaults to model; set it to refined to keep the existing local intensity-refined tip while using the model-predicted turn/end points. One-point detector models keep the mask-derived solver path. Both paths publish the display LineSegment3D, triangulated tip, turn_point, end_point, and evaluated tip-line distance under context["debug"]["targets"]["instrument_tip"]. At normal or higher run levels, T1 also publishes complete valid three-keypoint plus mask training targets under context["debug"]["training_targets"]["keypoint3_mask"], including per-camera tip, turn, end, and mask values. T1 keeps its output-pose gate in the native virtual camera-B basis, then serializes context["export"]["midpoint_3d"] and context["export"]["instrument_pose_quaternion"] into the visualizer basis at export time using the visualizer helper’s known B camera placement (camera at (0, 0, 0), facing +y, with +z as image-up). midpoint_3d is emitted in meters and is already in visualizer world coordinates, while instrument_pose_quaternion remains ordered as [x, y, z, w] and is already ready to apply directly to the original Three.js instrument asset in scene world. At normal-or-higher run levels, T1 also exports context["export"]["bgr_image_ll_base64"] and context["export"]["bgr_image_lr_base64"] as base64-encoded LL/LR bgr_image previews downsampled to one quarter of each source frame width and height; the fields are "" when preview generation is disabled. The fixed instrument model points loaded from configs/instrument-model-points-2.json are also converted from their Three.js/model-inspection basis into the native Sense/OpenCV basis before pose recovery. T1 also publishes tip_solve_valid in the instrument-tip debug payload. When all three observed pose points are present and rigid pose recovery succeeds, T1 also publishes pose_solve_valid = true; otherwise that flag remains false even if tip tracking itself is still valid. When tracking.enable_prediction is enabled, the LL/LR ROI windows are projected from the pipeline’s persistent world-space TrajectoryV2 spherical tracking frame. For T1, each accepted frame tracks the full observed tip_point/turn_point/end_point triplet only; if any one of those points is missing, the frame counts as an invalid tracker update instead of refreshing the sphere. TrajectoryV2 encloses the accepted triplet in a sphere, adds tracking.tracking_volume_radius as extra world-space padding, and T1 projects that sphere into each camera so the projected major axis becomes the square ROI dimension. When tracking.roi_prediction_use_extrapolation is false, T1 instead projects the last accepted TrajectoryV2.last_frame sphere for ROI following. ROI following is now driven only by trajectory state, not by OutputPoseGate status. T1 first tries the requested sphere frame, then falls back through older sphere-derived frames before resetting to the default LL/LR search windows after either tracking.roi_prediction_recenter_timeout_sec seconds without a valid projectable sphere, or tracking.roi_prediction_recenter_invalid_frames consecutive invalid tracker updates when that optional frame budget is set. During tolerated invalid updates, TrajectoryV2 keeps the last predicted sphere fixed instead of advancing it farther. Low-speed prediction jitter suppression is controlled by tracking.prediction_jitter_velocity_threshold and tracking.prediction_displacement_ema_alpha. The schema defaults remain 120.0 in the same world-space units as the triangulated tip velocity and 0.4, while the shipped T1 presets set these explicitly to 60.0 and 0.7 for lighter smoothing. T1 can also reject implausible solved tip samples when tracking.acceleration_rejection_threshold is set; rejected samples are treated as held misses in TrajectoryV2, while context["debug"]["targets"]["instrument_tip"]["tip_point_raw"] preserves the raw triangulation and context["debug"]["targets"]["instrument_tip"]["tip_tracking_filtered"] indicates when the solver/export stayed on the predicted tracked-point path instead. EMA smoothing is prediction-only and applies to the persistent pipeline-owned TrajectoryV2; its accepted displacement, velocity, and acceleration values remain raw. The ROI tracking box turns green only when extrapolation is enabled and that EMA-smoothed prediction path is in use. T1 now also gates exported pose telemetry through tracking.output_pose_gate: export enters tracking after stable_frames_required stable pose frames, tolerates max_drop_frames short dropouts by holding the last stable pose, leaves tracking after unstable_frames_to_lost unstable frames, smooths translation across smoothing_window stable samples, and smooths rotation across rotation_smoothing_window stable samples. export.flags.midpoint_valid is therefore true only when the gate is in tracking or hold-drop, while midpoint_3d and instrument_pose_quaternion continue to mirror the last gated pose until a newer gated pose replaces it or the pipeline state resets. TrajectoryV2 owns the ROI follow/recenter behavior from its persistent sphere state, while the output-pose gate remains export-only. TrajectoryV2 remains responsible only for tip-motion prediction and trajectory-level outlier rejection. Pose export validity now depends on the instrument-tip debug pose_solve_valid, so tip-only or two-point frames no longer count as valid poses.

Notes

  • The checked-in configs/pipeline.config.yaml currently selects PipelineT1, and it is the CLI default path used by s6 track and s6 monitor when --config is omitted.

  • configs/pipeline_t1.config.yaml is an additional repository preset, but it is not auto-discovered by the loader.

  • The pipeline loader reads YAML when the extension is .yml or .yaml and JSON otherwise.