Skip to content

Component: Runtime

Canonical source: SmrtApps/PythonApp/python_core/runtime/README.md (mirrored below)

python_core.runtime

Overview & Responsibilities

  • Coordinates SmrtHub's Python-side runtime loop: state management, clipboard absorption, save queues, filetype-aware routing, and status reporting for the WinUI bridge.
  • Packages the asynchronous save pipeline (trigger → queue → worker → SmrtSpace persistence) alongside prompt tooling and SmrtReader HTML generation.
  • Provides the canonical /status contract expected by bridge diagnostics and health probes.
  • Displays filetype detection results in terminal snapshots for immediate visibility.

Dependencies & Integrations

  • Shares python_core.utils.logger.get_logger_for("python-runtime") for structured/text logging in %LOCALAPPDATA%/SmrtHub/Logs/python-runtime/ per Operational Data Policy.
  • Reads SmrtSpace configuration via python_core.config.load_or_seed_smrtspace_config() only when an active SmrtSpace is selected; otherwise defaults stay in-memory.
  • Cooperates with smrtdetect modules (filetype detection engine, filename extraction) and smrtspace_mgt for workspace routing.
  • Called by HTTP routes (python_core.net.routes) and C# supervisors that enqueue saves through trigger endpoints.
  • Receives clipboard fragments with pre-attached detection metadata from clipboard_routes.py.

Runtime Flows / Operational Workflows

Clipboard Ingestion & Detection Pipeline

┌───────────────────┐     ┌──────────────────────┐     ┌─────────────────────┐
│  C# Clipboard     │ ──► │  /clipboard route    │ ──► │  shared_state       │
│  Monitor          │     │  (clipboard_routes)  │     │  results["clipboard"]│
└───────────────────┘     └──────────────────────┘     └─────────────────────┘
                                    │                            │
                                    ▼                            ▼
                          ┌──────────────────────┐     ┌─────────────────────┐
                          │  filetype_detect     │     │  Terminal Snapshot  │
                          │  (for text fragments)│     │  shows detection    │
                          └──────────────────────┘     └─────────────────────┘
  • Filetype detection happens at ingestion in clipboard_routes.py, not at save time.
  • Text fragments arrive with detection metadata attached (type_id, group, confidence, extension).
  • shared_state preserves detection through deepcopy and displays it in terminal snapshots.
  • Detection is visible immediately when content is copied—no save required.

Save Pipeline

┌───────────────────┐     ┌──────────────────────┐     ┌─────────────────────┐
│  User triggers    │ ──► │  save_trigger        │ ──► │  save_queue         │
│  SmrtSave hotkey  │     │  snapshots state     │     │                     │
└───────────────────┘     └──────────────────────┘     └─────────────────────┘
                                                                 │
                                                                 ▼
┌───────────────────┐     ┌──────────────────────┐     ┌─────────────────────┐
│  SmrtSpace +      │ ◄── │  file_saver          │ ◄── │  save_worker        │
│  SmartSpace       │     │  (reuses detection)  │     │  daemon thread      │
│  Archive DB       │     │                      │     │                     │
└───────────────────┘     └──────────────────────┘     └─────────────────────┘
  • save_trigger.user_triggered_save() snapshots state under lock, normalizes payloads, and pushes work items into shared_state.save_queue only when a SmrtSpace is active. When no SmrtSpace is selected, it emits a shared-state alert for the UI and skips the save.
  • save_worker.start_save_worker() launches a daemon thread executing save_and_route_worker(), pulling payloads and delegating to file_saver.save_to_file().
  • file_saver.save_to_file() reuses pre-attached detection metadata from clipboard ingestion for consistency. Falls back to fresh detection only if metadata is missing (and defaults to .txt for multi-item clipboards to avoid noisy routing).
  • SmrtReader (HTML) is off by default for single-item saves (SmrtSpace default_format defaults to txt). It automatically turns on for multi-item saves and writes a unified save folder containing the .txt, .html, and any pasted images/files together (no internal asset subfolders). When SmrtReader is explicitly enabled for single-item saves, the unified folder is still used. SmartSpace Archive entries are appended for each saved artifact.
  • SmrtReader’s H1/title is the source title (metadata.source, e.g., web page title or local filename), while the SmrtSave-derived folder name is shown in the footer breadcrumb. Breadcrumb timestamps prefer structured saved_at_iso + saved_at_tz (fallback saved_at).
  • Single-item, text-only saves (SmrtReader disabled) write a lone .txt directly under SmrtSpace/<Source>/ (no .txt filetype folder, no date folder).
  • Multi-item saves collapse all content into one .txt with numbered [image n placeholder] markers and one SmrtReader .html file; images/files are stored beside the .txt and referenced in the HTML.
  • promptsave supplies GUI fallbacks for non-.txt saves when automation needs user selection; must run on the main thread.
  • status.get_status() returns an ok payload with version, uptime, and PID for /status endpoints.
  • Clipboard encounters provided by ClipboardMonitor are stored in shared_state.results['encounter'], propagated through the save queue, and logged to %ProgramData%/SmrtHub/Logs/encounter-log/encounter-log-<date>.ndjson for provenance exports (no per-file sidecars).

Structured Output Save Pipeline (ExtractStructuredText)

TriggerManager’s local structured OCR hotkey (HOTKEY_EXTRACT_STRUCTURED_TEXT) can POST structured OCR results to the Python bridge for persistence.

Flow:

┌─────────────────────┐     ┌──────────────────────────┐     ┌─────────────────────┐
│  TriggerManager     │ ──► │  POST /structured-output │ ──► │  save_queue         │
│  (local OCR)        │     │  (structured_output)     │     │                     │
└─────────────────────┘     └──────────────────────────┘     └─────────────────────┘
                                                                 │
                                                                 ▼
                                                     ┌─────────────────────┐
                                                     │  save_worker        │
                                                     │  structured saver   │
                                                     └─────────────────────┘
                                                                 │
                                                                 ▼
                                                     ┌─────────────────────┐
                                                     │  SmrtSpace          │
                                                     │  Structured Data    │
                                                     │  Output/<YYYY-MM-DD>│
                                                     └─────────────────────┘

Key rules:

  • The HTTP route validates payload shape and enqueues work; it does not block on disk IO.
  • Outputs are written under the active SmrtSpace root in Structured Data Output/<YYYY-MM-DD>/.
  • Filenames are requestId-based when available (idempotent) and may incorporate a sanitized origin filename stem; extracted content is never used for naming.
    • This prevents duplicate files when TriggerManager retries persistence POSTs.
    • If requestId is missing, a timestamp-based fallback name is used and collisions are deconflicted with (... ) suffixes.
  • Logs remain metadata-only (never log extracted text or structured document JSON).

Telemetry (structured JSON logs)

  • All structured-output telemetry events use a shared cross-component schema:
    • Canonical reference: README.Files/System/Policies/SmrtHub-StructuredOutput-Telemetry-v1.0.README.md
    • kind: always structured-output
    • eventVersion: 1
    • runId (optional) and requestId (optional) for correlation
    • outcome for result classification
  • python-runtime emits the following event names:
    • structured-output.worker.completed: save queue work item processed (includes relativePath, never absolute paths)
    • structured-output.persisted: persistence outcome (outcome is saved or duplicate-suppressed)

Clipboard Item Structure with Detection

Text items in shared_state.results["clipboard"] may include detection metadata:

{
    "type": "text",
    "content": "def hello(): print('world')",
    "detection": {
        "type_id": "code.python",   # Canonical type identifier
        "group": "code",             # Top-level group (code/data/doc/markup/etc.)
        "confidence": 0.85,          # Detection confidence (0.0–1.0)
        "extension": ".py"           # Recommended file extension
    }
}

Detection metadata is: - Attached at ingestion by clipboard_routes.py - Displayed in terminal snapshots by shared_state._format_snapshot_for_terminal() - Reused at save time by file_saver._prepare_text_fragment() - Available to HubWindow UI via shared_state.results["clipboard"]

Configuration & Log Paths

  • SmrtSpace preferences read from %APPDATA%/SmrtHub/Config/smrt-space/<Space>/smrtspace-config.json via config helpers (only when a SmrtSpace is selected).
  • User overrides for save format can be provided through python_core.config.user_flags (runtime overrides; not persisted).
  • Runtime logs (text + JSONL) land under %LOCALAPPDATA%/SmrtHub/Logs/python-runtime/ and are included in Support Bundles.
  • Encounter logs live beside other ProgramData evidence: %PROGRAMDATA%/SmrtHub/Logs/encounter-log/encounter-log-YYYYMMDD.ndjson (overridable via SMRTHUB_COMMON_APPDATA_OVERRIDE).
  • SmartSpace Archive DB lives under %LOCALAPPDATA%/SmrtHub/SmrtDB/smart-space-archive/archive.db (append-only).
  • Archive entries include metadata plus content text/JSON for search/export; image bytes are never stored.
  • Full-text search is backed by SQLite FTS inside the archive database.
  • SmrtReader template lives at runtime/HTML_output/smrtreader_template.html with inline assets (Cal Sans, logos) for offline viewing.

Testing & Validation Expectations

  • Run pytest -k runtime (or pytest for the full suite) when modifying routing rules, save flows, or shared state helpers.
  • Exercise the clipboard → save pipeline by launching the PythonApp in debug, triggering a save, and confirming output in the active SmrtSpace directory plus appended records in the SmartSpace Archive DB.
  • Verify detection visibility: Copy code to clipboard and confirm the terminal snapshot displays the detected filetype immediately.
  • For promptsave changes, perform manual GUI validation on Windows to ensure the dialog opens and returns expected paths.

Fire Drill Harness

  • python_core.runtime.fire_drill.run_fire_drill() seeds shared state with sample fragments, triggers user_triggered_save(), and blocks until the queue drains. Run with python -m python_core.runtime.fire_drill --text "hello" (and optional --image-path <file>) to validate the pipeline outside of WinUI.

Support Bundle / Diagnostics Touchpoints

  • Support Bundles ingest the python-runtime logs and SmrtSpace configuration; keep file paths consistent so exports remain accurate.
  • status.get_status() feeds the /status endpoint surfaced in diagnostics dashboards—document contract changes before roll-out.
  • Shared state snapshots are printed only on change to avoid console noise; operators rely on those summaries during live troubleshooting.
  • Detection results in terminal snapshots help operators verify filetype inference without triggering saves.