System architecture¶

This document provides a comprehensive overview of the GOLIAT system architecture, showing all major components and their relationships.

Architecture diagram¶

The diagram below is interactive: hold Alt (or Option on Mac) and use your mouse wheel to zoom in/out, or hold Alt and drag to pan around.

graph TB
    subgraph "Entry points"
        CLI[CLI Commands
study, analyze, parallel
worker, super_study]
    end
    
    subgraph "Main process"
        subgraph "GUI layer"
            ProgressGUI[ProgressGUI
PySide6]
            QueueHandler[QueueHandler
processes messages]
            WebBridgeMgr[WebBridgeManager
connection lifecycle]
        end
        
        subgraph "GUI components"
            StatusMgr[StatusManager
log display]
            ProgressMgr[ProgressManager
progress bars]
            DataMgr[DataManager
CSV tracking]
            GraphMgr[GraphManager
plot updates]
            UtilMgr[UtilizationManager
system metrics]
            SystemMon[SystemMonitor
CPU/RAM/GPU]
            Plots[Plots
matplotlib]
            Timings[TimingsTable
execution stats]
            PieCharts[PieChartsManager
time breakdown]
            TrayMgr[TrayManager
system tray]
        end
        
        subgraph "Web monitoring"
            WebBridge[WebGUIBridge
message forwarding]
            HTTPClient[HTTPClient
API requests]
            MsgSanitizer[MessageSanitizer
serialization]
        end
    end
    
    subgraph "Study process"
        subgraph "Orchestration"
            BaseStudy[BaseStudy
common logic]
            NearFieldStudy[NearFieldStudy]
            FarFieldStudy[FarFieldStudy]
            Config[Config
hierarchical JSON]
            Profiler[Profiler
timing & ETA]
        end
        
        subgraph "Core processing"
            ProjectMgr[ProjectManager
.smash files]
            Setup[NearFieldSetup
FarFieldSetup]
            SetupModules[Setup Modules
Phantom, Placement
Material, Gridding
Boundary, Source]
            SimRunner[SimulationRunner
iSolve / oSPARC]
            ResultsExt[ResultsExtractor
SAR extraction]
            Extractors[Extractor Modules
Power, SAR, Sensor
Reporter, Cleaner]
        end
        
        subgraph "Analysis"
            Analyzer[Analyzer
strategy pattern]
            NearStrategy[NearFieldStrategy]
            FarStrategy[FarFieldStrategy]
            Plotter[Plotter
visualization]
        end
        
        QueueGUI[QueueGUI
proxy interface]
    end
    
    subgraph "Worker system"
        SuperStudy[Super Study
config splitting]
        WorkerCLI[Worker CLI
assignment fetch]
        Assignment[Assignment
claim & run]
    end
    
    subgraph "Cloud execution"
        oSPARCBatch[oSPARC Batch
job submission]
        oSPARCClient[oSPARC Client
API wrapper]
        oSPARCWorker[oSPARC Worker
job monitoring]
    end
    
    subgraph "External services"
        S4L[Sim4Life Engine
iSolve.exe]
        oSPARC[oSPARC Platform
cloud compute]
        Dashboard[Monitoring Dashboard
PostgreSQL + Next.js]
    end
    
    subgraph "Communication"
        Queue[multiprocessing.Queue
GUI messages]
        InternalQueue[Internal Queue
web bridge]
    end
    
    %% Entry point flows
    CLI -->|study| ProgressGUI
    CLI -->|analyze| Analyzer
    CLI -->|parallel| ProgressGUI
    CLI -->|worker| WorkerCLI
    CLI -->|super_study| SuperStudy
    
    %% Main process flows
    ProgressGUI --> QueueHandler
    QueueHandler --> StatusMgr
    QueueHandler --> ProgressMgr
    QueueHandler --> DataMgr
    QueueHandler --> GraphMgr
    QueueHandler --> UtilMgr
    QueueHandler --> WebBridgeMgr
    
    ProgressMgr --> ProgressGUI
    GraphMgr --> Plots
    UtilMgr --> SystemMon
    DataMgr --> GraphMgr
    
    WebBridgeMgr --> WebBridge
    WebBridge --> InternalQueue
    WebBridge --> HTTPClient
    HTTPClient --> MsgSanitizer
    HTTPClient -->|POST| Dashboard
    
    %% Study process flows
    ProgressGUI -->|spawns| QueueGUI
    QueueGUI --> Queue
    Queue --> QueueHandler
    
    QueueGUI --> BaseStudy
    BaseStudy --> Config
    BaseStudy --> Profiler
    BaseStudy --> ProjectMgr
    
    BaseStudy --> NearFieldStudy
    BaseStudy --> FarFieldStudy
    
    NearFieldStudy --> Setup
    FarFieldStudy --> Setup
    Setup --> SetupModules
    Setup --> ProjectMgr
    Setup --> S4L
    
    NearFieldStudy --> SimRunner
    FarFieldStudy --> SimRunner
    SimRunner --> S4L
    SimRunner -->|batch mode| oSPARCBatch
    oSPARCBatch --> oSPARCClient
    oSPARCClient --> oSPARC
    
    NearFieldStudy --> ResultsExt
    FarFieldStudy --> ResultsExt
    ResultsExt --> Extractors
    ResultsExt --> S4L
    
    ResultsExt --> Analyzer
    Analyzer --> NearStrategy
    Analyzer --> FarStrategy
    Analyzer --> Plotter
    
    %% Worker system flows
    SuperStudy -->|uploads| Dashboard
    WorkerCLI -->|fetches| Dashboard
    WorkerCLI --> Assignment
    Assignment -->|runs| BaseStudy
    Assignment -->|reports| Dashboard
    
    %% Styling
    style ProgressGUI fill:#E1BEE7
    style WebBridge fill:#BBDEFB
    style BaseStudy fill:#BBDEFB
    style NearFieldStudy fill:#BBDEFB
    style FarFieldStudy fill:#BBDEFB
    style Setup fill:#C5E1A5
    style SimRunner fill:#FFE082
    style ResultsExt fill:#FFCCBC
    style Analyzer fill:#D1C4E9
    style Plotter fill:#D1C4E9
    style Dashboard fill:#E1BEE7
    style Queue fill:#FFE082
    style S4L fill:#FFCCBC
    style oSPARC fill:#FFCCBC

Hold "Alt" / "Option" to enable pan & zoom

Component descriptions¶

Entry points¶

CLI Commands: Main entry points (goliat study, goliat analyze, goliat parallel, goliat worker, goliat super_study)

Main process (GUI)¶

ProgressGUI: Main PySide6 window, manages all GUI components
QueueHandler: Polls multiprocessing queue (every 100ms), dispatches messages to GUI components and web bridge
WebBridgeManager: Manages web bridge lifecycle, collects system info, handles connection status

GUI Components:

StatusManager: Color-coded log display
ProgressManager: Overall and stage progress bars
DataManager: CSV file management for time series data
GraphManager: Coordinates plot updates
UtilizationManager: System resource monitoring (CPU, RAM, GPU)
SystemMonitor: Low-level system metrics via psutil and nvidia-smi
Plots: Matplotlib-based visualizations (time remaining, progress, utilization)
TimingsTable: Execution statistics display
PieChartsManager: Time breakdown visualization
TrayManager: System tray integration

Web Monitoring:

WebGUIBridge: Forwards GUI messages to dashboard API, throttles and batches
HTTPClient: HTTP request handling for /api/gui-update and /api/heartbeat
MessageSanitizer: Serializes non-serializable objects before sending

Study process¶

Orchestration:

BaseStudy: Common study logic, phase management, error handling
NearFieldStudy: Near-field simulation orchestration
FarFieldStudy: Far-field simulation orchestration
Config: Hierarchical JSON configuration with inheritance
Profiler: Timing tracking, ETA estimation, weighted progress

Core Processing:

ProjectManager: Sim4Life .smash file lifecycle, verification, metadata
Setup: Coordinates scene building (NearFieldSetup or FarFieldSetup)
Setup Modules: Specialized setup tasks (PhantomSetup, PlacementSetup, MaterialSetup, GriddingSetup, BoundarySetup, SourceSetup)
SimulationRunner: Executes simulations via iSolve.exe or oSPARC
ResultsExtractor: Orchestrates result extraction
Extractor Modules: Specialized extraction (PowerExtractor, SarExtractor, SensorExtractor, Reporter, Cleaner)

Analysis:

Analyzer: Strategy-based analysis orchestration
NearFieldStrategy: Near-field specific analysis logic
FarFieldStrategy: Far-field specific analysis logic
Plotter: Generates plots and visualizations

Communication:

QueueGUI: Proxy interface that mimics GUI but sends to multiprocessing queue

Worker system¶

Super Study: Splits config into assignments, uploads to dashboard
Worker CLI: Fetches assignments from dashboard, claims them, runs studies
Assignment: Individual config slice executed by a worker

Cloud execution¶

oSPARC Batch: Manages batch job submission and monitoring
oSPARC Client: Wraps oSPARC API for job submission
oSPARC Worker: Monitors job status, downloads results

External services¶

Sim4Life Engine: FDTD solver (iSolve.exe), scene building API
oSPARC Platform: Cloud compute platform for parallel execution
Monitoring Dashboard: Web-based monitoring (PostgreSQL database, Next.js frontend)

Communication channels¶

multiprocessing.Queue: Inter-process communication between GUI and study process
Internal Queue: WebGUIBridge internal queue for message throttling

Data flow¶

User starts study: CLI command → ProgressGUI spawns study process
Study process: QueueGUI → multiprocessing.Queue → QueueHandler → GUI components
Web monitoring: QueueHandler → WebGUIBridge → HTTPClient → Dashboard API
Worker mode: Worker CLI → Dashboard API → Assignment → Study process
Cloud execution: SimulationRunner → oSPARC Batch → oSPARC Platform

Key design patterns¶

Multiprocessing: GUI and study run in separate processes for responsiveness
Queue-based communication: Decoupled message passing between processes
Strategy pattern: Analysis strategies (near-field vs far-field)
Bridge pattern: WebGUIBridge forwards messages without coupling
Modular components: GUI and setup use modular, replaceable components
Configuration-driven: JSON configs drive simulation parameters