PSI DataSync

PSI DataSync synchronizes Intego measurement data (geometry CSVs and daily statistics) from six shop-floor ARGO PCs to a centralized Azure Files share. It replaces GoodSync Control Center with a custom-built system designed to eliminate Azure Storage transaction costs.

Architecture

┌─────────────────────────────────────────────────────┐
│                PSI Shop Floor (On-Prem)               │
│                                                       │
│  ┌────────┐ ┌────────┐ ┌────────┐                    │
│  │ ARGO1  │ │ ARGO2  │ │ ARGO3  │  ... ARGO4-6      │
│  │ Agent  │ │ Agent  │ │ Agent  │                    │
│  └───┬────┘ └───┬────┘ └───┬────┘                    │
│      └──────────┼──────────┘                          │
│                 │ VPN                                  │
└─────────────────┼─────────────────────────────────────┘
                  │
┌─────────────────┼─────────────────────────────────────┐
│           Azure VNet (PS-VNMAIN)                       │
│                 │                                      │
│         ┌───────▼────────┐    ┌──────────────────┐    │
│         │  DataSync      │    │  Azure Files     │    │
│         │  Server        │    │  (psargostorage) │    │
│         │  (App Service) │◄──►│  argodatastore   │    │
│         │                │    │                  │    │
│         │ • REST API     │    │  /Argo-Export/   │    │
│         │ • Dashboard    │    │  /Daily-Stats/   │    │
│         │ • Alert Engine │    │                  │    │
│         └────────────────┘    └──────────────────┘    │
│                 │                                      │
│         ┌───────▼────────┐                             │
│         │  Azure SQL     │                             │
│         │  (DataSync DB) │                             │
│         └────────────────┘                             │
└────────────────────────────────────────────────────────┘

Components

Component	Technology	Hosting
Server API	Node.js / Express	Azure App Service (psi-datasync)
Dashboard	React / TypeScript / TailwindCSS	Served by Express (same App Service)
State Database	Azure SQL	DataSync on procserv-proddata
Agent	.NET 8 Windows Service	Each ARGO PC
File Storage	Azure Files	\\psargostorage.file.core.windows.net\argodatastore

How It Works

The system avoids expensive Azure Files list/read transactions by maintaining a server-side state index in Azure SQL. Agents never query Azure Files to determine what needs syncing — they ask the server.

1. Agent scans local source folders for new/changed files
2. Agent sends a file manifest to the Server
3. Server compares the manifest against its state index database
4. Server returns only the delta (new/changed files)
5. Agent copies delta files directly to Azure Files via SMB
6. Agent reports completion; Server updates its index

Folder sealing prevents re-scanning completed date folders. Once a geometry date folder has been inactive for 24 hours, it’s sealed — the agent skips it entirely on future cycles.

URLs and Access

Resource	URL	Access
Dashboard	https://psi-datasync.azurewebsites.net	Azure AD SSO (internal network via private endpoint)
API Health	https://psi-datasync.azurewebsites.net/api/health	Unauthenticated
GHE Repository	https://progressivesurface.ghe.com/ProgressiveSurface/PSISync	GHE access

Jobs

DataSync manages 20 sync jobs across 6 ARGO PCs:

Category	Per ARGO	Total	Schedule	Type
Geometry (ARGO1-2, Pattern A)	3	6	Every 4 hours	Append-only CSVs
Geometry (ARGO3-6, Pattern B)	2	8	Every 4 hours	Append-only CSVs
Daily Stats (all)	1	6	Every 15 min	Mutable .txt files

Dashboard

The dashboard provides:

• Health Overview — All runners with traffic-light status, sync volume, active alerts
• Runner Detail — Per-runner status, heartbeat, disk space, assigned jobs
• Job Detail — Sync history, folder seal status, manual seal/unseal
• Sync Run Log — Per-file drill-down for any sync run
• Alert Management — Active alerts, configurable rules, suppression windows
• Credential Management — Azure Files credential tracking with expiry alerts
• Cost — Per-job Azure Files transaction cost (see below)

Cost Accounting

The Cost dashboard tab attributes Azure Files spend to individual jobs. Phase 1 is self-attributed: each completed sync_run records the billable write operations (1 CreateFile + ceil(bytes/4MiB) PutRange per OK file) and bytes written; each triangulation_run records the ListFilesAndDirectories pages consumed by the walk. Costs apply the Azure Files Standard LRS Hot North Central US pricing card (verify on contract renewal).

Phase 2 will pipe storage diagnostic logs to a Log Analytics workspace and reconcile per-URI requests back to job ids; variance >5% between Phase 1 estimate and Phase 2 truth raises an alert.

Endpoints (dashboard auth or X-CI-Secret):

• GET /api/admin/cost?since=&until= — per-job rollup
• GET /api/admin/cost/jobs/:jobId?since=&until= — daily series for one job

Local SSD Backup (last-resort layer)

Each ARGO PC also has a USB SSD as a second, fully independent redundant copy of Intego data — a last resort if cloud sync, the network, or the DataSync server itself is down. It runs from its own Scheduled Task (PSIDataSync-SSDBackup, daily at 02:00), executing SSDBackup.ps1 — a generic script with no hardcoded paths; it decrypts the same DPAPI state.dat the agent uses to pull its RunnerId, then fetches that runner’s actual job/source-path config from the server at runtime.

• Retention: 12-month floor (vendor requirement), prunes oldest only if free space drops below 5%.
• Self-updating: the script polls the server for newer versions each run, with last-known-good fallback if the check itself fails — the backup must never be skipped because an update check failed.
• Reports to: ssd_backup_runs (per-run history) — visible on Health Overview’s “Local SSD Backup” table (drive letter, free space, last run, status, and a “Scheduled Task” column showing Registered/Not registered/checking…).
• Provisioning: the installer registers the Scheduled Task under the same account configured for the PSIDataSync service — the only point in the system where the account name and its password are ever in scope together (schtasks /Create /RU <account> needs the password up front; neither the running service nor the agent’s own maintenance worker can supply one afterward, and that’s intentional — it’s never stored). The agent’s SsdBackupProvisionWorker keeps the script current on disk and reports task presence every ~30 min, but does not recreate a missing task itself (would require SYSTEM-context recreation, reintroducing the network-share access problem service mode exists to solve).
• Alerts (not gated on alert_rules — same reasoning as backup_at_risk, this is the last-resort layer and must not be silently disable-able): ssd_task_missing (critical, immediate), ssd_missing (critical — the drive itself isn’t present), ssd_backup_stale (warning), ssd_space_critical (critical).

Pilot: ARGO1 + ARGO6 (service mode required — LocalSystem can’t reach network-path sources, so this is gated on the same service-account conversion tray-mode PCs need for agent self-update too). See GHE #35 for status and #41 for the fleet-wide tray→service conversion needed before ARGO2/3/4/5 can run this or self-update at all.

Alerting

Alert	Trigger	Severity / Delivery
Consecutive Failures	3 failed sync runs	Critical (pushed: Teams + Email)
Runner Offline	No heartbeat for 6 hours	Critical (pushed)
Stale Sync	Last success > 2x schedule	Warning (dashboard only)
No New Data	Geometry job synced zero new files for 6x its schedule while a sibling job on the same runner is producing — the “source moved/stopped” detector (added after the 2026-08-31 ARGO5 path change)	Critical (pushed)
Credential Expiring	Runner SAS <12h to expiry (agent refreshes at <24h, so this means refresh is failing) or expiry unknown	Warning (dashboard only)
Credential Critical	Runner SAS <4h to expiry or already expired	Critical (pushed)
Backup At Risk	Geometry job failed + real unsynced backlog, unrecovered for a full cycle — see ops runbook	Critical (pushed)
Chronic File Failure	Same file failed in every one of the last 3 completed runs; also raised immediately if a seal is blocked by a residual failure	Critical (pushed)
Retention Behind	agent_logs rows past the retention cutoff exceed threshold (the prune sweep is failing or behind)	Warning (dashboard only)
SSD Task Missing	Local SSD backup’s Scheduled Task reported unregistered	Critical (pushed)
SSD Missing	Local backup SSD drive not present	Critical (pushed)
SSD Backup Stale	No local SSD backup run reported within threshold	Warning (dashboard only)
SSD Space Critical	Local backup SSD free space below 5%	Critical (pushed)

Delivery follows the push policy (auditable at /api/diag/fleet → pushPolicy): critical alerts push to Teams + email; warnings are dashboard-only (PSI Alerting Standard alert-fatigue gate). Only consecutive_failures, runner_offline, and stale_sync honor the alert_rules table (enabled + threshold); every other type is deliberately always-on — they protect against data loss / total backup failure and must not be silently disable-able. partial_sync and disk_space_low were removed 2026-08-31 (documented types that had no implementation — GHE #150).

Alerts auto-resolve when conditions clear. Suppression windows can be configured for planned maintenance. backup_at_risk and every ssd_* alert are deliberately not gated on alert_rules — they protect against data loss / total backup failure and must not be silently disable-able.

Related

• Deploy to Azure — App Service infrastructure
• PRD and architecture docs in the PSISync repository under docs/