Skip to content

data-imports.md

Latest commit

 

History

History
50 lines (33 loc) · 5.14 KB

data-imports.md

File metadata and controls

50 lines (33 loc) · 5.14 KB
Raw

Data Imports

openclaw-rtt imports normalized artifacts from OpenClaw channel QA runs. The measurement harness lives in openclaw/openclaw; this repo stores the rows and dashboard artifacts.

The Main RTT workflow builds and packs OpenClaw before measuring Telegram. Its packaging step requires the complete Node/npm toolchain, installed after pnpm's runtime setup. A runtime-only Node installation can fall back to the runner's older npm, which drops native-package libc constraints and fails OpenClaw's package-lock validation.

Import Commands

Run importers from the repo root:

node scripts/import-result.mjs ../openclaw/.artifacts/qa-e2e/npm-telegram-live/<run>/qa-evidence.json --spec <package-spec> --version <version> --started-at <iso> --finished-at <iso>
node scripts/import-result.mjs ../openclaw/.artifacts/qa-e2e/npm-telegram-live/<run>/qa-evidence.json --version <version> --started-at <iso> --finished-at <iso> --resource-metrics resource-metrics.env
node scripts/import-discord-rtt.mjs samples.tsv --spec openclaw@main --version <ref>
(cd ../openclaw && node --import tsx ../openclaw-rtt/scripts/measure-rpc-rtt.mjs --output-dir ../openclaw-rtt/.artifacts/rpc-rtt/sample-1)
node scripts/import-surface-rtt.mjs rpc-samples.tsv --surface rpc --spec openclaw@main --version <ref> --provider-mode gateway-rpc --scenario rpc-gateway-smoke --require-pass
node scripts/import-surface-rtt.mjs samples.tsv --surface control-ui --spec openclaw@main --version <ref>
node scripts/backfill-release-rss.mjs --family discord --spec openclaw@2026.5.16 --version 2026.5.16 --sample-paths samples.tsv
node scripts/summary.mjs

Sample-paths TSV files accept LF or CRLF line endings and ignore empty lines. Tabs separate paths; spaces within each path are preserved. Surface TSV columns are summary, resource metrics, and performance events; channel import and RSS backfill columns are summary, observed messages, and resource metrics. Leave an optional column empty when supplying a later column.

Telegram release imports expect the aggregate qa-evidence.json shape emitted by the OpenClaw package Telegram live lane. The OpenClaw harness checkout must include the package Telegram RTT evidence path; older OpenClaw packages can still be the system under test.

OPENCLAW_NPM_TELEGRAM_PACKAGE_SPEC=openclaw@beta \
pnpm test:docker:npm-telegram-live

Data Layout

  • data/channels/<channel>/<version>.jsonl: append-only graph source, one normalized run per line for that channel/version bucket.
  • runs/<channel>/<run-id>/result.json: copied per-run record for audit/debug.
  • data/surfaces/<surface>/<version>.jsonl: non-channel RTT surfaces such as RPC and Control UI.
  • runs/surfaces/<surface>/<run-id>/result.json: copied per-surface record for audit/debug.

Current channel folders are telegram, discord, slack, and whatsapp. Telegram and Discord still have specialized importers because their source artifact shapes differ; they now share the same storage contract as generic live-transport channels.

Readers also accept legacy data/channels/<channel>.jsonl files alongside versioned channel directories. Aggregate reports read each source file once, including when both layouts exist for the same channel.

Raw Telegram QA artifacts stay in the OpenClaw repo artifact directory unless explicitly copied here later. Historical Telegram rows imported from the old package RTT wrapper may include rtt.warmSamples with every successful sample value. New Telegram imports from qa-evidence.json preserve the aggregate dashboard metrics (canaryMs, mentionReplyMs, avgMs, p50Ms, p95Ms, maxMs, sampleCount, and failedSamples) but do not reconstruct individual sample RTT arrays because the evidence artifact stores aggregate timing.

Release RSS backfills only write resources onto an existing Telegram or Discord row and its copied result.json. The backfill command asserts the stored RTT p50 and p95 values are unchanged before it rewrites that version's JSONL file. RSS is process-level data around the sampled command, not isolated channel transport memory.

RPC rows come only from scripts/measure-rpc-rtt.mjs, which starts an isolated loopback Gateway, warms each method once, and measures persistent WebSocket calls such as health and config.get. Channel request/reply timings are not valid RPC measurements. Control UI rows should be imported with explicit scenario RTT measurements or control-ui.* performance events.

Surface imports validate every sample's start and finish timestamps before writing data. A finish may equal its start but must not precede it. Run bounds cover the earliest start through the latest finish, regardless of sample order, and are stored in UTC; sample order and RTT measurements are preserved.

Release QA compatibility patches serialize candidate auth stores in separate processes, each with its own QA state directory. The patcher stages all source edits before replacing files and attempts to restore the originals if a replacement fails; any files requiring manual recovery are named in the error. Each file replacement is atomic, but the multi-file patch is not a filesystem transaction and should run before starting the QA harness.