← Back to clyva.com
Engineering Note · Catalog ETL, FFmpeg & Visual QA

Shipping High-Volume eCommerce Pipelines Without Silent Failures

Most eCommerce automation scripts break quietly at SKU #400—corrupted CSV variants, broken CDN images, unescaped Liquid markup, or horizontal overflow on 375px phones. Here is the production architecture I use across multi-thousand-SKU storefronts.

150,000+ SKUsCatalog Extraction & Migration

Built resumable Python + SQLite ETL workers with deterministic checkpointing, schema validation, variant deduplication, and automated retry backoffs.

5,841 SKUsLive Shopify Storefront Upgrade

Rebuilt a live Dawn-based Shopify theme and processed thousands of product images with strict aspect-ratio, resolution, and broken-link gates prior to publish.

1280px + 375pxAutomated Browser Visual QA

Headless Chromium + CDP scripts capture desktop and mobile viewports, assert zero horizontal overflow, flag broken images (naturalWidth === 0), and verify live DOM state.

1. Programmatic Media & Storefront QA Gate (Python + FFprobe / CDP)

Before shipping storefront updates, email templates, or batch ad creatives, every asset runs through a lightweight command-line verifier that blocks defective builds early:

{
  "storefront_check": {
    "viewports": ["1280x820 desktop", "375x812 mobile@2x"],
    "mobile_horizontal_overflow": false,
    "broken_images_natural_width_0": 0,
    "klaviyo_tracking_verified": true,
    "schema_and_meta_complete": true
  },
  "media_gate_ffprobe": {
    "aspect_ratio_verified": "9:16 | 1:1 | 4:5",
    "audio_stream_present": true,
    "bitrate_and_codec": "h264 / aac"
  }
}

2. Typical Fixed-Scope Engineering Sprints