Shopify to BigCommerce catalog migration for a US lighting retailer

Overview

A command-line tool that moves a lighting catalog exported from Shopify into a BigCommerce store: it creates the missing products with their variants, images, categories, brands and SEO fields, syncs images onto products that already exist, and comes with a script that can wipe the catalog for a fresh run.

It was freelance work for a US lighting retailer, and it also included preparing the retailer's customer list for BigCommerce's import format.

Context and problem

The retailer was moving its catalog from Shopify to BigCommerce, and the job appears to have been about carrying variants, images, taxonomy, SEO and technical-file metadata across without re-entering them by hand, in runs that could be checked first and picked up again after a failure.

My role

This was my freelance client work.

Neither folder is under git, so judging by where the files sit, the migration CLI, its BigCommerce API client, the catalog wipe script and the README that documents them are all mine.

Solution and user experience

The README leads with a dry run: the operator previews what would be created and which images would be added, without writing anything, and gets a report for that preview.

From there a run can be narrowed with a product limit for testing and a vendor filter, so the catalog can go across one vendor first and then the rest with --resume; dry runs never touch the saved state.

Three modes cover the work: import-new creates only missing products, sync-images brings existing products up to date, and both does the two in one pass.

Every run writes a per-product CSV and a JSON summary, and failed products can be re-run on their own with --retry-failed.

Key features

• Matching: existing products are found by SKU first, with an optional exact-name fallback.
• Taxonomy: category paths such as "A > B > C" are resolved or created parent to child to leaf, with the Shopify product type as a fallback; brands come from the Shopify vendor and are created when missing.
• Field mapping: SEO title and description, SKU to SKU and MPN, barcode to UPC, weight from grams to pounds, installation-instruction metafields to a "Technical Files" block in the description, and search boosts to search keywords.
• Variants: multi-variant products are built from the Shopify option columns.
• Existing products: they are never re-created; their SEO fields are updated from the export and missing images are added.
• Visibility: Shopify drafts and unpublished products are created hidden in BigCommerce.
• Idempotent image sync: images are de-duplicated by a filename key, so a rerun does not add the same image twice.
• Resumable runs: checkpoint and resume, retry of failed products, and per-run CSV and JSON reports.
• Customer list: a separate step reformatted the customer export to BigCommerce's import layout and filled missing required fields such as email, name and country.

Architecture and stack

The tool is a single-file pipeline: it parses every Shopify products export into product and variant dataclasses, indexes the products, categories and brands already in BigCommerce, creates or updates each product, then saves its state and writes the reports.

• Language: Python 3.9+ with only the standard library (urllib, csv, sqlite3, argparse, dataclasses) and no third-party dependencies.
• API: a small client class for the BigCommerce Catalog API v3 covering products, variants, images, categories and brands.
• State: SQLite holds one row per mode and product handle, which is what resume and retry read.
• Wipe script: a separate script lists or deletes catalog products through the same API.

Implementation details

Each state row keeps separate statuses for product creation, image sync and SEO update, the matched or created product ID and the last error; a failure in any step marks the product as failed, so --retry-failed picks it up on the next run.

API calls retry with exponential backoff on 429 and 5xx responses, and wait at least as long as the Retry-After header asks.

The wipe script deletes nothing unless --confirm-delete-all is passed, even when --dry-run is left off; --limit deletes only the first N products, and the README says to reset the local state database after a full wipe so the next import starts clean.

Challenges

Long catalog runs against the API likely ran into rate limits and transient errors; backoff that honours Retry-After, per-product state and resumable runs suggest the tool was built so a failed run could carry on where it stopped.

Trade-offs

Using only the standard library likely makes the script easy to run on any machine with Python, at the cost of hand-written HTTP, retry and pagination code.

Links

Client work: links and code are not public.

Skills demonstrated

• E-commerce platform migration from Shopify to BigCommerce, including catalog field mapping.
• REST API client design with retries and backoff.
• Resumable batch data processing with SQLite state.
• Python CLI tooling with safe defaults: dry runs, staged rollout and a confirm flag for destructive steps.