Work / Data pipeline · 2023

Amazon marketplace feed automation for an EU seller of VidaXL products

For an Amazon EU seller of VidaXL products, I built a Lumen pipeline that downloads the VidaXL catalogue each night, stages it in MySQL with change detection, and submits offer listings through the Selling Partner API Feeds flow. Each marketplace runs its own copy of the app with its own markup on the B2B price. The listings went live on Amazon Belgium, France, Italy and Spain.

  • case study
  • live
  • client work
  • data pipeline
  • client anonymized
Client
an Amazon EU seller of VidaXL products
Type
Data pipeline · E-commerce integration: Amazon SP-API listing feeds
Built with
PHP, Lumen, Amazon Selling Partner API (Feeds), double-break/spapi-php, league/csv, MySQL, cron
Also uses
Python
Last active
2023

Overview

This pipeline turns the VidaXL dropship catalogue into Amazon offer flat-file listings and submits them through the Selling Partner API Feeds endpoint, so one seller can list and reprice VidaXL stock on several Amazon marketplaces.

It was freelance work for an Amazon EU seller of VidaXL products, and the listings went live on Amazon Belgium, France, Italy and Spain.

Context and problem

The seller appears to have needed the whole VidaXL catalogue listed, priced and stocked on several Amazon EU marketplaces at once, from a single supplier feed.

My role

I built the integration: feed download and splitting, staging in MySQL, the Amazon offer template, SP-API submission and the cron schedule. Every commit in the Belgium repository is mine.

The France, Italy and Spain apps are not in git; they appear to be copies of the Belgium app that differ mainly in marketplace ID and markup.

Solution

Every night a crontab calls the app's routes in order: it downloads the VidaXL CSV at midnight, splits it into 8 parts, processes each part on its own staggered minute, then builds and submits offer files every 21 to 22 minutes until 06:59.

Rows are staged in a MySQL table with a status flag. New SKUs are inserted, and existing rows are updated only when the webshop price, B2B price or stock changed, so only new or changed offers go back to Amazon.

The offer file follows Amazon's offer template, keyed on EAN, with price, stock, weight and top-level category, and goes up in batches of 5,000 rows.

Key features

  • Split ingest: the catalogue is processed in 8 parts by separate routes on staggered cron minutes.
  • Change detection: only new SKUs and rows whose price or stock changed are listed again.
  • Per-marketplace pricing: the B2B price plus a fixed markup set for each marketplace.
  • Exclusion rules: skip-lists of SKUs and EANs, two brands filtered out by name, and rows with incomplete columns skipped.
  • Batched submission: 5,000 rows per feed, with the status flag stopping rows from being sent twice.

Architecture and stack

The design is pull, stage, template, submit, with one near-identical Lumen app per marketplace, each setting its own marketplace ID.

  • App: PHP 8 on Lumen 9.
  • Amazon: the SP-API Feeds flow through double-break/spapi-php: createFeedDocument, uploadFeedDocument, createFeed with POST_FLAT_FILE_LISTINGS_DATA, and getFeedDocument for the processing report.
  • Files: league/csv writes the tab-delimited offer template.
  • Data: a MySQL staging table whose status column marks rows as needing a listing or listed.
  • Scheduling: a crontab of curl calls to the app's routes, which also runs a Python split script.

Challenges

  • Reading the SP-API processing report back was never fully automated: the result document ID is a hardcoded placeholder, so checking the report was a manual step.
  • Fitting a whole supplier catalogue into the nightly window is likely why the CSV is split into 8 parts on staggered schedules.

Trade-offs

Running one app copy per marketplace probably kept each marketplace simple and independent, but it means every fix has to be made four times.

Client work: links and code are not public.

Skills demonstrated

  • Amazon Selling Partner API integration through the Feeds flow.
  • Supplier feed processing with split ingest and change detection.
  • Marketplace listing generation from flat-file templates, with pricing and exclusion rules.
  • Scheduled batch jobs with cron.
  • PHP backend development with Lumen and MySQL.
All projects →