Skip to content
Back to Docs

Pipelines & Sources

Create data pipelines, configure sync options, and monitor job status.

What is a Pipeline?

A pipeline is a repeatable workflow that moves data from one or more sources to one or more destinations. Along the way, data passes through field mapping and optional transforms and filters.

Source→Field Mapping→Transforms→Filters→Destination

Creating a Pipeline

1

Navigate to Pipelines

Go to Pipelines in the sidebar and click Create Pipeline.

2

Name your pipeline

Give it a descriptive name like “Shopify → Google Merchant Center”. Optionally assign it to a project.

3

Add a source

Select an existing connection, choose the data type (e.g., Products), and set the sync trigger (Manual or Scheduled).

4

Add a destination

Select a destination connection, configure field mapping, and set the export trigger (Manual, Scheduled, or Dependency — runs automatically after source sync completes).

5

Run your pipeline

Click Run to trigger the first sync immediately, or wait for the next scheduled run.

Sources

Sources define where data comes from. Each source references a connection and specifies:

  • Data type — Products, Orders, or Customers
  • Sync trigger — Manual or Scheduled
  • Source type — Primary (main data) or Join (supplementary data merged during export)

Destinations

Destinations define where data goes. Each destination references a connection and specifies:

  • Export format — CSV, NDJSON, or native API
  • Export trigger — Manual, Scheduled, or Dependency
  • Field mapping — Maps source fields to destination fields

Scheduling

Schedule frequency depends on your billing plan:

PlanMin. Interval
FreeDaily
StarterHourly
GrowthEvery 15 minutes
BusinessEvery 5 minutes

Dependency triggers allow a destination export to run automatically after its source sync completes, regardless of schedule.

Pipeline Statuses

ACTIVE

Pipeline is enabled and will run on schedule.

PAUSED

Pipeline is temporarily paused. No jobs will be created until you activate it again.

ERROR

The last run failed. Check the Jobs tab for details. The pipeline will still attempt future scheduled runs.

IDLE

Pipeline has no schedule. It only runs when triggered manually.

SYNCING

A source sync is currently in progress.

EXPORTING

A destination export is currently in progress.

Pipeline Tabs

Each pipeline detail page has the following tabs:

Overview

Pipeline status, last run time, next scheduled run, and quick actions.

Sources

Manage source connections, data types, and sync triggers.

Destinations

Manage destination connections, export formats, and export triggers.

Data

Preview the data that was fetched in the last source sync.

Schema

View detected source fields and track schema changes over time.

Transforms

Configure field-level transformations applied during export.

Jobs

View all past and current jobs with status, duration, and error details.

Need help?

Reach out at support@snowforge.dev