ETL / ELT & Ingestion

The "E" and "L" of ELT. After data lands, transform it with Data Transformation & Modeling. For row-level streaming change capture, see Change Data Capture. For pushing modeled data back out to SaaS, see Reverse ETL.

Connector platforms (ELT)

• ★ Airbyte — 400+ connectors, Python / low-code CDK; ELv2 license (free to self-host, restrictions on hosted competitors). Hosted Airbyte Cloud has a small free tier. The default OSS connector platform.
• ★ Meltano — Singer-protocol-based; CLI-first, Git-versioned config; Apache 2.0. Best when you live in YAML and prefer composable taps / targets.
• Estuary Flow — real-time CDC + connectors; commercial, generous free tier (10 GB/month). Streaming-first; good when you want sub-minute latency.
• Hevo Data, Stitch (Talend), Fivetran — paid ELT SaaS; Stitch and Fivetran have small free tiers; Fivetran is the enterprise default and expensive.
• Apache SeaTunnel — Apache 2.0; Java/Spark/Flink connectors; popular in Asia and for k8s-native ELT.

Code-first ingestion (Python)

• ★ dlt (dlthub) — Python library: pip install dlt, write a generator, schema is inferred and evolved automatically; loads to Postgres / BigQuery / Snowflake / DuckDB / Iceberg. Apache 2.0. The most-recommended pick for "I'd rather write Python than configure a UI" in 2026.
• Singer — open standard for taps and targets; many community taps exist. Powers Meltano. The protocol is more relevant than running raw Singer scripts in 2026.
• PyAirbyte — Airbyte connectors as a Python package; useful when you want Airbyte sources without the Airbyte server.
• Sling — Go-based CLI for moving data between DBs / files / S3 / warehouses; very fast. MIT.

Warehouse-native loaders

• Snowflake COPY INTO, BigQuery LOAD DATA, Redshift COPY — when the data is already in object storage, native bulk loaders beat any tool.
• DuckDB COPY / httpfs — pull Parquet / CSV directly from S3.
• ClickHouse s3() / url() — same idea.

Streaming ingestion

• Kafka Connect — JDBC, Debezium, S3 sink/source; battle-tested.
• Apache NiFi — visual DAG of dataflow; Apache 2.0; older enterprise classic.
• Benthos / RedPanda Connect — stream-processor-meets-ingestion; great for event-shaped sources. MIT (Benthos was renamed RedPanda Connect after acquisition).
• Vector (Datadog) — observability data, but capable for general structured ingestion; see Log Aggregation.

SaaS-specific connectors worth knowing

• stripe-cli + webhooks → S3 → DuckDB — common DIY pattern that beats a "Stripe → Snowflake" connector for hobby budgets.
• GitHub Archive / gharchive — public dataset; load directly with DuckDB.
• Segment OSS replacement: RudderStack — see Event Tracking & CDP.

License & licensing watch-outs

• Airbyte is ELv2 — fine to self-host, can't run a competing hosted Airbyte service.
• Meltano, dlt, Sling, SeaTunnel, NiFi, Singer are permissive (Apache 2.0 / MIT).
• Fivetran, Stitch, Hevo, Estuary are commercial SaaS — read free-tier limits.

Pick this if…

• Default OSS connector breadth: Airbyte.
• Code-first, Pythonic, evolving schemas: dlt.
• YAML-config / Git-flow: Meltano.
• Real-time / sub-minute, willing to pay: Estuary Flow or Fivetran HVR.
• Just move bytes, no UI: Sling.
• Source is a database; you want every change: see Change Data Capture (Debezium / PeerDB).
• Source is event streams: Kafka Connect or RedPanda Connect.