Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
-
Updated
Aug 19, 2024 - Python
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
An orchestration platform for the development, production, and observation of data assets.
Turns Data and AI algorithms into production-ready web applications in no time.
SeaTunnel is a next-generation super high-performance, distributed, massive data integration tool.
🧙 Build, run, and manage data pipelines for integrating and transforming data.
Infinitely scalable, event-driven, language-agnostic orchestration and scheduling platform to manage millions of workflows declaratively in code.
The open source high performance ELT framework powered by Apache Arrow
Flink CDC is a streaming data integration tool
Upserts, Deletes And Incremental Processing on Big Data.
Privacy and Security focused Segment-alternative, in Golang and React
Jitsu is an open-source Segment alternative. Fully-scriptable data ingestion engine for modern data teams. Set-up a real-time data pipeline in minutes, not days
A data integration framework
Community-curated list of software packages and data resources for single-cell, including RNA-seq, ATAC-seq, etc.
Lean and mean distributed stream processing system written in rust and web assembly. Alternative to Kafka + Flink in one.
Apache DevLake is an open-source dev data platform to ingest, analyze, and visualize the fragmented data from DevOps tools, extracting insights for engineering excellence, developer experience, and community growth.
ingestr is a CLI tool to copy data between any databases with a single command seamlessly.
A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow
BitSail is a distributed high-performance data integration engine which supports batch, streaming and incremental scenarios. BitSail is widely used to synchronize hundreds of trillions of data every day.
Hop Orchestration Platform
Add a description, image, and links to the data-integration topic page so that developers can more easily learn about it.
To associate your repository with the data-integration topic, visit your repo's landing page and select "manage topics."