Average Ratings 0 Ratings

Total
ease
features
design
support

No User Reviews. Be the first to provide a review:

Write a Review

Average Ratings 0 Ratings

Total
ease
features
design
support

No User Reviews. Be the first to provide a review:

Write a Review

Description

Airflow is a community-driven platform designed for the programmatic creation, scheduling, and monitoring of workflows. With its modular architecture, Airflow employs a message queue to manage an unlimited number of workers, making it highly scalable. The system is capable of handling complex operations through its ability to define pipelines using Python, facilitating dynamic pipeline generation. This flexibility enables developers to write code that can create pipelines on the fly. Users can easily create custom operators and expand existing libraries, tailoring the abstraction level to meet their specific needs. The pipelines in Airflow are both concise and clear, with built-in parametrization supported by the robust Jinja templating engine. Eliminate the need for complex command-line operations or obscure XML configurations! Instead, leverage standard Python functionalities to construct workflows, incorporating date-time formats for scheduling and utilizing loops for the dynamic generation of tasks. This approach ensures that you retain complete freedom and adaptability when designing your workflows, allowing you to efficiently respond to changing requirements. Additionally, Airflow's user-friendly interface empowers teams to collaboratively refine and optimize their workflow processes.

Description

A framework for distributed data integration that streamlines essential functions of Big Data integration, including data ingestion, replication, organization, and lifecycle management, is designed for both streaming and batch data environments. It operates as a standalone application on a single machine and can also function in an embedded mode. Additionally, it is capable of executing as a MapReduce application across various Hadoop versions and offers compatibility with Azkaban for initiating MapReduce jobs. In standalone cluster mode, it features primary and worker nodes, providing high availability and the flexibility to run on bare metal systems. Furthermore, it can function as an elastic cluster in the public cloud, maintaining high availability in this setup. Currently, Gobblin serves as a versatile framework for creating various data integration applications, such as ingestion and replication. Each application is usually set up as an independent job and managed through a scheduler like Azkaban, allowing for organized execution and management of data workflows. This adaptability makes Gobblin an appealing choice for organizations looking to enhance their data integration processes.

API Access

Has API No 

API Access

Has API No 

Screenshots View All

Screenshots View All

Integrations

Apache Druid Yes 
Ardent Yes 
Ascend Yes 
Azure Marketplace Yes 
Control-M Yes 
CrateDB Yes 
DataBuck Yes 
DataHub Yes 
DoubleCloud Yes 
IBM watsonx.data integration Yes 
Kedro Yes 
Mode Yes 
Netdata Yes 
Prophecy Yes 
Snowflake CoCo Yes 
Tobiko Yes 
Yandex Data Proc Yes 
emma Yes 
lakeFS Yes 
rudol Yes 

Integrations

Apache Druid No 
Ardent No 
Ascend No 
Azure Marketplace No 
Control-M No 
CrateDB No 
DataBuck No 
DataHub No 
DoubleCloud No 
IBM watsonx.data integration No 
Kedro No 
Mode No 
Netdata No 
Prophecy No 
Snowflake CoCo No 
Tobiko No 
Yandex Data Proc No 
emma No 
lakeFS No 
rudol No 

Pricing Details

No price information available.
Free Trial No 
Free Version No 

Pricing Details

No price information available.
Free Trial No 
Free Version No 

Deployment

Web-Based Yes 
On-Premises No 
iPhone App No 
iPad App No 
Android App No 
Windows No 
Mac No 
Linux No 
Chromebook No 

Deployment

Web-Based No 
On-Premises No 
iPhone App No 
iPad App No 
Android App No 
Windows Yes 
Mac No 
Linux Yes 
Chromebook No 

Customer Support

Business Hours No 
Live Rep (24/7) No 
Online Support No 

Customer Support

Business Hours No 
Live Rep (24/7) No 
Online Support Yes 

Types of Training

Training Docs No 
Webinars No 
Live Training (Online) No 
In Person No 

Types of Training

Training Docs Yes 
Webinars No 
Live Training (Online) No 
In Person No 

Vendor Details

Company Name

The Apache Software Foundation

Founded

1999

Country

United States

Website

airflow.apache.org

Vendor Details

Company Name

Apache Software Foundation

Country

United States

Website

gobblin.apache.org

Product Features

Workflow Management

Access Controls/Permissions No 
Approval Process Control No 
Business Process Automation No 
Calendar Management No 
Compliance Tracking No 
Configurable Workflow No 
Customizable Dashboard No 
Document Management No 
Forms Management No 
Graphical Workflow Editor No 
Mobile Access No 
No-Code No 
Task Management No 
Third Party Integrations No 
Workflow Configuration No 

Product Features

Big Data

Collaboration No 
Data Blends No 
Data Cleansing No 
Data Mining No 
Data Visualization No 
Data Warehousing No 
High Volume Processing No 
No-Code Sandbox No 
Predictive Analytics No 
Templates No 

Alternatives

Alternatives

Apache Spark Reviews

Apache Spark

Apache Software Foundation
JAMS Scheduler Reviews

JAMS Scheduler

JAMS Software
E-MapReduce Reviews

E-MapReduce

Alibaba
Amazon MWAA Reviews

Amazon MWAA

Amazon
dbt Reviews

dbt

dbt Labs
MLlib Reviews

MLlib

Apache Software Foundation