Average Ratings 0 Ratings
Average Ratings 0 Ratings
Description
Airflow is a community-driven platform designed for the programmatic creation, scheduling, and monitoring of workflows. With its modular architecture, Airflow employs a message queue to manage an unlimited number of workers, making it highly scalable. The system is capable of handling complex operations through its ability to define pipelines using Python, facilitating dynamic pipeline generation. This flexibility enables developers to write code that can create pipelines on the fly. Users can easily create custom operators and expand existing libraries, tailoring the abstraction level to meet their specific needs. The pipelines in Airflow are both concise and clear, with built-in parametrization supported by the robust Jinja templating engine. Eliminate the need for complex command-line operations or obscure XML configurations! Instead, leverage standard Python functionalities to construct workflows, incorporating date-time formats for scheduling and utilizing loops for the dynamic generation of tasks. This approach ensures that you retain complete freedom and adaptability when designing your workflows, allowing you to efficiently respond to changing requirements. Additionally, Airflow's user-friendly interface empowers teams to collaboratively refine and optimize their workflow processes.
Description
Dagster is the cloud-native open-source orchestrator for the whole development lifecycle, with integrated lineage and observability, a declarative programming model, and best-in-class testability.
It is the platform of choice data teams responsible for the development, production, and observation of data assets.
With Dagster, you can focus on running tasks, or you can identify the key assets you need to create using a declarative approach. Embrace CI/CD best practices from the get-go: build reusable components, spot data quality issues, and flag bugs early.
API Access
Has API
No
API Access
Has API
Yes
Integrations
DataHub
Yes
Great Expectations
Yes
OpenMetadata
Yes
SDF
Yes
Tobiko
Yes
AT&T Alien Labs Open Threat Exchange
Yes
Apache Druid
Yes
Azure Databricks
No
BentoML
Yes
Control-M
Yes
Integrations
DataHub
Yes
Great Expectations
Yes
OpenMetadata
Yes
SDF
Yes
Tobiko
Yes
AT&T Alien Labs Open Threat Exchange
No
Apache Druid
No
Azure Databricks
Yes
BentoML
No
Control-M
No
Pricing Details
No price information available.
Free Trial
No
Free Version
No
Pricing Details
$0
Pricing starts at $0.04/min for Serverless and $0.03/min for Hybrid, and tapers down from there based on usage.
Free Trial
Yes
Free Version
Yes
Deployment
Web-Based
Yes
On-Premises
No
iPhone App
No
iPad App
No
Android App
No
Windows
No
Mac
No
Linux
No
Chromebook
No
Deployment
Web-Based
Yes
On-Premises
Yes
iPhone App
No
iPad App
No
Android App
No
Windows
Yes
Mac
Yes
Linux
Yes
Chromebook
No
Customer Support
Business Hours
No
Live Rep (24/7)
No
Online Support
No
Customer Support
Business Hours
No
Live Rep (24/7)
Yes
Online Support
Yes
Types of Training
Training Docs
No
Webinars
No
Live Training (Online)
No
In Person
No
Types of Training
Training Docs
Yes
Webinars
No
Live Training (Online)
No
In Person
No
Vendor Details
Company Name
The Apache Software Foundation
Founded
1999
Country
United States
Website
airflow.apache.org
Vendor Details
Company Name
Dagster Labs
Founded
2019
Country
United States
Website
dagster.io
Product Features
Workflow Management
Access Controls/Permissions
No
Approval Process Control
No
Business Process Automation
No
Calendar Management
No
Compliance Tracking
No
Configurable Workflow
No
Customizable Dashboard
No
Document Management
No
Forms Management
No
Graphical Workflow Editor
No
Mobile Access
No
No-Code
No
Task Management
No
Third Party Integrations
No
Workflow Configuration
No
Product Features
Data Fabric
Data Access Management
No
Data Analytics
Yes
Data Collaboration
Yes
Data Lineage Tools
Yes
Data Networking / Connecting
Yes
Metadata Functionality
Yes
No Data Redundancy
No
Persistent Data Management
No
Data Management
Customer Data
No
Data Analysis
Yes
Data Capture
No
Data Integration
Yes
Data Migration
Yes
Data Quality Control
No
Data Security
No
Information Governance
No
Master Data Management
No
Match & Merge
Yes
ETL
Data Analysis
Yes
Data Filtering
Yes
Data Quality Control
No
Job Scheduling
Yes
Match & Merge
No
Metadata Management
No
Non-Relational Transformations
No
Version Control
No
Machine Learning
Deep Learning
No
ML Algorithm Library
No
Model Training
Yes
Natural Language Processing (NLP)
No
Predictive Modeling
Yes
Statistical / Mathematical Tools
Yes
Templates
No
Visualization
No