Average Ratings 0 Ratings
Average Ratings 0 Ratings
Description
Managed Service for Apache Airflow is a cloud-based workflow orchestration service that simplifies the creation and management of complex data pipelines. Built on the open-source Apache Airflow framework, it allows users to define workflows using Python-based DAGs. The platform is fully managed, removing the need to provision or maintain infrastructure, which helps teams focus on pipeline development and execution. It integrates with a wide range of Google Cloud services, including BigQuery, Dataflow, Cloud Storage, and Managed Service for Apache Spark. The service supports hybrid and multi-cloud environments, enabling organizations to orchestrate workflows across different platforms. It offers advanced monitoring and troubleshooting tools, including visual workflow representations and logs. New features such as DAG versioning and improved scheduling enhance reliability and control. The platform also supports CI/CD pipelines and DevOps automation use cases. Its open-source foundation ensures flexibility and avoids vendor lock-in. Overall, it provides a powerful and scalable solution for managing data workflows and automation processes.
Description
You determine the cluster size, node specifications, and a range of services, while Yandex Data Proc effortlessly sets up and configures Spark, Hadoop clusters, and additional components. Collaboration is enhanced through the use of Zeppelin notebooks and various web applications via a user interface proxy. You maintain complete control over your cluster with root access for every virtual machine. Moreover, you can install your own software and libraries on active clusters without needing to restart them. Yandex Data Proc employs instance groups to automatically adjust computing resources of compute subclusters in response to CPU usage metrics. Additionally, Data Proc facilitates the creation of managed Hive clusters, which helps minimize the risk of failures and data loss due to metadata issues. This service streamlines the process of constructing ETL pipelines and developing models, as well as managing other iterative operations. Furthermore, the Data Proc operator is natively integrated into Apache Airflow, allowing for seamless orchestration of data workflows. This means that users can leverage the full potential of their data processing capabilities with minimal overhead and maximum efficiency.
API Access
Has API
No
API Access
Has API
Yes
Integrations
Apache Airflow
Yes
Python
Yes
APERIO DataWise
Yes
Apache HBase
No
Apache Hive
No
Apache Spark
No
Control-M
Yes
Google Cloud AI Infrastructure
Yes
Google Cloud BigQuery
Yes
Google Cloud Dataflow
Yes
Integrations
Apache Airflow
Yes
Python
Yes
APERIO DataWise
No
Apache HBase
Yes
Apache Hive
Yes
Apache Spark
Yes
Control-M
No
Google Cloud AI Infrastructure
No
Google Cloud BigQuery
No
Google Cloud Dataflow
No
Pricing Details
$0.074 per vCPU hour
Free Trial
No
Free Version
No
Pricing Details
$0.19 per hour
Free Trial
Yes
Free Version
No
Deployment
Web-Based
Yes
On-Premises
No
iPhone App
No
iPad App
No
Android App
No
Windows
No
Mac
No
Linux
No
Chromebook
No
Deployment
Web-Based
Yes
On-Premises
No
iPhone App
No
iPad App
No
Android App
No
Windows
No
Mac
No
Linux
No
Chromebook
No
Customer Support
Business Hours
Yes
Live Rep (24/7)
No
Online Support
Yes
Customer Support
Business Hours
No
Live Rep (24/7)
Yes
Online Support
Yes
Types of Training
Training Docs
Yes
Webinars
No
Live Training (Online)
No
In Person
No
Types of Training
Training Docs
Yes
Webinars
Yes
Live Training (Online)
Yes
In Person
No
Vendor Details
Company Name
Founded
1998
Country
United States
Website
cloud.google.com/products/managed-service-for-apache-airflow
Vendor Details
Company Name
Yandex
Founded
1997
Country
Russia
Website
cloud.yandex.com/en/services/data-proc