AWS unveals managed big data service of Apache Airflow

Get a free Techzine subscription!

Amazon Web Services launched a new managed product to help its users conduct data processing workloads in the cloud. The new service is named Amazon Managed Workflows for Apache Airflow. Users can deploy the open-source Apache Airflow tool to author, schedule, and monitor data workflows.

The chief evangelist at AWA for Europe, the Middle East, and Africa, Danilo Poccia, said that Apache Airflow could simplify the data processing pipelines by decomposing them into several smaller tasks that are then executed as part of a workflow.

A clear and present need

Any company that uses machine learning and analytics to get insights from massive amounts of data needs data processing.

Apache Airflow is excellent at doing this, but it has some challenges. The manual installation, maintenance, and scaling of Airflow, while dealing with security, authentication, and authorization for users, takes a lot of time.

Amazon MWAA is a fully managed platform that removes these time-consuming tasks out of the way. Users can deploy this platform, with which they are familiar, to manage their data processing workloads, with all the perks and no worries about the underlying infrastructure.

The product is available

Amazon MWAA is available in the Asia Pacific (Singapore, Tokyo, Sydney), Europe (Ireland, Frankfurt, Stockholm), the US East (Northern Virginia), US West (Oregon), and US East (Ohio).

Amazon has also developed a new app programming interface for financial organizations to track events like bankruptcy, leadership changes at companies, mergers, and acquisitions. The new API is called Amazon Comprehend Events.

It can scan text documents like new reports and press releases to pick up on the critical information and track it.

Tip: AWS emphasises the importance of a Well Architected Framework