Byte Ebi's Logo

Byte Ebi 🍀

A Bit everyday A Byte every week

A DAG as a Variable Desk

Designated parameter changes at the desk. All other variables, not our department.

Ray

The requesting team wants to tweak a parameter whenever they need to, without going through an engineer every time.
Giving them a database table to edit is a maintenance burden, and Airflow Variables can’t be opened up one at a time.
I don’t want to grant overly broad permissions, and I don’t want to be on call to change a value either. So what now?

Investigating Connection Encryption on Amazon MWAA

Investigating whether Connections can be stored in plaintext on Amazon MWAA

Ray

The Airflow docs actually state that setting the encryption key to an empty string makes Connections fall back to plaintext.
But does the same thing hold true on AWS-managed MWAA?
This post traces that question from Airflow’s native Fernet mechanism all the way to MWAA’s reserved environment variables.

Deploy Your Own Openclaw AI Assistant for Free

Spin up your own AI assistant on GitHub Codespaces at zero cost

Ray

No GPU? Don’t feel like paying for a server?
This guide shows you how to deploy Openclaw on GitHub Codespaces using free resources, so you can start running your own AI assistant in minutes.

Triggering Airflow DAG from Another

Use operator to easily trigger downstream DAGs in Airflow

Ray

When building dependencies between DAGs, you don’t always have to rely on RESTful APIs.
Airflow also provides a built-in Operator for this purpose.

Recent Posts

Categories

Tags