I built a data platform entirely from installable agent skills: Airflow, ClickHouse, dbt and Metabase

I’ve been experimenting with packaging data infrastructure as skills that an AI coding agent can install and operate.

The result is an open-source data platform composed of two Package Skills:

Airflow skill

• Apache Airflow 3 with LocalExecutor
• PostgreSQL metadata database
• continuous WAL-G backups to object storage
• Caddy TLS and authentication
• GitHub Actions DAG deployment over restricted rsync

ClickHouse skill

• 3 replicated ClickHouse nodes
• 3-member ClickHouse Keeper quorum
• Metabase with PostgreSQL metadata storage
• local dbt project with sample data and tests
• Hetzner private networking
• WireGuard-only service access
• Cloudflare DNS

Each deployment is defined by a single non-secret colors.yml. The skills generate and run the OpenTofu, Ansible, dbt and acceptance workflows behind four commands:

./green build ./green create --dry-run ./green create ./green delete 

build and create --dry-run require no credentials and make no provider calls, so the generated infrastructure and execution graph can be reviewed before provisioning.

Security was a major constraint. ClickHouse, Keeper, PostgreSQL and Metabase are not publicly exposed. Airflow sits behind Caddy authentication, while DAG deployment uses a disposable SSH key restricted to rsync.

The ClickHouse workflow also seeds data with dbt, tests the model, verifies replication and Keeper health, checks Metabase connectivity, and finishes with zero-drift OpenTofu plans.

Install the skills:

npx skills add getcolors/airflow npx skills add getcolors/clickhouse 

Source and documentation:

• Airflow: https://github.com/getcolors/airflow
• ClickHouse: https://github.com/getcolors/clickhouse
• Colors: https://www.getcolors.ai/

I’d appreciate feedback from data engineers—especially on the topology, security boundaries, and what you would need before trusting agent-installable infrastructure with a real workload.

One explicit limitation: ClickHouse replication is not backup. The current ClickHouse skill focuses on reproducible infrastructure and sample workloads rather than providing a production backup strategy.