Senior Data Engineer with 4+ years building production data platforms across AWS, GCP, and Azure. I've orchestrated 200+ ELT/ETL pipelines processing 10+ TB of data, built real-time CDC streaming systems, and architected AI-powered analytics with LangGraph, MCP, and multi-agent LLM frameworks. Reduced cloud costs by 70% through hybrid warehouse architecture and enabled self-service analytics for 200+ users.
- 🔭 Provisioning Azure infrastructure (AKS, Databricks, AI Foundry, VNet, KeyVault) with Terraform/OpenTofu + Terragrunt across dev/staging/prod
- 🤖 Building a conversational analytics platform with LangGraph + 6 MCP servers + Anthropic Managed Agent on ClickHouse retail data
- 🔗 Shipping a federated Data Catalogue with Trino gateway, OpenMetadata lineage, and multi-tier RBAC
- 🧪 Running a dual-mode stream + batch ingestion platform (PeerDB → Kafka → ClickHouse + Airflow batch path)
- 🎓 Mentoring 80+ data engineering students at Dibimbing.id (Airflow, dbt, Spark, Data Warehousing)
- 💬 Ask me about: Airflow orchestration, dbt modeling, streaming CDC, ClickHouse, LangGraph agents, or Terraform on Azure
- ⚡ Fun fact: started out mapping ocean currents (B.Sc. Oceanography), now I pipe data instead
| Project | What it does |
|---|---|
| lakehouse-iceberg-stream | Flagship. End-to-end streaming lakehouse: Postgres CDC → Apache Fluss → Apache Iceberg (Polaris REST catalog) on SeaweedFS, with a streaming bronze/silver/gold medallion in Flink SQL and Trino on top. One make verify proves it end to end, and the README documents the five things that broke on the way |
| BI Platform | Conversational analytics for retail ops. LangGraph agent loop across 6 MCP servers querying ClickHouse, with Anthropic Managed Agent for automated PPTX report generation and Langfuse LLM observability |
| Data Catalogue | Federated data portal with Trino gateway across ClickHouse/BigQuery/Databricks/Athena, multi-tier RBAC, OpenMetadata lineage, and AI-powered table descriptions via Gemini |
| Stream-Batch Platform | Dual-mode ingestion: PeerDB CDC → Kafka → ClickHouse for sub-minute freshness, plus Airflow → Parquet → Azure Data Lake → dbt for batch/replay |
| llm-wiki | Open-source template for autonomous multi-agent LLM wiki maintenance. 5 specialized agents (orchestrator, scout, researcher, ingestor, linter), MIT licensed |
| Kaleidoskop Hoax | Multi-source scraper (Python + Playwright) collecting 5,000+ articles from Indonesian fact-checking platforms, enriched with LLM-based sentiment & topic analysis |
Styled after Andrew6rant/Andrew6rant's dark/light-mode card layout. No custom stats engine or PAT required.