Rafael Kansy

Transition from an ETL architecture to an Event Sourcing architecture with Spring Boot, Confluent Kafka, Apache Spark and MongoDB, deployed on OpenShift / Kubernetes. Conception and implementation of architecture blueprints and PoCs on Spring Boot and Confluent Kafka. Definition and enforcement of Data Contracts across Kafka event streams (JSON Schemas and Avro via Schema Registry) as the foundation for cross-domain Data Mesh interfaces between producing and consuming teams. Training and support of internal employees, implementation of various use cases, code reviews, improvement of test coverage and data lineage across distributed systems.

Design and implementation of a Medallion Data Architecture as a Data Lakehouse on Azure Databricks (managed Kubernetes clusters on AKS), including the end-to-end migration of several business-critical use cases from SAP into Databricks. Transformation and streaming of data with Apache Kafka and Apache Flink. Hands-on work with Unity Catalog (governance, lineage; evaluated against Collibra as the enterprise catalog alternative), Delta Lake (bronze/silver/gold layers), PySpark and Structured Streaming (Kafka → Databricks). Deployment and environment promotion via Databricks Asset Bundles (DABs), analytics transformations modelled with dbt, contract-first approach with Data Contracts between SAP source systems and the Databricks target, CI/CD with GitHub Actions driving DABs rollouts.