Data Engineer

 

Project description:

The project aims to develop and implement a modern data platform for an automotive company based on Azure, Databricks and a lakehouse architecture.

Success is measured by successful implementation, improved efficiency and data quality, and positive user feedback.

 

Service description:

  • Requirements analysis
  • Implementation of infrastructure and data pipelines
  • Testing, validation, training and handover

 

Must-have:

  • Experience in Azure (ADLv2, ADF, Azure Functions)
  • Databricks Platform (Unity Catalog/Governance, SQL, PySpark, Delta)
  • Very good Python knowledge
  • Experience with an orchestration tool
  • Basic understanding of a lakehouse architecture
  • Very good knowledge of German

 

Nice to have:

  • Dagster
  • DBT
  • Terraform
  • Jenkins
  • Kubernetes