Data Engineer (Analytics)
Carl Zeiss AG
Publication date:
15 August 2024Workload:
100%- Place of work:Budapest - ZDI
Your Task
- Participating in data analytics projects from end to end in Microsoft Azure environment
- Build and maintain data pipelines (Azure Data Factory)
- Implement data transformation logic using Apache Spark (Databricks, Synapse)
- Turning raw data into relevant datasets and insights
- Mentoring junior colleges
- Building highly scalable data products for the automated consolidation, process and analyse large amounts of data for machine learning - from conception to prototyping to operation, batch and streaming
- Evaluating the latest trends and technologies in data engineering and drive innovative solutions for data management in the machine learning context
- Contribute to ensuring continuous quality, reusability and performance improvement and take responsibility for applications and stakeholders during development, commissioning and operation
- Automating and deploying data transformations and pipelines on the provided Data Platform
- Working in an accurate (clean-code) and test-driven software development environment with a devops mindset
- In close cooperation with data scientists, you implement exciting AI use cases (e.g. entity extraction, recommender, chatbots) with data of all kinds (e.g. time series, images, documents)
Your Profile
- At least 3 years of relevant professional experience as Data Engineer
- 2+ years of experience with Python
- Hands-on software development skills
- Solid knowledge of SQL and coding skills in general
- Very good experience in object-oriented programming and application modularization
- Expertise in designing and implementing data models (principles, methods, best practices)
- Experience in implementation of different data integration frameworks: batch-oriented or sync/async real-time integration
- Demonstrated ability to understand and communicate technical details at all levels of the organization
- Experience in multiple forms of Data Storages (relational, analytical or in-memory databases, objects stores, search engines, graph databases) and their interrogation, physical data modelling
- Best-practice and implementation for continuous integration / delivery: Unit-testing, CICD Pipelines and deployment (preferably Azure DEVOPS, Gitlab)
- Ability to work in an independent, conscientious and solution-oriented way
- Fluency in English
- Teamplayer (agile team experience is a plus)
Nice to have:
- University degree in computer science, business informatics, mathematics or in a related field
- Experience with CRM/ERP source systems
- Experience with Continuous Integration/Continuous Delivery (CI/CD)
- Experience with clean-code, configurability and reusability
- Experience with Cloud technologies like Microsoft Azure or similar environment
- Development of serverless (Azure Data Factory, Synapse Analytics) or container-based applications (Kubernetes) provisioning and deployment
- Distributed processing frameworks: Spark-based data transformations (Databricks, Data Factory)
- Productionalising and publishing data services as scalable, performant Data APIs
- Experience in Pipeline Orchestration and Scheduling (ex: Airflow)
- Monitoring and logging (Elastic, Datadog), telemetry
- Technical development and integration of User-Request Workflows through reusable Rest APIs
- Experience in machine learning and deep learning algorithms productionalisation in Cloud
- Up to date with emerging technologies and trends in data engineering
Your ZEISS Recruiting Team:
Bartók Tímea, Fedor Fanni, Hrehuss Orsolya, Kiss Bernadett, Meláth Laura, Sturcz Noémi, Sztaskó Daniella, Wenner Lili