Specialist Software Engineer - Data Engineer Spark Scala

a { text-decoration: none; color: #464feb; } tr th, tr td { border: 1px solid #e6e6e6; } tr th { background-color: #f5f5f5; }

Specialist Software Engineer – Data Engineer (Spark / Scala) Responsibilities

• Understand business and technical requirements and translate them into scalable, maintainable data engineering solutions.
• Own the design and development of robust, high‑performance data pipelines using Spark/Scala.
• Build and optimize data flows within the Hadoop ecosystem (Hive, HDFS, Starburst, Oozie).
• Ensure performance tuning, resource optimization, and efficient cluster usage on Cloudera platforms.
• Contribute to data architecture decisions related to storage, processing, security, and exposure.
• Apply industrialization best practices including CI/CD, unit testing, integration testing, and pipeline validation.
• Model data based on business needs and maintain shared data components, dictionaries, and governance rules.
• Perform functional and technical analysis, document specifications, and validate implemented solutions.
• Participate in Agile ceremonies and collaborate with Product Owners, architecture, business, and Ops teams.
• Support production processes, troubleshoot issues, update runbooks, and maintain system stability.
• Ensure compliance with data quality, traceability, and security standards across the data lifecycle.
• Engage in continuous improvement activities, proposing upgrades, refactoring, and automation opportunities.
• Work closely with global teams and support cross‑location coordination when required.
• Demonstrate ownership during key delivery phases, including planned on‑call or deployment activities.