Principal Software Engineer (AWS/Python)

Principal Software Engineer | London, UK | Department: Engineering | Hybrid (8 days per month onsite) | Full-time, Permanent | Industry: SaaS, Energy, Commodities & Fin. Tech | Visa Sponsorship: Not available | REQ: 0031.

About Us

Founded in 1995, Zema Global Data Corporation empowers organizations to simplify complexity, reduce risk, and make faster, more confident decisions that drive measurable results. Over the past two years, Zema Global has accelerated its growth through strategic investment and acquisition to strengthen our global leadership.

Together we’re helping our customers gain a Decisioning Advantage - one bold idea at a time. With a presence across global energy, commodity, and financial markets, Zema Global empowers customers to simplify complexity, reduce risk, and make faster, more confident decisions that drive measurable results. At Zema Global, we Think Big, Make It Happen, and Win as One.

We thrive on collaboration, creativity, and respect, united by a shared drive to innovate and deliver meaningful impact for our customers and communities. If you’re inspired by solving complex challenges and contributing to a culture that values purpose and performance, we invite you to join us.

Position Overview

We are looking for a Principal Software Engineer to define and own the technical architecture of our Data Surface programme, designing the end-to-end data platform on AWS including the Lakehouse layer (Apache Iceberg on S3), ingestion patterns, data modelling standards, and governance frameworks that the engineering team builds against.

The architecture you define will also underpin our AI/ML ambitions: storing data in open source Apache Iceberg format so that AWS platforms such as Amazon Bedrock and Sage. Maker can consume it directly. The Principal Software Engineer is a foundational role. The standards, patterns, and models you establish will shape how the platform is built and how our customers access commodities and energy market data for years to come.

You will work alongside the UK engineering team and with global teams across the US, Canada, and India, with the autonomy to make architectural decisions on a programme that has senior sponsorship and genuine strategic weight. What You'll Be Doing; Principal Software Engineer:

• As the Pr. Define and maintain the overall data architecture for the AWS Lakehouse platform, including Apache Iceberg table design, partitioning strategies, and storage layout on S3.
• Design data ingestion patterns from DynamoDB, Oracle, and streaming sources (Kinesis, Kafka) into Bronze-Silver-Gold transformation layers.
• Establish data modelling standards, naming conventions, and schema evolution policies across the platform.
• Architect data governance frameworks covering lineage tracking, access control, data quality rules, and metadata management using AWS Glue Data Catalog and related tooling.
• Evaluate and recommend technologies for query performance optimisation across Spark, Trino, Athena, and other SQL engines.
• Design the platform so that data held in Apache Iceberg is consumable by AWS AI/ML services, including Amazon Bedrock and Sage. Maker, covering table layout, access patterns, and feature engineering workflows.
• Use Claude Code to generate architecture specifications and design documents, and review specifications produced by other team members to ensure AI-generated code accounts for all operational nuances of the technology.
• Collaborate with engineering teams across the UK, US, Canada, and India to ensure architectural consistency and provide technical guidance on implementation.
• Define infrastructure-as-code patterns and reference architectures for EKS, Lambda, and other AWS services used by the platform.
• Produce and maintain architecture decision records (ADRs), data flow diagrams, and technical roadmaps for the Data Surface programme.

What You'll Bring

Essential As the Principal Software Engineer, you will have 8+ years of experience as a Principal Software Engineer in data engineering, data platform design, or data architecture roles, including several years hands-on as a software or data engineer building production systems Proven track record of designing and delivering large-scale data platforms on AWS (S3, Glue, EMR, Athena, Kinesis, Lambda, EKS) or equivalent cloud environments Experience architecting for high-volume, high-frequency data, such as market, trading, or timeseries data, including partitioning, compaction, and retention strategies for very large tables Deep expertise in Lakehouse architecture patterns including Apache Iceberg, Delta Lake, or Apache Hudi Strong data modelling skills across dimensional, Data Vault, and schema-on-read paradigms Advanced SQL proficiency and experience optimising complex analytical queries across distributed engines (Trino, Presto, Athena) Hands-on experience with data governance and cataloguing tools (AWS Glue Data Catalog, Apache Atlas, Collibra, or similar) Experience with infrastructure as code (Terraform, Cloud.

Formation) and CI/CD pipelines Strong communication skills with the ability to translate complex technical concepts for both engineering and business audiences Bachelor’s or Master’s degree in Computer Science, Engineering, or STEM related field Desirable Experience with Apache Spark for batch and streaming data processing at scale Experience with Apache Kafka, Confluent Platform, or AWS Kinesis for real-time streaming architectures Background in migrating Oracle or legacy database workloads to cloud-native architectures Familiarity with data mesh or data product principles Experience working as an ML engineer, or hands-on experience with ML platform integration (Amazon Bedrock, Sage.

Maker, Databricks) and feature store design Knowledge of financial services, commodities, or energy market data, and/or experience working with timeseries data AWS Solutions Architect Professional or Data Analytics Specialty certification Why Join Zema Global? Global Impact Shape the future of energy, commodities, and financial markets through innovative technology.

Innovation at Scale Work with cutting-edge software, advanced analytics, and complex data that solve real-world challenges. Career Growth Join a fast-growing global organization where your ideas, development, and career progression are valued. Exceptional People Collaborate with talented colleagues and industry experts across North America, Europe, and Asia.

Meaningful Ownership Take ownership of your work, influence strategic initiatives, and make a measurable impact on our customers and business. One Global Team Be part of a collaborative culture built on trust, respect, and our values: Think Big. Make It Happen. Win as One. Application Information Please submit your English CV in PDF format.

If your experience aligns with our requirements, our Talent Acquisition team will contact you regarding the next steps. Please note: *Visa sponsorship is not available for this position. * Equality, Diversity & Inclusion Zema Global is committed to building an inclusive workplace where everyone can thrive. We encourage applications from all qualified candidates regardless of background or protected characteristic.