Sr. Bigdata Engineer

UST Tittle: Lead I

• Data Engineering Who we are: At UST, we help the world’s best organizations grow and succeed through transformation. Bringing together the right talent, tools, and ideas, we work with our client to co-create lasting change. Together, with over 30,000 employees in 30+ countries, we build for boundless impact—touching billions of lives in the process. Visit us at . Summary: UST is looking for Sr. Big Data Engineer The Opportunity:
• Hands-on senior engineer and technical lead who has personally built and shipped large-scale, production-grade ETL pipelines on AWS.
• The resource will architect and deliver batch/streaming pipelines, optimize cost and performance at scale, and set the team's engineering standards. Required:
• Minimum 10 years in data engineering / big data is mandatory.
• Mandatory Qualifications Production experience (not POCs) required in all of: - o AWS Glue — Glue ETL jobs, Data Catalog, crawlers, job bookmarks, dynamic frames; tuning for cost/runtime. - o Amazon EMR — Cluster lifecycle (transient, EMR Serverless / on EKS), Spark tuning, instance fleets / spot. - o Py. Spark — Data. Frame APIs, UDFs, partitioning, join strategies, skew handling, and Spark memory/executor tuning. - o AWS Step Functions — Orchestrating multi-stage ETL with retries, error handling, and parallel/map states. - o Production ETL delivery — Delivered ETL at scale with data quality controls, schema evolution, idempotency, and backfills. Core Skills:
• Apache Spark internals; Parquet/ORC/Avro and Iceberg/Hudi/Delta; partitioning and small-file management
• S3 data lake, Athena, Redshift/Spectrum, Lake Formation; Lambda, SQS/SNS, Event Bridge; Kinesis/MSK (plus)
• Expert Python and strong SQL; IAC (Terraform / CDK / Cloud. Formation); CI/CD; Git; Docker
• Dimensional modelling, Lakehouse / medallion architectures, batch and CDC ingestion
• Production-Readiness Expectations
• Reliability (retries, checkpointing, safe re-runs, SLAs/SLOs)
• Observability (Cloud. Watch/Datadog metrics, logging, lineage)
• Data quality (validation frameworks, late/malformed data handling)
• Cost and performance optimization at multi-terabyte scale
• Security/compliance (PII, encryption, masking, access governance)
• Operational ownership (on-call, incident response, RCA) Leadership:
• Mentor engineers, run design/code reviews, and lead architecture decisions.
• Partner with data scientists, analysts, and product to deliver multi-quarter initiatives.

Nice to Have

• AWS certifications; streaming (Kafka/Kinesis/Structured Streaming/Flink);
• Airflow/MWAA; dbt; regulated-domain experience (financial / credit / rental data).

Education

• Bachelor's or Master's in CS/Engineering or equivalent practical experience. What we believe : We’re proud to embrace the same values that have shaped UST since the beginning. Since day one, we’ve been building enduring relationships and a culture of integrity. And today, it's those same values that are inspiring us to encourage innovation from everyone, to champion diversity and inclusion and to place people at the centre of everything we do. Humility: We will listen, learn, be empathetic and help selflessly in our interactions with everyone. Humanity: Through business, we will better the lives of those less fortunate than ourselves. Integrity: We honour our commitments and act with responsibility in all our relationships. Equal Employment Opportunity Statement UST is an Equal Opportunity Employer. We believe that no one should be discriminated against because of their differences, such as age, disability, ethnicity, gender, gender identity and expression, religion, or sexual orientation. All employment decisions shall be made without regard to age, race, creed, colour, religion, sex, national origin, ancestry, disability status, veteran status, sexual orientation, gender identity or expression, genetic information, marital status, citizenship status or any other basis as protected by federal, state, or local law. UST reserves the right to periodically redefine your roles and responsibilities based on the requirements of the organization and/or your performance.
• To support and promote the values of UST.
• Comply with all Company policies and procedures