Data Engineer- Databricks

Overview

The Data Engineer builds and tests data pipelines in Databricks, covering ingestion, transformation, and Gold-layer modeling, against tasks assigned by the Lead Engineer at the start of each sprint and refined daily. This role suits an early-to-mid career engineer seeking rapid growth. The Data Engineer is expected to ask questions when requirements are unclear and to raise blockers promptly rather than delay progress.

Responsibilities

Build and test Databricks pipelines (PySpark/SQL, Delta Lake) for ingestion, transformation, and modeling tasks assigned during sprint planning. Write and maintain data quality checks and tests for pipelines built. Participate in daily standups, reporting progress accurately, asking questions when requirements are unclear, and flagging blockers promptly.

Follow the platform's data security model (RLS/CLS) and coding/documentation standards set by the Lead and Senior Engineers. Support integration testing and reconciliation checks for Boomi-based data flows (e.g., JDE-Concur, JDE-UKG) as assigned. Apply feedback from code reviews to subsequent work. Document pipelines clearly enough for another engineer to assume ownership.

Qualifications

Required Qualifications 2–4 years of data engineering experience, including hands-on work with Spark/PySpark and SQL. Hands-on Databricks experience (Delta Lake, notebooks, jobs/workflows). Working understanding of data warehousing/lakehouse concepts (Bronze/Silver/Gold or comparable layered modeling). Experience working in an agile team with daily standups and sprint-based delivery.

Strong written and verbal English communication skills.

Preferred Qualifications

Databricks Certified Data Engineer Associate. Exposure to financial or ERP source data — JD Edwards (JDE), EcoSys, or Microsoft Dynamics — or general ledger/cost data. Exposure to an EPC, construction, energy, or other asset-heavy industry. Familiarity with Unity Catalog-based security (RLS/CLS) or Power BI/Genie reporting on governed data.

Exposure to Boomi or another iPaaS/integration middleware platform. Professional Attributes: Asks questions early when a task is unclear rather than proceeding on assumptions. Demonstrates visible progress daily rather than only at sprint boundaries. Applies feedback from the Lead and Senior Engineers quickly and consistently.

Seeks increasing ownership and more complex problems over time. Build and test Databricks pipelines (PySpark/SQL, Delta Lake) for ingestion, transformation, and modeling tasks assigned during sprint planning. Write and maintain data quality checks and tests for pipelines built. Participate in daily standups, reporting progress accurately, asking questions when requirements are unclear, and flagging blockers promptly.

Follow the platform's data security model (RLS/CLS) and coding/documentation standards set by the Lead and Senior Engineers. Support integration testing and reconciliation checks for Boomi-based data flows (e.g., JDE-Concur, JDE-UKG) as assigned. Apply feedback from code reviews to subsequent work. Document pipelines clearly enough for another engineer to assume ownership.

Required Qualifications 2–4 years of data engineering experience, including hands-on work with Spark/PySpark and SQL. Hands-on Databricks experience (Delta Lake, notebooks, jobs/workflows). Working understanding of data warehousing/lakehouse concepts (Bronze/Silver/Gold or comparable layered modeling). Experience working in an agile team with daily standups and sprint-based delivery.

Strong written and verbal English communication skills.

Preferred Qualifications

Databricks Certified Data Engineer Associate. Exposure to financial or ERP source data — JD Edwards (JDE), EcoSys, or Microsoft Dynamics — or general ledger/cost data. Exposure to an EPC, construction, energy, or other asset-heavy industry. Familiarity with Unity Catalog-based security (RLS/CLS) or Power BI/Genie reporting on governed data.

Exposure to Boomi or another iPaaS/integration middleware platform. Professional Attributes: Asks questions early when a task is unclear rather than proceeding on assumptions. Demonstrates visible progress daily rather than only at sprint boundaries. Applies feedback from the Lead and Senior Engineers quickly and consistently.

Seeks increasing ownership and more complex problems over time.