Job description
About the company : BSE Tech is an established player in e-enabling businesses in the financial services sector. BSE Tech is a leading provider of cutting-edge IT solutions with a focus on Commodities, Banking and Financial Services markets in India. To know about BSE Tech, Head to: https://www.bsetech.in/ Role Overview: As a Data Engineer at BTPL, you will design, build, and maintain scalable data pipelines and infrastructure to support analytics, reporting, and business intelligence in the BFSI and capital markets domains.
You will work closely with data scientists, analysts, and application teams to ensure reliable, high-quality data delivery.
Key Responsibilities
Design, develop, and optimise ETL/ELT pipelines for structured and unstructured data. Build and manage data lakes, data warehouses, and real-time data streaming solutions. Integrate data from multiple sources (databases, APIs, files, external feeds) using tools like Kafka, MinIO, and message brokers. Work with distributed storage and compute platforms (Clickhouse, Apache Pinot, Hadoop, Spark, etc.)
Implement data modelling, data quality, and data governance best practices. Collaborate with DevOps for CI/CD, monitoring, and automation (Docker, Jenkins, Azure DevOps). Ensure data security, privacy, and compliance with regulatory standards. Troubleshoot and resolve data pipeline and infrastructure issues. Document data flows, architecture, and technical specifications.
Requirements
5+ years of experience in data engineering or related roles. Strong programming skills in Python, Java, or Scala. Experience with distributed data systems: Kafka, Clickhouse, Pinot, MinIO, Hadoop, Spark, or similar. Proficiency in SQL and working with relational (MS SQL, PostgreSQL, Oracle) and NoSQL databases. Hands-on experience with ETL/ELT tools and data integration frameworks.
Experience with DevOps tools: Docker, Jenkins, Git, Azure DevOps. Understanding of data modelling, data warehousing, and real-time data processing. Good communication and teamwork skills. Preferred: Experience in BFSI, capital markets, or fintech domains. Knowledge of data governance, data quality, and regulatory compliance.
Certifications in relevant data engineering, cloud, or big data technologies.