Lead Software Engineer - DevOps

Experience: 10+ Years

Location: Bangalore

Role: Full-time / Technical Lead

Role Summary

We are seeking a highly capable Technical Lead with deep expertise in Big Data (Spark, Hadoop, Hive), Kubernetes platforms, Azure cloud, PostgreSQL, CI/CD (GitHub Actions/Jenkins), and DevOps automation. The ideal candidate will be a strong hands-on engineer who also brings mature leadership capabilities, sound architectural judgment, and exceptional software craftsmanship practices.

Key Responsibilities

1. Software Engineering & Craftsmanship

• Develop high‑quality, maintainable code using Software Craftsmanship principles such as:

• Test-Driven Development (TDD)

• Continuous Integration / Continuous Delivery / Continuous Deployment

• Clean code, refactoring, SOLID principles

• Participate actively in design reviews and code reviews to ensure engineering excellence.

• Implement quick POCs using the latest tech stack to validate ideas and technical feasibility.

• Apply design patterns and engineering best practices for scalable and maintainable solutions.

• Collaborate in all phases of the SDLC, with strong understanding of Agile/Scrum or continuous delivery environments.

2. Big Data Engineering & Distributed Systems

• Lead development of large-scale ETL/ELT pipelines using Apache Spark (Java/Scala).
• Oversee data processing frameworks in Hadoop and Hive, especially within Azure-based ecosystems.
• Optimize Spark/Hive workloads for cost, performance, and reliability.

3. Cloud, Kubernetes & Infrastructure Engineering

• Lead architecture and operations of distributed applications running on:

• Kubernetes (On-Prem + AKS)

• Docker

• Drive adoption of Helm, container standards, and cluster best practices.

• Build and maintain Azure infrastructure using Terraform (IaC) with reusable, scalable modules.

• Ensure end-to-end observability of clusters using Prometheus, ELK, APM tools, and custom Python scripts.

4. DevOps Engineering & CI/CD Leadership

• Define and own the platform CI/CD strategy using:

• GitHub Actions

• Jenkins

• Terraform automation

• Build standardized CI/CD frameworks for 50+ microservices, leveraging Helm and automated build workflows.

• Automate DNS creation, certificate management, APM integration, user management, Vault policy creation, and repository migrations (200+ repos).

• Champion an automation-first mindset across the engineering and SRE teams.

• Coordinate with developers and testers for smooth deployment and production readiness.

5. Database Engineering

• Provide leadership on PostgreSQL cluster management:

• HA setup

• Performance tuning

• Failover mechanisms

• Monitoring dashboards

• Automate PostgreSQL operations using Python and infrastructure scripts.

6. Observability, Stability & Support

• Ensure reliability and availability of services across stacks (Java, Angular, .NET).
• Implement monitoring and alerting using Prometheus, Grafana, ELK, Elastic APM.
• Oversee production stability through proactive dashboards and automated health checks.
• Provide guidance for L2/L3 support and participate in incident review and resolution.

7. Leadership & Collaboration

• Lead a cross-functional team of developers, DevOps engineers, and data engineers.
• Translate business requirements into technical designs and actionable backlogs.
• Mentor team members on coding practices, DevOps processes, and cloud-native patterns.
• Collaborate with distributed teams and communicate effectively with technical and non-technical stakeholders.

Required Skills & Qualifications

• 10+ years of experience in software engineering / data engineering / DevOps roles.

• Strong expertise in:

• Apache Spark (Java/Scala)

• Hadoop, Hive on Azure

• Kubernetes, Docker

• Azure Kubernetes Service (AKS)

• Ansible

• Jenkins & GitHub Actions

• PostgreSQL (HA + automation)

• Strong understanding of CI/CD, SDLC, Agile, and continuous delivery.

• Good understanding of networking fundamentals, DNS, LBs, ingress controllers.

• Experience working on production support and high-availability systems.

Soft Skills

• Strong leadership and mentoring capabilities.
• Excellent communication skills with distributed teams.
• Ability to analyze complex problems and deliver quick, practical solutions.
• High sense of ownership and accountability.

Nice-to-Have

• Experience with Azure Data Lake, Databricks, or Synapse.
• Exposure to microservices architecture and domain-driven design.