Senior Data Engineer
Job description
Description Prisma Photonics, a fast-growing startup, transforms infrastructure monitoring with optical fibers. We eliminate the need for extra sensors by offering sensor-free solutions to monitor electrical power grids and oil & gas pipelines across thousands of kilometers. Our fiber-sensing technology, integrated with AI and machine learning, enables our customers to achieve environmental and renewable energy targets, ensuring smooth utility operations on their path to net-zero emissions. We are looking for the best minds and spirits to join us in our journey. We know our product is only as great as the individuals building the hardware and software and harnessing data for good causes. Being a great team member means being eager to learn and grow, able to challenge while accepting being challenged, and working for the team and the product with enthusiasm and passion. We are now hiring a talented, self-driven and passionate Senior Data Engineer to build and maintain optimized and highly available data pipelines that facilitate deeper analysis and reporting.
Requirements
Design and implement data collection processes for various data types, including textual and multidimensional data. Develop and implement data search capabilities and access APIs across edge computing and cloud environments. Design, develop, and optimize data and metadata storage solutions to enable efficient access to training and testing datasets for AI and Algorithm teams. Analyze existing data schemas and processes, identify areas for improvement, and implement optimization solutions. Monitor and analyze cloud data usage, optimizing storage and operational costs while maintaining low-latency data access using standard and proprietary services. Collaborate closely with Software, AI, and Algorithm teams on system integration, ongoing data requirements, data accessibility, and related initiatives.
What You’ll Do
What You Bring 5+ years of professional experience in Data Engineering or related roles, including hands-on experience designing and implementing data pipelines using orchestration frameworks such as Apache Airflow, Prefect, or similar tools. Experience with event-driven architectures and asynchronous messaging systems (e.g., Kafka, RabbitMQ). 5+ years of experience and a deep understanding of SQL and NoSQL databases (e.g., MySQL, Elasticsearch, MongoDB, PostgreSQL, ClickHouse). 5+ years of software development experience, including at least 2 years of hands-on experience with Python. Strong proficiency in version control systems (e.g., Git), containerization technologies (e.g., Docker), and CI/CD pipelines. Strong collaboration and communication skills, with the ability to work effectively across teams. An innovative mindset, a proactive approach to problem-solving, and a strong desire to learn and grow.
Great to Have Experience with streaming technologies, unstructured data, and diverse data types beyond traditional text and tabular data. Experience with distributed computing frameworks such as Hadoop, MapReduce, Apache Spark, or similar technologies. Experience working with Kubernetes-based systems. Experience with data visualization, monitoring, and analytics tools such as Grafana, the ELK Stack, Kibana, or similar platforms.
Description copied from Prisma Photonics's careers page. Read the full posting before you apply.
More jobs at Prisma Photonics
Electro Optics Systems Engineer
Prisma Photonics· Tel AvivExecutive Sales Director, US
Prisma Photonics· Remote, United StatesSenior Backend Engineer
Prisma PhotonicsSenior DevOps
Prisma Photonics· Tel Aviv
More jobs in Tel Aviv
Director of Finance & Financial Control
IsraAID· Tel Aviv, IsraelClient Partner, Ads
Unity· Tel Aviv, IsraelElectrician
Integral UK· Tel Aviv, IsraelSenior Software Engineer
Cisco Systems· Tel Aviv-Yafo, IsraelResearch Scientists - Quantum Error Correction & Quantum Architectures
Anyon Systems· Tel Aviv, Israel
Senior Data Engineer jobs at other companies
Senior Data Engineer
TaskUs· Chennai, IndiaSenior Data Engineer
Paramount Skydance· Los Angeles, CA, US, 90038· $110k – $166kSenior Data Engineer
Connor, Clark & Lunn Financial Group· Toronto, Ontario, CanadaSenior Data Engineer
Persistent Systems, LLC· New Jersey, United StatesSenior Data Engineer
Tripledot Studios· London