Senior Data Infrastructure Engineer

תיאור המשרה

Design, build, and maintain scalable, reliable data pipelines and AWS-based infrastructure (EMR, S3, Redshift, Glue, RDS, Spark) for Rise’s ad-serving platform.
Develop and optimize real-time streaming data workflows using tools like Kafka, Kinesis, or MSK.
Optimize data pipelines for performance, low latency, high throughput, and cost efficiency.
Work closely with data engineers, data scientists, ML ops, analysts, and other stakeholders to understand data needs and improve data processes.
Automate workflows to ensure high performance in production and development environments.
Monitor and maintain data systems, ensuring uptime, reliability, and stability.

דרישות התפקיד

5+ years of experience as a Data Engineer/Big Data Developer or in a similar data infrastructure role.
Strong hands-on experience with AWS data infrastructure (EMR, S3, Glue, Spark) and Google Cloud services (BigQuery, GCS, Dataflow, Pub/Sub).
Experience with data processing frameworks such as Apache Spark and streaming tools like Kafka, Kinesis, Spark Streaming, Flink, or MSK for real-time data ingestion and transformation.
Proficiency with tools like Airflow for managing workflows and Python for building and optimizing data workflows.
Expertise in optimizing data pipelines for performance, cost efficiency, and scalability in cloud environments (AWS, GCP, etc.), including networking and VPC setups.
Strong communication skills and the ability to collaborate effectively with cross-functional teams.
Advantages
Knowledge of GoLang programming language
Hands-on experience with IAC tools like: Terraform, Cloudformation, Pulumi
Experience with MLops methodologies and tools

לכל המשרות