Big Data Engineer

Job Summary:

We are looking for a skilled Big Data Engineer with strong experience in PySpark and data engineering. The ideal candidate will have hands-on experience in building scalable data pipelines, working with streaming data, and cloud platforms. Experience with Palantir Foundry is highly preferred

Experience Required: 5 – 10 years

Key Responsibilities:

• Design, develop, and maintain ETL/data pipelines using Python and PySpark
• Work with real-time/streaming data systems
• Build and optimize data processing solutions on cloud platforms
• Collaborate with cross-functional teams to understand business requirements
• Develop and maintain CI/CD pipelines
• Ensure data quality, performance, and scalability

Required Skills:

• 5–10+ years of experience in Data Engineering
• Strong expertise in Python and PySpark
• Experience with Kafka, Flink, and Kubernetes
• Hands-on with Big Data technologies (Hadoop, Spark, MongoDB)
• Experience with cloud platforms (Azure, AWS, or GCP)
• Good understanding of data architecture and pipeline design

Preferred Skills:

• Experience with Palantir Foundry
• Knowledge of Java
• Experience in consulting or Utilities domain