Key Responsibilities:
- Design develop and maintain robust scalable and highperformance backend services for processing large datasets.
- Work with distributed systems and big data frameworks such as Spark Kafka or Flink to ensure the efficient processing of massive data pipelines.
- Collaborate with the data engineering team to optimize data ingestion storage and query performance.
- Implement RESTful APIs and/or gRPC services to expose backend functionality to other services or frontend applications.
- Ensure system scalability reliability and security through performance tuning monitoring and regular testing.
- Integrate and optimize databases (SQL NoSQL) for big data environments such as ClickHouse or PostgreSQL.
- Collaborate with the DevOps team to automate deployment manage cloud infrastructure (e.g. AWS) and monitor system performance.
- Participate in code reviews writing unit tests and improving coding standards to ensure the quality of code and scalability of systems.
- Stay updated on the latest big data trends tools and technologies and drive the adoption of best practices in backend development.
- Troubleshoot and resolve production issues related to big data pipelines and backend systems.
Qualifications:
- 45 years of experience as a backend developer working in big data environments.
- Proficiency in one or more programming languages such as PythonGo or Node.js.
- Handson experience with SQL and NoSQL databases optimized for big data such as ClickHouse MongoDB or PostgreSQL.
- Familiarity with containerization (Docker Kubernetes) and working in cloud environments (AWS).
- Experience with building RESTful APIs or gRPC services.
- Understanding of data modeling ETL pipelines and data warehousing concepts.
- Strong problemsolving skills and experience with debugging largescale distributed systems.
Preferred Qualifications:
- Experience with ClickHouseHypertables or InfluxDB for handling timeseries or highthroughput data.
- Knowledge of security best practices in big data environments.
node.js,grpc,docker,spark,mongodb,sql,go,flink,postgresql,nosql,python,kafka,clickhouse,aws,backend development,restful apis,kubernetes