Data Engineer Pyspark Python AWS
Data Engineer Pyspark Python AWS
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
Detailed JD *(Roles and Responsibilities)
- Understanding of core Business Intelligence BI data warehousing technology ETL RDBMS etc.
- Work with the application development teams throughout the development life cycle.
- Create Airflow DAG pipelines for data ingestion.
- Hands on knowledge of Scripting languages like Python PySpark Spark SQL Data Frames etc.
- Experience in AWS cloud services like S3 GLUE REDSHIFT LAMBDA IAM EMR ATHENA SPECTRUM.
- Expert in quickly understanding business needs and requirement gathering.
- Ability to take initiative and ownership on tasks.
- Good written and oral communication.
- SQL knowledge and performance tuning experience needed.
Minimum 5 to 6 years of experience and having done a lead experience with at least 4 members in the team.
Mandatory skills*
Python Pyspark and AWS
Desired skills*
Python Pyspark and AWS
Employment Type
Full Time
Company Industry
Key Skills
- Apache Hive
- S3
- Hadoop
- Redshift
- Spark
- AWS
- Apache Pig
- NoSQL
- Big Data
- Data Warehouse
- Kafka
- Scala
