Pyspark Developer
Pyspark Developer
Posted on :
11-11-2022
Employer Active
The job posting is outdated and position may be filled
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Share
Send me jobs like this
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Job Description
Job Title: Pyspark Developer
Location: Louiseville, KY
Type: Full Time
Job Description:
- Experience building complex, highly scalable and reliable data pipelines using technologies like Databricks, spark, Kafka, Hive, Parquet, Avro
- Software development experience with High level languages like Python, Scala, Java, SQL, No-SQL
- Experience working with pipeline orchestration tools to automate data flows like Azure Data Factory, Airflow, Perfect
- Experienced in Establishing and promoting high standards in DevOps, Security, CI/CD, Monitoring, data validation, testing
- Experience with cloud platforms (preferably Azure) and experience building enterprise level data lakes
- Experience working with Terraform
- Experience using Docker and Kubernetes
- Experience with Microservices and RESTful APIs
- Understanding Data Catalog, Data Governance, Data Lineage
- Experience with integration of data from multiple data sources
- Knowledge of various ETL techniques
- 5+ years of experience in data engineering
Employment Type
Full Time
Report This Job
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.
