5 years of experience in the design and implementation of Big Data systems using PySpark database migration transformation and integration solutions for any Data warehousing project.
Must have excellent knowledge in Apache Spark and Python programming experience
Deep experience in developing data processing tasks using PySpark such as reading data from external sources merging data performing data enrichment and loading into target data destinations.
Experience in deployment and operationalizing the code knowledge of scheduling tools like Airflow ControlM etc. is preferred
Working experience on Cloud technology architecture like AWS ecosystem Google Cloud BigQuery etc. is an added advantage
Understanding of Unix/Linux Shell Scripting
Data modeling experience using advanced statistical analysis unstructured data processing
Experience with building APIs for provisioning data to downstream systems by leveraging different frameworks.
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.