Lead DatabricksPyspark Developer
Lead DatabricksPyspark Developer
Posted on :
20-12-2024
Employer Active
1 Vacancy
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Share
Send me jobs like this
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Job Description
Role: Lead Databricks/Pyspark Developer
Exp: 7years
Location: Hyderabad/Chennai
Work Mode: Hybrid
Required Skills:
Must have
SCALA
SQL
Spark/Spark Streaming
Big Data Tool Set
Linux
Python/PySpark
Kafka
Essential Job Functions:
Design and development of data ingestion pipelines (Databricks background preferred).
Performance tune and optimize the databricks jobs
Evaluated new features and refractors existing code
Mentor junior developers and makes sure all patterns are documented
Perform data migration and conversion activities.
Develop and integrate software applications using suitable development methodologies and standards applying standard architectural patterns taking into account critical performance characteristics and security measures.
Collaborate with Business Analysts Architects and Senior Developers to establish the physical application framework (e.g. libraries modules execution environments).
Perform end to end automation of ETL process for various datasets that are being ingested into the big data platform.
* Maintain and support the application.
* Must be willing to flex work hours accordingly to support application launches and manage production outages if necessary
* Ensures to understand the requirements thoroughly and in detail and identify gaps in requirements
* Ensures that detailed unit testing is done handles negative scenarios and document the same
* Work with QA and automation team.
* Works on best practices and documenting the process
* code merges and releases (Bitbucket)
* Works with architect and manager on designs and best practices
* Good data analysis skills
scala,sql,spark,big data,python,kafka
Employment Type
Full Time
Key Skills
Report This Job
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.
