Sr Data Engineer
Sr Data Engineer
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
Only W2
Job Role : Data Engineer
Location : Dallas TX
Job description:
Data Engineer
Required Skills : Unix Python PySpark SQLAWS
Job summary:
Required Skills : Unix Shell Scripting Unix PySpark Spark Pyspark
Roles & Responsibilities :
Analyze business requirements to develop data solutions that align with organizational goals and objectives.
Collaborate with crossfunctional teams to gather and document detailed business and technical requirements.
Design and implement data processing workflows using PySpark and SparkPyspark to ensure efficient data handling.
Develop and maintain Unix and UnixShellScripting scripts to automate data processing tasks and improve operational efficiency.
Oversee the integration of data from various sources to create comprehensive datasets for analysis and reporting.
Provide technical support and troubleshooting for data processing issues to ensure smooth operations.
Conduct data quality assessments and implement measures to ensure data accuracy and consistency.
Create and maintain detailed documentation of data processes workflows and technical specifications.
Collaborate with data scientists and analysts to develop and optimize data models and algorithms.
Monitor and optimize the performance of data processing workflows to ensure timely and accurate data delivery.
Lead efforts to identify and implement process improvements to enhance data processing efficiency and effectiveness.
Provide training and support to team members on data processing tools and techniques.
Ensure compliance with data governance and security policies to protect sensitive information.
Qualifications:
Possess a Bachelors degree in Computer Science Information Technology or a related field.
Demonstrate strong proficiency in PySpark and SparkPyspark for data processing and analysis.
Have extensive experience with Unix and UnixShellScripting for automating data workflows.
Exhibit excellent analytical and problemsolving skills to address complex data challenges.
Show strong communication skills to effectively collaborate with crossfunctional teams.
Display a keen attention to detail to ensure data accuracy and consistency.
Employment Type
Full Time
