Overview:
The Data Engineer (Python) plays a crucial role in collecting storing processing and analyzing large sets of data. They work closely with the data science and analytics teams to build and maintain scalable data pipelines and create efficient methods for analytics and reporting. The position is essential in ensuring that the organization can make datadriven decisions and derive valuable insights from the data.
Key Responsibilities:
- Design and develop robust data pipelines using Python for extracting transforming and loading (ETL) data from various sources.
- Collaborate with data scientists to understand data requirements and implement efficient solutions for data processing and analysis.
- Implement data modeling and database design ensuring data integrity and performance.
- Optimize and maintain existing data pipelines and processes to ensure scalability and reliability.
- Work with largescale complex data sets to solve challenging business problems.
- Build and maintain data warehouses and data lakes to store and organize data for analysis and reporting purposes.
- Develop and implement data quality standards and best practices.
- Automate and streamline data processes improving efficiency and reliability.
- Conduct performance tuning and troubleshooting of datarelated issues.
- Stay uptodate with the latest technologies and trends in data engineering and analytics.
Required Qualifications:
- Bachelors or Masters degree in Computer Science Engineering or a related field.
- Proven experience as a Data Engineer with strong proficiency in Python.
- Expertise in SQL and database technologies for data manipulation and querying.
- Solid understanding of data modeling ETL development and data warehousing concepts.
- Experience with big data technologies such as Hadoop Spark or Kafka.
- Ability to design and implement scalable highperformance data solutions.
- Strong analytical and problemsolving skills for dealing with complex data challenges.
- Excellent communication and collaboration skills to work effectively in crossfunctional teams.
- Understanding of data security and privacy concerns in handling sensitive data.
- Proficiency in agile development methodologies and version control systems.
- Familiarity with cloud platforms such as AWS Azure or Google Cloud Platform.
- Experience with data visualization tools and techniques is a plus.
- Ability to work in a fastpaced dynamic environment with a focus on delivering results.
- Certifications in data engineering or related fields are desirable.
etl,cloud platforms,google cloud platform,data modeling,agile development methodologies,data engineering,data security,data visualization,kafka,data processing,big data,big data technologies,python,certifications,spark,azure,etl development,hadoop,cloud,data warehousing,database design,aws,sql