AI Big Data Engineer
AI Big Data Engineer
Posted on :
13-10-2024
Employer Active
The job posting is outdated and position may be filled
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Share
Send me jobs like this
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Job Description
Job Title: AI Big Data Engineer
Duration: 12 Months
Location: Troy MI 48098
Position Summary:
Client is Looking for a Data Engineer to join the AI team. In this role you will focus on creating a Unified Data Platform designing developing and maintaining data pipelines data lakes and platforms that meet the analytics and business intelligence needs. To tackle largescale data challenges you will work with advanced technologies like Spark Kafka AWS Azure and Kubernetes. Collaboration with fullstack developers data scientists analysts and stakeholders will be essential to ensure data quality and usability.
Responsibilities:
- Build automated pipelines to extract and process data from various legacy platforms primarily SQL Server.
- Implement datarelated business logic on modern data platforms utilizing AWS Glue Databricks and Azure adhering to best practices and industry standards.
- Create vector databases data marts and develop the corresponding data models.
- Optimize and monitor data systems and processes performance reliability and security.
- Integrate and transform data from diverse sources and formats including structured unstructured streaming and batch data.
- Develop and maintain data quality checks tests and comprehensive documentation.
- Support data analysis reporting and visualization using SQL Python Tableau and Quicksight tools.
- Research and evaluate emerging data technologies and trends to enhance data solutions and capabilities.
- Bachelors degree or higher in Computer Science Engineering Mathematics or a related field.
- At least 5 years of experience in data engineering or a similar role (previous DBA experience is a plus).
- Proficient with big data frameworks and tools such as Spark Hadoop Kafka and Hive.
- Expert in SQL with strong knowledge of efficient query and schema design DDL data modeling and stored procedures.
- Proficient in at least one programming language such as Python Go or Java.
- Experience with CI/CD containerization (e.g. Docker Kubernetes) and orchestration (e.g. Airflow).
- Familiarity with modern ETL ELT and data systems including AWS Glue Databricks Snowflake Elastic and Azure Cognitive Search.
- Experience deploying data infrastructure on cloud platforms (AWS Azure or GCP).
- Strong understanding of data quality governance and security principles and practices.
- Excellent communication collaboration and problemsolving skills
Employment Type
Full Time
Key Skills
Report This Job
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.
