Principal Data Engineer

Job Title: Principal Data Engineer

Work Location: Atlanta GA (Remote option available)

Duration: 4 6month contract

Please submit best quality 12 profiles today.

What We Are Looking For:

Responsibilities:

• As a senior/principal engineer you will be responsible for ideation architecture design and development of a new enterprise data platform. You will collaborate with other cloud and security architects to ensure seamless alignment within our overarching technology strategy.
• Architect and design core components with a microservices architecture abstracting platform and infrastructure intricacies.
• Create and maintain essential data platform SDKs and libraries adhering to industry best practices.
• Design and develop connector frameworks and modern connectors to source data from disparate systems both onprem and cloud.
• Design and optimize data storage processing and querying performance for largescale datasets using industry best practices while keeping costs in check.
• Architect and design the best security patterns and practices.
• Design and develop data quality frameworks and processes to ensure the accuracy and reliability of data.
• Collaborate with data scientists analysts and cross functional teams to design data models database schemas and data storage solutions.
• Design and develop advanced analytics and machine learning capabilities on the data platform.
• Design and develop observability and data governance frameworks and practices.
• Stay up to date with the latest data engineering trends technologies and best practices.
• Drive the deployment and release cycles ensuring a robust and scalable platform.

Requirements:

• 10 (for senior) 15 (for principal) of proven experience in modern cloud data engineering broader data landscape experience and exposure and solid software engineering experience.
• Prior experience architecting and building successful enterprise scale data platforms in a green field environment is a must.
• Proficiency in building endtoend data platforms and data services in GCP is a must.
• Proficiency in tools and technologies: BigQuery Cloud Functions Cloud Run Dataform Dataflow Dataproc SQL Python Airflow PubSub.
• Experience with Microservices architectures Kubernetes Docker and Cloud Run
• Experience building Symantec layers.
• Proficiency in architecting and designing and development experience with batch and real time streaming infrastructure and workloads.
• Solid experience with architecting and implementing metadata management including data catalogues data lineage data quality and data observability for big data workflows.
• Handson experience with GCP ecosystem and data lakehouse architectures.
• Strong understanding of data modeling data architecture and data governance principles.
• Excellent experience with DataOps principles and test automation.
• Excellent experience with observability tooling: Grafana Datadog.

Nice to have:

• Experience with Data Mesh architecture.
• Experience building Semantic layers for data platforms.
• Experience building scalable IoT architectures.