Full-Stack Data Engineer
Full-Stack Data Engineer
Employer Active
Job Alert
You will be updated with latest job alerts via email
Share
Job Alert
You will be updated with latest job alerts via email
Job Description
We are seeking a highly skilled FullStack Data Engineer to lead the design development and management of a datadriven solution that integrates diverse carrelated data sources and ensure that all integrated data is validated structured and accurate serving as a single source of truth for all carrelated data services.
This role combines expertise in backend engineering data science and knowledge in frontend development to build a scalable and efficient and maintainable platform. You will be responsible for building APIs implementing data aggregation pipelines creating a rule engine for intelligent data unification and routing and extending a monitoring/admin panel for solution management.
The ideal candidate has a strong understanding of AWS services data classification and data modeling techniques as well as fundamental web development knowledge.
Key Responsibilities
Data Integration and Processing
- Design implement and maintain pipelines to gather data from various sources including Web sites APIs DBs FTPs etc.
- Classify and categorize data into a wellstructured taxonomy for use in analytics and decisionmaking systems.
- Develop and configure a rule engine to unify incoming data and make intelligent routing decisions based on predefined rules.
Backend Development
- Design and maintain a robust database schema in PostgreSQL (hosted on AWS RDS) to store and retrieve data efficiently.
- Build scalable RESTful APIs using AWS API Gateway and AWS Lambda.
- Implement a data aggregation pipeline using the ELK Stack for analytics.
FullStack Development
- Extend and maintain an intuitive and responsive Admin Panel using Node.js and JavaScript for solution monitoring managing data pipelines rules and configurations.
Cloud Infrastructure and Deployment
- Utilize AWS services to deploy and manage APIs ensuring scalability and security.
- Implement AWS S3 CloudFront for efficient storage and delivery of image assets.
- Extend and implement data security and access control measures using tools like IAM API Keys and AWS Cognito.
Collaboration
- Work closely with data scientists product managers and other developers to ensure the platform meets business requirements.
- Document processes workflows and designs for team collaboration and knowledge sharing.
Required Skills & Qualifications
Technical Skills :
- Experience with PostgreSQL including schema design query optimization and AWS RDS deployment.
- Experience with classification and categorization of complex datasets.
- Expertise in data aggregation tools like the ELK Stack (Elasticsearch Logstash Kibana).
- Familiarity with AWS services including API Gateway Lambda S3 RDS and CloudFront.
- Knowledge of building and using rule engines for data unification and routing.
- Knowledge in Python (for data gathering and pipeline development).
- Knowledge of Node.js and JavaScript (for the admin panel extension)
Soft Skills:
- Strong analytical and problemsolving skills.
- Ability to work independently and collaboratively in a team environment.
- Excellent communication and documentation skills.
Nicetohave:
- Familiarity with VIN decoding and car industryspecific datasets.
- Experience with some of the modern data engineering tools like Apache Airflow AWS Glue or Step Functions
Interested in leading the design development and management of a datadriven solution that integrates diverse carrelated data sources ensuring they are validated structured and accurate
Apply Now! Send us your CV in English to
Employment Type
Full Time
