Overview
The Big Data Engineer plays a pivotal role in TCS driving initiatives to leverage data in innovative ways to address complex business challenges. As a key member of the data team the Big Data Engineer is responsible for designing developing and maintaining robust data pipeline architectures. This position is crucial as organizations increasingly rely on datadriven decisionmaking. The Big Data Engineer collaborates with crossfunctional teams to transform raw data into actionable insights enabling TCS to stay competitive in a rapidly evolving technological landscape. Additionally the role encompasses evaluating new technologies optimizing current processes and ensuring the integrity and privacy of data. The Big Data Engineer directly contributes to TCSs mission of delivering highquality services and solutions making this position integral to the organizations success.
Key Responsibilities
- Design and implement scalable data processing frameworks.
- Develop data pipelines for ETL (Extract Transform Load) processes.
- Work with Hadoop ecosystem (HDFS MapReduce Hive) to process large datasets.
- Utilize Apache Spark for data processing and analysis.
- Collaborate with data scientists to analyze and interpret data.
- Integrate data from various sources ensuring data quality and consistency.
- Create data models that support data analytics and reporting.
- Optimize performance of existing data systems and procedures.
- Monitor and troubleshoot data flows and pipeline performance.
- Ensure data privacy and compliance with regulations.
- Build visualization tools and dashboards for data presentation.
- Research and evaluate new technologies to enhance data capabilities.
- Support team members with best practices in data engineering.
- Document processes and tools for operations and support.
- Provide insights on trends and patterns to stakeholders.
Required Qualifications
- Bachelors degree in Computer Science Information Technology or related field.
- Proven experience as a Big Data Engineer or similar role.
- Strong knowledge of Hadoop ecosystem and related technologies.
- Expertise in programming languages such as Java Scala or Python.
- Experience with SQL and NoSQL databases.
- Familiarity with cloud services like AWS Azure or Google Cloud.
- Understanding of data warehousing concepts and architectures.
- Experience with data visualization tools such as Tableau or Power BI.
- Knowledge of machine learning algorithms and frameworks.
- Ability to work with large complex data sets.
- Good understanding of data governance and security requirements.
- Strong analytical and problemsolving skills.
- Excellent communication and collaboration abilities.
- Ability to quickly learn new technologies and tools.
- Experience with Agile development methodologies.
tableau,data governance,nosql,machine learning,pipeline,hadoop,problem solving,python,hive,java,apache spark,analytical skills,big data,ecosystem,data analysis,data security,agile development methodologies,scala,cloud,power bi,azure,hadoop ecosystem,spark,etl processes,communication skills,problem-solving skills,aws,data visualization tools,machine learning algorithms,collaboration skills,data,google cloud,sql,data visualization