Senior Data Analytics Back End

Title: Senior Data Analytics Back End

Location: Remote

Contract

Job Description:

The Senior Data Engineer is responsible for overseeing junior data engineering activities and aiding in building the business' data collection systems and processing pipelines. The role of the Senior Data Engineer is responsible for building and maintaining optimized and highly available data pipelines that facilitate deeper analysis and reporting by the Data and Analytics department.

The Senior Data Engineer builds data processing frameworks that handle the business's growing database. They work with senior data science leadership as well as other Data and Analytics teams in leveraging data with reporting and scientific tools, for example, Tableau, Python, and Spark. The Senior Data Engineer strives to continuously develop new and improved data engineering capabilities.

Required and Nice-to-Have Competencies:

Required: Languages: PySpark, SQL, Python (Pandas)

Nice To Have: Raw data living in Palantir Foundry

Nice to Domain Experience: Telecom

Skillsets Nice to haves: Merge and transform datasets

Skillsets Nice to haves: Create machine learning record

Objectives and Responsibilities of the Role:

Management and Strategy:

The managerial role of the Senior Data Engineer is primarily for overseeing activities of the junior data engineering teams, ensuring proper execution of their duties and alignment with business vision and objectives. Provides senior-level contribution to a team that is responsible for the design, deployment, and maintenance of the business's data platforms.

However, the Senior Data Engineer will also implement strategies directed at acquiring data and promoting the development of new insights across the business. The Senior Data Engineer owns and extends the business's data pipeline through the collection, storage, processing, and transformation of large data-sets.

It is their duty to monitor the existing metrics, analyze data, and lead partnership with other Data and Analytics teams in an effort to identify and implement system and process improvements. The Senior Data Engineer will additionally develop queries for ad hoc business projects, as well as ongoing reporting. In this capacity, the Senior Data Engineer builds a metadata system where all available data is maintained and cataloged. The Senior Data Engineer also plays a major role in the development of reliable data pipelines that translate raw data into powerful features and signals

They design, architect, implement, and support key datasets that avail structured and timely access to actionable business insights. The Senior Data Engineer is additionally tasked with developing ETL processes that convert data into formats through a team of data analysts and dashboard charts.

Collaboration and Support:

The Senior Data Engineer plays a collaborative role where they work closely with the business's Data and Analytics teams, gathering technical requirements for exceptional data governance across the department and the business at large.

In this collaboration, the Senior Data Engineer works the data analysts, data warehousing engineers, and data scientists in finding and applying best practices within the Data and Analytics department as well as defining the business's data requirements, which will ensure that the collected data is of a high quality and optimal for use across the department and the business at large.

The Senior Data Engineer will also work with senior data science management and departments beyond the Data and Analytics department in analyzing and understanding data sources, participating in design, and providing insights and guidance on database technology and data modeling best practices.

In this capacity, the Senior Data Engineer will further be required to draw performance reports and strategic proposals form his gathered knowledge and analyses results for senior data science leadership.

Analytics:

The Senior Data Engineering plays an analytical role where they develop and manage scalable data processing platforms that they use for exploratory data analysis and real-time analytics. It is also the role of the Senior Data Engineer to oversee, design, and develop algorithms for real-time data processing within the business and to create the frameworks that enable quick and efficient data acquisition.In this capacity, the Senior Data Engineer retrieves and analyzes data through the use of SQL, Excel, among other data management systems. They also build data loading services for the purpose of importing data from numerous disparate data sources, inclusive of APIs, logs, relational, and non-relational databases.

Knowledge and Opportunity:

The Senior Data Engineer is tasked with the responsibility of contributing to the continual improvement of the business's data platforms through their observations and well-researched knowledge. They keep track of industry best practices and trends and through their acquired knowledge, take advantage of process and system improvement opportunities.

Other Duties:

The Senior Data Engineer performs similar duties as thhe deem fit for the proper execution of their duties and duties as delegated by the Head of Data Science, Director Data Science, Chief Data Officer, or the Employer.

Required Qualifications of the Role:

• Education: The Senior Data Engineer must have a bachelor's degree (masters' preferred) in Computer Science, Applied Mathematics, Engineering, or any other technology related field. An equivalent of this educational requirement in working experience is also acceptable.
• Experience: A candidate for this position must have had at least 5 years of working experience working in a data engineering department, preferably as a Data Engineer in a fast-paced environment and complex business setting. The candidate must have a demonstrated experience in building and maintaining reliable and scalable ETL on big data platforms as well as experience working with varied forms of data infrastructure inclusive of relational databases such as SQL, Hadoop, Spark and column-oriented databases such as Redshift, MySQL, and Vertica. The candidate must also have had experience in data warehousing inclusive of dimensional modeling concepts and demonstrate proficiency in scripting languages, for example, Python, R, and so forth. A suitable candidate will also demonstrate machine learning experience and experience with big data infrastructure inclusive of MapReduce, Hive, HDFS, YARN, HBase, Oozie, etc. The candidate will additionally demonstrate substantial experience and a deep knowledge of data mining techniques, relational, and non-relational databases.
• Communication Skills: Communication Skills for the Senior Data Engineer are just as important as they are for the Data Engineer, both in verbal and written form. The Senior Data Engineer oversees and manages junior data engineering teams and to ensure effective management, they must be capable of conveying information and instructions clearly down the line to the junior team.
• Communication skills are also imperative for the Senior Data Engineer in their collaborative role where they will have to interact cross-functionally with non-technical departments. To enable effective collaborations, the Senior Data Engineer will have an exceptional ability to convey complex messages in a clear, simplified, and understandable manner. They will also be required to draft reports and prepare presentations for senior data science leadership. These reports and presentation must be clear, concise, unambiguous, engaging, and convincing, which will demand exceptional communication skills on the Senior Data Engineer's part.
• MS Office/Software: A candidate for this position must be highly proficient in the use of MS Word, MS Excel, PowerPoint, and Outlook, which will all be necessary for the creation of both visually and verbally engaging reports and presentations, for senior data science leadership. The Senior Data Engineer must further have exceptional skills in SQL server reporting services, analysis services, Tableau, Salesforce, integration services, or any other data visualization tools.
• Technological Savvy/Analytical Skills: A candidate for this position will also demonstrate strong computer skills and a deep passion for analytics. The candidate for this position must possess an ability to perform complex data analyses with large data volumes. They will be an expert in SQL, Java, and have a keen understanding of data models and data warehouse concepts.
• The candidate will demonstrate an ability translate algorithms provided by senior data science management and implement them in as well as strong knowledge in Linux, OS tools, and file-system level troubleshooting. The candidate must have substantial experience working with big data infrastructure tools such as Python, SQS, and Redshift. A suitable candidate will also be proficient Scala, Spark, Spark Streaming, AWS, and EMR.
• Interpersonal Skills: The Senior Data Engineer must have certain preferable personal attributes that will make them that much more suited for the position. The Senior Data Engineer will be a result-driven individual, be passionate and a self-starter, be proactive requiring minimal supervision, be highly organized, have an ability to handle multiple tasks and meet tight deadlines, be a creative and strategic thinker, work comfortably work in a collaborative setting, work comfortably with senior departmental leadership, and demonstrate an ability to remain calm during times of uncertainty and stress, inspiring the same in his team.
• People Skill: The candidate must be a people person who is able to form strong, meaningful, and lasting connections with others, enabling smooth and continued collaborative relationships, earning them the trust of his juniors who will readily follow in their directives, and gaining the confidence of senior data science leadership.