Python pyspark Developer

10 years Proficiency in Python and experience with Langchain pyspark PyTorch Tensorflow Streamlit and other relevant tools Prompt Engineering and AI Chatbot Acumen: Developing sophisticated AI chatbot solutions including QA/chatbots translation and search/summarization functionalities. 5 years of experience in GenAI Foundation Models and Vector DB: Leveraging foundational AI models and vector database technologies (multi vector) for advanced AI capabilities. 5 years of experience in RAG (RetrievalAugmented Generation) and Model Fine Tuning: Employing RAG techniques for enhanced AI responses and finetuning AI models for optimal performance. Use of Orchestration Tools: Utilizing advanced tools like Semantic Kerner Langchain and others for efficient AI model management. Expertise in either OpenAI or Google Vertex and/or other models from Hugging face for implementing advanced language models. Managing high availability and efficient deployment of cloudnative applications Experience with hosting LLMs onpremises. Experience with GitOps principles and tools such as Git Tekton Flux or ArgoCD for managing infrastructure and application deployments. Strong problemsolving skills and the ability to work in a collaborative and fastpaced environment. 5 years of experience in Automated Artificial Intelligence Tools 5 years of experience in Machine Learning / AI 5 years of experience in AWS Azure and other Cloud Platforms. 5 years of experience in Docker and Kubernetes Bachelors or masters degree in computer science AI or a related field This role is crucial in achieving the best RAG results on client data and contributing to the broader AI and machine learning community. Collaborate with crossfunctional teams to understand business requirements and design AI solutions based on language models. Develop train and optimize language models using PyTorch Tensorflow and other relevant frameworks. Utilize expertise in either OpenAI or Google Vertex to implement stateoftheart language models. Responsible for EndtoEnd RAG Solution Design and Development Lead the experimentation of different chunking and retrieval methods to enhance the efficiency and effectiveness of RAG systems. Developing training finetuning LLMs for RAG Conduct thorough evaluations of model and application performance. This entails a deep dive into model accuracy bias identification and the formulation of strategies for ongoing enhancement. Analytical skills that drive continuous improvements and set new benchmarks for excellence. Engage with customers from various sectors to facilitate the successful adoption of Unstructured APIs