- Proven experience of 5 years as a Web Scraping Specialist or similar role with a track record of successful web scraping projects.
- Expertise in handling dynamic content useragent rotation CAPTCHAs rate limits and utilization of proxy services.
- Proficiency in programming languages commonly used for web scraping such as Python BeautifulSoup Scrapy Scrapebox or Selenium.
- Knowledge of Proxies (static free and commercial/Residential)
- Past experience scraping linkedin data is a huge plus.
- Experience with one or more scraping tools and providers like Apify Diffbot Bright Data or Oxylabs
- Experience with cloudbased platforms like AWS GCP or Azure is a plus
- Strong knowledge of JavaScript API database HTML CSS XPath and other web technologies relevant to web scraping.
- Knowledge and experience in best of class data storage and retrieval of large volume of scraped data.
- Understanding of web scraping best practices including handling dynamic content useragent rotation and IP address management.
- Attention to detail and the ability to handle and process large volumes of data accurately.
- Familiarity with data cleansing techniques and data validation processes.
- Knowledge of web scraping ethics legal considerations and compliance with website terms of service.
- Strong problemsolving skills and the ability to adapt to changing web environments
selenium,python,javascript,database,web scraping,scrapy,scrapebox,diffbot,beautifulsoup,api,apify,linkedin,bright data,css,proxies,aws,xpath,oxylabs,gcp,azure,html