Hello Nairalanders,

I want to share my experience in crawling leads from Jiji. As a web scraping enthusiast, I've tackled various sites, but this one proved to be difficult. This is because the phone numbers are in the product detail page and you have to be logged in to get it. Another issue I faced was that the website is loaded with java script and uses infinite scrolling.

I was able to circumvent these road blocks by using selenium to login to the site with my personal details after which I saved the cookies as a json file. As the cite uses infinite scrolling I scraped all the sellers url links to a csv file. I then read the url links so as to crawl the site using the links.

Remember my login details have been saved as cookies in a json file, In the process of crawling the site again using the URLs, I applied the cookies from the json file. The crawler visited each seller's detail page using the saved URL links and clicked the "Show Contact" button to extract the contact information.

Retailers specializing in second-hand items like laptops can leverage this approach coupled with applied analytics on the scraped the data to gain swift access to mouth-watering deals, faster than a regular visitor.

You can check out the code for executing the project in my github repo:
https://github.com/StephDAnalyst/JijiLeadScraping