How web scraping makes your job easier: three cases

Lena Volk
Lena Volk May 12, 2020

In my webinar about Web Scraping I like to say that Web Scraping is not a magic pill for all occasions. But there are some tasks where scraping can make your life easier and help do everything faster.

First of all, it’s really important to analyze what task you have, and then you can decide which resource (source) can work for it and choose a tool for scraping.

In this article I will share three cases that we and our school participants solved using Web Scraping. But in fact there are many more such cases in recruiting.

Case 1. Collect emails for mailing.

The easiest way is to use Iryna Shamaeva's Email extractor tool. It quickly copies all emails and then pastes them into the table. Once, thanks to this method, I extracted 500 mails in 5 minutes.

Where can it be used?

  • Github

  • X-ray requests (for example: site: linkedin.com/in "gmail.com" "Java developer" Ukraine)

  • TurboHiring

In order not to go to each page of the selection separately, you can use one of the tools (which, while scrolling, combines the results into one long page): AutoPagerize or PageZipper

If, in addition to mails, you also need names and additional information, try to use Instant Data Scraper or Data Miner

Case 2. Extract participants of communities/pages/event into one list

Where can it be used?

  • Facebook Groups

  • Meetup

  • LinkedIn Pages

  • Slack

A great tool for Facebook is DIG. Just keep in mind that if you are going to extract a list of participants of the event, it will extract only people you can see, i.e. your friends. Therefore, it is convenient to use it for events only if you organize them yourself.

But in groups where the list of participants is open, you can extract everyone.

Talking about Meetup, I prefer Spider. It is easy to use, you just have to highlight and select the required data for extracting.

Case 3. Automate some routine task for social networks.

What did we do?

  • Automatically accepted friend requests on Linkedin

  • Automatically sent LinkedIn requests to expand contacts

The best tool for these aims is PhantomBuster. And this is my favorite tool, or rather a store with tools, where you can find anything you want for different purposes and resources.

If you need to work with large amounts of data or automate your repetitive actions, take a look at these tools. They will definitely make your life easier and save some time.