Skip to content

Script to scrape comments (including name, profile link, pfp, designation, email(if present), and comment) from a LinkedIn post from the URL of the post.

Notifications You must be signed in to change notification settings

gurbaaz27/linkedin-comments-scraper

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

35 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

LinkedIn Comments Scraper

Updates 🚀

  • Fix utf-8 encoding error
  • Shift configuration from Config.py to config.json
  • Add support for profile pictures of commentors
  • Add support for pulling images in comment section
  • Add headless support
  • Chrome extension
  • Show all replies support

Brief Overview

If you have used LinkedIn, you must have encountered many posts regarding helpful resources, and they would ask email address,to which replies in comments are usually like

Interested!
<email-address>

I don't like this thing, like you can already share link in the post itself 😠. Nevertheless, I thought to automate this work of collecting all emails.
All the comments, with columns of

  • Name of the person commented
  • Designation of the person
  • Profile Picture URL
  • Email address (if present in comment)
  • Comment (UTF-8 encoded)

are stored in a csv.

Requirements

Install the dependencies:

pip install -r requirements.txt

Usage

  • In config.json, enter the required url of LinkedIn Post in post_url variable:
post_url = ""

NOTE: If you forget to enter here, it will be asked during execution of script itself.

  • You can also change csv file name (in which scraped data will be stored) and dir name (in which profile pics will be downloaded) in config.json .
  • Help:
usage: main.py [-h] [--headless] [--show-replies] [--download-pfp]

Linkedin Scraping.

options:
  -h, --help      show this help message and exit
  --headless      Go headless browsing
  --show-replies  Load all replies to comments
  --download-pfp  Download profile pictures of commentors

NOTE: Even if the flag --download-pfp isn't provided, URLs of image would get stored in the output csv.

  • Run the script:
python main.py

Login email and password for your LinkedIn account will be asked and process would start.

Configuration

config.json contains various fields, containing information about scraping the HTML elements by name or xpath, and other metadata

Suggestions and contributio ns are alwasy welcome!:smile:

Star History

Star History Chart

About

Script to scrape comments (including name, profile link, pfp, designation, email(if present), and comment) from a LinkedIn post from the URL of the post.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published