Skip to content

Text Extraction with POS Tagging and Deep Learning(LSTMs)

Notifications You must be signed in to change notification settings

RemeAjayi/ds-job-detective

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

51 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Data Science Job Detective

Getting your dream Data Science Job is a great motivation for developing a Data Science Roadmap. How do you develop a Roadmap without knowing the relevant skills and tools to Learn. In this project, I will explore over 800 Data Science Job postings in Canada. The data used was collected from postings on Glassdoor and Indeed in early June, 2021.

Link to [Medium] - (https://medium.com/@Olohireme/job-skills-extraction-from-data-science-job-posts-38fd58b94675)

Table of contents

Installation

(Back to top)

pip install requirements.txt

Usage

(Back to top)

  • Stage 1: Scraping Data using selenium - Run glassdoor.py and indeed.py to scrape jobs.
  • Stage 2: Performed Exploratory Data analysis in EDA.ipynb
  • Stage 3: Performed Rule Based Skill Extraction using Spacy in skill_extraction.ipynb
  • Stage 4: Training and testing skill extraction LSTM model in job_skills_prediction.ipynb
  • Stage 5: Use saved LSTM and Rule-Based model to predict unseen text (i.e. Text that was not in the dataset) in job_skills_extraction_pipeline.ipynb
  • Stage 6: Streamlit deployment code in deploy.py

To run locally - streamlit run deploy.py

Demo

(Back to top)

You can find the Demo here.

Leave a star in GitHub, give a clap in Medium and share this guide if you found this helpful.

About

Text Extraction with POS Tagging and Deep Learning(LSTMs)

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published