Skip to content

greamown/web_crawler

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

6 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Web Crawler

Getting Started

  • Support "PCHOME" GPU INFO in this code.

Pre-requirements

sudo chmod u+x ./env/*.sh
sudo ./env/env.sh

Setting mariadb

sudo service mariadb start
sudo mysql -u ${USER} -p

DB GUI - Phpmyadmin

sudo apt update
sudo apt-get install -y phpmyadmin
sudo nano /etc/apache2/apache2.conf ->

# add this line somewhere
Include /etc/phpmyadmin/apache.conf

sudo phpenmod mbstring
sudo systemctl restart apache2

Demo

conda activate crawler
python3 main.py -u https://24h.pchome.com.tw/ -v "NVIDIA GPU"
  • u: The URL of the crawler.
  • v: The value of search.

Reference

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages