This Crawler is for crawl an online shop Bhinneka. It will save the item name, link, categories and price in MySQL.
-
Clone this repository
git clone https://github.com/clasense4/scrapy-bhinneka-crawler.git
-
Edit
bhinneka_crawler/settings.py
change yourscrapy
,redis
andMySQL
setting -
Insert this SQL query :
CREATE TABLE `bhinneka` ( `bhinneka_id` int(11) NOT NULL AUTO_INCREMENT, `name` tinytext NOT NULL, `link` tinytext NOT NULL, `categories` tinytext NOT NULL, `price` tinytext NOT NULL, PRIMARY KEY (`bhinneka_id`) ) ENGINE=MyISAM DEFAULT CHARSET=latin1 COMMENT='latin1_swedish_ci'
-
Start your crawler with this command
$> scrapy crawl bhinneka_spider
-
At 19 January 2013, this script give me
14567 Items
.
The script is still sucks, just for fun, not follow scrapy standards, use at your own risks.
mail me at clasense4[at]gmail[dot]com