Skip to content

ojd2/json-print

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

py-json-converter

Installation

The following modules are required for the program to be succesfully executed:

  • re (For regular expressions)
  • urllib2 (For url / request parsing)
  • json (For pretty printing objects as json)
  • BeautifulSoup (For html parsing)
  • 'request' (For Unit Testing)
  • ``unittest` (For Unit Testing)

Simply returning the following at the command line should install these for you:

> python setup.py

Run Program

The following command should run the program:


> python model.py

Program Architecture

The program follows a simple OOP architecture and the fundamental operations are called using a class mechanisim that adopts multiple methods through inheritence.

The Base Class: Scrape()

Class Methods:

- `retrieveData(self)`
- `getProdAttr(self)`
- `getProdUri(self)`
- `getRequest(self)`

Global Objects: json_obj, product_attr

Each method communicates with eachother and recursively returns objects for each method to communicate and interact with.

Testing

Some Unit Testing has been implemented and can be found within the test_module.py file.

Installation

The following modules need to be installed on your system:

  • unittest
  • request

Run Tests

After installing both unittest and request, the testing program can be executed by returning the following command:

> python -m unittest test_model

About

Simple Web Crawler to export results as Json

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages