Skip to content

Rexdzx/url-to-code

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

4 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Getting Started

Prerequisites

Make sure you have Node.js installed on your machine.

Installation

  1. Clone the repository:

    git clone https://github.com/Rexdzx/url-to-code.git
  2. Navigate to the project directory:

    cd your-repo-name
  3. Install dependencies:

    npm install

Usage

  1. Run the scrapper:

    node index.mjs
  2. Set the Website URL

    Masukkan URL : https://example.com
  3. Customize other options if needed (e.g., maxDepth, directory, etc.).

  4. Check the ./result directory for the downloaded website.

Configuration

  • urls: An array of URLs to scrape.
  • urlFilter: A function to filter URLs. The example filters URLs that start with the specified websiteUrl.
  • recursive: If true, the scraper will follow links recursively.
  • maxDepth: Maximum recursion depth.
  • prettifyUrls: If true, URLs will be prettified.
  • filenameGenerator: File naming strategy, set to 'bySiteStructure' in the example.
  • directory: Output directory for the downloaded website.

Acknowledgments

Happy Scrapping! 🕸️

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published