GitHub - helllamer/ebot: Ebot, an Opensource Web Crawler built on top of a nosql database (apache couchdb, riak), AMQP database (rabbitmq), webmachine and mochiweb. Ebot is written in Erlang and it is a very scalable, distribuited and highly configurable web cawler. See wiki pages for more details

helllamer / ebot Public

Notifications You must be signed in to change notification settings
Fork 4
Star 4

Ebot, an Opensource Web Crawler built on top of a nosql database (apache couchdb, riak), AMQP database (rabbitmq), webmachine and mochiweb. Ebot is written in Erlang and it is a very scalable, distribuited and highly configurable web cawler. See wiki pages for more details

www.redaelli.org/matteo-blog/projects/ebot

GPL-3.0 license

4 stars 55 forks Branches Tags Activity

Star

Notifications

Branches Tags

Name		Name	Last commit message	Last commit date
Latest commit History 243 Commits
deps		deps
doc		doc
ebin		ebin
priv		priv
src		src
utils		utils
.gitignore		.gitignore
.hgignore		.hgignore
Emakefile		Emakefile
LICENSE.txt		LICENSE.txt
Makefile		Makefile
README		README
start-dev.sh		start-dev.sh
start.sh		start.sh
sys.config		sys.config

Repository files navigation

EBOT (https://www.redaelli.org/matteo-blog/projects/ebot/)
USAGE: see the wiki homepage (https://wiki.github.com/matteoredaelli/ebot/) and ebot_test.erl for details
LICENSE: GPL v3+, see file LICENSE
Copyright (c) 2010, 2011 - Matteo Redaelli at libero dot it

Tested with Linux (Ubuntu and Debian) and Erlang R13B03 and R14A

*******************************************************************************
* DOWNLOAD "TAG" releases and not the latest sources that could be unstable...
*******************************************************************************


=== Compliling ===

By now, you need to fill the ./deps by hands. Resulting deps should be like this:

$ ls -l ./deps
/couchbeam			(git:https://github.com/benoitc/couchbeam.git)
/rabbitmq-erlang-client		(hg clone https://hg.rabbitmq.com/rabbitmq-erlang-client/)
/rabbit_common			symlink to ./rabbitmq-erlang-client/dist/rabbit_common-0.0.0
/webmachine			(git:https://github.com/basho/webmachine.git)
/riak-erlang-client		(git:https://github.com/basho/riak-erlang-client.git)
/ibrowse			(git:https://github.com/cmullaparthi/ibrowse.git)

See https://github.com/matteoredaelli/ebot/wiki/Installation for more info about compiling.


=== Architecture ===

https://github.com/matteoredaelli/ebot/wiki/


=== More info ===

https://github.com/matteoredaelli/ebot/wiki/_pages