This repository contains satelite projects of ache, a focused crawler.
To install it, you need:
-
install git sudo apt-get install git
-
Install Java, like the OpenJDK such as java-7-openjdk sudo apt-get install openjdk-7-jdk
-
Make sure python 2.7 is installed
-
virtualenv sudo pip2 install virtualenv
-
install fabrick pip2 install fabric
-
install numpy and scipy sudo apt-get install pyton-numpy python-scipy
-
install elasticsearch wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.5.2.deb sudo dpkg -i elasticsearch-1.5.2.deb rm elasticsearch-1.5.2.deb
-
Start elasticsearch, for now: sudo /etc/init.d/elasticsearch start
-
On this directory (seed_crawler), type: fab setup
It will take some time to proceed, download everything you need, check that things are properly installed, and stop.
-
Run the program: fab runvis
-
Open a web browser and connect to the vis server at the following url: http://localhost:8084/seedcrawler