Python distributed web crawling scraper
More info to come...
David Albertson
##Execution
- Take a look at the configuration file and :
- change the server
listeningAddress / listeningPort
to the right info; - change the client
hostAddr / hostPort
to the right info.
- Launch the server on the master node
python server.py
- Launch the client on the working nodes
python client.py
##References