Web Crawler

1. "crawler.py" will search the absolute links and data and frequency of words for the webpage mentioned within the code.(static web crawler)
2. "crawler.py" will save the links and frequency of words in links.txt and data in text.txt
3. Here, we have used an array visted to keep the links we have visited. URL Server:
4. Here we have used 2 files "my_server.py" and "i_client.py" to implement URL server along with "crawler.py"
5. Here we have implemented one URL server for one Web Crawler.
6. The website address is given within "my_server.py" How to Run:
7. pyton my_server.py
8. python i_client.py