Tools for crawling the novels, currently support the following websites.
Setup Virtual Environment
python -m venv .venv
source .venv/bin/activate
Install dependencies
pip install -r requirements.txt
Install current project as well
pip install -e .
Modify the config.json, change the save directory
{
"store_dir" : "C:\\path\\you\\want\\to\\save\\novels",
"file_format" : "txt"
}
Execute the crawler
python src/novel_crawler/main.py [novel_url]