• NumPy
• Pandas
• Transformers
• Streamlit
• Torch
• Sentence_Transformers
• SimpleTransformers
• scikit-learn

Run this command from shell to install all libraries

pip install -r requirements.txt

Data can be downloaded from Kaggle here and must be placed in the data directory

• Can also be downloaded through Kaggle API by following these instructions

pip install kaggle
kaggle datasets download -d Cornell-University/arxiv
unzip arxiv.zip

• First make sure the full .json data file is saved in the data directory
• Run generate_embeddings.py

python generate_embeddings.py --save 'y'

• Then you can run the app with the below code

streamlit run app.py