An End-to-End network for Multi Modal Deep Learning.
To train the unsupervised approach, run the file python3 singleNet.py
.
The model is label independent and uses the pairwise loss function to maximize the distance between negative pairs from the positive ones.
To train the supervised appraoch, run the file Supervised_Model/train.py
.
The models uses the label in form of one-hot vectors to make sure that the images and positive descriptions are nearer than images and their negative counterparts.