This work aims to assess the value of semantic (ontological) image comparison. We conduct the following analyses:

• classification value: Our main analysis is testing the ability of semantic image comparison scores on our ability to classify images. We show that semantic annotations can be used succesfully in a classification framework, can be used for decoding, and thus that there is value in performing semantic image comparison.
• wang graph similarity: The Wang Graph metric is a commonly used strategy to assess the similarity of gene sets, and we apply this to compare images based on their Cognitive Atlas tags. We compare an image similarity matrix of this metric against traditional spatial image comparison to assess if the method captures something that we understand about images.

All analyses are driven by the NeuroVault and Cognitive Atlas APIs, meaning that they can be reproduced as these databases are updated. While this particular workflow requires a cluster environment for performing many of the tests, the resources are all publicly available by way of the NeuroVault, Cognitive Atlas APIs (data), and the pybraincompare and CogatSimilaR packages (methods). This means that the data and algorithms, if useful to you, can be easily integrated into tools for development of applications that could use semantic image comparison.

While this is not (yet) set up to be a completely, one-click reproducible workflow, I intend to give this a shot. For now, assume that everything in "preprocessing" should be run in a cluster environment, and analysis (can be) run locally. All compiled outputs from the preprocessing steps are provided in this repo, and detailed where they come from in the documentation. For now, please use the requirements.txt file to install python dependencies, locally and on your cluster.

  pip install -r requirements.txt --user

For the R analyses, you will need R version 3.02 or higher, and to install the plyr, dplyr, and cogat-similaR package.

It is recommended to perform the preprocessing analyses in a cluster environment (SLURM was the cluster use to generate these results, and a local machine is only appropriate for the analyses.

Installation (for now) means cloning the repo in your cluster environment:

  git clone http://www.github.com/vsoch/semantic-image-comparison

and then following the instructions in the README.md, which coincide with example commands in run_analyses.sh. It is unfortunately still recommended to work through the steps in sequence, manually using the scripts, as many rely on submitting jobs, waiting for completion, and then compiling output. Let's get started with preprocessing!

You will notice that the cluster output is all relative to a base path, which is not necessarily relative to the repo you clone with scripts. You will need to move compiled outputs into their appropriate spots in the analysis directory (likely you will want to clone the repo and run this locally). Detailed instructions are provided in both the scripts and README.md in the preprocess folder.