GRAph-based Finding of Individual Motif Occurrences

Regulatory proteins, such as Transcription Factors (TFs), are key genomic elements which promote or reduce the expression of genes by binding short, evolutionary conserved DNA sequences, often referred to as motifs. Mutations occurring in DNA motifs have been shown to have deleterious effects on the transcriptional landscape of the cell (Li & Ovcharenko, 2015; Guo et al., 2018). The recent introduction of Genome Variation Graphs (VG) (Garrison et al., 2018) allowed to represent in a single and efficient data-structure the genomic variation present within a population of individuals.

GRAFIMO (GRAph-based Finding of Individual Motif Occurrences) is a command-line tool that extends the traditional Position Weight Matrix (PWM) scanning procedure to VGs. GRAFIMO can search the occurrences of a given PWM in many genomes in a single run, accounting for the effects that SNPs, indels and potentially any structural variation (handled by VG) have on found potential motif occurrences. As result, GRAFIMO produces a report containing the statistically significant motif candidates found, reporting their frequency within the haplotypes embedded in the scanned VG and if they contain genomic variants or belong to the reference genome sequence.

GRAFIMO depends on a number of external tools and Python packages. Before installing GRAFIMO you should install

• VG, v1.27.1 or later (https://github.com/vgteam/vg)
• Tabix (https://github.com/samtools/htslib)
• Graphviz (https://www.graphviz.org/)

Be sure that all are reachable in Unix PATH.

Note that is suggested to use samtools Tabix and not the one coming with VG. If the user is not sure about which tabix is running, he/she can type which tabix, to retrieve which is the currently used Tabix.

GRAFIMO is written in both Python3 and Cython. Thus, the user will need Cython to be installed to correctly build GRAFIMO. Cython can be obtained via pip

pip3 install Cython

To build GRAFIMO are used setuptools and wheel. The user should make sure he/she has the latest version of both setuptools and wheel

python3 -m pip install --user --upgrade setuptools wheel

GRAFIMO depends on a number of Python packages. If the dependencies are not satisfied, by building GRAFIMO from source or via pip they should be automatically solved.

To install all the required packages:

pip3 install pandas
pip3 install numpy
pip3 install statsmodels
pip3 install sphinx
pip3 install numba
pip3 install colorama

GRAFIMO can be built and installed via pip, from source code or via Bioconda. Note that the latter option is available only for Linux users.

For further details on how to install GRAFIMO visit our Wiki.

Build and install via pip

To build and install GRAFIMO via pip

pip3 install grafimo

To test if GRAFIMO have been correctly installed

grafimo -h

If the help is correctly printed, then GRAFIMO have been installed and can be called from any location.

Build and install from source code

To build and install GRAFIMO from source code

git clone https://github.com/pinellolab/GRAFIMO.git 
cd GRAFIMO
python3 setup.py install --user

To quickly test GRAFIMO installation

grafimo -h

If the help is correctly printed, then GRAFIMO have been installed and can be called from any location.

It is also possible to test all the main functionalities of GRAFIMO using pytest (https://docs.pytest.org/en/stable/).

To install pytest

pip3 install pytest

Once pytest have been installed, enter the tests directory and launch pytest

cd tests
pytest

If no test fails, then all GRAFIMO functionalities work properly.

Build and install via Bioconda (Linux users only)

To install GRAFIMO via Bioconda the user should first have the conda package installed. If the user have an Anaconda Python installation, conda is already available, otherwise it can be installed with Miniconda package. For further details (https://bioconda.github.io/user/install.html).

Once conda is available, to install GRAFIMO

conda install grafimo

To update GRAFIMO

conda update grafimo

For MacOS and Windows users is suggested to run GRAFIMO via Docker.

The user can both pull an already built GRAFIMO docker image or build it from scratch. The user has also to ensure that Docker is currently installed and there are no too strict limits on the number of CPUs and amount of memory that Docker can use (https://docs.docker.com/config/containers/resource_constraints/ for further details).

To pull the pre-built Docker image:

docker pull pinellolab/grafimo

To test if the image is correctly running, type:

docker run -i pinellolab/grafimo grafimo -h

If the help is correctly displayed, then the image has been correctly pulled.

To build GRAFIMO Docker image from scratch, the user should clone or download GRAFIMO's github repository:

git clone https://github.com/pinellolab/GRAFIMO.git
cd GRAFIMO

and build the image:

docker build -t grafimo .

To test if the image has been correctly built, type:

docker run -i grafimo grafimo -h

If the help is correctly displayed, then the image has been correctly built.

For hands-on tutorials on how to use GRAFIMO check out our tutorials.

For further details on GRAFIMO usage refer to our Wiki.

GRAFIMO requires three mandatory arguments:

• path to a directory containing the chromosomes VGs (XG and GBWT indexes) or path to the whole genome variation graph (XG and GBWT indexes). See VG's wiki for further details on XG and GBWT indexes.

• path to PWM motif file in MEME or JASPAR format

• BED file containing a set of genomic regions where GRAFIMO will search the motif occurrences

The main functionality of GRAFIMO is to perform a haplotype and variant-aware search of potential DNA motif occurrences in genome variation graph.

Here we assume that the genome variation graph (VG) has been built constructing a VG for each chromosome. If working with a single whole genome variation graph just substitute the argument -d with -g followed by the path to the whole genome VG. In the next section will be presented how to build a VG with GRAFIMO.

Note that in both cases the XG and GBWT indexes of the VG must be stored in the same location.

For further details refer to our Wiki.

If you are working in the tutorials/findmotif_tutorial directory, to run GRAFIMO

grafimo findmotif -d data/mygenome/ -m  data/example.meme -b data/regions.bed

By default GRAFIMO will create a directory called grafimo_out_PID_MOTIFID, containing the results. For further details on result files see Results description section.

Using background distributions

For each potential motif occurrence GRAFIMO computes a log-likelihood score, a P-value and a q-value. Such measures are weighted by a background probability distribution. By default, GRAFIMO assumes a uniform background distribution for nucleotides. The user can specify a different background distribution in a text file and give it to GRAFIMO using -k option. An example of background file is

A	0.2951
C	0.2047
T	0.2955
G	0.2048

For an example of background files accepted by GRAFIMO, take a look at bg_nt in tutorials/findmotif_tutorial/data directory.

If you are working in tutorials/findmotif_tutorial directory, to run GRAFIMO with a background distribution

grafimo findmotif -d data/mygenome/ -m data/example.meme -b data/regions.bed -k data/bg_nt 

Setting thresholds on motif occurrences statistical significance

By default GRAFIMO applies a threshold of 1e-4 on the P-value of each retrieved potential motif occurrence. So, will be reported the motif candidates with an associated P-value smaller than 1e-4. The threshold can be changed by using the -t option. For example, let us set a threshold of 0.05 on the P-values.

If you are working in tutorials/findmotif_tutorial directory, to run GRAFIMO applying a different threshold on P-values

grafimo findmotif -d data/mygenome -m data/example.meme -b data/regions.bed -t 0.05

GRAFIMO, besides P-values, computes q-values for the motif occurrences candidates. The user can apply a threshold on q-values, rather than on P-values, by using the --qvalueT option. --qvalueT option can be used in combination with -t to define a threshold value different from 1e-4. Let us apply a threshold of 1e-4 on q-values.

If you are working in tutorials/findmotif_tutorial directory, to run GRAFIMO applying a threshold on q-values

grafimo findmotif -d data/mygenome -m data/example.meme -b data/regions.bed --qvalueT -t 1e-4

For more options refer to our Wiki or type

grafimo -h

GRAFIMO results are reported in three files (stored in output directory):

• tab-delimited report (TSV report)
• HTML report
• GFF3 report

The TSV report contains all the statistically significant potential motif occurrence found by GRAFIMO (according to the applied threshold). Each retrieved motif occurrence has a log-likelihood score, a P-value, a q-value, its DNA sequence, a flag value stating if a sequence is part of the reference or has been found in the haplotypes and the number of haplotype sequences where the motif candidate sequence occurs. An example of TSV report is the following

	motif_id	motif_alt_id	sequence_name	start	stop	strand	score	p-value	q-value	matched_sequence	haplotype_frequency	reference
1	MA0139.1	CTCF	chr22:43481590-43481860	43481733	43481714	-	21.26229508196724	4.403657357543095e-08	0.004175283686980911	AAGCCAGCAGGGGGCACAG	5096	ref
2	MA0139.1	CTCF	chr22:19038291-19038561	19038422	19038441	+	19.245901639344254	1.9442011615088443e-07	0.005962538354344465	TGGCCAGCAAGGGGCACTG	4	non.ref
3	MA0139.1	CTCF	chr22:19038291-19038561	19038422	19038441	+	19.114754098360663	2.1268826066771178e-07	0.005962538354344465	CGGCCAGCAAGGGGCACTG	5092	ref
4	MA0139.1	CTCF	chr22:40856678-40856948	40856891	40856910	+	18.295081967213093	3.6764803446618004e-07	0.005962538354344465	TCCCCTCCAGGGGGCGACG	5096	ref
5	MA0139.1	CTCF	chr22:11285607-11285877	11285804	11285785	-	18.213114754098342	3.8774723287177635e-07	0.005962538354344465	ATACCGCCAGGTGGCAGCA	5096	ref
6	MA0139.1	CTCF	chr22:22125904-22126174	22126044	22126063	+	18.13114754098359	4.088625891963074e-07	0.005962538354344465	CAGCCTGCAGATGGCACAG	5096	ref
7	MA0139.1	CTCF	chr22:20146797-20147067	20147010	20147029	+	17.688524590163922	5.4295945317287e-07	0.005962538354344465	CGGCCCGCAGGGGGCGGAT	5092	ref
8	MA0139.1	CTCF	chr22:34842682-34842952	34842827	34842846	+	17.672131147540995	5.486120126825257e-07	0.005962538354344465	GAGCCAGTAGGGGACAGCG	146	non.ref
9	MA0139.1	CTCF	chr22:42532903-42533173	42533062	42533081	+	17.622950819672155	5.659801842459994e-07	0.005962538354344465	GGGCCACCAGAGGGCTCCT	5096	ref
10	MA0139.1	CTCF	chr22:34842682-34842952	34842827	34842846	+	17.44262295081967	6.331282484526275e-07	0.006002942174878742	GAGCCAGTAGGGGACAGTG	4950    ref

This report can be easily processed for a downstream analysis.

The HTML report has the same content of the TSV, but it can be loaded and viewed on the most commonly used web browsers.

The GFF3 report can be loaded on the UCSC Genome Browser as a custom track. For example, this allows a fast linking between the genomic variants used to build the VG and those present in annotated databases like dbSNP or ClinVar.

GRAFIMO allows also to build a genome variation graph from user data. To construct the VG are required

• a genome reference (in FASTA format)
• VCF file containing the genomic variants to enrich the reference sequence.

GRAFIMO builds the genome variation graph by constructing a VG for each chromosome. This allows a faster and more efficient motif search on the genome variation graph.

Note that this genome variation graph building approach is suggested by VG developers.

GRAFIMO will construct the XG and the GBWT index for each chromosome. The XG and GBWT indexes allow a faster and haplotype-aware motif search on VG.

Before attempting to build the VG it is very important to make sure that the chromosome names in the VCF and in the reference FASTA sequence headers match. For example, if in the VCF the chromosome 1 is named 1, the header of chromosome 1 sequence on the reference FASTA file should be >1, and not something like >chr1.

If you are in tutorials/buildvg_tutorial directory, to build a VG with GRAFIMO

grafimo buildvg -l data/xy.fa -v data/xy2.vcf.gz    

For further details refer to our Wiki.

Li, Shan, and Ivan Ovcharenko. "Human enhancers are fragile and prone to deactivating mutations." Molecular biology and evolution 32.8 (2015): 2161-2180.

Guo, Yu Amanda, et al. "Mutation hotspots at CTCF binding sites coupled to chromosomal instability in gastrointestinal cancers." Nature communications 9.1 (2018): 1-14.

Garrison, Erik, et al. "Variation graph toolkit improves read mapping by representing genetic variation in the reference." Nature biotechnology 36.9 (2018): 875-879.

All the scripts and IPython notebooks required to reproduce the experiments and the analysis presented in GRAFIMO's paper are available here.

If you use GRAFIMO in your research, please cite us:

Tognon M, Bonnici V, Garrison E, Giugno R, Pinello L (2021) GRAFIMO: Variant and haplotype aware motif scanning on pangenome graphs. PLOS Computational Biology 17(9): e1009444. https://doi.org/10.1371/journal.pcbi.1009444

MIT