Ngs crumbs is a collection os small ngs file processing utilities.

Seq Crumbs is a collection of small sequence processing utilities.

Seq Crumbs is modeled after the Unix command line text processing utilities, so every utility tries to perform a specific task. Most of them take a sequence file as input and create a new processed sequence file as output. This design encourages chaining the operation of multiple Seq Crumbs utilities using Unix pipes.

You can find more information about seq_crumbs in the seq_crumbs web site.

sff_extract

Extracts the reads from a SFF file used by 454 and Ion Torrent.

split_matepairs

Splits the mate-pairs separated by an oligo sequence.

filter_by_quality

Filters the sequences according to its mean quality.

filter_duplicates

Filter out identical sequences.

filter_by_length

Filters sequences according to a length threshold.

filter_by_name

Filters the sequences with a list of names given in a file.

filter_by_blast

Filters the sequences using BLAST.

filter_by_bowtie2

Filters the sequences using bowtie2.

filter_by_complexity

Filters the sequences according to their complexity.

trim_by_case

Trims the sequences according to case.

trim_edges

Removes a fixed number of residues from the sequence edges.

trim_quality

Removes, using a sliding window, regions of low quality in the edges.

trim_blast_short

Removes oligonucleotides by using the blast-short algorithm.

convert_format

Converts between different supported sequence formats.

guess_seq_format

Guesses the format of a file, including Sanger and Illumina fastq formats.

cat_seqs

Concatenates one or more input sequence files, possibly in different formats, into one output.

seq_head

Outputs only the first sequences of the given input.

sample_seqs

Outputs a random sampling of the input sequences.

change_case

Modifies the letter case of the sequences. The case can be converted to lower or upper case or swapped.

pair_matcher

Filters out orphaned read pairs.

interleave_pairs

Interleaves two ordered paired read files.

deinterleave_pairs

Splits an ordered file of paired reads into two files, one for each end.

calculate_stats

Generates basic statistics for the given sequence files.

count_seqs

Counts the number of sequences and the total sequence length for the given files.

orientate_transcripts

Reverse complements transcripts according to polyA, ORF or blast hits.

All seq crumbs try to share a consistent interface. By default most Seq Crumbs read from standard input and write to standard output, allowing them to to be easily combined using Unix pipes. Alternatively, several input sequence files can be provided as a list of arguments. Output can also be directed to specific files with the -o parameter (or --outfile).

seq_crumbs supports compressed gzip, BGZF and bzip2 files. When used as input it autodetects the compressed files. It can also generate compressed outputs.

The sequence formats accepted by seq_crumbs are those supported by Biopython's SeqIO module. As output only Sanger and Illumina fastq and fasta files are supported.

seq_crumbs can take advantage of multiprocessor computers by splitting the computational load into several processes.

The filtering seq crumbs can be made aware of paired reads and can filter both reads of pairs at once.

seq_crumbs depends on Python 2.7. Biopython is a recommended dependency. The installation manual is located in the doc/install.rst document.

seq_crumbs relies heavily on Biopython and without this free software project it wouldn't be able to provide some of its functionality.

Biopieces is a project with a scope similar to seq_crumbs. Biopieces is a great software project for working with different kinds of biological data using Unix pipes. seq_crumbs tries to be more limited in its scope, limiting itself only to sequence files and thus providing a somewhat simpler interface.

Another software package very similar in approach to seq_crumbs is the nice fastx collection.

Other related software: PRINSEQ, ea-utils, Pyrocleaner, Sequence Cleaner, lucy, NGS QC Toolkit, scythe, sickle, cutadapt, trimomatic and FastQC.

Seq Crumbs is free software. Licensed mainly under the General Public License (GPL). For more details on the licensing take a look at the LICENSE.txt file.