Analysis template v4-3

Introduction and initialization

This is an R Markdown file containing code to parse the results of a dada2 analysis into phyloseq for further analysis. It is separated into chunks that may be run independently by pressing the play button. You will need 3 files in the same location in order to run this pipeline successfully:

• A sequence table called ‘seqtab_nochim.rds’ (automatically generated by the dada2 pipeline)
• A taxonomy table called ‘taxa.rds’ (automatically generated by the dada2 pipeline)
• A table describing the sample properties called ‘descriptors.txt’ (Has to be provided)
• Optional (some plots require this): A phylogenic tree file called ‘phylotree.rds’ (can be calculated if it is not provided, takes a long time)
• Optional: Instead of ‘taxa.rds’, a custom taxonomy file may be provided (tab-delimited text)

Recommended use: Set the individual chunks until you are content with the ouput, then knit the whole document into a PDF/html, so you have a full record of a successful run.

Optional custom Taxonomy file

A custom taxonomy file may be provided instead of using the taxonomy output from dada2. This may be used to supply taxonomy derived e.g. from BLAST searches of the ASVs. Custom taxonomy files must be tab-delimited text with as many rows as the original, colum headers (for all columns except for the first column). For example:

Kingdom Phylum Class Order Family Genus Species
ESV1 Kingdomx Phylumx Classx Orderx Familyx Genusx Speciesx
ESV2 Kingdomy Phylumy Classy Ordery Familyy Genusy Speciesy
ESV3 Kingdomz Phylumz Classz Orderz Familyz Genusz Speciesz
… ESVn Kingdomy Phylumy Classy Ordery Familyy Genusy Speciesy

Friendly warning: Parsing the results of a BLAST search into this format may require some effort.

Descriptor table

‘descriptors.txt’ should be a tab-delimited .txt table describing the samples. It must have the same length and order as the samples in seqtab_nochim.rds. To check the order and length of samples in seqtab_nochim.rds and generate a template to fill out, you may run the chunk below with “optional_sample_check” set to “TRUE”.

Any number of descriptors is possible. The sample names may be retained as one descriptor, but this is not necessary, as they will be added during parsing. For example, if there are 4 samples (order: s1, s2, s3, s4), the txt file could look as follows:

Subject Species Time
Kar1 A.thaliana 24hpi
Kar1 A.thaliana 72hpi
Mec2 S.tuberosum 24hpi
Mec3 S.tuberosum 24hpi

Finally, the file should end with an empty line, since it may throw an error otherwise. However, this is usually not a serious problem.

If you choose to use the blank file, you MUST retain the original order of the samples!

Setup

This chunk also loads required packages and defines the location of the input files. It requires the correct path as input, and allows setting the pruning of control samples and choosing generation of a phylogenetic tree. Beware: The generation of a phylogenetic tree may take several days for >1000 sequences, it is therefore recommended to only use this feature for the final analysis or small sample sets. This scricpt assumes the packages Biostrings, dada2, DECIPHER, ggplot2, ggsci, phangorn, phyloseq and stringr to be installed.

# CHANGE ME to the directory that contains 'seqtab_nochim.rds'
path = "OITS1-bigtest/"

# CHANGE ME to TRUE to list all samples and generate an empty metadata file 
optional_sample_check = TRUE

# CHANGE ME to TRUE to update cuphyr
update_cuphyr = TRUE

# Initiate by loading packages and setting knit options
################# NO CHANGES NECESSARY BELOW #################
knitr::opts_chunk$set(echo = TRUE)
knitr::opts_chunk$set(root.dir = paste0(path))
knitr::opts_chunk$set(message = FALSE)
knitr::opts_chunk$set(warning = FALSE)

if (update_cuphyr) {
  devtools::install_github("simeross/cuphyr")
}

# Sequence and microbiome specific libraries
library(dada2)
library(Biostrings)
library(DECIPHER)
library(cuphyr)
# The export of phyloseq objects to a BIOM format and the generation of fancier 
# ordination plots require the phyloseq-extended package. The first command 
# installs the package that is currently on the dev brach of the author's 
# repository, the second command sources some extra functions, including the 
# better ordination plot implementation.
remotes::install_github("mahendra-mariadassou/phyloseq-extended", ref = "dev")
source("https://raw.githubusercontent.com/mahendra-mariadassou/phyloseq-extended/master/load-extra-functions.R" )

library(phyloseq)
library(SIAMCAT)

# Phylogeny libraries
library(phangorn)
library(ape)

# Plotting and figure export
library(gridExtra)
library(viridis)
library(ggpubr)

# Tidyverse
library(tidyverse)
library(stringr)


# Checks whether output path exists and creates it if not. Throws warning if 
# directory exists.
outp <- paste0(path,"/analysis_output")
dir.create(file.path(outp))

if (optional_sample_check) {
  seqtabcheck <- readRDS(paste0(path,"/seqtab_nochim.rds")) 
  samps <- rownames(seqtabcheck)
  lensamps <- length(samps)
  blankcol <- vector(mode = "character", length = lensamps)
  blanktable <- data.frame(SampleIDs = samps, ExampleProperty1 = blankcol, 
                           ExampleProperty2 = blankcol, 
                           ExampleProperty3 = blankcol)
  write.table(blanktable, file = paste0(path, "/descriptors_blank.txt"), 
              sep = "\t", row.names = F)
  cat("'seqtab_nochim.rds' contains samples in the following order:\n", 
      samps, "\nThe number of samples in the file is:", lensamps, sep = "\n")
  rm(optional_sample_check, seqtabcheck, samps, 
     lensamps, blankcol, blanktable, update_cuphyr)
  }else{rm(optional_sample_check, update_cuphyr)}

## 'seqtab_nochim.rds' contains samples in the following order:
## 
## 01A-OITS1
## 01B-OITS1
## 02A-OITS1
## 02B-OITS1
## 03A-OITS1
## 03B-OITS1
## 04A-OITS1
## 04B-OITS1
## 05A-OITS1
## 05B-OITS1
## 06A-OITS1
## 06B-OITS1
## 07A-OITS1
## 07B-OITS1
## 08A-OITS1
## 08B-OITS1
## 09B-OITS1
## 10A-OITS1
## 10B-OITS1
## 11A-OITS1
## 11B-OITS1
## 12A-OITS1
## 12B-OITS1
## 13A-OITS1
## 13B-OITS1
## 14A-OITS1
## 14B-OITS1
## 15A-OITS1
## 15B-OITS1
## 16A-OITS1
## 16B-OITS1
## 17A-OITS1
## 17B-OITS1
## 18A-OITS1
## 18B-OITS1
## 19A-OITS1
## 19B-OITS1
## 20A-OITS1
## 20B-OITS1
## OITS1-H2O2
## OITS1-H2O4
## OITS1-Pos1
## OITS1-Pos2
## OITS1-Pos3
## OITS1-Pos4
## OITS1-Pos5
## 21A-OITS1
## 21B-OITS1
## 22A-OITS1
## 22B-OITS1
## 24A-OITS1
## 24B-OITS1
## 25A-OITS1
## 25B-OITS1
## 26A-OITS1
## 26B-OITS1
## 27A-OITS1
## 27B-OITS1
## 28A-OITS1
## 28B-OITS1
## 29B-OITS1
## 30A-OITS1
## 30B-OITS1
## 31A-OITS1
## 31B-OITS1
## 33A-OITS1
## 33B-OITS1
## 34A-OITS1
## 34B-OITS1
## 35A-OITS1
## 35B-OITS1
## 36A-OITS1
## 36B-OITS1
## 37A-OITS1
## 37B-OITS1
## 38A-OITS1
## 38B-OITS1
## 39A-OITS1
## 39B-OITS1
## 40A-OITS1
## 40B-OITS1
## 41A-OITS1
## 41B-OITS1
## 42A-OITS1
## 42B-OITS1
## 43A-OITS1
## 43B-OITS1
## 44A-OITS1
## 44B-OITS1
## 45A-OITS1
## 45B-OITS1
## 46A-OITS1
## 46B-OITS1
## 47A-OITS1
## 47B-OITS1
## 48A-OITS1
## 48B-OITS1
## 49A-OITS1
## 49B-OITS1
## 50A-OITS1
## 50B-OITS1
## 53A-OITS1
## 53B-OITS1
## 54-1A-OITS1
## 54-1B-OITS1
## 54-2A-OITS1
## 54-2B-OITS1
## 54-4A-OITS1
## 54-4B-OITS1
## 55A-OITS1
## 55B-OITS1
## 56-1A-OITS1
## 56-1B-OITS1
## 56-2A-OITS1
## 56-2B-OITS1
## 57A-OITS1
## 57B-OITS1
## 58A-OITS1
## 58B-OITS1
## 59A-OITS1
## 59B-OITS1
## 60A-OITS1
## 60B-OITS1
## 62A-OITS1
## 62B-OITS1
## 64A-OITS1
## 64B-OITS1
## 65A-OITS1
## 65B-OITS1
## 67A-OITS1
## 67B-OITS1
## 68A-OITS1
## 68B-OITS1
## 69A-OITS1
## 69B-OITS1
## H2O-OITS1
## Positiv1-OITS1
## Positiv2-OITS1
## Positiv3-OITS1
## Positiv4-OITS1
## Positiv5-OITS1
## 
## The number of samples in the file is:
## 141

Parameters

This chunk allows the adjustment of several parameters, such as setting the pruning of control samples based on keywords, requiring that a phylogenetic tree be provided or generated, defining a minimum ASV count and providing an alternative taxonomy.

# Dedicated environment containing all global analysis settings for better
# overview and collected export of settings
parameters <- new.env()

# CHANGE ME to 'TRUE' to remove control samples from the analysis or 'FALSE' to
# analyse all samples.
parameters$prune_controls = "TRUE"
# CHANGE ME to a list of unique identifiers that only occur in the names of
# samples you do NOT want to analyse. Common examples are provided.
parameters$controls = c("Pos", "H2O", "Neg", "Kontr", "Contr", "POSK")

# CHANGE ME to 'TRUE' to remove certain taxonomic groups from the analysis by
# name. This is useful to exclude non-target organisms or noise from organelles
# such as Chloroplasts and Mitochondria. It is recommended to first look at all
# data before using this setting.
parameters$prune_noise_taxgroups = "FALSE"
# CHANGE ME to define the taxonomic groups to be removed as noise.
parameters$noise_taxgroups = c("Chloroplast", "Mitochondria")

# CHANGE ME to a number of ASV counts [~reads] that analyzed samples should
# minimally have. Samples with lower ASV counts than 'minread' will be pruned.
# Set to 0 to not prune any samples.
parameters$minASVcount = 3000

# CHANGE ME to 'TRUE', if you want to provide a custom taxonomy table instead of
# using the default dada2 output ('taxa.rds').
parameters$customTax = "TRUE"
# CHANGE ME to the location of the custom taxonomy file. This only matters if
# parameters$customTax='TRUE', otherwise it will be ignored.
parameters$taxfile = "OITS1-bigtest/custom_BLAST_tophits_taxonomy.txt"

# CHANGE ME to 'TRUE' to generate a phylogenetic tree. This process takes a long
# time depending on the number of sequences (up to days for thousands).  If a
# tree is provided as 'phylotree.rds' in 'path', then it will be used regardless
# of the value of 'parameters$maketree'
parameters$maketree = "FALSE"

# CHANGE ME to 'TRUE' to root the used phylogenetic tree (if one exists) on the
# leaf with the longest branch (outgroup). This makes analyses that rely on the
# phylogenetic tree reproducible instead of picking a random leaf as root when
# calculating UNIFRAC distances. Implementation based on
# http://john-quensen.com/r/unifrac-and-tree-roots/ and answers in
# https://github.com/joey711/phyloseq/issues/597
parameters$roottree = "TRUE"

## CHANGE ME to 'TRUE' to export all generated phyloseq objects as .biom objects
parameters$biom_export = "FALSE"

Parsing input data

This chunk loads the input data into a usable format.This chunk does not require any user inputs. If no phylogenetic tree with the name ‘phylotree.rds’ was provided and ‘parameters$maketree=“TRUE”’, it will be calculated here. The phylogenetic tree is necessary for certain plots that incorporate ‘true’ taxonomic relationships beyond the annotations, such as PCoA.

############### NO NEED FOR CHANGES BELOW ############### Make dedicated environments to
############### contain temporary values and manage other objects
tmp <- new.env()
plots <- new.env()
set <- new.env()

# Read in variables
tmp$seqtabp <- readRDS(paste0(path, "/seqtab_nochim.rds"))
if (parameters$customTax == "TRUE") {
    tmp$taxap <- read.delim(parameters$taxfile, header = TRUE, sep = "\t")
    rownames(tmp$taxap) <- colnames(tmp$seqtabp)
    tmp$taxap <- as.matrix(tmp$taxap)
} else {
    tmp$taxap <- readRDS(paste0(path, "/taxa.rds"))
}
tmp$samp_table <- read.delim(paste0(path, "/descriptors.txt"), header = TRUE, sep = "\t")
tmp$samp_list <- rownames(tmp$seqtabp)

# Check if descriptors has the same samples as seqtabp
if (length(tmp$samp_table[, 1]) != length(tmp$samp_list)) {
    stop("There are ", length(tmp$samp_table[, 1]), " samples in 'descriptors.txt', but ", 
        length(tmp$samp_list), " samples in 'seqtab_nochim.rds'. Please make sure that the correct samples 
    are contained in descriptors.txt.
       
    You may use 'optional_sample_check <- TRUE' in the first chunk to generate an 
    empty template for 'descriptors.txt'")
} else if (!identical(tmp$samp_table[, 1], tmp$samp_list)) {
    warning("Warning: The samples in 'descriptors.txt' do not have the same names 
          or order as the samples in 'seqtab_nochim.rds'. This may be fine if 
          abbreviated names were used or the sample names are not contained in 
          the first column of 'descriptors.txt'. Double-checking never hurts!")
}


# generate phylogenetic tree of ASVs only if there is no file called
# 'phylotree.rds' in the working directory and 'parameters$maketree' is 'TRUE'
if (!file.exists(paste0(path, "/phylotree.rds"))) {
    if (parameters$maketree == "TRUE") {
        tmp$ASVs <- getSequences(tmp$seqtabp)
        names(tmp$ASVs) <- tmp$ASVs
        tmp$ASV_align <- AlignSeqs(DNAStringSet(tmp$ASVs), anchor = NA)
        tmp$ASV_phang <- phyDat(as(tmp$ASV_align, "matrix"), type = "DNA")
        tmp$dm <- dist.ml(tmp$ASV_phang)
        tmp$treeNJ <- NJ(tmp$dm)
        tmp$fit <- pml(tmp$treeNJ, data = tmp$ASV_phang)
        tmp$fitGTR <- update(tmp$fit, k = 4, inv = 0.2)
        tmp$fitGTR <- optim.pml(tmp$fitGTR, model = "GTR", optInv = TRUE, optGamma = TRUE, 
            rearrangement = "stochastic", control = pml.control(trace = 0))
        saveRDS(tmp$fitGTR, file = paste0(path, "/phylotree.rds"))
    }
}

## parse into phyloseq object
row.names(tmp$samp_table) <- tmp$samp_list
if (file.exists(paste0(path, "/phylotree.rds"))) {
    tmp$treep <- readRDS(paste0(path, "/phylotree.rds"))
    p <- phyloseq(otu_table(tmp$seqtabp, taxa_are_rows = FALSE), sample_data(tmp$samp_table), 
        tax_table(tmp$taxap), phy_tree(tmp$treep$tree))
} else {
    p <- phyloseq(otu_table(tmp$seqtabp, taxa_are_rows = FALSE), sample_data(tmp$samp_table), 
        tax_table(tmp$taxap))
}

## Adding nucleotide info and giving sequences ASV## identifiers
tmp$ASV_sequences <- Biostrings::DNAStringSet(taxa_names(p))
taxa_names(p) <- paste0("ASV", seq(ntaxa(p)))
names(tmp$ASV_sequences) <- taxa_names(p)
p <- merge_phyloseq(p, tmp$ASV_sequences)

## optional pruning
if (parameters$prune_controls == "TRUE") {
    if (!is.null(parameters$controls)) {
        tmp$samp_clean <- tmp$samp_list[!tmp$samp_list %in% grep(paste0(parameters$controls, 
            collapse = "|"), tmp$samp_list, value = T)]
        tmp$contr_pruned <- setdiff(tmp$samp_list, tmp$samp_clean)
        ps <- prune_samples(tmp$samp_clean, p)
        # Physeq object for Just controls
        ps.contr <- prune_samples(tmp$contr_pruned, p)
        ps.contr <- prune_taxa(taxa_sums(ps.contr) > 0, ps.contr)
        ps.transcontr <- transform_sample_counts(ps.contr, function(ASV) ASV/sum(ASV))
        
        message(cat("\n", "Number of control samples that were pruned and will not be analysed:\n", 
            length(tmp$samp_list) - length(tmp$samp_clean), "\n", "The following controls were pruned:\n", 
            tmp$contr_pruned, "The controls are contained in a separate phyloseq object: ps.contr", 
            "\n", sep = "\n"))
    } else {
        warning(cat("\n\nparameters$prune_controls is TRUE but 'parameters$controls' is empty. 
    No samples were pruned.\n\n"))
    }
} else {
    ps <- p
}

## 
## 
## Number of control samples that were pruned and will not be analysed:
## 
## 13
## 
## 
## The following controls were pruned:
## 
## OITS1-H2O2
## OITS1-H2O4
## OITS1-Pos1
## OITS1-Pos2
## OITS1-Pos3
## OITS1-Pos4
## OITS1-Pos5
## H2O-OITS1
## Positiv1-OITS1
## Positiv2-OITS1
## Positiv3-OITS1
## Positiv4-OITS1
## Positiv5-OITS1
## The controls are contained in a separate phyloseq object: ps.contr

# Prune ASVs defined as noise
if (parameters$prune_noise_taxgroups == "TRUE") {
    tmp$ps_taxlvls <- colnames(tax_table(ps))
    tmp$noise_ASVs <- character(0)
    for (lvl in tmp$ps_taxlvls) {
        tmp$noise_ASVs <- c(tmp$noise_ASVs, cuphyr::list_subset_ASVs(physeq = ps, 
            subv = parameters$noise_taxgroups, taxlvlsub = lvl))
    }
    tmp$noise_ASVs <- unique(tmp$noise_ASVs)
    tmp$no_noise_ASVs <- colnames(otu_table(ps))
    tmp$no_noise_ASVs <- setdiff(tmp$no_noise_ASVs, tmp$noise_ASVs)
    if (length(tmp$noise_ASVs) > 0) {
        ps <- prune_taxa(tmp$no_noise_ASVs, ps)
        tmp$no_noise_ps <- ps
        cat(length(tmp$noise_ASVs), "ASVs were pruned because they belonged to the following 
        taxonomic groups:\n")
        cat(parameters$noise_taxgroups, "\n", sep = "\n")
    } else {
        cat("No ASVs were recognized as belonging to the following taxonomic groups 
        defined as noise:\n")
        cat(parameters$noise_taxgroups, "\n", sep = "\n")
    }
}

# Prune samples with fewer than reads than minASVcount
if (parameters$minASVcount > 0) {
    tmp$samp_pruned <- names(which(sample_sums(ps) < parameters$minASVcount))
    ps <- prune_samples(sample_sums(ps) >= parameters$minASVcount, ps)
    if (length(tmp$samp_pruned) > 0) {
        cat("The following samples were pruned because ASV counts were lower than", 
            parameters$minASVcount, ":\n")
        cat(tmp$samp_pruned, "\n", sep = "\n")
    }
}

## The following samples were pruned because ASV counts were lower than 3000 :
## 10A-OITS1
## 12A-OITS1
## 17A-OITS1
## 17B-OITS1
## 18B-OITS1
## 19B-OITS1

# Remove 0 count ASVs (e.g. control ASVs that remain) from the base object
ps <- prune_taxa(taxa_sums(ps) > 0, ps)

# Get a tbl of the base object for easier access in some phyloseq-independent
# analyses. Takes some seconds, potentially up to minutes.
ps_tbl <- as_tibble(psmelt(ps))

# Transformed per sample (per-sample relative abundance)
ps.trans <- transform_sample_counts(ps, function(ASV) ASV/sum(ASV))

if (parameters$roottree == "TRUE" && parameters$maketree == "TRUE") {
    phyloseq::phy_tree(ps) <- cuphyr::root_tree_in_outgroup(physeq = ps)
}

if (parameters$biom_export == "TRUE") {
    suppressWarnings(phyloseq.extended::write_phyloseq(p, biom_file = paste0(path, 
        "all_samples.biom"), biom_format = "standard"))
    suppressWarnings(phyloseq.extended::write_phyloseq(ps, biom_file = file.path(path, 
        "samples_without_controls.biom"), biom_format = "standard"))
    suppressWarnings(phyloseq.extended::write_phyloseq(ps.trans, biom_file = file.path(path, 
        "samples_without_controls_rel_abundance.biom"), biom_format = "standard"))
    suppressWarnings(phyloseq.extended::write_phyloseq(ps.contr, biom_file = file.path(path, 
        "just_controls.biom"), biom_format = "standard"))
}

ps

## phyloseq-class experiment-level object
## otu_table()   OTU Table:         [ 2685 taxa and 122 samples ]
## sample_data() Sample Data:       [ 122 samples by 5 sample variables ]
## tax_table()   Taxonomy Table:    [ 2685 taxa by 7 taxonomic ranks ]
## phy_tree()    Phylogenetic Tree: [ 2685 tips and 2683 internal nodes ]
## refseq()      DNAStringSet:      [ 2685 reference sequences ]

Output

The chunks below will produce various plots and other output. Each chunk is headed by a description of the output and may contain some parameters to adjust the output.

Plot looks

This chunk sets the background structure and color palette. Viridis was chosen because it is optimized for grey-scale printing and various types of color blindness and More info on the Viridis palette can be found on the Viridis info page. It also establishes save_plot as a shorter variant of ggsave with customized date-time structure to save plots with the same name mulitple times instead of overwriting them (overwriting can be triggered with overwrite=TRUE).

##### Optional settings (sensible defaults) #####
# Can be changed to adjust the output format for all plots. Default "pdf", 
# possible "eps"/"ps", "tex" (pictex), "jpeg", "tiff", "png", "bmp" and "svg"
parameters$output_format = "pdf"

# Can be changed to preferred ggplot2 theme. Recommended: "theme_bw()".
theme_set(theme_bw())

############### NO NEED FOR CHANGES BELOW ###############

my_scale_col <- scale_color_viridis(discrete = TRUE)
my_scale_fill <- scale_fill_viridis(discrete = TRUE)

# Custom, more narrow color ranges based on viridis
# Base order to have adjacent colors be distinct from each other
tmp$sort_colors <- c(rbind(c(1:5), c(6:10), c(11:15), c(16:20)))

# Customized vectors
tmp$n_col <- 20
tmp$viridis_greens <- viridis(tmp$n_col,  option = "D", begin = 0.85, 
                              end = 0.7)[tmp$sort_colors]
tmp$viridis_reds <- viridis(tmp$n_col,  option = "B", begin = 0.7, 
                            end = 0.5)[tmp$sort_colors]
tmp$viridis_blues <- viridis(tmp$n_col,  option = "D", begin = 0.2, 
                             end = 0.4)[tmp$sort_colors]
tmp$viridis_yellows <- viridis(tmp$n_col,  option = "D", begin = 1, 
                               end = 0.9)[tmp$sort_colors]
tmp$viridis_dark <- viridis(tmp$n_col,  option = "A", begin = 0, 
                            end = 0.1)[tmp$sort_colors]
tmp$viridis_light <- viridis(tmp$n_col,  option = "A", begin = 1, 
                             end = 0.9)[tmp$sort_colors]
# Collected list that is available in the global environment
sub_viridis <- list(tmp$viridis_greens, tmp$viridis_blues, tmp$viridis_yellows, 
                    tmp$viridis_light, tmp$viridis_reds, tmp$viridis_dark)
names(sub_viridis) <- c("greens", "blues", "yellows", "lights", "reds", "darks")

tmp$out <- paste0(".", parameters$output_format)

#################### Function ############################

# Generic save function for plots that checks whether file exists and if so, 
# creates a new one with d/m/y+time info to avoid overwriting. Overwriting can 
# be triggered with overwrite = TRUE. Width, height and resolution are taken 
# from parameters in the 'set' environment or set to 20x20 cm with 300dpi.
save_plot <- function(
  pl, filetype = ".pdf", plot_name = "my_plot", overwrite=FALSE){
  wp <- if (!is.null(set$wp)) set$wp else 20
  hp <- if (!is.null(set$hp)) set$hp else 20
  res <- if (!is.null(set$res)) set$res else 300
  name <- paste0("/", plot_name,filetype)
  if (file.exists(paste0(outp, name)) & !overwrite) {
  name <- paste0("/", plot_name, "_", 
                 format(Sys.time(), "%d-%m-%y_%H%M%S"),filetype)}
  ggsave(file.path(outp, name), pl, 
         width = wp, height = hp, unit = "cm", dpi = res)
}

################################################

Total ASV counts ranked

This chunk plots the absolute abundance of all samples (including controls) and all samples without controls and other trimmed samples.

# CHANGE ME to the sample group for color coding. Accepted values are the column 
# headers in the descriptor file.
set$color_by = "Run"

##### Optional settings (sensible defaults) #####

# Can be changed to change the width (in cm) of the saved plot.
set$wp = 17
# Can be changed to change the height (in cm) of the saved plot.
set$hp = 20  
# Can be changed to change the resolution (in dpi) of the saved plot.
set$res = 300

############### NO NEED FOR CHANGES BELOW ###############
# Rank samples
set$ranked <- cuphyr::make_ranked_sums(p, myset = tmp$subset_id)
set$ranked_ps <- cuphyr::make_ranked_sums(ps, myset = tmp$subset_id)
set$ymax <- max(set$ranked$Abundance)
set$ymax <- set$ymax + round(set$ymax/10)
set$xmax <- nrow(set$ranked) + 1
set$title2 <- "Samples (without controls)"

# Stabilize colors
set$color_vars <- set$ranked[,set$color_by]  %>% 
  unlist() %>% as.character() %>% unique()
set$color_vars <- sort(set$color_vars)
set$color_varsPalette <- viridis(length(set$color_vars))
names(set$color_varsPalette) <- set$color_vars
set$my_scale_fill <- scale_fill_manual(values = set$color_varsPalette)

# plot
plots$overview_all <- ggplot(data = set$ranked, aes(x = Rank, y = Abundance)) + 
  aes_string(fill = set$color_by) + 
  geom_col() + set$my_scale_fill + ggtitle("All samples") + ylim(0, set$ymax) + 
  xlim(0,set$xmax) + ylab("ASV counts ('reads')")

if (length(tmp$noise_ASVs) > 0) {
  set$ranked_nonoise <- cuphyr::make_ranked_sums(
    tmp$no_noise_ps, myset = tmp$subset_id)
  plots$overview_noise <- ggplot(
    data = set$ranked_nonoise, aes(x = Rank, y = Abundance)) + 
  aes_string(fill = set$color_by) + 
  geom_col() + set$my_scale_fill + 
    ggtitle("Samples (without controls), noise ASVs removed") + 
    ylim(0, set$ymax) + 
    xlim(0,set$xmax) + ylab("ASV counts ('reads')")
}

if (parameters$minASVcount > 0) {
plots$overview_all <- plots$overview_all + 
  geom_hline(yintercept = parameters$minASVcount, linetype = "dashed") + 
    ggtitle("All samples (ASV count cutoff indicated)")
set$title2 <- "Samples (without controls and low count samps)"
}

plots$overview_ps <- ggplot(data = set$ranked_ps, aes(x = Rank, y = Abundance)) + 
  aes_string(fill = set$color_by) + 
  geom_col() + set$my_scale_fill + ggtitle(set$title2) + ylim(0, set$ymax) + 
  xlim(0,set$xmax) + ylab("ASV counts ('reads')")
plots$combo_overview <- ggarrange(
  plots$overview_all, plots$overview_ps, nrow = 2, align = "v", 
  common.legend = TRUE, legend = "right")

if (parameters$minASVcount > 0) {
plots$combo_overview <- ggarrange(
  plots$overview_all, plots$overview_noise, plots$overview_ps,
  nrow = 3, align = "v", 
  common.legend = TRUE, legend = "right")
}

#Save plots
save_plot(plots$combo_overview, plot_name = "Overview_all_and_pruned", 
          filetype = tmp$out)

#Clean up plot parameters
rm(list = ls(set), envir = set)

#Print plots
plots$combo_overview

Controls

This chunk generates an overview over the controls (positive AND negative)

# CHANGE ME to the desired sample categories on the x-axis. In this case it 
# should be the Sample names.
set$x_axis_value = "SampleIDs"

# CHANGE ME to the taxonomic level for color coding. Use "OTU" for ASVs, 
# "Genus", "Species" or "OTU" recommended to compare pos. controls.
set$color_by_taxlvl = "Species"

# CHANGE ME to the taxonomic level for labeling the tree tips (if phylogenetic 
# tree is available). Use "OTU" for ASVs.
set$label_by_taxlvl = "OTU"

# CHANGE ME to a sample category to shape the tree tip labels by (if 
# phylogenetic tree is available).
set$label_shape_by = "Run"

##### Optional settings (sensible defaults) #####

# Can be changed to generate a tree for just the control sequences IF no 
# phylogenetic tree for all seuquences is provided. This may slow down this 
# chunk when running it for the first time
set$control_tree = TRUE

# Can be changed to change the width (in cm) of the saved plot.
set$wp = 17
# Can be changed to change the height (in cm) of the saved plot.
set$hp = 20  
# Can be changed to change the resolution (in dpi) of the saved plot.
set$res = 300

############### NO NEED FOR CHANGES BELOW ###############
if (set$control_tree & class(try(phy_tree(ps.transcontr), 
                                 silent = TRUE)) == "try-error") {
  # generate phylogenetic tree of ASVs only if there is no file called 
  # 'phylotree.rds' in the working directory and 'parameters$maketree' is "TRUE"
  if (!file.exists(paste0(path, "/controls_phylotree.rds"))) {
    set$ASVs <- phyloseq::refseq(ps.transcontr)
    set$ASV_align <- AlignSeqs(set$ASVs, anchor = NA)
    set$ASV_phang <- phyDat(as(set$ASV_align, "matrix"), type = "DNA")
    set$dm <- dist.ml(set$ASV_phang)
    set$treeNJ <- NJ(set$dm)
    set$fit <- pml(set$treeNJ, data = set$ASV_phang)
    set$fitGTR <- update(set$fit, k = 4, inv = 0.2)
    set$fitGTR <- optim.pml(set$fitGTR, model = "GTR", 
                            optInv = TRUE, optGamma = TRUE,
                            rearrangement = "stochastic", 
                            control = pml.control(trace = 0))
    saveRDS(set$fitGTR, file = paste0(path, "/controls_phylotree.rds"))}
  set$fitGTR <- readRDS(paste0(path, "/controls_phylotree.rds"))
  phyloseq::phy_tree(ps.transcontr) <- set$fitGTR$tree
}

plots$topnpplot <- plot_bar(ps.contr, x = set$x_axis_value, 
                            fill = set$color_by_taxlvl) + my_scale_fill + 
  theme(axis.title.x = element_blank(), legend.position = "none", 
        legend.key.size = unit(3, "mm")) + 
  ylab("ASV counts") + guides(col = guide_legend(ncol = 3))

plots$topntplot <- plot_bar(ps.transcontr, x = set$x_axis_value, 
                            fill = set$color_by_taxlvl) + my_scale_fill + 
  theme(axis.title.x = element_blank(), legend.position = "none", 
        legend.key.size = unit(3, "mm")) + 
  ylab("Relative abundance") + guides(col = guide_legend(ncol = 3))

plots$combo_contr <- ggarrange(plots$topnpplot, plots$topntplot, ncol = 2, 
                               labels = c("A", "B"), align = "hv", 
                               common.legend = TRUE, legend = "right")

if (class(try(phy_tree(ps.transcontr), silent = TRUE)) != "try-error") {
plots$tre <- plot_tree(
          ps.transcontr, ladderize = "left", label.tips = set$label_by_taxlvl, 
          color = "abundance", text.size = 2.5, shape = set$label_shape_by) + 
          scale_color_viridis_c(aesthetics = c("color","fill")) + 
          theme(legend.position = "left", panel.border = element_blank())
plots$combo_contr <- ggarrange(plots$tre, ggarrange(plots$topnpplot, 
                                                    plots$topntplot, ncol = 2, 
                               labels = c("B", "C"), align = "hv", 
                               common.legend = TRUE, legend = "right"), 
                               nrow = 2, legend = "right", labels = c("A")) 
}

# save
save_plot(plots$combo_contr, plot_name = "Controls", filetype = tmp$out)

plots$combo_contr

Richness plot

This chunk plots the Alpha-Diversity according to the Shannon and Simpson indices. The chunk does not require any input, but it is possible to adjust the width, height and resolution of the PDF-output if necessary.

# CHANGE ME to the desired sample categories on the x-axis. Accepted values are
# the column headers in the descriptor file.
set$x_axis_value = "Bait"
# CHANGE ME to the sample group for color coding. Accepted values are the column
# headers in the descriptor file.
set$color_by = "Soil"

##### Optional settings (sensible defaults) #####

# Can be changed to change the width (in cm) of the saved plot.
set$wp = 17
# Can be changed to change the height (in cm) of the saved plot.
set$hp = 12
# Can be changed to change the resolution (in dpi) of the saved plot.
set$res = 300

############### NO NEED FOR CHANGES BELOW ############### Plot all diversity measures
plots$richP <- try(plot_richness(ps, x = set$x_axis_value, color = set$color_by) + 
    my_scale_col, silent = TRUE)
# Just Shannon and Simpson
plots$richShSi <- plot_richness(ps, x = set$x_axis_value, measures = c("Shannon", 
    "Simpson"), color = set$color_by) + my_scale_col

# Save
if (!class(plots$richP) == "try-error") {
    save_plot(plots$richP, plot_name = "Alpha_diversity_all", filetype = tmp$out)
}
save_plot(plots$richShSi, plot_name = "Alpha_diversity_all_ShSi", filetype = tmp$out)

# Clean up plot parameters
rm(list = ls(set), envir = set)
# Print to standard out
if (!class(plots$richP) == "try-error") {
    plots$richP
}

plots$richShSi

Bray-Curtis NMDS plot

This chunk generates a non-metric multidimensional scaling (NMDS) plot of the Bray-Curtis dissimililarity, giving a two-dimensional measure of community diversity. This is done for the primary parameter and the taxonomic level separately. The chunk does not require any input, but it is possible to adjust the width, height and resolution of the PDF-output if necessary, as well as the max. number of taxa to be displayed at taxlvl. Friendly warning: This chunk may not perform for lower order taxlvl, such as ‘species’, if they are not sufficiently abundant in all samples

# CHANGE ME to the sample group for shape coding. Accepted values are the column
# headers in the descriptor file.
set$shape_by = "Bait"
# CHANGE ME to the sample group for color coding. Accepted values are the column
# headers in the descriptor file.
set$color_by = "Bait"
# CHANGE ME to the taxonomic level of interest (color coding). Accepted values
# are the column headers in your descriptor file.
set$taxlvl = "Genus"

##### Optional settings (sensible defaults) ##### Can be changed to change the number
##### of Top n taxa plotted at taxlvl in separate panels, a maximum of 9 is
##### recommended for good readability.
set$top_n = 9
# Can be changed to change the width (in cm) of the saved plot.
set$wp = 17
# Can be changed to change the height (in cm) of the saved plot.
set$hp = 12
# Can be changed to change the resolution (in dpi) of the saved plot.
set$res = 300

############### NO NEED FOR CHANGES BELOW ############### Transform data for Bray-Curtis
############### distance
tmp$ord_nmds <- ordinate(ps.trans, method = "NMDS", distance = "bray")

## Run 0 stress 0.2136107 
## Run 1 stress 0.2226072 
## Run 2 stress 0.2142082 
## Run 3 stress 0.2190356 
## Run 4 stress 0.2162249 
## Run 5 stress 0.217838 
## Run 6 stress 0.2163812 
## Run 7 stress 0.2158658 
## Run 8 stress 0.2230285 
## Run 9 stress 0.2186416 
## Run 10 stress 0.2122362 
## ... New best solution
## ... Procrustes: rmse 0.03680055  max resid 0.2173066 
## Run 11 stress 0.2154819 
## Run 12 stress 0.2186523 
## Run 13 stress 0.220061 
## Run 14 stress 0.2137288 
## Run 15 stress 0.2188156 
## Run 16 stress 0.2147923 
## Run 17 stress 0.2242811 
## Run 18 stress 0.2238814 
## Run 19 stress 0.2195426 
## Run 20 stress 0.2103208 
## ... New best solution
## ... Procrustes: rmse 0.0561615  max resid 0.3045269 
## *** No convergence -- monoMDS stopping criteria:
##     20: stress ratio > sratmax

tmp$ps.topn <- cuphyr::abundant_tax_physeq(physeq = ps.trans, lvl = set$taxlvl, top = set$top_n)
tmp$top_ord_nmds <- ordinate(tmp$ps.topn, method = "NMDS", distance = "bray")

## Run 0 stress 0.2148938 
## Run 1 stress 0.2238265 
## Run 2 stress 0.219069 
## Run 3 stress 0.2171295 
## Run 4 stress 0.2220894 
## Run 5 stress 0.2175849 
## Run 6 stress 0.2125965 
## ... New best solution
## ... Procrustes: rmse 0.04724952  max resid 0.3230443 
## Run 7 stress 0.2173571 
## Run 8 stress 0.2240511 
## Run 9 stress 0.2251322 
## Run 10 stress 0.216587 
## Run 11 stress 0.2152722 
## Run 12 stress 0.2134213 
## Run 13 stress 0.2177421 
## Run 14 stress 0.2174177 
## Run 15 stress 0.2146429 
## Run 16 stress 0.2148969 
## Run 17 stress 0.218171 
## Run 18 stress 0.2182099 
## Run 19 stress 0.2123141 
## ... New best solution
## ... Procrustes: rmse 0.03798303  max resid 0.3385933 
## Run 20 stress 0.2237142 
## *** No convergence -- monoMDS stopping criteria:
##     20: stress ratio > sratmax

# Plot
plots$nmds <- plot_samples(ps.trans, tmp$ord_nmds, color = set$color_by, shape = set$shape_by, 
    title = paste0("Bray NMDS")) + my_scale_col + guides(color = FALSE, shape = FALSE)

plots$nmds_tax <- plot_ordination(tmp$ps.topn, tmp$top_ord_nmds, type = "taxa", color = set$taxlvl, 
    title = paste0("Bray NMDS ", set$taxlvl)) + my_scale_col

plots$nmds_taxpanels <- plots$nmds_tax + facet_wrap(paste0("~", set$taxlvl), scales = "free_x") + 
    my_scale_col

# Save
save_plot(plots$nmds, plot_name = paste0("NMDS_", set$shape_by, "_", set$color_by), 
    filetype = tmp$out)
save_plot(plots$nmds_tax, plot_name = paste0("NMDS_", set$taxlvl), filetype = tmp$out)
save_plot(plots$nmds_taxpanels, plot_name = paste0("NMDS_top", set$top_n, "_", set$taxlvl), 
    filetype = tmp$out)

# Clean up plot parameters
rm(list = ls(set), envir = set)

# Print to standard out
plots$nmds

plots$nmds_tax

plots$nmds_taxpanels

PcoA (requires phylogenetic tree)

This chunk generates an alternative common ordination plot, called ‘PcoA’, based on the primary variable, giving a two-dimensional measure of community diversity by considering the phylogenetic tree. The chunk does not require any input, although it is possible to adjust the width, height and resolution of the PDF-output if necessary. If the provided tree is not rooted, Phyloseq will root it to a random ASV. Root the tree to a given ASV to get consistent plots here (implementation will follow, until then, see: this github issue.

# CHANGE ME to the sample group for shape coding. Accepted values are the column
# headers in the descriptor file.
set$shape_by = "Bait"
# CHANGE ME to the sample group for color coding. Accepted values are the column
# headers in the descriptor file.
set$color_by = "Soil"

##### Optional settings (sensible defaults) #####

# Can be changed to change the width (in cm) of the saved plot.
set$wp = 20
# Can be changed to change the height (in cm) of the saved plot.
set$hp = 20
# Can be changed to change the resolution (in dpi) of the saved plot.
set$res = 300

############### NO NEED FOR CHANGES BELOW ############### Stopping this code from running, if
############### there is no phylogenetic tree
if (class(try(phy_tree(ps), silent = TRUE)) == "try-error") {
    # Message could be more informative
    cat("This plot could not be generated because no Phylogenetic tree was 
      provided.\n\n")
} else {
    # Transform and ordinate
    tmp$ord_pcoa <- ordinate(ps.trans, "PCoA", "unifrac", weighted = TRUE)
    # Plot
    plots$pcoa <- plot_ordination(ps.trans, tmp$ord_pcoa, color = set$color_by, shape = set$shape_by) + 
        my_scale_col
    # Save
    save_plot(plots$pcoa, plot_name = paste0("PCOA_", set$color_by, "_", shape = set$shape_by), 
        filetype = tmp$out)
    
    # Clean up plot parameters
    rm(list = ls(set), envir = set)
    
    # Print to standard out
    plots$pcoa
}

Get a list of Top N taxa at a given level

This chunk lists the top n most abundant taxonomic terms at a given level. Change the function parameters to the desired values. For more info, check help page of the function with ?cuphyr::abundant_tax_physeq(). Change ‘ignore_na’ to include/exclude NA values at the given level.

#The character vector can later be accessed by calling 'tmp$tops'
tmp$tops <- cuphyr::abundant_tax_physeq(physeq = ps, 
                            lvl = "Genus",
                            top = 10,
                            output_format = "tops",
                            ignore_na = TRUE,
                            silent = FALSE)

## 
## The top 10 most abundant annotated groups at the taxonomic level 'Genus' are:
## Pythium
## Pedospumella
## Phytophthora
## Globisporangium
## Spumella
## Aphanomyces
## Pythiogeton
## Saprolegnia
## Hirpicium
## Prorocentrum

Top N ASVs/taxa Bar plot

This chunk plots abundance of the Top n ASVs or taxa at a given level as a bar plot, giving an insight into the presence of the n ASV and most common taxa for the primary and secondary parameters. The default for n is set at 20, a larger n may lead to delay/skipping of the plot in standard out, but it should be saved as a PDF regardless for ASVs. For taxa, a large n may lead to unreadable plots. The chunk does not require any input, but it is possible to adjust the default ‘n’, and to change width, height and resolution of the PDF-output if necessary.

# CHANGE ME to the sample category that will be shown in separate panels.
# Accepted values are the column headers in your descriptor file.
set$panel_by = "Bait"

# CHANGE ME to the desired sample categories on the x-axis.  Accepted values are
# the column headers in the descriptor file.
set$x_axis_value = "Soil"

# CHANGE ME to the count of top ASVs you want to plot (e.g. 'set$topASVs = 20'
# plots the 20 most abundant ASVs)
set$topASVs = 200

# CHANGE ME to the taxonomic level of interest (color coding). Accepted values
# are the column headers in your descriptor file.
set$taxlvl = "Genus"

# CHANGE ME to change the number of Top n taxa to be plotted at taxlvl.
set$top_n = 10

# CHANGE ME to an entry at the chosen taxonomic level you want to highlight.
# Comment out to not highlight anything.
set$highlight = "Pythium"

##### Optional settings (sensible defaults) #####

# Can be changed to turn unified coloring on or off (same taxonomy term = same
# color in both plots). Highlighting will unify colors even if unify_colors is
# FALSE.
set$unify_colors = TRUE

# Can be changed to include (FALSE) or exclude (TRUE) NA values in the barplot
set$ignore_na = FALSE

# Can be changed to remove ASV segmentation in the top n taxlvl plot. This
# improves visual clarity when a bar segment appears black due to the border of
# many small ASVs overlapping.
set$fuse_ASVs = FALSE

# Can be changed to change the width (in cm) of the saved plot.
set$wp = 20
# Can be changed to change the height (in cm) of the saved plot.
set$hp = 20
# Can be changed to change the resolution (in dpi) of the saved plot.
set$res = 300

# Can be changed to change the y-axis label
set$y_axis_label = "Relative abundance"

############### NO NEED FOR CHANGES BELOW ###############

# Make physeq objects of top n taxa and top n ASVs
set$ps.topnTax <- cuphyr::abundant_tax_physeq(ps.trans, lvl = set$taxlvl, top = set$top_n, 
    ignore_na = set$ignore_na)
set$topnASVs <- names(sort(taxa_sums(ps), decreasing = TRUE))[1:set$topASVs]
set$ps.topnASVs <- prune_taxa(set$topnASVs, ps.trans)

if (set$unify_colors | exists("highlight", envir = set) | set$fuse_ASVs) {
    set$toptax <- union(phyloseq::tax_table(set$ps.topnTax)[, set$taxlvl], phyloseq::tax_table(set$ps.topnASVs)[, 
        set$taxlvl])
    set$toptax <- sort(set$toptax)
    set$taxlvlPalette <- viridis(length(set$toptax))
    names(set$taxlvlPalette) <- set$toptax
    if (exists("highlight", envir = set)) {
        # It is possible to change the highlight color here by substituting
        # 'sub_viridis$reds[4]' with a hexcode-string, e.g. '#ff7f7f''
        set$taxlvlPalette[set$highlight] <- sub_viridis$reds[4]
    }
    set$my_scale_fill <- scale_fill_manual(values = set$taxlvlPalette, na.value = "grey")
} else {
    set$my_scale_fill <- my_scale_fill
}

# Plot
plots$topn_tax <- plot_bar(set$ps.topnTax, x = set$x_axis_value, fill = set$taxlvl, 
    title = paste0("Top", set$top_n, "_", set$taxlvl)) + facet_wrap(paste0("~", set$panel_by), 
    scales = "free_x") + set$my_scale_fill + ylab(set$y_axis_label)

if (set$fuse_ASVs) {
    plots$topn_tax <- plots$topn_tax + geom_bar(aes_string(color = set$taxlvl, fill = set$taxlvl), 
        stat = "identity", position = "stack") + scale_color_manual(values = set$taxlvlPalette, 
        na.value = NA)
}

plots$topn_ASVs <- plot_bar(set$ps.topnASVs, x = set$x_axis_value, fill = set$taxlvl, 
    title = paste0("Top", set$topASVs, "_ASVs")) + facet_wrap(paste0("~", set$panel_by), 
    scales = "free_x") + set$my_scale_fill + ylab(set$y_axis_label)

# save
save_plot(plots$topn_tax, plot_name = paste0("Top", set$top_n, "_", set$taxlvl), 
    filetype = tmp$out)
save_plot(plots$topn_ASVs, plot_name = paste0("Top", set$topASVs, "_ASVs"), filetype = tmp$out)

# Clean up plot parameters
rm(list = ls(set), envir = set)

# Print to standard out
plots$topn_tax

plots$topn_ASVs

Siamcat

This chunk implements statistical testing of ASVs that are differentially abundant for a given biological train (column in descriptors.txt). It can also test whether grouping variables other than the tested one is associated with the abundance data in a similar or different way than the chosen train (confounders). The chunk is largely based on the SIAMCAT “Get started” vignette. There are several options that can be chosen.

# CHANGE ME to the sample category that will be used as the test group. ASVs 
# that are differentially abundant according to this grouping will be detected. 
# Accepted values are the column headers in your descriptor file.
set$test_label = "Bait"
# CHANGE ME to the desired sample categories on the x-axis. Accepted values are 
# the column headers in the descriptor file.
set$case_value = "Baited"

# CHANGE ME to the cutoff p-value for selecting significant ASVs (FDR-adjusted 
# p-value)
set$p_val_cutoff = 0.05

# CHANGE ME to the taxonomic level of interest for more informative ASV 
# annotation (format: taxlv-ASV)
set$taxlvl = "Genus"

##### Optional settings (sensible defaults) #####

# Can be changed to filter low-abundance ASVs. Sequence variants with lower 
# abundance will not be analysed to reduce artifacts
set$filter_abundance = 0.001

# Can be changed to include (TRUE) or exclude (FALSE) an output file where all 
# possible confounders are checked. This will analyse the confounding effect of 
# other factors in 'descriptors' over the chosen test group and produce a pdf 
# file containing several plots.
set$check_confounders = TRUE

############### NO NEED FOR CHANGES BELOW ###############
# Make a copy of the transformed physeq object and parse taxonomic information 
# for the chosen taxlvl into ASV names to give more informative plots.
ps.siam <- ps.trans
taxa_names(ps.siam) <- tax_table(ps.trans) %>% 
  as.data.frame() %>% 
  rownames_to_column(var = "OTU") %>% 
  unite(col = OTU, set$taxlvl, OTU) %>% 
  select(OTU) %>% 
  unlist() %>% 
  unname()

# Read in transformed physeq object as SIAMCAT object and choose trait
sc.trans <- siamcat(phyloseq = ps.siam, 
                    label = set$test_label, 
                    case = set$case_value)

# print the generated Siamcat object to check for valid parsing
show(sc.trans)

## siamcat-class object
## label()                Label object:         60 Untreated and 62 Baited samples
## 
## contains phyloseq-class experiment-level object @phyloseq:
## phyloseq@otu_table()   OTU Table:            [ 2685 taxa and 122 samples ]
## phyloseq@sam_data()    Sample Data:          [ 122 samples by 4 sample variables ]
## phyloseq@tax_table()   Taxonomy Table:       [ 2685 taxa by 7 taxonomic ranks ]
## phyloseq@phy_tree()    Phylogenetic Tree:    [ 2685 tips and 2683 internal nodes ]

# Filter ASVs with less than set$filter_abundace
sc.filt <- filter.features(sc.trans,
    filter.method = 'abundance',
    cutoff = set$filter_abundance)

# check confounders if the option is TRUE
if (set$check_confounders) {  
  sc.conf <- check.confounders(
      sc.filt,
      fn.plot = file.path(outp, 'confounder_plots.pdf'),
      meta.in = NULL,
      feature.type = 'filtered',
      verbose = 1)
  cat("Confounders checked, results stored in", 
      file.path(outp, 'confounder_plots.pdf'))
}

## Confounders checked, results stored in OITS1-bigtest//analysis_output/confounder_plots.pdf

# Plot asscoiations and save the analysis to the siamcat object
sc.filt <- check.associations(
    sc.filt,
    sort.by = 'fc',
    alpha = set$p_val_cutoff,
    mult.corr = "fdr",
    detect.lim = 10^-6,
    plot.type = "quantile.box",
    panels = c("fc", "prevalence", "auroc"),
    prompt = FALSE,
    fn.plot = file.path(outp, 
                  paste0("Differential_abundance_", set$test_label,
                         "_", format(Sys.time(), "%d-%m-%y_%H%M%S"), ".pdf")))

# Plot asscoiations again to standard out
sc.filt <- check.associations(
    sc.filt,
    sort.by = 'fc',
    alpha = set$p_val_cutoff,
    mult.corr = "fdr",
    detect.lim = 10^-6,
    plot.type = "quantile.box",
    panels = c("fc", "prevalence", "auroc"),
    prompt = FALSE,
    verbose = 0)

# record plot from standard out
plots$siam_assoc <- recordPlot()

# Turn significant hits into tbl, if there are any, generate a vector containing 
# significant tax groups at taxlvl and a vector containing significant ASVs
tbl_me_this <- associations(sc.filt) %>%
  filter(p.adj < set$p_val_cutoff) %>%
  rownames_to_column("tax_ASV") %>%
  separate(col = "tax_ASV", into = c("tax", "ASV"), sep = "_") %>%
  select(tax, ASV, p.adj)

if (nrow(tbl_me_this) > 0) {
 significant_tax_groups <- select(tbl_me_this, tax) %>%
   unique() %>% unlist() %>% unname()
 significant_ASVs <- select(tbl_me_this, ASV) %>%
   unique() %>% unlist() %>% unname()
 
 cat(sep = "\n", "The following taxonomic groups were found to be differentially 
     abundant and stored in 'significant_tax_groups':",
     significant_tax_groups, 
     "This object can be used to set a subgroup in the chunk below.")
}

Subset the Phyloseq object by taxonomic group(s)

This chunk gives the option to create a subset of the general Phyloseq object by providing a vector of search terms and a taxonomic level to search at. It requires one or more search terms, a taxonomic level to search at and a description of the subset. The description will only be used for the titles of plots generated from the subsets.

# Vector to subset on
set$subv = c("Pythium", "Phytophthora")
# CHANGE ME to the taxonomic level at which you want to search for matching
# entries
set$taxlvlsub = "Genus"
# CHANGE ME to a descriptor for the subset
tmp$subset_id = "Oomycetes of interest"
# CHANGE ME if you want to use the significant groups found by SIAMCAT. If TRUE,
# those groups will be used in addition to the groups specified in set$subv.
set$use_siamcat_results = TRUE

# CHANGE ME to the sample group for color coding in the summary plot.  Accepted
# values are the column headers in the descriptor file.
set$color_by = "Run"

##### Optional settings (sensible defaults) #####

# Can be changed to change the width (in cm) of the saved plot.
set$wp = 17
# Can be changed to change the height (in cm) of the saved plot.
set$hp = 12
# Can be changed to change the resolution (in dpi) of the saved plot.
set$res = 300

############### NO NEED FOR CHANGES BELOW ###############
set$subv <- if (set$use_siamcat_results && exists("significant_tax_groups")) {
    unique(c(set$subv, significant_tax_groups))
} else {
    set$subv
}
set$subASVs <- cuphyr::list_subset_ASVs(subv = set$subv, taxlvlsub = set$taxlvlsub)
ps.subs <- prune_taxa(set$subASVs, ps)
ps.subs.trans <- prune_taxa(set$subASVs, ps.trans)

# plot the reads in the subset
set$ranked <- cuphyr::make_ranked_sums(ps.subs, myset = tmp$subset_id)
set$avg <- mean(set$ranked$Abundance)
set$avg_round <- format(round(set$avg, 0), nsmall = 0)
plots$subset <- ggplot(data = set$ranked, aes(x = Rank, y = Abundance)) + aes_string(fill = set$color_by) + 
    geom_col() + my_scale_fill + geom_hline(yintercept = set$avg, linetype = "dashed") + 
    ylab("ASV counts ('reads')") + ggtitle(paste0("Subset: ", tmp$subset_id, " (average ASV count ", 
    set$avg_round, ")"))

# Save plot
save_plot(plots$subset, plot_name = "Subset_overview", filetype = tmp$out)
# Print plot
plots$subset

# print info on generated object
cuphyr::summarise_physeq(ps, ASV_sublist = set$subASVs, sublist_id = tmp$subset_id, 
    samp_names = FALSE)

## There are 2685 ASVs in the phyloseq object 'ps'.
## Of this, 686 belong to the provided subset (Oomycetes of interest), representing 62.18 percent of abundance per sample on average.

# Optional export as biom-file
if (parameters$biom_export == "TRUE") {
    tmp$subset_id <- tmp$subset_id %>% str_replace_all(" ", "_")
    suppressWarnings(phyloseq.extended::write_phyloseq(p, biom_file = file.path(path, 
        paste0("subset_", tmp$subset_id, ".biom")), biom_format = "standard"))
}

# Clean up plot parameters
rm(list = ls(set), envir = set)

Bar plots for subsets of taxonomic group(s)

The chunk is very similar to the vanilla bar plot chunk above but takes the subset data instead of the complete Phyloseq object. This chunk plots abundance of the Top n ASVs or taxa at a given level as a bar plot, giving an insight into the presence of the n ASV and most common taxa for the primary and secondary parameters. The default for n is set at 100 for subsets. The range of n is for subsets is larger, since the taxonomic variety was reduced by the subsetting already, meaning the Top 100 ASVs likely belong to few species. For taxa, a large n may lead to unreadable plots. The chunk does not require any input, but it is possible to adjust the default ‘n’, and to change width, height and resolution of the PDF-output if necessary.

# CHANGE ME to the sample category that will be shown in separate panels.
# Accepted values are the column headers in your descriptor file.
set$panel_by = "Bait"
# CHANGE ME to the desired sample categories on the x-axis. Accepted values are
# the column headers in the descriptor file.
set$x_axis_value = "Soil"

# CHANGE ME to the count of top ASVs you want to plot (e.g. 'set$topASVs = 20'
# plots the 20 most abundant ASVs)
set$topASVs = 200

# CHANGE ME to the taxonomic level of interest (color coding). Accepted values
# are the column headers in your descriptor file.
set$taxlvl = "Genus"
# CHANGE ME to change the number of Top n taxa to be plotted at taxlvl.
set$top_n = 10

# CHANGE ME to an entry at the chosen taxonomic level you want to highlight.
# Comment out to not highlight anything. set$highlight <- 'Pythium'

##### Optional settings (sensible defaults) #####

# Can be changed to turn unified coloring on or off (same taxonomy term = same
# color in both plots). Highlighting will unify colors even if unify_colors is
# FALSE.
set$unify_colors = TRUE

# Can be changed to include (FALSE) or exclude (TRUE) NA values in the barplot
set$ignore_na = FALSE

# Can be changed to remove ASV segmentation in the top n taxlvl plot. This
# improves visual clarity when a bar segment appears black due to the border of
# many small ASVs overlapping.
set$fuse_ASVs = FALSE

# Can be changed to change the width (in cm) of the saved plot.
set$wp = 20
# Can be changed to change the height (in cm) of the saved plot.
set$hp = 20
# Can be changed to change the resolution (in dpi) of the saved plot.
set$res = 300

# Can be changed to change the y-axis label
set$y_axis_label = "Relative abundance"

############### NO NEED FOR CHANGES BELOW ############### Make physeq objects of top n taxa and
############### top n ASVs
set$ps.topnTax <- cuphyr::abundant_tax_physeq(ps.subs.trans, lvl = set$taxlvl, top = set$top_n, 
    ignore_na = set$ignore_na)
set$topnASVs <- names(sort(taxa_sums(ps), decreasing = TRUE))[1:set$topASVs]
set$ps.topnASVs <- prune_taxa(set$topnASVs, ps.subs.trans)

if (set$unify_colors | exists("highlight", envir = set) | set$fuse_ASVs) {
    set$toptax <- union(phyloseq::tax_table(set$ps.topnTax)[, set$taxlvl], phyloseq::tax_table(set$ps.topnASVs)[, 
        set$taxlvl])
    set$toptax <- sort(set$toptax)
    set$taxlvlPalette <- viridis(length(set$toptax))
    names(set$taxlvlPalette) <- set$toptax
    if (exists("highlight", envir = set)) {
        # It is possible to change the highlight color here by substituting
        # 'sub_viridis$reds[4]' with a hexcode-string, e.g. '#ff7f7f''
        set$taxlvlPalette[set$highlight] <- sub_viridis$reds[4]
    }
    set$my_scale_fill <- scale_fill_manual(values = set$taxlvlPalette, na.value = "grey")
} else {
    set$my_scale_fill <- my_scale_fill
}

# Plot
plots$topn_tax_subset <- plot_bar(set$ps.topnTax, x = set$x_axis_value, fill = set$taxlvl, 
    title = paste0("Top", set$top_n, "_", set$taxlvl)) + facet_wrap(paste0("~", set$panel_by), 
    scales = "free_x") + set$my_scale_fill + ylab(set$y_axis_label)

if (set$fuse_ASVs) {
    plots$topn_tax_subset <- plots$topn_tax_subset + geom_bar(aes_string(color = set$taxlvl, 
        fill = set$taxlvl), stat = "identity", position = "stack") + scale_color_manual(values = set$taxlvlPalette, 
        na.value = NA)
}
plots$topn_ASVs_subset <- plot_bar(set$ps.topnASVs, x = set$x_axis_value, fill = set$taxlvl, 
    title = paste0("Top", set$topASVs, "_ASVs")) + facet_wrap(paste0("~", set$panel_by), 
    scales = "free_x") + set$my_scale_fill + ylab(set$y_axis_label)

# save
save_plot(plots$topn_tax_subset, plot_name = paste0("Top", set$top_n, "_", set$taxlvl, 
    "_subset"), filetype = tmp$out)
save_plot(plots$topn_ASVs_subset, plot_name = paste0("Top", set$topASVs, "_ASVs_subset"), 
    filetype = tmp$out)

# Clean up plot parameters
rm(list = ls(set), envir = set)

# Print to standard out
plots$topn_tax_subset

plots$topn_ASVs_subset

Other phylogenetic trees

For these chunks, the ggtree library is required. If you are not sure whether it is installed, run the following chunk.

if (!requireNamespace("BiocManager", quietly = TRUE)) {
  install.packages("BiocManager")}
if (!requireNamespace("ggtree", quietly = TRUE)) {
  BiocManager::install("ggtree")}

Generic phylogenetic (from a subset)

This chunk allows the generation of a generic phylogenetic tree for a given subset of the phyloseq object, even if none is provided for the whole set.

# Vector to subset on (the larger the subset, the longer the tree generation 
# will take!)
set$subv = c("Phytopythium", "Hirpicum")
# CHANGE ME to the taxonomic level at which you want to search for matching 
# entries
set$taxlvlsub = "Genus"

# CHANGE ME to the width of the tree shown (depends on its size, start large, 
# make smaller untill best fit is achieved)
set$tree_width = 11
# CHANGE ME to the position you want the tree's scale to be shown. Must be 
# smaller than set$tree_width, larger = more to the right.
set$tree_scale_pos = 10

##### Optional settings (sensible defaults) #####

# Can be changed to change the width (in cm) of the saved plot.
set$wp = 20
# Can be changed to change the height (in cm) of the saved plot.
set$hp = 20
# Can be changed to change the resolution (in dpi) of the saved plot.
set$res = 300

############### NO NEED FOR CHANGES BELOW ###############
set$subASVs <- cuphyr::list_subset_ASVs(subv = set$subv, 
                                        taxlvlsub = set$taxlvlsub)
set$ps.treesubs <- prune_taxa(set$subASVs, ps)
set$seqs <- phyloseq::refseq(set$ps.treesubs)
set$align <- AlignSeqs(DNAStringSet(set$seqs), anchor = NA)

## Determining distance matrix based on shared 8-mers:
## ================================================================================
## 
## Time difference of 0 secs
## 
## Clustering into groups by similarity:
## ================================================================================
## 
## Time difference of 0.01 secs
## 
## Aligning Sequences:
## ================================================================================
## 
## Time difference of 0.08 secs
## 
## Iteration 1 of 2:
## 
## Determining distance matrix based on alignment:
## ================================================================================
## 
## Time difference of 0 secs
## 
## Reclustering into groups by similarity:
## ================================================================================
## 
## Time difference of 0.01 secs
## 
## Realigning Sequences:
## ================================================================================
## 
## Time difference of 0.07 secs
## 
## Iteration 2 of 2:
## 
## Determining distance matrix based on alignment:
## ================================================================================
## 
## Time difference of 0 secs
## 
## Reclustering into groups by similarity:
## ================================================================================
## 
## Time difference of 0 secs
## 
## Realigning Sequences:
## ================================================================================
## 
## Time difference of 0.01 secs
## 
## Refining the alignment:
## ================================================================================
## 
## Time difference of 0.02 secs

set$seqs_phang <- phangorn::phyDat(as(set$align, "matrix"), type = "DNA")
set$seqs_dm <- phangorn::dist.ml(set$seqs_phang)
set$seqs_treeNJ <- NJ(set$seqs_dm)
set$seqs_fit = pml(set$seqs_treeNJ, data = set$seqs_phang)
set$fitGTRseqs <- update(set$seqs_fit, k = 4, inv = 0.2)
set$fitGTRseqs <- optim.pml(set$fitGTRseqs, model = "GTR", optInv = TRUE, 
                            optGamma = TRUE,
                            rearrangement = "stochastic", 
                            control = pml.control(trace = 0))

# Part of cuphyr::root_tree in outgroup, consider implementing a generic tree 
# version instead of just physeq.
root_generic_tree <- function(tree.unrooted){
      if (requireNamespace(c("ape", "data.table"), quietly = TRUE)) {
        phylo_tree <- tree.unrooted
        tips <- ape::Ntip(phylo_tree)
        tree_data <- base::cbind(data.table::data.table(phylo_tree$edge), 
            data.table::data.table(length = phylo_tree$edge.length))[1:tips,]
        tree_data <- base::cbind(
          tree_data, data.table::data.table(id = phylo_tree$tip.label))
        out_group <- dplyr::slice(tree_data, which.max(length)) %>% 
            select(id) %>% as.character()
        new_tree <- ape::root(phylo_tree, outgroup = out_group, 
            resolve.root = TRUE)
        message("Tree successfully rooted.")
    }else {
        stop("The function 'root_tree_in_outgroup' requires the packages 
             'ape' and 'data.table' to be installed. Please make sure those 
             packages can be loaded.") }
  return(new_tree)
}

set$fitGTRseqs$tree <- root_generic_tree(set$fitGTRseqs$tree)
plots$subset_ASV_tree <- ggtree::ggtree(set$fitGTRseqs$tree) + 
  ggtree::geom_tree() + 
  ggtree::geom_treescale(x = set$tree_scale_pos) + 
  ggtree::geom_tiplab() + 
  xlim(0,set$tree_width)

# save
save_plot(plots$subset_ASV_tree, 
          plot_name = "subset_phylogenetic_tree", 
          filetype = tmp$out)
# Clean up plot parameters
rm(list = ls(set), envir = set)
# plot
plots$subset_ASV_tree

Generic phylogenetic tree (from any FASTA)

This chunk allows the generation of a generic phylogenetic tree for any given fasta. This may be useful to compare the phylogeny of a given set of ASVs and some reference sequences.

# CHANGE ME to the path of the FASTA file you want to make a phylogenetic tree 
# for (the larger the fasta, the longer the tree generation will take!)
set$fasta = "OITS1-bigtest/for_phylogeny.fasta"

# CHANGE ME to the width of the tree shown (depends on its size, start large, 
# make smaller untill best fit is achieved)
set$tree_width = 11
# CHANGE ME to the position you want the tree's scale to be shown. Must be 
# smaller than set$tree_width, larger = more to the right.
set$tree_scale_pos = 10

##### Optional settings (sensible defaults) #####

# Can be changed to change the width (in cm) of the saved plot.
set$wp = 20
# Can be changed to change the height (in cm) of the saved plot.
set$hp = 20
# Can be changed to change the resolution (in dpi) of the saved plot.
set$res = 300

############### NO NEED FOR CHANGES BELOW ###############
set$seqs <- readDNAStringSet(set$fasta)
set$align <- AlignSeqs(DNAStringSet(set$seqs), anchor = NA)

## Determining distance matrix based on shared 8-mers:
## ================================================================================
## 
## Time difference of 0 secs
## 
## Clustering into groups by similarity:
## ================================================================================
## 
## Time difference of 0.01 secs
## 
## Aligning Sequences:
## ================================================================================
## 
## Time difference of 0.11 secs
## 
## Iteration 1 of 2:
## 
## Determining distance matrix based on alignment:
## ================================================================================
## 
## Time difference of 0 secs
## 
## Reclustering into groups by similarity:
## ================================================================================
## 
## Time difference of 0 secs
## 
## Realigning Sequences:
## ================================================================================
## 
## Time difference of 0.05 secs
## 
## Iteration 2 of 2:
## 
## Determining distance matrix based on alignment:
## ================================================================================
## 
## Time difference of 0 secs
## 
## Reclustering into groups by similarity:
## ================================================================================
## 
## Time difference of 0 secs
## 
## Realigning Sequences:
## ================================================================================
## 
## Time difference of 0.05 secs
## 
## Refining the alignment:
## ================================================================================
## 
## Time difference of 0.04 secs

set$seqs_phang <- phangorn::phyDat(as(set$align, "matrix"), type = "DNA")
set$seqs_dm <- phangorn::dist.ml(set$seqs_phang)
set$seqs_treeNJ <- NJ(set$seqs_dm)
set$seqs_fit = pml(set$seqs_treeNJ, data = set$seqs_phang)
set$fitGTRseqs <- update(set$seqs_fit, k = 4, inv = 0.2)
set$fitGTRseqs <- optim.pml(set$fitGTRseqs, model = "GTR", 
                            optInv = TRUE, optGamma = TRUE,
                            rearrangement = "stochastic", 
                            control = pml.control(trace = 0))

# Part of cuphyr::root_tree in outgroup, consider implementing a generic tree 
# version instead of just physeq.
root_generic_tree <- function(tree.unrooted){
      if (requireNamespace(c("ape", "data.table"), quietly = TRUE)) {
        phylo_tree <- tree.unrooted
        tips <- ape::Ntip(phylo_tree)
        tree_data <- base::cbind(data.table::data.table(phylo_tree$edge), 
            data.table::data.table(length = phylo_tree$edge.length))[1:tips,]
        tree_data <- base::cbind(
          tree_data, data.table::data.table(id = phylo_tree$tip.label))
        out_group <- dplyr::slice(tree_data, which.max(length)) %>% 
            select(id) %>% as.character()
        new_tree <- ape::root(phylo_tree, outgroup = out_group, 
            resolve.root = TRUE)
        message("Tree successfully rooted.")
    }else {
        stop("The function 'root_tree_in_outgroup' requires the packages 
             'ape' and 'data.table' to be installed. Please make sure those 
             packages can be loaded.") }
  return(new_tree)
}

set$fitGTRseqs$tree <- root_generic_tree(set$fitGTRseqs$tree)
plots$fasta_ASV_tree <- ggtree::ggtree(set$fitGTRseqs$tree) + 
  ggtree::geom_tree() + 
  ggtree::geom_treescale(x = set$tree_scale_pos) + 
  ggtree::geom_tiplab() + xlim(0,set$tree_width)

# save
save_plot(plots$fasta_ASV_tree, 
          plot_name = "phylogenetic_tree_from_fasta", 
          filetype = tmp$out)
# Clean up plot parameters
rm(list = ls(set), envir = set)
# plot
plots$fasta_ASV_tree

Machine learning with SIAMCAT

This is an experimental chunk implementing the machine learning functions of SIAMCAT following the tutorial steps and settings from the “Get started” vignette. There is no convenient way to change the settings yet, because the usefulness and different optimal ways to run these models needs to be tested further. The chunk can be run as is and will produce a result if the basic SIAMCAT chunk above was run. However, this should be handled skeptically and not given extraordinary weight, if the user is not confident that they understand the used method.

# Count normalization by log-transforming and adding pseudocounts
sc.norm <- normalize.features(
    sc.filt,
    norm.method = "log.unit",
    norm.param = list(
        log.n0 = 1e-06,
        n.p = 2,
        norm.margin = 1
    )
)
# splitting data into training and test sets to validate the model
sc.obj <-  create.data.split(
    sc.norm,
    num.folds = 5,
    num.resample = 2
)

# Train a model on the training set
sc.obj <- train.model(
     sc.obj,
     method = "lasso"
)

# Store model into separate object and check first entry
models <- models(sc.obj)
models[[1]]

## Model for learner.id=classif.cvglmnet; learner.class=classif.cvglmnet
## Trained on: task.id = data; obs = 97; features = 1968
## Hyperparameters: nlambda=100,alpha=1

# Run model on the data and check output prediction matrix
sc.obj <- make.predictions(sc.obj)
pred_matrix <- pred_matrix(sc.obj)
head(pred_matrix)

##             CV_rep1   CV_rep2
## 01A-OITS1 0.3666409 0.3705328
## 01B-OITS1 0.5644322 0.6173407
## 02A-OITS1 0.4810090 0.4516314
## 02B-OITS1 0.4440480 0.4656959
## 03A-OITS1 0.2239698 0.2924859
## 03B-OITS1 0.7869916 0.7646198

# Save model results to plot
model.interpretation.plot(
     sc.obj,
     fn.plot = file.path(outp, 'model_interpretation.pdf'),
     consens.thres = 0.5,
     limits = c(-3, 3),
     heatmap.type = 'zscore',
 )

cat("Model results stored to:", file.path(outp, 'model_interpretation.pdf'))

## Model results stored to: OITS1-bigtest//analysis_output/model_interpretation.pdf

Credit

This script is based on ideas and code from the dada2 Tutorial by Benjamin Callahan, the publication “Bioconductor Workflow for Microbiome Data Analysis: from raw reads to community analyses” by Callahan et al. (2016) and various pages of the official phyloseq website by Paul J. McMurdie.