1 Introduction

MetMashR is an R package designed to facilitate the cleaning, filtering and combining of annotations from different sources. MetMashR defines an“annotation source” as a piece of software, proprietary or otherwise, that takes the raw input of an analytical instrument and attempts to assign molecule names to the peaks in the data, usually by comparison to a library. MetMashR was primarily designed for use with metabolomics data measured by LCMS (hence “metabolite” in the package name) but could be extended to include other platforms (e.g. NMR, DIMS etc.), or other analytical approaches.

In this vignette we describe commonly used annotation workflow steps and show how to use them in detail.


2 Statistics in R using Class Templates (struct)

All of the objects defined in MetMashR use or extend the class templates defined by the struct package. Although originally intended for statistics applications, the templates in the struct package have proven to be adaptable to many different scenarios and types of analysis/workflow step.

The use of struct templates allows workflow steps to be applied in sequence and intermediate outputs to be retained for further analysis if required. The templates include ontology definitions for both the object and its input/output parameters. This makes the workflows more “FAIR” which is critical alongside FAIR data to making workflows repeatable, transparent and reproducible.

A general summary of extending struct templates is provided in the package vignette.


3 Getting Started

The latest versions of struct and MetMashR that are compatible with your current R version can be installed using BiocManager.

# install BiocManager if not present
if (!requireNamespace("BiocManager", quietly = TRUE)) {
    install.packages("BiocManager")
}

# install MetMashR and dependencies
BiocManager::install("MetMashR")

Once installed you can activate the packages in the usual way:

# load the packages
library(struct)
library(MetMashR)
library(metabolomicsWorkbenchR)
library(ggplot2)


4 Annotation Sources

annotation_source objects are the dataset used by all MetMashR workflow steps. If you have used our structToolbox package before, then annotation sources used equivalently to DatasetExperiment objects, except that they hold a single data.frame of metabolite annotation data.

The annotation_source object is not very specific, and not intended for general use. Instead we have extended them to two main types of source:

  • annotation_table
  • annotation_database

Although all annotation_sources contain a single a data.frame, the intended use of annotation_table and annotation_database is different.


4.1 Annotation Tables

A annotation_table is defined by us as a data.frame of metabolite annotations for experimentally collected data. For example, we have provided lcms_table objects which ensure that both m/z and retention time data is included in the data.frame for LCMS data. Usually this table of annotations is acquired after the application software to generate annotations for an experimental data set.

Note:
It is not the aim of MetMashR to generate these annotations. Instead we aim to provide tools to process, filter, clean and otherwise “mash” this table of annotations generated elsewhere.


annotation_table objects should have a read_source method specific to the source. For example the read_source method for ls_source object reads in the exported data file from LipidSearch by stripping the header and parsing the rest of the file into a table.

# prepare source object
AT <- ls_source(
    source = system.file(
        paste0("extdata/MTox/LS/MTox_2023_HILIC_POS.txt"),
        package = "MetMashR"
    )
)

# read source
AT <- read_source(AT)

# show info
AT
#> A "ls_source" object
#> --------------------
#> name:          LCMS table
#> description:   An LCMS table extends [`annotation_table()`] to represent annotation data for an LCMS
#>                  experiment. Columns representing m/z and retention time are required for an
#>                  `lcms_table`.
#> input params:  mz_column, rt_column 
#> annotations:   62 rows x 12 columns

The imported annotation_table object is compatible with MetMashR workflow steps.


4.2 Annotation Databases

An annotation_database is a table of additional metabolite meta data. For example it might contain identifiers and/or InChIKeys for different metabolites. Usually (but not always) this table is used in a read-only fashion and is used to augment an annotation_table with additional information.

Like other sources, annotation_database objects have a read_source method specific to the database.

# prepare source object
MT <- MTox700plus_database()

# read
MT <- read_source(MT)

# show
MT
#> A "MTox700plus_database" object
#> -------------------------------
#> name:          MTox700plus_database
#> description:   Imports the MTox700+ database, which is made available under the ODC Attribution License.
#>                  MTox700+ is a list of toxicologically relevant metabolites derived from
#>                  publications, public databases and relevant toxicological assays.
#> input params:  version 
#> annotations:   722 rows x 15 columns

annotation_database objects also have a read_database method to read the table directly to a data.frame.

# prepare source object
MT <- MTox700plus_database()

# read to data.frame
df <- read_database(MT)

# show
.DT(df)

Some annotation_database objects also have a write_database method, that allows you to update the table on disk. For example, in MetMashR the rds_database has a write_database method. It is useful in combination with rest_api objects to cache results and reduce the number of requests to the api.


4.3 Cached databases

An annotation_database class has been included that uses functionality provided by BiocFileCache. Although not used directly, many of the annotation_database objects provided by MetMashR extend the BiocFileCache_database object so that the web resources they retrieve are cached locally.


5 Annotation Mashing

We define annotation mashing as the importing, cleaning, filtering and combining of multiple annotation sources. This is useful for metabolomics datasets where there might be several assays and/or sources of information/annotations.


5.1 Importing sources

Although annotation_sources all have a read_source method, it is convenient to be able to read in a source as part of a workflow.

The import_source model (workflow step) allows you to do this. Note that using this object will replace the existing annotation_source and is really intended to be used as the first step in a workflow.

# prepare source object
AT <- ls_source(
    source = system.file(
        paste0("extdata/MTox/LS/MTox_2023_HILIC_POS.txt"),
        package = "MetMashR"
    )
)

# prepare workflow
WF <- import_source()

# apply workflow to annotation source
WF <- model_apply(WF, AT)

# show
predicted(WF)
#> A "ls_source" object
#> --------------------
#> name:          LCMS table
#> description:   An LCMS table extends [`annotation_table()`] to represent annotation data for an LCMS
#>                  experiment. Columns representing m/z and retention time are required for an
#>                  `lcms_table`.
#> input params:  mz_column, rt_column 
#> annotations:   62 rows x 12 columns


5.2 Filtering / Cleaning

MetMashR provides a number of commonly used workflow steps to filter, clean and process annotation sources. Some of these steps, such as filter_range are applicable to any annotation source, while others are specific to a source. For example mz_rt_match is only applicable to an lcms_table as it requires that both an m/z and a retention time column are present. This property is only enforced for lcms_table objects.

Workflow steps use the model class from struct. We can build up a workflow by “adding” steps together to form a model sequence (model_seq). See the vignettes for struct for more details.

Both models and model sequences can be applied to an annotation_source objects using the model_apply method. In this example we import the source, and then apply a filtering step to remove records with a lower Grading.

# prepare source object
AT <- ls_source(
    source = system.file(
        paste0("extdata/MTox/LS/MTox_2023_HILIC_POS.txt"),
        package = "MetMashR"
    )
)

# prepare workflow
WF <-
    # step 1 import source from file
    import_source() +
    # step 2 filter the "Grade" column to only include "A" and "B"
    filter_labels(
        column_name = "Grade",
        labels = c("A", "B"),
        mode = "include"
    )

# apply workflow to annotation source
WF <- model_apply(WF, AT)

# show
predicted(WF)
#> A "ls_source" object
#> --------------------
#> name:          LCMS table
#> description:   An LCMS table extends [`annotation_table()`] to represent annotation data for an LCMS
#>                  experiment. Columns representing m/z and retention time are required for an
#>                  `lcms_table`.
#> input params:  mz_column, rt_column 
#> annotations:   29 rows x 12 columns

The predicted method returns the processed annotation_source after applying all steps of the workflow.

Indexing can also be used with a model sequence to extract the processed annotation source after that step in the workflow.

# source after import and before filtering
predicted(WF[1])
#> A "ls_source" object
#> --------------------
#> name:          LCMS table
#> description:   An LCMS table extends [`annotation_table()`] to represent annotation data for an LCMS
#>                  experiment. Columns representing m/z and retention time are required for an
#>                  `lcms_table`.
#> input params:  mz_column, rt_column 
#> annotations:   62 rows x 12 columns


5.3 LCMS peak matching

The following methods are restricted to lcms_table sources:

  • mz_match
  • rt_match
  • mz_rt_match
  • calc_ppm_diff
  • calc_rt_diff

The _match objects align features and annotations by comparing m/z and/or retention time values between two sources. If the values fall within a window then this is considered to be a match.

Often times one of the sources will be a library or database of reference m/z and/or retention time values, and the other will be a table of peaks from an experiment. In this case the reference database might be considered as the gold standard, while the experimentally determined values will have some degree of uncertainty. In this case you may want to only consider a window applied to the experimental data. The diagram below illustrates this for retention time matching.


In other cases both sources might be obtained experimentally. For example when matching MS2 peaks to MS1 peaks. In this case a window can be applied to both sources, reflecting the uncertainty in the values for both sources.


6 REST APIs

MetMashR provides a rest_api object that implements some base methods to query an api and return a data.frame of results.

The template has been extended to include api lookup objects for the following:

  • ClassyFire
  • HMDB
  • KEGG
  • LipidMaps
  • Metabolmics Workbench
  • OPSIN
  • PubChem

Note these are not necessarily a complete wrapper for all functionality provided by the api; we have only implemented simple wrappers for the most useful parts e.g. querying for molecular identifiers.

The rest_api template object includes the ability to cache results locally, in order to reduce the number of api queries. This means a rest_api object can be included in a workflow and updated as more results are collected over time.

# prepare source object
AT <- ls_source(
    source = system.file(
        paste0("extdata/MTox/LS/MTox_2023_HILIC_POS.txt"),
        package = "MetMashR"
    )
)

# prepare cache
TF <- rds_database(
    source = tempfile()
)

# prepare workflow
WF <-
    # step 1 import source from file
    import_source() +
    # step 2 filter the "Grade" column to only include "A" and "B"
    filter_labels(
        column_name = "Grade",
        labels = c("A", "B"),
        mode = "include"
    ) +
    # step 3 query lipidmaps api for inchikey
    lipidmaps_lookup(
        query_column = "LipidName",
        context = "compound",
        context_item = "abbrev",
        output_item = "inchi_key",
        cache = TF,
        suffix = ""
    )

# apply workflow to annotation source
WF <- model_apply(WF, AT)

# show
predicted(WF)
#> A "ls_source" object
#> --------------------
#> name:          LCMS table
#> description:   An LCMS table extends [`annotation_table()`] to represent annotation data for an LCMS
#>                  experiment. Columns representing m/z and retention time are required for an
#>                  `lcms_table`.
#> input params:  mz_column, rt_column 
#> annotations:   39 rows x 14 columns

Note that the cache is stored as an annotation_database object, and can be used in workflows like any other annotation_source.

# retrieve cache
TF <- read_source(TF)

# filter records with no inchikey
FI <-
    filter_na(
        column_name = "inchi_key"
    )

# apply
FI <- model_apply(FI, TF)

# show
.DT(predicted(FI)$data)

An alternative to using rest_api objects in every workflow is to create separate workflow to generate a local database of relevant data. This database can then be used in other workflows without needing to query the api every time the workflow is run.


7 Dictionaries

The normalise_strings object uses a special list format, referred to as a “dictionary”, to provide conversion between string patterns. In MetMashR workflows we use it to e.g. convert adducts into a standardised format across sources, and to tidy/clean strings before using them as search terms in rest api queries. MetMashR currently provides the following dictionaries:

  • Greek character dictionary (greek_dictionary) to convert greek characters to their romanised name.
  • Racemic notation dictionary (racemic_dictionary) to remove certain types of racemic notation from molecule names (e.g. “(+/-)”).
  • A tripeptide dictionary (tripeptide_dictionary) to convert three-letter tripeptide abbreviations into a format more commonly used as a synonym on PubChem e.g. “ACD” becomes “Ala-Cys-Asp”.

A custom dictionary can be created on the fly as a list where each element has the following fields:

  • pattern: used as input to [grepl()] to detect matches to the input pattern
  • replace: a string, or a function that returns a string, to replace the pattern with in the matching string.

Additional fields in the list item can be any of the additional inputs to grepl(), such as fixed = TRUE.

For example here we create a dictionary to convert some of the lipid abbreviations to the LipidMaps standard, and replace underscores with forward slashes:

custom_dict <- list(
    list(
        pattern = "AcCa",
        replace = "CAR",
        fixed = TRUE
    ),
    list(
        pattern = "AEA",
        replace = "NAE",
        fixed = TRUE
    ),
    list(
        pattern = "_",
        replace = "/",
        fixed = TRUE
    )
)

We can now use this dictionary in a workflow to create a new column of “normalised” lipid names and (hopefully) get fewer NA when querying LipidMaps:

# prepare workflow
WF <-
    # step 1 import source from file
    import_source() +
    # step 2 filter the "Grade" column to only include "A" and "B"
    filter_labels(
        column_name = "Grade",
        labels = c("A", "B"),
        mode = "include"
    ) +
    # step 3 normalise lipid names using the custom dictionary:
    normalise_strings(
        search_column = "LipidName",
        output_column = "normalised_name",
        dictionary = custom_dict
    ) +
    # step 4 query lipidmaps api for inchikey using the names provided by
    # LipidSearch
    lipidmaps_lookup(
        query_column = "LipidName",
        context = "compound",
        context_item = "abbrev",
        output_item = "inchi_key",
        suffix = "_LipidName",
        cache = TF
    ) +
    # step 5 query lipidmaps api for inchikey using the names provided by
    # LipidSearch
    lipidmaps_lookup(
        query_column = "normalised_name",
        context = "compound",
        context_item = "abbrev",
        output_item = "inchi_key",
        suffix = "_normalised"
    )

# apply workflow to annotation source
WF <- model_apply(WF, AT)

#  show result table for relevant columns
.DT(predicted(WF)$data[, c(
    "LipidName", "normalised_name",
    "inchi_key_LipidName", "inchi_key_normalised"
)])

You can see that we obtained inchikey for more of the Lipids after normalising the lipid names.

MetMashR also provides an interface to the rgoslin package to assist with Lipid annotations.


8 Combining Records

In the previous output LipidMaps returned multiple matches to the same lipid. This is because Lipid names can be ambiguous regarding the location of double bonds, for example.

Sometimes it is useful to collapse multiple entries (records) into a single record. MetMashR provides the combine_records object and a number of helper functions to facilitate this.

The combine records object is a wrapper around dplyr::reframe (formally dplyr::summarise). You can provide a default function to apply to all columns, and then specify transformations for individual columns by name.

For the Lipids example above, we collapse multiple records for the same LipidName into a single record, and collapse e.g. multiple inchikeys into a single string separated by semi colons.

# prepare workflow
CR <- combine_records(
    group_by = "LipidName",
    default_fcn = fuse_unique(separator = "; "),
    fcns = list(
        count = count_records()
    )
)

# apply to previous output
CR <- model_apply(CR, predicted(WF))

# show output for relevant columns
.DT(predicted(CR)$data[, c(
    "LipidName", "normalised_name",
    "inchi_key_normalised", "count"
)])

You can see that there is now a single record (row) for each LipidName, and that the multiple inchikeys associated with that LipidName have been collapsed into a single entry separated by semicolons.

The helper function fuse_unique ensures that each inchikey only appears once in the collapsed string, and is applied as default to all columns.

The add_count helper function adds a new column of counts for each LipidName. Note that AcCa(20:4) has 8 counts but only 4 inchikey. This means that AcCa(20:4) appeared twice in the original table, each time with the same 4 inchikey.

There are a number of other helper functions to suit different requirements see ?combine_records_helper_functions for a complete list.

9 Session Info

sessionInfo()
#> R version 4.4.1 (2024-06-14)
#> Platform: x86_64-pc-linux-gnu
#> Running under: Ubuntu 24.04.1 LTS
#> 
#> Matrix products: default
#> BLAS:   /home/biocbuild/bbs-3.20-bioc/R/lib/libRblas.so 
#> LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.12.0
#> 
#> locale:
#>  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
#>  [3] LC_TIME=en_GB              LC_COLLATE=C              
#>  [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
#>  [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
#>  [9] LC_ADDRESS=C               LC_TELEPHONE=C            
#> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       
#> 
#> time zone: America/New_York
#> tzcode source: system (glibc)
#> 
#> attached base packages:
#> [1] stats     graphics  grDevices utils     datasets  methods   base     
#> 
#> other attached packages:
#> [1] DT_0.33                       dplyr_1.1.4                  
#> [3] structToolbox_1.18.0          ggplot2_3.5.1                
#> [5] metabolomicsWorkbenchR_1.16.0 MetMashR_1.0.0               
#> [7] struct_1.18.0                 BiocStyle_2.34.0             
#> 
#> loaded via a namespace (and not attached):
#>  [1] DBI_1.2.3                   gridExtra_2.3              
#>  [3] rlang_1.1.4                 magrittr_2.0.3             
#>  [5] matrixStats_1.4.1           compiler_4.4.1             
#>  [7] RSQLite_2.3.7               vctrs_0.6.5                
#>  [9] stringr_1.5.1               pkgconfig_2.0.3            
#> [11] crayon_1.5.3                fastmap_1.2.0              
#> [13] dbplyr_2.5.0                magick_2.8.5               
#> [15] XVector_0.46.0              labeling_0.4.3             
#> [17] utf8_1.2.4                  rmarkdown_2.28             
#> [19] UCSC.utils_1.2.0            tinytex_0.53               
#> [21] purrr_1.0.2                 bit_4.5.0                  
#> [23] xfun_0.48                   MultiAssayExperiment_1.32.0
#> [25] zlibbioc_1.52.0             cachem_1.1.0               
#> [27] GenomeInfoDb_1.42.0         jsonlite_1.8.9             
#> [29] RVenn_1.1.0                 blob_1.2.4                 
#> [31] highr_0.11                  DelayedArray_0.32.0        
#> [33] R6_2.5.1                    bslib_0.8.0                
#> [35] stringi_1.8.4               ComplexUpset_1.3.3         
#> [37] GenomicRanges_1.58.0        jquerylib_0.1.4            
#> [39] Rcpp_1.0.13                 bookdown_0.41              
#> [41] SummarizedExperiment_1.36.0 knitr_1.48                 
#> [43] IRanges_2.40.0              Matrix_1.7-1               
#> [45] tidyselect_1.2.1            abind_1.4-8                
#> [47] yaml_2.3.10                 ggVennDiagram_1.5.2        
#> [49] curl_5.2.3                  lattice_0.22-6             
#> [51] tibble_3.2.1                plyr_1.8.9                 
#> [53] Biobase_2.66.0              withr_3.0.2                
#> [55] evaluate_1.0.1              ontologyIndex_2.12         
#> [57] BiocFileCache_2.14.0        zip_2.3.1                  
#> [59] pillar_1.9.0                BiocManager_1.30.25        
#> [61] filelock_1.0.3              MatrixGenerics_1.18.0      
#> [63] stats4_4.4.1                ChemmineOB_1.44.0          
#> [65] generics_0.1.3              sp_2.1-4                   
#> [67] S4Vectors_0.44.0            munsell_0.5.1              
#> [69] scales_1.3.0                glue_1.8.0                 
#> [71] tools_4.4.1                 data.table_1.16.2          
#> [73] openxlsx_4.2.7.1            cowplot_1.1.3              
#> [75] grid_4.4.1                  tidyr_1.3.1                
#> [77] crosstalk_1.2.1             colorspace_2.1-1           
#> [79] patchwork_1.3.0             GenomeInfoDbData_1.2.13    
#> [81] cli_3.6.3                   rsvg_2.6.1                 
#> [83] fansi_1.0.6                 ggthemes_5.1.0             
#> [85] S4Arrays_1.6.0              gtable_0.3.6               
#> [87] sass_0.4.9                  digest_0.6.37              
#> [89] BiocGenerics_0.52.0         SparseArray_1.6.0          
#> [91] htmlwidgets_1.6.4           farver_2.1.2               
#> [93] memoise_2.0.1               htmltools_0.5.8.1          
#> [95] lifecycle_1.0.4             httr_1.4.7                 
#> [97] bit64_4.5.2