---
title: "benchmarks"
author: "<h4>Author: <i>Brian M. Schilder</i></h4>" 
date: "<h4>Most recent update: <i>`r format( Sys.Date(), '%b-%d-%Y')`</i></h4>"
output: rmarkdown::html_vignette
editor_options: 
  chunk_output_type: inline
vignette: >
  %\VignetteIndexEntry{benchmarks}
  %\VignetteEncoding{UTF-8}
  %\VignetteEngine{knitr::rmarkdown}
---

```{r, include = FALSE}
knitr::opts_chunk$set(echo = TRUE, root.dir=here::here())
knitr::opts_knit$set(root.dir=here::here())
```


Benchmark the following strategies: 

1. **uppercase** : Simply making the genes uppercase. 
2. **homologene** : `method="homologene"` 
3. **gorth** : `method="gorth"`  


For each method, benchmark the following metrics: 
 
- % genes mapped within species (`all_genes`).
- % genes mapped to human (`map_orthologs`).  
- Speed of each metric. 

```{r setup}
library(orthogene)

library(dplyr)
library(ggplot2)
library(patchwork)
```

# Define species 

Repeat tests across various common model organisms.

```{r species_mapped} 
species_order <- c(
  "Homo sapiens",
  "Pan troglodytes",
  "Macaca mulatta",
  "Canis lupus familiaris",
  "Bos taurus",
  "Mus musculus",
  "Rattus norvegicus",
  "Gallus gallus",
  "Xenopus (Silurana) tropicalis",
  "Danio rerio",
  "Drosophila melanogaster",
  "Anopheles gambiae",
  "Caenorhabditis elegans",
  "Arabidopsis thaliana",
  "Oryza sativa",
  "Saccharomyces cerevisiae",
  "Schizosaccharomyces pombe",
  "Kluyveromyces lactis",
  "Eremothecium gossypii",
  "Neurospora crassa",
  "Magnaporthe oryzae"
)


species_mapped <- orthogene::all_species(method="homologene")

# Set species order
data.table::setkey(species_mapped,scientific_name)
species_mapped <- species_mapped[species_order]
```


# Benchmark  

*Note* the `orthogene:::function` notation is needed to use these benchmarking functions,
as they are internal.

## Run benchmark 

`run_benchmark()` will run the full benchmarking pipeline. 

You can set `mc.cores` to speed this up with multi-core parallelisation.

**WARNING**: This step can take a long time. For the purposes of this example, 
we'll not run the full benchmark and instead provide some pre-computed results. 

```{r, eval=FALSE} 

bench_res <- orthogene::: run_benchmark(species = species_mapped$scientific_name, 
                                       run_convert_orthologs = TRUE, 
                                       force = 2,
                                       save_path = here::here("inst/benchmark/bench_res_example.csv"),
                                       mc.cores = 1) 
```

## Print results 

Load stored benchmark results.

```{r}
if(!exists("bench_res")) {
  bench_res <- read.csv(system.file(package = "orthogene","benchmark/bench_res_example.csv"))
}
knitr::kable(bench_res)
```

## Plot results

### Bar plot

For each method, plot the run time (a) and the number of genes returned (b).

```{r, fig.height=10, fig.width=8} 
bench_barplot <- orthogene::: plot_benchmark_bar_grouped(bench_res = bench_res, remove_failed_times=TRUE)
ggplot2::ggsave(here::here("inst/benchmark/bench_barplot.pdf"),bench_barplot, height = 10)
```


```{r}
avg_time <- bench_res[,list(time=mean(time, na.rm = TRUE)),
                      by=c("test","method")]|>
    data.table::setkey(test)
avg_time

```


### Scatter plot 

For each method, plot the relationship between number of genes returned and run time.

```{r}
bench_scatterplot <- orthogene:::plot_benchmark_scatter(bench_res = bench_res)
# ggsave(here::here("inst/benchmark/bench_scatterplot.pdf"),bench_scatterplot)
```


```{r, fig.height=15}
tree_plot <- orthogene:: plot_orthotree(width = 15)
```


# Session Info 

<details> 

```{r Session Info}
utils::sessionInfo()
```

</details>  

