# Detecting exact nearest neighbors

Aaron Lun1

1Cancer Research UK Cambridge Institute, Cambridge, United Kingdom

#### Revised: 2 December 2018

#### Package

BiocNeighbors 1.5.1

# 1 Introduction

The BiocNeighbors package implements a few algorithms for exact nearest neighbor searching:

- The k-means for k-nearest neighbors (KMKNN) algorithm (Wang 2012) uses k-means clustering to create an index. Within each cluster, the distance of each of that cluster’s points to the cluster center are computed and used to sort all points. Given a query point, the distance to each cluster center is determined and the triangle inequality is applied to determine which points in each cluster warrant a full distance calculation.
- The vantage point (VP) tree algorithm (Yianilos 1993) involves constructing a tree where each node is located at a data point and is associated with a subset of neighboring points. Each node progressively partitions points into two subsets that are either closer or further to the node than a given threshold. Given a query point, the triangle inequality is applied at each node in the tree to determine if the child nodes warrant searching.

Both methods involve a component of randomness during index construction, though the k-nearest neighbors result is fully deterministic11 Except in the presence of ties, see `?findKNN` for details..

# 2 Identifying k-nearest neighbors

The most obvious application is to perform a k-nearest neighbors search. We’ll mock up an example here with a hypercube of points, for which we want to identify the 10 nearest neighbors for each point.

    nobs <- 10000
    ndim <- 20
    data <- matrix(runif(nobs*ndim), ncol=ndim)

The `findKNN()` method expects a numeric matrix as input with data points as the rows and variables/dimensions as the columns. We indicate that we want to use the KMKNN algorithm by setting `BNPARAM=KmknnParam()` (which is also the default, so this is not strictly necessary here). We could use a VP tree instead by setting `BNPARAM=VptreeParam()`.

    fout <- findKNN(data, k=10, BNPARAM=KmknnParam())
    head(fout$index)

    ##      [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
    ## [1,]  443 5534 2819 4491 3319 5044 8108 1872 8343  7779
    ## [2,] 1871  473 4848 6941 7820  219 3023 4168 7922  4380
    ## [3,] 7307 6429 8867 1627 2387 8586 9701 6580 2992  5371
    ## [4,] 4104  543   66 1787 5196   47 1451 5188 2491  5385
    ## [5,] 8609 8688 8101 3912 2833 9608 2286 3149 5428  6098
    ## [6,] 1976 3531 5849 7158 3454 4055 9387  342 1200  3761

    head(fout$distance)

    ##           [,1]      [,2]      [,3]     [,4]      [,5]      [,6]      [,7]
    ## [1,] 0.8498439 0.9389631 0.9857974 1.004528 1.0529663 1.0533910 1.0559884
    ## [2,] 0.9630193 1.0641081 1.0677634 1.070712 1.0814036 1.0888527 1.1004052
    ## [3,] 1.0402729 1.0735655 1.0961041 1.104724 1.1133433 1.1165352 1.1323641
    ## [4,] 0.9358014 0.9575405 0.9639459 0.977944 0.9807007 0.9868828 0.9949084
    ## [5,] 0.7687973 1.0249677 1.0512036 1.065656 1.0768615 1.1254789 1.1267192
    ## [6,] 0.9128077 0.9342950 0.9619985 1.018286 1.0401043 1.0418174 1.0553538
    ##           [,8]     [,9]    [,10]
    ## [1,] 1.0783543 1.081047 1.083787
    ## [2,] 1.1243240 1.126671 1.127643
    ## [3,] 1.1503935 1.177099 1.205111
    ## [4,] 0.9996142 1.004220 1.014464
    ## [5,] 1.1303504 1.136442 1.164114
    ## [6,] 1.0598305 1.069526 1.076167

Each row of the `index` matrix corresponds to a point in `data` and contains the row indices in `data` that are its nearest neighbors. For example, the 3rd point in `data` has the following nearest neighbors:

    fout$index[3,]

    ##  [1] 7307 6429 8867 1627 2387 8586 9701 6580 2992 5371

… with the following distances to those neighbors:

    fout$distance[3,]

    ##  [1] 1.040273 1.073565 1.096104 1.104724 1.113343 1.116535 1.132364 1.150393
    ##  [9] 1.177099 1.205111

Note that the reported neighbors are sorted by distance.

# 3 Querying k-nearest neighbors

Another application is to identify the k-nearest neighbors in one dataset based on query points in another dataset. Again, we mock up a small data set:

    nquery <- 1000
    ndim <- 20
    query <- matrix(runif(nquery*ndim), ncol=ndim)

We then use the `queryKNN()` function to identify the 5 nearest neighbors in `data` for each point in `query`.

    qout <- queryKNN(data, query, k=5, BNPARAM=KmknnParam())
    head(qout$index)

    ##      [,1] [,2] [,3] [,4] [,5]
    ## [1,] 4694 6398 4320 3160 5623
    ## [2,] 3343 6316 8860 2198 5545
    ## [3,] 6892 6990 3579 7018 6951
    ## [4,] 5762 4060 9057 6257 3003
    ## [5,] 8278 8388 6779 8710  859
    ## [6,] 6908 4585  242 2875 5552

    head(qout$distance)

    ##           [,1]      [,2]      [,3]      [,4]      [,5]
    ## [1,] 0.8140070 0.8608475 0.8990512 0.9325698 0.9466198
    ## [2,] 0.9426442 0.9741116 1.0204131 1.0640365 1.0672929
    ## [3,] 0.9582085 1.0093231 1.0226169 1.0365479 1.0616490
    ## [4,] 1.0096666 1.0545721 1.0630065 1.0684061 1.1023994
    ## [5,] 0.9156484 0.9196761 0.9197024 0.9217155 0.9241611
    ## [6,] 0.7578627 0.7848712 0.8372202 0.9272435 0.9366036

Each row of the `index` matrix contains the row indices in `data` that are the nearest neighbors of a point in `query`. For example, the 3rd point in `query` has the following nearest neighbors in `data`:

    qout$index[3,]

    ## [1] 6892 6990 3579 7018 6951

… with the following distances to those neighbors:

    qout$distance[3,]

    ## [1] 0.9582085 1.0093231 1.0226169 1.0365479 1.0616490

Again, the reported neighbors are sorted by distance.

# 4 Further options

Users can perform the search for a subset of query points using the `subset=` argument. This yields the same result as but is more efficient than performing the search for all points and subsetting the output.

    findKNN(data, k=5, subset=3:5)

    ## $index
    ##      [,1] [,2] [,3] [,4] [,5]
    ## [1,] 7307 6429 8867 1627 2387
    ## [2,] 4104  543   66 1787 5196
    ## [3,] 8609 8688 8101 3912 2833
    ##
    ## $distance
    ##           [,1]      [,2]      [,3]     [,4]      [,5]
    ## [1,] 1.0402729 1.0735655 1.0961041 1.104724 1.1133433
    ## [2,] 0.9358014 0.9575405 0.9639459 0.977944 0.9807007
    ## [3,] 0.7687973 1.0249677 1.0512036 1.065656 1.0768615

If only the indices are of interest, users can set `get.distance=FALSE` to avoid returning the matrix of distances. This will save some time and memory.

    names(findKNN(data, k=2, get.distance=FALSE))

    ## [1] "index"

It is also simple to speed up functions by parallelizing the calculations with the BiocParallel framework.

    library(BiocParallel)
    out <- findKNN(data, k=10, BPPARAM=MulticoreParam(3))

For multiple queries to a constant `data`, the pre-clustering can be performed in a separate step with `buildIndex()`. The result can then be passed to multiple calls, avoiding the overhead of repeated clustering22 The algorithm type is automatically determined when `BNINDEX` is specified, so there is no need to also specify `BNPARAM` in the later functions..

    pre <- buildIndex(data, BNPARAM=KmknnParam())
    out1 <- findKNN(BNINDEX=pre, k=5)
    out2 <- queryKNN(BNINDEX=pre, query=query, k=2)

The default setting is to search on the Euclidean distance. Alternatively, we can use the Manhattan distance by setting `distance="Manhattan"` in the `BiocNeighborParam` object.

    out.m <- findKNN(data, k=5, BNPARAM=KmknnParam(distance="Manhattan"))

Advanced users may also be interested in the `raw.index=` argument, which returns indices directly to the precomputed object rather than to `data`. This may be useful inside package functions where it may be more convenient to work on a common precomputed object.

# 5 Session information

    sessionInfo()

    ## R Under development (unstable) (2019-11-03 r77362)
    ## Platform: x86_64-apple-darwin15.6.0 (64-bit)
    ## Running under: OS X El Capitan 10.11.6
    ##
    ## Matrix products: default
    ## BLAS:   /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRblas.0.dylib
    ## LAPACK: /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRlapack.dylib
    ##
    ## locale:
    ## [1] C/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
    ##
    ## attached base packages:
    ## [1] stats     graphics  grDevices utils     datasets  methods   base
    ##
    ## other attached packages:
    ## [1] BiocParallel_1.21.0 BiocNeighbors_1.5.1 knitr_1.25
    ## [4] BiocStyle_2.15.0
    ##
    ## loaded via a namespace (and not attached):
    ##  [1] Rcpp_1.0.2          bookdown_0.14       lattice_0.20-38
    ##  [4] digest_0.6.22       grid_4.0.0          stats4_4.0.0
    ##  [7] magrittr_1.5        evaluate_0.14       rlang_0.4.1
    ## [10] stringi_1.4.3       S4Vectors_0.25.0    Matrix_1.2-17
    ## [13] rmarkdown_1.16      tools_4.0.0         stringr_1.4.0
    ## [16] parallel_4.0.0      xfun_0.10           yaml_2.2.0
    ## [19] compiler_4.0.0      BiocGenerics_0.33.0 BiocManager_1.30.9
    ## [22] htmltools_0.4.0

# References

Wang, X. 2012. “A Fast Exact k-Nearest Neighbors Algorithm for High Dimensional Search Using k-Means Clustering and Triangle Inequality.” Proc Int Jt Conf Neural Netw 43 (6): 2351–8.

Yianilos, P. N. 1993. “Data Structures and Algorithms for Nearest Neighbor Search in General Metric Spaces.” In SODA, 93:311–21. 194.