1 Introduction

The BiocNeighbors package provides an implementation of the k-means for k-nearest neighbors (KMKNN) algorithm, as described by Wang (2012). For a dataset with \(N\) points, the pre-training is done as follows:

  1. Apply k-means clustering to all points, partitioning the data into \(\sqrt{N}\) clusters.
  2. Compute the distance from each data point to its cluster center.
  3. Store the cluster identities and distances.

For each query point, identification of the nearest neighbors is done as follows:

  1. Start with a threshold distance \(d\) to the current kth-nearest neighbor (this can be set with arbitrary points).
  2. Compute the distance from the query to each cluster center.
  3. For any given cluster center, apply the triangle inequality on the query-center distance, the center-point distances and \(d\). Only compute query-point distances for points where the triangle inequality holds.
  4. Update \(d\) with the new closest kth-nearest neighbor and repeat for the next cluster.

The pre-clustering arranges the points in a manner that effectively reduces the search space, even in high-dimensional data. Note that, while kmeans itself is random, the k-nearest neighbors result is fully deterministic1 Except in the presence of ties, see ?findKNN for details..

The algorithm is implemented in a combination of R and C++, derived from code in cydar (Lun, Richard, and Marioni 2017). We observe 2-5-fold speed-ups in 20- to 50-dimensional data, compared to KD-trees in FNN and RANN (see https://github.com/LTLA/OkNN2018 for timings). This is consistent with results from Wang (2012).

2 Identifying k-nearest neighbors

The most obvious application is to perform a k-nearest neighbors search. We’ll mock up an example here with a hypercube of points, for which we want to identify the 10 nearest neighbors for each point.

nobs <- 10000
ndim <- 20
data <- matrix(runif(nobs*ndim), ncol=ndim)

The findKNN() method expects a numeric matrix as input with data points as the rows and variables/dimensions as the columns. We indicate that we want to use the KMKNN algorithm by setting BNPARAM=KmknnParam() (which is also the default, so this is not strictly necessary here).

fout <- findKNN(data, k=10, BNPARAM=KmknnParam())
head(fout$index)
##      [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
## [1,]  491 8844 6987  909 7724 6616 7610 7762 9358  5636
## [2,] 9924 8885 9585 7387 8637 5472 4223 8740 2003  5947
## [3,] 3974 2206 5942 3675 7054 3571 4402  637 8402  9347
## [4,] 6946 5169 5875 1878 5901 7100 5232 6937 2526  2073
## [5,] 5622 5273 9025 8677   48 7793 6365 4678 1167  4588
## [6,] 6100  684 7048 8297 6733 1233 5672 3029 6996  4555
head(fout$distance)
##           [,1]      [,2]      [,3]      [,4]     [,5]     [,6]     [,7]
## [1,] 0.9244110 0.9772503 0.9780374 0.9873968 1.010858 1.014041 1.014974
## [2,] 0.7837896 0.9500115 0.9956408 1.0121876 1.020259 1.026712 1.035924
## [3,] 0.8778828 0.9530510 1.0168607 1.0496337 1.069225 1.071454 1.072035
## [4,] 0.9392798 0.9830096 0.9974784 1.0121211 1.033360 1.036489 1.040670
## [5,] 0.8846882 0.9148439 0.9685309 0.9724015 1.009772 1.016357 1.036703
## [6,] 0.7762335 1.0017938 1.0231434 1.0530299 1.056910 1.072419 1.086293
##          [,8]     [,9]    [,10]
## [1,] 1.031932 1.036426 1.038599
## [2,] 1.042073 1.049463 1.050105
## [3,] 1.088347 1.089627 1.090839
## [4,] 1.046043 1.046782 1.065121
## [5,] 1.037697 1.048872 1.061362
## [6,] 1.088754 1.098093 1.104476

Each row of the index matrix corresponds to a point in data and contains the row indices in data that are its nearest neighbors. For example, the 3rd point in data has the following nearest neighbors:

fout$index[3,]
##  [1] 3974 2206 5942 3675 7054 3571 4402  637 8402 9347

… with the following distances to those neighbors:

fout$distance[3,]
##  [1] 0.8778828 0.9530510 1.0168607 1.0496337 1.0692253 1.0714538 1.0720354
##  [8] 1.0883472 1.0896274 1.0908386

Note that the reported neighbors are sorted by distance.

3 Querying k-nearest neighbors

Another application is to identify the k-nearest neighbors in one dataset based on query points in another dataset. Again, we mock up a small data set:

nquery <- 1000
ndim <- 20
query <- matrix(runif(nquery*ndim), ncol=ndim)

We then use the queryKNN() function to identify the 5 nearest neighbors in data for each point in query.

qout <- queryKNN(data, query, k=5, BNPARAM=KmknnParam())
head(qout$index)
##      [,1] [,2] [,3] [,4] [,5]
## [1,] 4802 7850  495 2999 3704
## [2,] 9299 7748 4193 3518 2582
## [3,] 4293 5117  266 4864 3261
## [4,] 4063 1345 5216 4784 7436
## [5,] 4684 6204 2167 2303 8350
## [6,] 4398 2519 7681  697 7513
head(qout$distance)
##           [,1]      [,2]      [,3]      [,4]      [,5]
## [1,] 0.8653166 0.8805966 0.9016671 0.9164906 0.9241810
## [2,] 0.7573697 0.7652216 0.7660648 0.8355930 0.8706586
## [3,] 0.9713096 1.0203341 1.0214711 1.0293597 1.0298720
## [4,] 0.6912581 1.0550427 1.0563564 1.0855658 1.1067250
## [5,] 1.0380893 1.0539590 1.0919226 1.0997027 1.1127108
## [6,] 0.8976966 0.9642629 0.9696387 1.0164593 1.0398051

Each row of the index matrix contains the row indices in data that are the nearest neighbors of a point in query. For example, the 3rd point in query has the following nearest neighbors in data:

qout$index[3,]
## [1] 4293 5117  266 4864 3261

… with the following distances to those neighbors:

qout$distance[3,]
## [1] 0.9713096 1.0203341 1.0214711 1.0293597 1.0298720

Again, the reported neighbors are sorted by distance.

4 Further options

Users can perform the search for a subset of query points using the subset= argument. This yields the same result as but is more efficient than performing the search for all points and subsetting the output.

findKNN(data, k=5, subset=3:5)
## $index
##      [,1] [,2] [,3] [,4] [,5]
## [1,] 3974 2206 5942 3675 7054
## [2,] 6946 5169 5875 1878 5901
## [3,] 5622 5273 9025 8677   48
## 
## $distance
##           [,1]      [,2]      [,3]      [,4]     [,5]
## [1,] 0.8778828 0.9530510 1.0168607 1.0496337 1.069225
## [2,] 0.9392798 0.9830096 0.9974784 1.0121211 1.033360
## [3,] 0.8846882 0.9148439 0.9685309 0.9724015 1.009772

If only the indices are of interest, users can set get.distance=FALSE to avoid returning the matrix of distances. This will save some time and memory.

names(findKNN(data, k=2, get.distance=FALSE))
## [1] "index"

It is also simple to speed up functions by parallelizing the calculations with the BiocParallel framework.

out <- findKNN(data, k=10, BPPARAM=MulticoreParam(3))

For multiple queries to a constant data, the pre-clustering can be performed in a separate step with buildNNIndex(). The result can then be passed to multiple calls, avoiding the overhead of repeated clustering2 The algorithm type is automatically determined when BNINDEX is specified, so there is no need to also specify BNPARAM in the later functions..

pre <- buildNNIndex(data, BNPARAM=KmknnParam())
out1 <- findKNN(BNINDEX=pre, k=5)
out2 <- queryKNN(BNINDEX=pre, query=query, k=2)

Advanced users may also be interested in the raw.index= argument, which returns indices directly to the precomputed object rather than to data. This may be useful inside package functions where it may be more convenient to work on a common precomputed object.

5 Session information

sessionInfo()
## R version 3.5.1 Patched (2018-07-12 r74967)
## Platform: x86_64-apple-darwin15.6.0 (64-bit)
## Running under: OS X El Capitan 10.11.6
## 
## Matrix products: default
## BLAS: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRblas.0.dylib
## LAPACK: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib
## 
## locale:
## [1] C/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
## [1] BiocNeighbors_1.0.0 BiocParallel_1.16.0 knitr_1.20         
## [4] BiocStyle_2.10.0   
## 
## loaded via a namespace (and not attached):
##  [1] Rcpp_0.12.19        bookdown_0.7        digest_0.6.18      
##  [4] rprojroot_1.3-2     backports_1.1.2     stats4_3.5.1       
##  [7] magrittr_1.5        evaluate_0.12       stringi_1.2.4      
## [10] S4Vectors_0.20.0    rmarkdown_1.10      tools_3.5.1        
## [13] stringr_1.3.1       parallel_3.5.1      xfun_0.4           
## [16] yaml_2.2.0          compiler_3.5.1      BiocGenerics_0.28.0
## [19] BiocManager_1.30.3  htmltools_0.3.6

References

Lun, A. T. L., A. C. Richard, and J. C. Marioni. 2017. “Testing for differential abundance in mass cytometry data.” Nat. Methods 14 (7):707–9.

Wang, X. 2012. “A Fast Exact k-Nearest Neighbors Algorithm for High Dimensional Search Using k-Means Clustering and Triangle Inequality.” Proc Int Jt Conf Neural Netw 43 (6):2351–8.