# PC Directions in dppca

In ordinary PCA, the principal component directions are obtained from the eigenvectors of the sample covariance matrix. In `dppca`, these directions can be computed in two different ways.

1. Non-private PC directions: eigenvectors of the sample covariance matrix.
2. Differentially private PC directions: private principal component directions obtained through the g-DPPCA procedure.

## Notation

Let

\[ X = \begin{bmatrix} X_1^\top \\ X_2^\top \\ \vdots \\ X_n^\top \end{bmatrix} \in \mathbb{R}^{n \times p} \]

be the data matrix used for PCA, where \(X_i \in \mathbb{R}^p\)is the \(i\)-th observation. We assume that \(X\) has been centered, and optionally standardized.

The principal component direction matrix is denoted by

\[ V_k = [v_1,\ldots,v_k] \in \mathbb{R}^{p \times k}, \]

where each column \(v_\ell\) is a unit vector representing the \(\ell\)-th pc direction.

The corresponding score matrix is \(Z = X V_k\).

## 1. Non-private PC directions

The classical sample covariance matrix is

\[ \hat\Sigma = \frac{1}{n-1}X^\top X. \]

The non-private PCA directions are obtained from the eigenvalue decomposition

\[ \hat\Sigma = \hat V \hat\Lambda \hat V^\top, \]

where

\[ \hat V = [\hat v_1,\ldots,\hat v_p], \quad \hat\Lambda = \operatorname{diag}(\hat\lambda_1,\ldots,\hat\lambda_p) \quad \text{with} \quad \hat\lambda_1 \geq \hat\lambda_2 \geq \cdots \geq \hat\lambda_p \geq 0. \]

The \(\ell\)-th sample principal component direction is \(\hat v_\ell\).

Equivalently,

\[ \hat v_\ell = \arg\max_{\|v\|_2 = 1} v^\top \hat\Sigma v \quad \text{subject to} \quad v^\top \hat v_j = 0, \qquad j = 1,\ldots,\ell-1. \]

In the non-private option of `dppca`, the direction matrix used for projection is

\[ \hat V_k = [\hat v_1,\ldots,\hat v_k]. \]

## 2. DP PC directions

Kim and Jung (2025) proposed `g-DPPCA` by adding matrix Gaussian mechanism on the generalized multivariate Kendall’s tau matrix which based on the robust data transformation called generalized spatial sign proposed by Raymakers and Rousseeuw (2019).

For a positive valued scale function \(\xi: (0, \infty) \to (0, \infty)\), consider a map \(g_\xi: \mathbb{R}^d \to \mathbb{R}^d\) defined as

\[ g_\xi(t) = \xi(\|t\|_2)\cdot \frac{t}{\|t\|_2}. \]

\(g_{\xi}\) is called as a generalized spatial sign with respect to \(\xi\).

The generalized multivariate Kendall’s tau matrix with respect to \(g_\xi\) is defined as

\[ K_{g_\xi} = \mathbb{E}_{X, X'}\left[ g_\xi\left( \frac{X - X'}{\sqrt{2}}\right) g_\xi\left( \frac{X - X'}{\sqrt{2}}\right)^\top ~ \right], \]

where \(X'\) is an independent copy of \(X\). Importantly, if \(X\) follows an elliptical distribution (which including Gaussian and multivariate \(t\)-distributions), \(K_{g_\xi}\) shares the same eigenvectors with same order to the \(\mbox{cov}(X)\). So, one can conduct a PCA by estimating \(K_{g_\xi}\) and then get eigenvectors of it.

For a convenience, we write \(g\) as the given sign function. For a random sample \(S = (X_1, \dots, X_n)\), the second order U-statistic of \(K_{g}\) can be written as

\[ \widehat{K}_g(S) = \frac{2}{n(n-1)} \sum_{i < j} g\left(\frac{X_j - X_i}{\sqrt{2}}\right) g\left(\frac{X_j - X_i}{\sqrt{2}}\right)^\top. \]

Note that the sensitivity of \(\widehat{K}_g\) with respect to the Frobenius norm can be upper bounded by

\[ \Delta_F(\widehat{K}_g) = \sup_{S \sim S'} \|\widehat{K}_g(S) - \widehat{K}_g(S')\|_F \le \frac{4\|g\|_\infty^2}{n}. \]

So, for a dataset \(S = (x_1, \dots, x_n)\) the randomized mechanism \(\bar{K}_g\) defined as

\[ \bar{K}_g(S) := \frac{2}{n(n-1)} \sum_{i < j} g\left(\frac{x_j-x_i}{\sqrt{2}}\right)g\left(\frac{x_j-x_i}{\sqrt{2}}\right)^\top + \mbox{vecd}^{-1}(\xi), \] where \(\xi \sim N_{d(d+1)/2}(0, \sigma_{\varepsilon, \delta}^2 I_{d(d+1)/2})\) and \(\sigma_{\varepsilon, \delta} = \frac{4\|g\|_{\infty}^2 \sqrt{2 \ln(1.25/\delta)}}{n\varepsilon}\), satisfies \((\varepsilon, \delta)\)-DP.

Define \(\bar{V}_{g, m}(S) \in \mathcal{O}(d, m)\) as the matrix of the first \(m\) eigenvectors of \(\bar{K}_g(S)\). Then, \(\bar{V}_{g, m}(S)\) satisfies \((\varepsilon, \delta)\)-DP due to the post-processing property, and it can be served as a DP principal components. Kim and Jung (2025) calls these process as a `g-DPPCA`.

In the implementation of the function `dp_pc_dir` with option `g_dppca=TRUE`, we use the spherical transformation \(g_{sph}(t) = t/\|t\|_2\) to output differentially private PC directions \(\bar{V}_{sph,m}\). In this case, it holds that \(\|g_{sph}\|_{\infty} = 1\), and thus the variance of additive Gaussian noise is set as \(\sigma_{\varepsilon, \delta} = \frac{4\sqrt{2 \ln(1.25/\delta)}}{n\varepsilon}\).

## Summary

The principal component direction step in `dppca` can be summarized as follows.

1. Start with a preprocessed data matrix \(X\).
2. Choose a direction estimation method.
3. Obtain a direction matrix \(V_k\).
4. Compute projected scores \(Y = X V_k\).
5. Use the scores for private scree estimation or private score visualization.

The main distinction is whether \(V_k\) is obtained from the ordinary sample covariance matrix or from a differentially private robust PC direction estimator.

## References

Minwoo Kim and Sungkyu Jung (2025), “Robust and differentially private principal component analysis,” Statistical Analysis and Data Mining, 18(6), https://doi.org/10.1002/sam.70053

Jakob Raymaekers and Peter Rousseeuw (2019), “A generalized spatial sign covariance matrix,” Journal of Multivariate Analysis, 171:94–111, https://doi.org/10.1016/j.jmva.2018.11.010