---
title: "Getting Started with tplyr2"
output:
  rmarkdown::html_vignette:
    toc: true
vignette: >
  %\VignetteIndexEntry{Getting Started with tplyr2}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r setup, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)
options(rmarkdown.html_vignette.check_title = FALSE)
library(tplyr2)
library(knitr)
```

# How tplyr2 Works

If you have worked with clinical data long enough, you know that summary tables --
demographics tables, adverse event tables, lab shift tables -- all share a common
structural pattern. Each section of the table represents some kind of summary: a
set of counts, a block of descriptive statistics, or a cross-tabulation.

tplyr2 is built around this idea. Rather than writing bespoke data manipulation
code for every table, you describe _what_ the table should contain using a
declarative specification, and tplyr2 handles the computing, formatting, and
assembly.

The key concepts are:

- **Spec**: A `tplyr_spec()` object is pure configuration. It describes the
  column structure, global filters, and a list of layers. No data processing
  happens when you create a spec.
- **Layers**: Each layer is one summary block. Count layers (`group_count()`)
  produce frequencies. Descriptive layers (`group_desc()`) compute means,
  medians, and other summaries. Shift layers (`group_shift()`) create
  cross-tabulations. Custom layers (`group_analyze()`) accept user-defined
  functions.
- **Build**: `tplyr_build(spec, data)` executes the spec against a dataset and
  returns a formatted data frame.

This separation of configuration from execution is intentional. A spec can be
saved to disk, reviewed, reused across studies, or applied to different datasets
at build time. If you are familiar with the PHUSE white paper *Analyses &
Displays Associated with Demographics, Disposition, and Medications in Phase 2-4
Clinical Trials and Integrated Summary Documents*, the layer model maps
naturally to the summary blocks described there.

# The tplyr_spec() Object

Every table starts with a `tplyr_spec()`. At minimum, you need the column
variable (typically the treatment arm) and at least one layer.

```{r}
spec <- tplyr_spec(
  cols = "TRT01P",
  layers = tplyr_layers(
    group_count("SEX"),
    group_desc("AGE")
  )
)
spec
```

The `cols` parameter determines which variable defines the output columns. Each
unique value becomes a result column. The function also accepts optional
parameters:

- `where`: A global filter expression (e.g., `where = SAFFL == "Y"`).
- `pop_data`: A `pop_data()` configuration for population-based denominators.
- `total_groups`: A list of `total_group()` objects for "Total" columns.
- `custom_groups`: A list of `custom_group()` objects for combined treatment arms.
- `layers`: Layer objects wrapped in `tplyr_layers()`.

The spec contains no data. Data is supplied at build time, so the same spec can
be reused across datasets.

# The tplyr_layer() Object

Layers are the building blocks of a tplyr2 table. You create them with the
`group_*()` family: `group_count()`, `group_desc()`, `group_shift()`, and
`group_analyze()`. Every layer constructor accepts:

- `target_var`: The variable(s) being summarized.
- `by`: Optional row grouping. Strings matching column names become grouping
  variables; other strings become text labels in the output. Use `label()` for
  explicit disambiguation.
- `where`: An optional layer-level filter expression.
- `settings`: A `layer_settings()` object for detailed configuration.

Multiple layers are collected with `tplyr_layers()`:

```{r}
layers <- tplyr_layers(
  group_count("SEX", by = "Sex n (%)"),
  group_count("RACE", by = "Race n (%)"),
  group_desc("AGE", by = "Age (Years)")
)
```

Layers are processed in order and stacked vertically in the output.

# Building a Table

Once you have a spec and a dataset, building is a single function call. Let's
look at a demographics-style table with both count and descriptive layers:

```{r}
spec <- tplyr_spec(
  cols = "TRT01P",
  where = SAFFL == "Y",
  layers = tplyr_layers(
    group_count("SEX", by = "Sex n (%)"),
    group_desc(
      "AGE",
      by = "Age (Years)",
      settings = layer_settings(
        format_strings = list(
          "n"         = f_str("xxx", "n"),
          "Mean (SD)" = f_str("xx.x (xx.xx)", "mean", "sd"),
          "Median"    = f_str("xx.x", "median"),
          "Min, Max"  = f_str("xx, xx", "min", "max")
        )
      )
    )
  )
)

result <- tplyr_build(spec, tplyr_adsl)
kable(result[, !grepl("^ord", names(result))])
```

The output is a standard data frame. Here is what the columns mean:

- **rowlabel1, rowlabel2, ...**: Row label columns. A `by` label adds one
  column; a `by` data variable adds another; and the target variable adds one
  more.
- **res1, res2, ...**: Result columns, one per unique level of `cols`. Each
  carries a `label` attribute with the column name and, when population data is
  available, an `(N=n)` suffix.
- **ord_layer_index, ord_layer_1, ...**: Ordering columns for sorting. These
  can be dropped for display but are useful for programmatic reordering.

# String Formatting in tplyr2

One of the most important features of tplyr2 is the format string system.
Clinical tables have specific alignment and precision requirements. The `f_str()`
function lets you declare exactly how numbers should appear.

An `f_str()` has two parts: a template string that defines the layout, and
variable names that map into the template's format groups. A format group is a
sequence of `x` characters (with optional decimal point) that determines width
and precision.

```{r}
# Two decimal places for mean, two for SD, with parentheses
f_str("xx.xx (xx.xx)", "mean", "sd")

# Integer count only
f_str("xxx", "n")

# Count with percentage
f_str("xx (xx.x%)", "n", "pct")
```

The number of `x` characters in the integer part sets the total width (including
leading spaces for alignment). Characters after the decimal point set decimal
precision. Everything between format groups is literal text.

For **descriptive statistics layers**, format strings are a named list and each name
becomes a row label. For **count and shift layers**, a single format string goes under
the reserved key `n_counts`:

```{r eval=FALSE}
# count layer: n (pct%) -- the default
list(n_counts = f_str("xx (xx.x%)", "n", "pct"))

# count layer: distinct subjects rather than records
list(n_counts = f_str("xx (xx.x%)", "distinct_n", "distinct_pct"))
```

Each layer type computes its own set of statistics, and `vignette("format_strings")`
carries the complete reference. Two more characters extend the template: `a` lets the
*data* determine the decimal width, and uppercase `X` closes the gap between a number
and its delimiter -- both covered in `vignette("precision_alignment")`.

Because the format and source variables are declared together, the package knows
exactly which numbers produced each cell -- this is the foundation of the
metadata and traceability system.

# Layer Types

## Descriptive Statistics Layers

Descriptive statistics layers summarize continuous variables. Without custom
format strings, the default produces six rows: n, Mean (SD), Median, Q1/Q3,
Min/Max, and Missing.

```{r}
spec <- tplyr_spec(
  cols = "TRT01P",
  layers = tplyr_layers(
    group_desc("AGE")
  )
)

result <- tplyr_build(spec, tplyr_adsl)
kable(result[, !grepl("^ord", names(result))])
```

### Multiple Target Variables

When you need to summarize more than one variable in the same layer -- for
example, Age and Average Daily Dose -- pass a vector of names to `target_var`.
Each variable gets its own block of rows, with the variable name as an
additional row label.

```{r}
spec <- tplyr_spec(
  cols = "TRT01P",
  layers = tplyr_layers(
    group_desc(
      c("AGE", "AVGDD"),
      settings = layer_settings(
        format_strings = list(
          "n"         = f_str("xxx", "n"),
          "Mean (SD)" = f_str("xx.x (xx.xx)", "mean", "sd"),
          "Min, Max"  = f_str("xx, xx", "min", "max")
        )
      )
    )
  )
)

result <- tplyr_build(spec, tplyr_adsl)
kable(result[, !grepl("^ord", names(result))])
```

## Count Layers

Count layers tabulate the frequency of categorical variable levels. By default,
they produce counts and percentages formatted as `"xx (xx.x%)"`:

```{r}
spec <- tplyr_spec(
  cols = "TRT01P",
  layers = tplyr_layers(
    group_count("RACE")
  )
)

result <- tplyr_build(spec, tplyr_adsl)
kable(result[, !grepl("^ord", names(result))])
```

### Total Rows

A total row sums the counts across all levels. Enable it with
`total_row = TRUE` in `layer_settings()`:

```{r}
spec <- tplyr_spec(
  cols = "TRT01P",
  layers = tplyr_layers(
    group_count(
      "RACE",
      settings = layer_settings(total_row = TRUE)
    )
  )
)

result <- tplyr_build(spec, tplyr_adsl)
kable(result[, !grepl("^ord", names(result))])
```

### Distinct Counting

In adverse event tables, you often need to count the number of _subjects_ who
experienced an event, not the number of event records. Use `distinct_by` to
specify the subject identifier. Because `ADAE` contains only subjects with
events, the denominator must come from the full population (`ADSL`), supplied
through `pop_data()` -- otherwise the percentages are computed against
subjects-with-events and come out too large (see `vignette("count")` and
`vignette("denom")`):

```{r}
spec <- tplyr_spec(
  cols = "TRTA",
  pop_data = pop_data(cols = c("TRTA" = "TRT01A")),
  layers = tplyr_layers(
    group_count(
      "AEBODSYS",
      settings = layer_settings(
        distinct_by = "USUBJID",
        format_strings = list(
          n_counts = f_str("xxx (xx.x%)", "distinct_n", "distinct_pct")
        )
      )
    )
  )
)

result <- tplyr_build(spec, tplyr_adae, pop_data = tplyr_adsl)
kable(result[, !grepl("^ord", names(result))])
```

### Nested Counts

Many clinical tables require hierarchical counts -- for example, adverse events
by body system and preferred term. Pass a vector of variable names to
`target_var`:

```{r}
spec <- tplyr_spec(
  cols = "TRTA",
  pop_data = pop_data(cols = c("TRTA" = "TRT01A")),
  layers = tplyr_layers(
    group_count(
      c("AEBODSYS", "AEDECOD"),
      settings = layer_settings(
        distinct_by = "USUBJID",
        format_strings = list(
          n_counts = f_str("xxx (xx.x%)", "distinct_n", "distinct_pct")
        )
      )
    )
  )
)

result <- tplyr_build(spec, tplyr_adae, pop_data = tplyr_adsl)
kable(head(result[, !grepl("^ord", names(result))], 15))
```

The outer level (body system) appears in `rowlabel1`, and the inner level
(preferred term) appears in `rowlabel2`. Outer rows contain aggregate
counts for the body system; inner rows contain per-term counts. Use
`collapse_row_labels(result, nest = TRUE)` to merge these into a single
indented column (see the Count Layer vignette for details).

## Shift Layers

Shift tables display a cross-tabulation of a baseline value against a
post-baseline value within each treatment arm. The `group_shift()` function
requires a named character vector with `row` and `column` elements:

```{r}
set.seed(12345)
shift_data <- data.frame(
  USUBJID = paste0("SUBJ", 1:30),
  TRTA = rep(c("Placebo", "Active"), each = 15),
  BNRIND = factor(
    sample(c("LOW", "NORMAL", "HIGH"), 30, replace = TRUE, prob = c(0.2, 0.6, 0.2)),
    levels = c("LOW", "NORMAL", "HIGH")
  ),
  ANRIND = factor(
    sample(c("LOW", "NORMAL", "HIGH"), 30, replace = TRUE, prob = c(0.15, 0.5, 0.35)),
    levels = c("LOW", "NORMAL", "HIGH")
  )
)

spec <- tplyr_spec(
  cols = "TRTA",
  layers = tplyr_layers(
    group_shift(
      c(row = "BNRIND", column = "ANRIND"),
      settings = layer_settings(
        format_strings = list(n_counts = f_str("xxx", "n"))
      )
    )
  )
)

result <- tplyr_build(spec, shift_data)
kable(result[, !grepl("^ord", names(result))])
```

In the output, `rowlabel1` contains the baseline (row) variable values, and the
result columns represent each combination of treatment arm and post-baseline
(column) level. Factor ordering is respected, so setting levels appropriately
(e.g., `c("LOW", "NORMAL", "HIGH")`) ensures the intended display order.

# Numeric Data

Formatted cells look good in a final report, but sometimes you need the raw
numbers -- for validation, further computation, or archiving. Every tplyr2 build
retains unformatted numeric data as an attribute. Use `tplyr_numeric_data()` to
retrieve it:

```{r}
spec <- tplyr_spec(
  cols = "TRT01P",
  layers = tplyr_layers(
    group_count("SEX"),
    group_desc(
      "AGE",
      settings = layer_settings(
        format_strings = list(
          "n" = f_str("xxx", "n"),
          "Mean (SD)" = f_str("xx.x (xx.xx)", "mean", "sd")
        )
      )
    )
  )
)

result <- tplyr_build(spec, tplyr_adsl)

# Raw numeric data for the count layer (layer 1)
kable(tplyr_numeric_data(result, layer = 1))
```

```{r}
# Raw numeric data for the desc layer (layer 2)
kable(tplyr_numeric_data(result, layer = 2))
```

This gives you the exact counts, percentages, means, and every other computed
statistic in raw numeric form, keyed by the same grouping variables used during
the build.

# Where to Go from Here

This vignette covers the fundamentals. Here are the topics covered in other
vignettes:

- `vignette("table")` -- Spec-level structure: column variables, table filters,
  total/custom groups, and population data.
- `vignette("count")` -- Count layers in depth: population-based denominators,
  distinct counts, nested summaries, stat columns, missing values.
- `vignette("desc")` -- Descriptive layers: custom summaries, auto-precision,
  stats-as-columns.
- `vignette("shift")` -- Shift layers, shift denominators, and the denominator
  row.
- `vignette("denom")` -- Denominator control: population data, header N,
  `denoms_by`, `denom_where`, single-proportion confidence intervals.
- `vignette("adverse-events")` -- A full adverse event table built end to end.
- `vignette("riskdiff")` and `vignette("binding-statistics")` -- Risk
  differences and association-test p-value columns.
- `vignette("sort")` -- Row ordering by frequency, factor levels, or a VARN
  companion.
- `vignette("format_strings")` -- The `f_str()` format-string system: the grammar,
  the complete statistic keyword reference per layer type, rounding, and
  missing-value handling.
- `vignette("precision_alignment")` -- Data-driven decimal precision (`a`/`A`) and
  parenthesis hugging (`X`/`A`).
- `vignette("display_conventions")` -- `<1%` and `>99%` percents, zero-count
  suppression, statistics as columns, missing rows.
- `vignette("post_processing")` -- Row masks, row label collapsing,
  conditional formatting, `as_display()`, and text wrapping.
- `vignette("metadata")` -- Cell-level metadata and traceability.
- `vignette("serialization")`, `vignette("ard")` -- Saving/loading specs and
  Analysis Results Data conversion.
- `vignette("analyze")` -- Custom analysis layers with user-defined functions.
- `vignette("options")` -- Package options (rounding, quantiles, precision).
- `vignette("migration")` -- Moving from Tplyr v1 to tplyr2.

# References

- PHUSE, *Analyses & Displays Associated with Demographics, Disposition, and
  Medications in Phase 2-4 Clinical Trials and Integrated Summary Documents*.
  Available from the PHUSE deliverables catalogue.
- [CDISC Analysis Data Model (ADaM)](https://www.cdisc.org/standards/foundational/adam)