---
title: "Data Formats"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Data Formats}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

This vignette illustrates cloud-native access of across a range of common formats, including netcdf, Zarr, hdf5, hdf4, and COGs.


```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  fig.path="img/",
  message = FALSE,
  warning = FALSE
)
```


(Right now these are somewhat generic examples, not necessarily based on NASA's EarthData sources and therefore not requiring authentication, but the workflow should be identical after calling `edl_set_token()`)


```{r setup}
library(stars)
library(terra)
```

## netcdf, 1 GB file 

Cloud-read data from a public AWS S3 bucket, over https:

```{r}
url <- paste0("https://era5-pds.s3.amazonaws.com",
              "/1979/01/data/air_pressure_at_mean_sea_level.nc")
r <- terra::rast(url, vsi=TRUE)
r
```

Cloud read of the same data, over S3 protocol:

```{r}
Sys.setenv("AWS_NO_SIGN_REQUEST"=TRUE)
s3uri <- "s3://era5-pds/1979/01/data/air_pressure_at_mean_sea_level.nc"
r <- terra::rast(s3uri)
Sys.unsetenv("AWS_NO_SIGN_REQUEST")
```

## Zarr, 31 TB

Cloud read of a 31 Terrabyte Zarr archive (S3, public https)

```{r}
url <- "https://mur-sst.s3.us-west-2.amazonaws.com/zarr-v1"
prefixes <- 'ZARR:\"/vsicurl/'
slice <- '\":/analysed_sst:0"'
addr <- paste0(prefixes, url, slice)
y = terra::rast(addr)
```


```{r}
library(tmap)
data(World)

india <-st_bbox( World$geometry[World$name == "India"] )
```


```{r zarr_sst}
st_as_stars(y) |>
  st_crop(india) |>
  plot()
```
