---
title: "Working with Loaders"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Working with Loaders}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r setup, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)
library(radiatR)
```

## Overview

`radiatR` provides a flexible loader framework that turns tracking exports
from a wide variety of systems into `Tracks` objects.  The helpers cover
three typical tasks:

1. Reading experiment manifests (`import_info()` and `import_tracks()`).
2. Ingesting trajectory tables with `read_tracks()` or
   `read_tracks_dir()`.
3. Extending the loader registry with `register_loader_dialect()` for
   custom formats.

This vignette walks through each step using the bundled millipede example
data and finishes with a user-defined loader for a custom format.

## Loading a Real Experiment

The package ships the trials from a *Cylindroiulus punctatus* (millipede)
visual orientation experiment. Tracks are stored as paired tab-separated text
files: `_point01.txt` holds two landmark rows (the origin + target location
on the circumference); `_point02.txt` holds the full per-frame xy trajectory.

### Step 1 — discover files

`import_tracks()` scans a directory for `_point01.txt` / `_point02.txt` pairs
and returns a tibble of basenames.

```{r import-tracks}
track_dir <- system.file("extdata", "tracks", package = "radiatR")
file_tbl  <- import_tracks(track_dir)
head(file_tbl)
```

### Step 2 — read the manifest

`import_info()` parses the trial manifest CSV.  Use `cond_cols` to generate
a compound condition column from multiple metadata fields.

```{r import-info}
manifest_path <- system.file("extdata", "millipede_trials.csv", package = "radiatR")
manifest <- import_info(manifest_path, cond_cols = c("type", "arc"))
head(manifest)
```

### Step 3 — join metadata

`load_tracks()` merges the manifest into `file_tbl`, matching on the `file`
column. The resulting tibble carries arc, type, obstacle, and id alongside
each file path.

```{r load-tracks}
file_tbl <- load_tracks(file_tbl, manifest, track_dir)
head(file_tbl[, c("basename", "arc", "type", "obstacle", "id")])
```

### Step 4 — extract and normalise trajectories

`get_all_object_pos()` reads each file pair, uses the landmark rows to
establish the unit-circle geometry, and returns a `Tracks` with normalised
unit-circle coordinates.

```{r get-all}
ts <- suppressWarnings(get_all_object_pos(file_tbl = file_tbl, track_dir = track_dir))
ts
```

## dtrack format

The bundled example data was tracked with **dtrack**
(<https://bitbucket.org/jochensmolka/dtrack>), a desktop tracking tool for
video recordings of freely-moving animals. The data are from:

> Kirwan J.D. & Nilsson D.-E. (2019). *A millipede compound eye mediating
> low-resolution vision.* Vision Research 165, 36–44.
> <https://doi.org/10.1016/j.visres.2019.09.003>

dtrack exports tab-separated text files with columns `frame`, `x`, `y`, and a
fourth confidence/flag column. Use `dtrack_read()` to load a single trajectory
file directly:

```{r dtrack-single}
track_file <- system.file(
  "extdata", "tracks", "10_10_point02.txt",
  package = "radiatR"
)
ts_raw <- dtrack_read(track_file)
head(ts_raw@data[, c("id", "frame", "x", "y")])
```

The `_point01` / `_point02` role split used in the bundled data — landmarks
in one file, trajectory in the other — is specific to this experiment and is
not a general dtrack convention.

## Tracking-tool format support

`radiatR` reads tabular (CSV/TSV) exports from many tracking tools through
named *dialects*. Each dialect maps a tool's column conventions onto the
`Tracks` model of one position per individual per frame. Pass the dialect
name to `read_tracks(path, dialect = "name")`; tool-specific options go
through `dialect_args`.

The table below summarises what is and is not currently handled. Two
limitations are general:

- **Tabular text only.** Dialects read CSV/TSV (and, where the optional
  packages are installed, JSON/XML/Parquet). Native binary formats — such
  as NumPy `.npz` or MATLAB `.mat` — are not read except where noted
  (`ctrax` reads `.mat` via the `R.matlab` package).
- **Position, not posture.** `radiatR` models a single position per
  individual per frame. Per-frame *posture* data (variable-length midline
  or outline polylines) is outside this model. Where a tool tracks discrete
  body keypoints, those can be loaded as extra columns and turned into a
  body-axis heading with the `bodypart_axis` rule or `pose_to_headings()`;
  dense posture polylines are not ingested.

| Dialect | Handled | Not handled |
|---|---|---|
| `trex` | Positional CSV: plain `X`/`Y`, the `#wcentroid` / `#centroid` / `#pcentroid` centroid variants, and TRex's `(cm)` / `(px)` unit annotations in headers. Per-individual `_fishN` files (id from filename) and aggregated files with an id column. | Native `.npz` export; posture export (`midline_points`, `outline_points`). |
| `deeplabcut` / `deeplabcut_multiheader` | Multi-bodypart CSV; single bodypart, or a likelihood-weighted centroid of several; three-row scorer/bodypart/coord header. Bodypart columns are preserved for `bodypart_axis`. | Native HDF5 (`.h5`) export. |
| `sleap` | Analysis CSV: `<node>.x` / `.y` / `.score`; multi-node centroid; `track` and `frame_idx`. | Native `.slp` / HDF5 export. |
| `ethovision` | Centre position, and multi-zone exports (nose, tail, etc.); zone centroid or single-zone selection. | Native `.xlsx` (export to CSV first). |
| `anymaze` | Centre position, optional nose/tail zones, units row. | — |
| `trackmate` | `TRACK_ID` / `FRAME` / `POSITION_X` / `POSITION_Y` CSV. | Native TrackMate `.xml`. |
| `ctrax` | `.mat` file with the `trx` struct (centroid + ellipse `theta`/`a`/`b`); needs `R.matlab`. | — |
| `idtrackerai_wide`, `toxtrac`, `boris_xy`, `tracktor`, `dtrack` | Their standard CSV/TSV column layouts. | Tool-specific binary or session formats. |

When a tool's export is not directly supported, two escape hatches remain:
convert to a tidy CSV and use `read_tracks()` with an explicit `mapping`,
or register a custom dialect (see *Registering a Custom Dialect* below).

## Bundled single-file examples

The package ships small real exports from three tools so the dialects can be
tried without any external data. Each loads with one `read_tracks()` call,
pointing the `dialect` argument at the matching tool.

**DeepLabCut** — a three-row scorer/bodypart/coord header; by default the
position is the likelihood-weighted centroid of all bodyparts, and each
bodypart's coordinates are retained for use with the `bodypart_axis` rule.

```{r dlc-example}
dlc_path <- system.file("extdata", "dlc_CollectedData_Pranav.csv",
                        package = "radiatR")
ts_dlc <- read_tracks(dlc_path, dialect = "deeplabcut_multiheader")
ts_dlc
```

**SLEAP** — an analysis CSV with `<node>.x` / `.y` / `.score` columns,
`track` as the individual and `frame_idx` as time.

```{r sleap-example}
sleap_path <- system.file("extdata", "sleap_example.csv", package = "radiatR")
ts_sleap <- read_tracks(sleap_path, dialect = "sleap")
ts_sleap
```

**Tracktor** — a tidy `frame` / `pos_x` / `pos_y` CSV (one individual here).

```{r tracktor-example}
tracktor_path <- system.file("extdata", "tracktor_example.csv",
                            package = "radiatR")
ts_tracktor <- read_tracks(tracktor_path, dialect = "tracktor")
ts_tracktor
```

The bundled dtrack example is larger and is shown in full in the
*Loading a Real Experiment* section above.

## Reading Tabular Trajectories

For data already in a data frame or CSV, `read_tracks()` is the central
entry point.

```{r basic-load}
example_df <- data.frame(
  id   = rep(c("A", "B"), each = 4),
  time = rep(seq(0, 3), times = 2),
  x    = c(1, 1.2, 1.4, 1.5, -0.2, -0.1, 0, 0.15),
  y    = c(0, 0.1, 0.3, 0.4, 1.0, 1.1, 1.3, 1.5)
)

ts_df <- read_tracks(example_df, mapping = list(id = "id", time = "time", x = "x", y = "y"))
ts_df
```

To batch a directory of CSV files, `read_tracks_dir()` calls `read_tracks()`
for each match and row-binds the result.

```{r dir-load}
csv_dir <- tempfile()
dir.create(csv_dir)

write.csv(example_df, file.path(csv_dir, "trial01.csv"), row.names = FALSE)
write.csv(transform(example_df, id = paste0(id, "_2")),
          file.path(csv_dir, "trial02.csv"), row.names = FALSE)

dset <- read_tracks_dir(csv_dir, pattern = "\\.csv$")
length(dset)

unlink(csv_dir, recursive = TRUE)
```

## Manifest Helpers (Synthetic Example)

For a quick demonstration without real files, `import_info()` works on any
manifest CSV, and `import_tracks()` works on any directory following the
`_point01` / `_point02` naming convention.

```{r manifest}
manifest_path <- tempfile(fileext = ".csv")
writeLines(
  c("file,type, arc", "trial01.csv, baseline,90", "trial02.csv,stimulus,135"),
  manifest_path
)
import_info(manifest_path, cond_cols = c("type", "arc"))
```

## Registering a Custom Dialect

When your exporter uses a non-standard format, register a custom function.
A dialect receives either a data frame or a file path and must return a data
frame with at least `id`, `time`, and either `angle` or `(x, y)`.

Below we define a loader for a hypothetical JSON file where each track is
stored under an ID key with arrays of `timestamp`, `px`, and `py`.

```{r custom-loader}
fake_json <- tempfile(fileext = ".json")
jsonlite::write_json(
  list(
    tracks = list(
      A = list(timestamp = c(0, 1, 2), px = c(0, 1, 2), py = c(0, 0.2, 0.5)),
      B = list(timestamp = c(0, 1, 2), px = c(1, 1.3, 1.5), py = c(0, -0.1, -0.2))
    )
  ),
  fake_json,
  auto_unbox = TRUE
)

json_tracks_fn <- function(x) {
  dat <- if (is.character(x) && file.exists(x)) jsonlite::fromJSON(x) else x
  entries <- purrr::imap(dat$tracks, function(track, id) {
    data.frame(
      id   = id,
      time = track$timestamp,
      x    = track$px,
      y    = track$py,
      stringsAsFactors = FALSE
    )
  })
  do.call(rbind, entries)
}
register_loader_dialect("json_tracks", json_tracks_fn)
```

After registration, call the dialect function directly on the file to get a
tidy data frame, then pass it to `read_tracks()`. (For standard tabular
formats that R's CSV/TSV readers handle natively, `read_tracks(path,
dialect = "name")` works directly—but non-tabular formats such as JSON
require this two-step approach.)

```{r use-custom}
ts_json <- read_tracks(json_tracks_fn(fake_json))
ts_json
```

Dialects live in-memory for the current session. To persist them across
projects, place the registration call in a package or in an `.Rprofile` that
you source before using `radiatR`.
