--- title: "Working with Loaders" output: rmarkdown::html_vignette vignette: > %\VignetteIndexEntry{Working with Loaders} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} --- ```{r setup, include = FALSE} knitr::opts_chunk$set( collapse = TRUE, comment = "#>" ) library(radiatR) ``` ## Overview `radiatR` provides a flexible loader framework that turns tracking exports from a wide variety of systems into `Tracks` objects. The helpers cover three typical tasks: 1. Reading experiment manifests (`import_info()` and `import_tracks()`). 2. Ingesting trajectory tables with `read_tracks()` or `read_tracks_dir()`. 3. Extending the loader registry with `register_loader_dialect()` for custom formats. This vignette walks through each step using the bundled millipede example data and finishes with a user-defined loader for a custom format. ## Loading a Real Experiment The package ships the trials from a *Cylindroiulus punctatus* (millipede) visual orientation experiment. Tracks are stored as paired tab-separated text files: `_point01.txt` holds two landmark rows (the origin + target location on the circumference); `_point02.txt` holds the full per-frame xy trajectory. ### Step 1 — discover files `import_tracks()` scans a directory for `_point01.txt` / `_point02.txt` pairs and returns a tibble of basenames. ```{r import-tracks} track_dir <- system.file("extdata", "tracks", package = "radiatR") file_tbl <- import_tracks(track_dir) head(file_tbl) ``` ### Step 2 — read the manifest `import_info()` parses the trial manifest CSV. Use `cond_cols` to generate a compound condition column from multiple metadata fields. ```{r import-info} manifest_path <- system.file("extdata", "millipede_trials.csv", package = "radiatR") manifest <- import_info(manifest_path, cond_cols = c("type", "arc")) head(manifest) ``` ### Step 3 — join metadata `load_tracks()` merges the manifest into `file_tbl`, matching on the `file` column. The resulting tibble carries arc, type, obstacle, and id alongside each file path. ```{r load-tracks} file_tbl <- load_tracks(file_tbl, manifest, track_dir) head(file_tbl[, c("basename", "arc", "type", "obstacle", "id")]) ``` ### Step 4 — extract and normalise trajectories `get_all_object_pos()` reads each file pair, uses the landmark rows to establish the unit-circle geometry, and returns a `Tracks` with normalised unit-circle coordinates. ```{r get-all} ts <- suppressWarnings(get_all_object_pos(file_tbl = file_tbl, track_dir = track_dir)) ts ``` ## dtrack format The bundled example data was tracked with **dtrack** (), a desktop tracking tool for video recordings of freely-moving animals. The data are from: > Kirwan J.D. & Nilsson D.-E. (2019). *A millipede compound eye mediating > low-resolution vision.* Vision Research 165, 36–44. > dtrack exports tab-separated text files with columns `frame`, `x`, `y`, and a fourth confidence/flag column. Use `dtrack_read()` to load a single trajectory file directly: ```{r dtrack-single} track_file <- system.file( "extdata", "tracks", "10_10_point02.txt", package = "radiatR" ) ts_raw <- dtrack_read(track_file) head(ts_raw@data[, c("id", "frame", "x", "y")]) ``` The `_point01` / `_point02` role split used in the bundled data — landmarks in one file, trajectory in the other — is specific to this experiment and is not a general dtrack convention. ## Tracking-tool format support `radiatR` reads tabular (CSV/TSV) exports from many tracking tools through named *dialects*. Each dialect maps a tool's column conventions onto the `Tracks` model of one position per individual per frame. Pass the dialect name to `read_tracks(path, dialect = "name")`; tool-specific options go through `dialect_args`. The table below summarises what is and is not currently handled. Two limitations are general: - **Tabular text only.** Dialects read CSV/TSV (and, where the optional packages are installed, JSON/XML/Parquet). Native binary formats — such as NumPy `.npz` or MATLAB `.mat` — are not read except where noted (`ctrax` reads `.mat` via the `R.matlab` package). - **Position, not posture.** `radiatR` models a single position per individual per frame. Per-frame *posture* data (variable-length midline or outline polylines) is outside this model. Where a tool tracks discrete body keypoints, those can be loaded as extra columns and turned into a body-axis heading with the `bodypart_axis` rule or `pose_to_headings()`; dense posture polylines are not ingested. | Dialect | Handled | Not handled | |---|---|---| | `trex` | Positional CSV: plain `X`/`Y`, the `#wcentroid` / `#centroid` / `#pcentroid` centroid variants, and TRex's `(cm)` / `(px)` unit annotations in headers. Per-individual `_fishN` files (id from filename) and aggregated files with an id column. | Native `.npz` export; posture export (`midline_points`, `outline_points`). | | `deeplabcut` / `deeplabcut_multiheader` | Multi-bodypart CSV; single bodypart, or a likelihood-weighted centroid of several; three-row scorer/bodypart/coord header. Bodypart columns are preserved for `bodypart_axis`. | Native HDF5 (`.h5`) export. | | `sleap` | Analysis CSV: `.x` / `.y` / `.score`; multi-node centroid; `track` and `frame_idx`. | Native `.slp` / HDF5 export. | | `ethovision` | Centre position, and multi-zone exports (nose, tail, etc.); zone centroid or single-zone selection. | Native `.xlsx` (export to CSV first). | | `anymaze` | Centre position, optional nose/tail zones, units row. | — | | `trackmate` | `TRACK_ID` / `FRAME` / `POSITION_X` / `POSITION_Y` CSV. | Native TrackMate `.xml`. | | `ctrax` | `.mat` file with the `trx` struct (centroid + ellipse `theta`/`a`/`b`); needs `R.matlab`. | — | | `idtrackerai_wide`, `toxtrac`, `boris_xy`, `tracktor`, `dtrack` | Their standard CSV/TSV column layouts. | Tool-specific binary or session formats. | When a tool's export is not directly supported, two escape hatches remain: convert to a tidy CSV and use `read_tracks()` with an explicit `mapping`, or register a custom dialect (see *Registering a Custom Dialect* below). ## Bundled single-file examples The package ships small real exports from three tools so the dialects can be tried without any external data. Each loads with one `read_tracks()` call, pointing the `dialect` argument at the matching tool. **DeepLabCut** — a three-row scorer/bodypart/coord header; by default the position is the likelihood-weighted centroid of all bodyparts, and each bodypart's coordinates are retained for use with the `bodypart_axis` rule. ```{r dlc-example} dlc_path <- system.file("extdata", "dlc_CollectedData_Pranav.csv", package = "radiatR") ts_dlc <- read_tracks(dlc_path, dialect = "deeplabcut_multiheader") ts_dlc ``` **SLEAP** — an analysis CSV with `.x` / `.y` / `.score` columns, `track` as the individual and `frame_idx` as time. ```{r sleap-example} sleap_path <- system.file("extdata", "sleap_example.csv", package = "radiatR") ts_sleap <- read_tracks(sleap_path, dialect = "sleap") ts_sleap ``` **Tracktor** — a tidy `frame` / `pos_x` / `pos_y` CSV (one individual here). ```{r tracktor-example} tracktor_path <- system.file("extdata", "tracktor_example.csv", package = "radiatR") ts_tracktor <- read_tracks(tracktor_path, dialect = "tracktor") ts_tracktor ``` The bundled dtrack example is larger and is shown in full in the *Loading a Real Experiment* section above. ## Reading Tabular Trajectories For data already in a data frame or CSV, `read_tracks()` is the central entry point. ```{r basic-load} example_df <- data.frame( id = rep(c("A", "B"), each = 4), time = rep(seq(0, 3), times = 2), x = c(1, 1.2, 1.4, 1.5, -0.2, -0.1, 0, 0.15), y = c(0, 0.1, 0.3, 0.4, 1.0, 1.1, 1.3, 1.5) ) ts_df <- read_tracks(example_df, mapping = list(id = "id", time = "time", x = "x", y = "y")) ts_df ``` To batch a directory of CSV files, `read_tracks_dir()` calls `read_tracks()` for each match and row-binds the result. ```{r dir-load} csv_dir <- tempfile() dir.create(csv_dir) write.csv(example_df, file.path(csv_dir, "trial01.csv"), row.names = FALSE) write.csv(transform(example_df, id = paste0(id, "_2")), file.path(csv_dir, "trial02.csv"), row.names = FALSE) dset <- read_tracks_dir(csv_dir, pattern = "\\.csv$") length(dset) unlink(csv_dir, recursive = TRUE) ``` ## Manifest Helpers (Synthetic Example) For a quick demonstration without real files, `import_info()` works on any manifest CSV, and `import_tracks()` works on any directory following the `_point01` / `_point02` naming convention. ```{r manifest} manifest_path <- tempfile(fileext = ".csv") writeLines( c("file,type, arc", "trial01.csv, baseline,90", "trial02.csv,stimulus,135"), manifest_path ) import_info(manifest_path, cond_cols = c("type", "arc")) ``` ## Registering a Custom Dialect When your exporter uses a non-standard format, register a custom function. A dialect receives either a data frame or a file path and must return a data frame with at least `id`, `time`, and either `angle` or `(x, y)`. Below we define a loader for a hypothetical JSON file where each track is stored under an ID key with arrays of `timestamp`, `px`, and `py`. ```{r custom-loader} fake_json <- tempfile(fileext = ".json") jsonlite::write_json( list( tracks = list( A = list(timestamp = c(0, 1, 2), px = c(0, 1, 2), py = c(0, 0.2, 0.5)), B = list(timestamp = c(0, 1, 2), px = c(1, 1.3, 1.5), py = c(0, -0.1, -0.2)) ) ), fake_json, auto_unbox = TRUE ) json_tracks_fn <- function(x) { dat <- if (is.character(x) && file.exists(x)) jsonlite::fromJSON(x) else x entries <- purrr::imap(dat$tracks, function(track, id) { data.frame( id = id, time = track$timestamp, x = track$px, y = track$py, stringsAsFactors = FALSE ) }) do.call(rbind, entries) } register_loader_dialect("json_tracks", json_tracks_fn) ``` After registration, call the dialect function directly on the file to get a tidy data frame, then pass it to `read_tracks()`. (For standard tabular formats that R's CSV/TSV readers handle natively, `read_tracks(path, dialect = "name")` works directly—but non-tabular formats such as JSON require this two-step approach.) ```{r use-custom} ts_json <- read_tracks(json_tracks_fn(fake_json)) ts_json ``` Dialects live in-memory for the current session. To persist them across projects, place the registration call in a package or in an `.Rprofile` that you source before using `radiatR`.