radiatR provides a flexible loader framework that turns
tracking exports from a wide variety of systems into Tracks
objects. The helpers cover three typical tasks:
import_info() and
import_tracks()).read_tracks() or
read_tracks_dir().register_loader_dialect() for custom formats.This vignette walks through each step using the bundled millipede example data and finishes with a user-defined loader for a custom format.
The package ships the trials from a Cylindroiulus punctatus
(millipede) visual orientation experiment. Tracks are stored as paired
tab-separated text files: _point01.txt holds two landmark
rows (the origin + target location on the circumference);
_point02.txt holds the full per-frame xy trajectory.
import_tracks() scans a directory for
_point01.txt / _point02.txt pairs and returns
a tibble of basenames. The example files ship as one archive, so unpack
it into a temporary directory first.
untar(system.file("extdata", "tracks.tar.gz", package = "radiatR"),
exdir = tempdir())
track_dir <- file.path(tempdir(), "tracks")
file_tbl <- import_tracks(track_dir)
head(file_tbl)
#> # A tibble: 6 × 3
#> basename landmark track
#> <chr> <chr> <chr>
#> 1 10_1 10_1_point01.txt 10_1_point02.txt
#> 2 10_10 10_10_point01.txt 10_10_point02.txt
#> 3 10_11 10_11_point01.txt 10_11_point02.txt
#> 4 10_12 10_12_point01.txt 10_12_point02.txt
#> 5 10_13 10_13_point01.txt 10_13_point02.txt
#> 6 10_14 10_14_point01.txt 10_14_point02.txtimport_info() parses the trial manifest CSV. Use
cond_cols to generate a compound condition column from
multiple metadata fields.
manifest_path <- system.file("extdata", "millipede_trials.csv", package = "radiatR")
manifest <- import_info(manifest_path, cond_cols = c("type", "arc"))
head(manifest)
#> file arc id stimulus_period recorded_radian recorded_degree type obstacle
#> 1 con_1 0 1 0 -1.473503 -84.42550 control none
#> 2 con_2 0 2 0 1.295963 74.25323 control none
#> 3 con_3 0 3 0 -0.314510 -18.02010 control none
#> 4 con_5 0 5 0 -2.070025 -118.60370 control none
#> 5 con_6 0 6 0 -2.907256 -166.57352 control none
#> 6 con_7 0 7 0 -1.601138 -91.73845 control none
#> cond
#> 1 control_0
#> 2 control_0
#> 3 control_0
#> 4 control_0
#> 5 control_0
#> 6 control_0get_all_object_pos() reads each file pair, uses the
landmark rows to establish the unit-circle geometry, and returns a
Tracks with normalised unit-circle coordinates. Passing
manifest joins its columns (arc, type, obstacle, id, …)
onto every trial, matching on the file column; use
manifest_cols to rename or restrict them.
ts <- suppressWarnings(get_all_object_pos(file_tbl = file_tbl, track_dir = track_dir,
manifest = manifest))
#> 478 points across 106 trials exceeded the unit circle boundary (radius > 1); coordinates left unscaled.
ts
#> Tracks: 235 trajectories, 44331 observations
#> Columns: id='trial_id', time='frame', angle='rel_theta' (radians), x='trans_x', y='trans_y', rel_x='rel_x', rel_y='rel_y'
#> Transform steps: unit_circle_mapping
#> # A tibble: 6 × 10
#> trial_id frame x y trans_x trans_y rel_theta video order vid_ord
#> <chr> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <chr> <chr> <chr>
#> 1 10_1_1 1 456. 372. 0.00511 0 6.21 10_1 1 10_1_1
#> 2 10_1_1 2 458. 370. 0.0128 0.00770 0.464 10_1 1 10_1_1
#> 3 10_1_1 3 456. 370. 0.00511 0.00770 0.908 10_1 1 10_1_1
#> 4 10_1_1 4 454. 370. 0 0.00770 1.49 10_1 1 10_1_1
#> 5 10_1_1 5 456. 377. 0.00511 -0.0153 4.96 10_1 1 10_1_1
#> 6 10_1_1 6 459. 382. 0.0154 -0.0307 5.10 10_1 1 10_1_1
head(ts@meta$trial_limits[, c("video", "arc", "type", "obstacle")])
#> # A tibble: 6 × 4
#> video arc type obstacle
#> <chr> <int> <chr> <chr>
#> 1 10_1 10 stimulus none
#> 2 10_10 10 stimulus none
#> 3 10_11 10 stimulus none
#> 4 10_12 10 stimulus none
#> 5 10_13 10 stimulus none
#> 6 10_14 10 stimulus noneThe bundled example data was tracked with dtrack (https://bitbucket.org/jochensmolka/dtrack), a desktop tracking tool for video recordings of freely-moving animals. The data are from:
Kirwan J.D. & Nilsson D.-E. (2019). A millipede compound eye mediating low-resolution vision. Vision Research 165, 36–44. https://doi.org/10.1016/j.visres.2019.09.003
dtrack exports headerless, tab-separated text files with columns
frame, x, y, and a fourth
confidence/flag column. radiatR has no built-in dtrack
dialect: the Registering a Custom Dialect section below writes
one in a few lines and uses it to read these files.
The _point01 / _point02 role split used in
the bundled data — landmarks in one file, trajectory in the other — is
specific to this experiment and is not a general dtrack convention.
radiatR reads tabular (CSV/TSV) exports from many
tracking tools through named dialects. Each dialect maps a
tool’s column conventions onto the Tracks model of one
position per individual per frame. Pass the dialect name to
read_tracks(path, dialect = "name"); tool-specific options
go through dialect_args.
The table below summarises what is and is not currently handled. Two limitations are general:
.npz or
MATLAB .mat — are not read.radiatR models
a single position per individual per frame. Per-frame posture
data (variable-length midline or outline polylines) is outside this
model. Where a tool tracks discrete body keypoints, those can be loaded
as extra columns and turned into a body-axis heading with the
bodypart_axis rule or pose_to_headings();
dense posture polylines are not ingested.| Dialect | Handled | Not handled |
|---|---|---|
trex |
Positional CSV: plain X/Y, the
#wcentroid / #centroid /
#pcentroid centroid variants, and TRex’s (cm)
/ (px) unit annotations in headers. Per-individual
_fishN files (id from filename) and aggregated files with
an id column. A file with a track column (Track,
TRACK_ID) is refused unless id_col names the
id. |
Native .npz export; posture export
(midline_points, outline_points). |
deeplabcut |
The predictions CSV: header rows scorer /
bodyparts / coords, with
individuals for multi-animal projects (one track per
individual). A single bodypart, or a likelihood-weighted centroid of
several; likelihood_min masks points and the count is kept
in the metadata. Bodypart columns are preserved for
bodypart_axis. |
Native HDF5 (.h5) export; labelled training data
(CollectedData_*.csv); 3-D output. |
sleap |
Analysis CSV: <node>.x / .y /
.score; a single node or a score-weighted centroid;
track and frame_idx (a table without
frame_idx is refused unless time_col is
given); score_min masks points and the count is kept in the
metadata. |
Native .slp / HDF5 export. |
ethovision |
The text export: a first line giving the number of header lines, the
trial details, a names row and a units row. One track per file,
Trial time in seconds. The centre point, or the body points
named in bodypart (nose, tail)
and their mean; a sample EthoVision marks - has no position
and is counted in the metadata. The header block and units are kept in
@meta$source_meta. |
Native .xlsx (export as text). A file cut off mid-row
(delete the partial last line). |
anymaze |
The CSV export: Time as a duration
(0:01:02.500), Centre position X /
Centre position Y. A sample with an empty position is
counted in the metadata. Pass the file path, or read Time
as text
(readr::read_csv(f, col_types = readr::cols(Time = "c"))):
readr’s own time parsing drops the fraction of a second. |
Head and tail positions; the In <zone> occupancy
columns. |
trackmate |
Spot tables with TRACK_ID / FRAME /
POSITION_T / POSITION_X /
POSITION_Y, including the three extra header rows of
version 7. Spots in no track (TRACK_ID None)
stop the load unless
dialect_args = list(unlinked = "drop"). |
Native TrackMate .xml. |
idtrackerai_wide |
trajectories.csv: one x<id> /
y<id> pair per animal, seconds as time,
nan for missing positions. |
.npy trajectory files. |
toxtrac |
Tracking_RealSpace.txt (mm, seconds) and the headerless
Tracking.txt (pixels, frames; 6 or 11 columns). The id is
<arena>_<track>. |
The statistics and project files. |
Built-in dialects are kept only for tracking software that is widely
used and still developed. When a tool’s export is not directly
supported, two escape hatches remain: convert to a tidy CSV and use
read_tracks() with an explicit mapping, or
register a custom dialect (see Registering a Custom Dialect
below).
The package ships small real exports from two tools so the dialects
can be tried without any external data. Each loads with one
read_tracks() call, pointing the dialect
argument at the matching tool.
DeepLabCut — the predictions CSV written when a
video is analysed (header rows scorer,
bodyparts, coords, plus
individuals for multi-animal projects). By default the
position is the likelihood-weighted centroid of all bodyparts, the time
is the frame index, and each bodypart’s coordinates are kept for the
bodypart_axis rule.
dlc_path <- system.file("extdata", "dlc_single_wasp_predictions.csv",
package = "radiatR")
ts_dlc <- read_tracks(dlc_path, dialect = "deeplabcut", fps = 40)
ts_dlc
#> Tracks: 1 trajectories, 300 observations
#> Columns: id='id', time='time', angle='angle' (radians), x='x', y='y'
#> id time x y head_x head_y stinger_x stinger_y angle
#> 1 1 0 1084.859 421.3867 1086.238 421.5013 1083.866 421.3042 0.3704886
#> 2 1 1 1084.742 421.5025 1086.020 421.5443 1083.370 421.4577 0.3706179
#> 3 1 2 1084.573 421.7333 1085.545 421.8074 1082.950 421.6096 0.3708553
#> 4 1 3 1083.631 423.8910 1085.084 422.5898 1081.628 425.6838 0.3728769
#> 5 1 4 1082.897 424.1358 1085.193 422.4883 1078.615 427.2081 0.3733029
#> 6 1 5 1082.439 424.0955 1084.933 422.5358 1077.953 426.9004 0.3734143A likelihood threshold masks the points below it. A frame with a
masked point has no position, so it is dropped only when
on_invalid = "drop" says so, and what was masked is
reported and kept:
ts_dlc_sure <- read_tracks(dlc_path, dialect = "deeplabcut", fps = 40,
dialect_args = list(likelihood_min = 0.9),
on_invalid = "drop")
#> deeplabcut: 104 of 300 row(s) have no position (threshold 0.9, partial = "na").
#> Masked points: head 104, stinger 57.
#> Warning: Dropped 104 row(s) with non-finite coordinates.
ts_dlc_sure@meta$attrition
#> source point masked frames frames_no_position
#> 1 dlc_single_wasp_predictions.csv head 104 300 104
#> 2 dlc_single_wasp_predictions.csv stinger 57 300 104
#> score_min partial
#> 1 0.9 na
#> 2 0.9 naSLEAP — an analysis CSV with
<node>.x / .y / .score
columns, track as the individual and frame_idx
as time (frames). The score_min and partial
arguments work as likelihood_min and partial
do for DeepLabCut.
sleap_path <- system.file("extdata", "sleap_three_mice.analysis.csv",
package = "radiatR")
ts_sleap <- read_tracks(sleap_path, dialect = "sleap")
ts_sleap
#> Tracks: 3 trajectories, 60 observations
#> Columns: id='id', time='time', angle='angle' (radians), x='x', y='y'
#> id time x y centroid_x centroid_y angle
#> 1 AEON3B_NTP 0 773.1893 1061.692 773.1893 1061.692 0.9413537
#> 4 AEON3B_NTP 1 773.2671 1061.651 773.2671 1061.651 0.9412873
#> 7 AEON3B_NTP 2 773.3159 1061.604 773.3159 1061.604 0.9412362
#> 10 AEON3B_NTP 3 773.2766 1061.557 773.2766 1061.557 0.9412393
#> 13 AEON3B_NTP 4 773.1517 1061.586 773.1517 1061.586 0.9413291
#> 16 AEON3B_NTP 5 771.1488 1061.770 771.1488 1061.770 0.9426457The bundled dtrack example is larger and is shown in full in the Loading a Real Experiment section above.
For data already in a data frame or CSV, read_tracks()
is the central entry point.
example_df <- data.frame(
id = rep(c("A", "B"), each = 4),
time = rep(seq(0, 3), times = 2),
x = c(1, 1.2, 1.4, 1.5, -0.2, -0.1, 0, 0.15),
y = c(0, 0.1, 0.3, 0.4, 1.0, 1.1, 1.3, 1.5)
)
ts_df <- read_tracks(example_df, mapping = list(id = "id", time = "time", x = "x", y = "y"))
ts_df
#> Tracks: 2 trajectories, 8 observations
#> Columns: id='id', time='time', angle='angle' (radians), x='x', y='y'
#> id time x y angle
#> 1 A 0 1.0 0.0 0.00000000
#> 2 A 1 1.2 0.1 0.08314123
#> 3 A 2 1.4 0.3 0.21109333
#> 4 A 3 1.5 0.4 0.26060239
#> 5 B 0 -0.2 1.0 1.76819189
#> 6 B 1 -0.1 1.1 1.66145621To batch a directory of CSV files, read_tracks_dir()
calls read_tracks() for each match and row-binds the
result.
csv_dir <- tempfile()
dir.create(csv_dir)
write.csv(example_df, file.path(csv_dir, "trial01.csv"), row.names = FALSE)
write.csv(transform(example_df, id = paste0(id, "_2")),
file.path(csv_dir, "trial02.csv"), row.names = FALSE)
dset <- read_tracks_dir(csv_dir, pattern = "\\.csv$")
length(dset)
#> [1] 4
unlink(csv_dir, recursive = TRUE)For a quick demonstration without real files,
import_info() works on any manifest CSV, and
import_tracks() works on any directory following the
_point01 / _point02 naming convention.
manifest_path <- tempfile(fileext = ".csv")
writeLines(
c("file,type, arc", "trial01.csv, baseline,90", "trial02.csv,stimulus,135"),
manifest_path
)
import_info(manifest_path, cond_cols = c("type", "arc"))
#> file type arc cond
#> 1 trial01.csv baseline 90 baseline_90
#> 2 trial02.csv stimulus 135 stimulus_135When no built-in dialect fits your exporter, register your own
loader: a function that receives what was passed to
read_tracks() (a file path or a data frame) and returns a
data frame with x and y (or
angle), and normally id and time.
The dialect is the only reader of the file, so it can handle layouts the
generic reader cannot: no header row, several header rows, or a format
that is not a table at all.
The bundled millipede tracks were made with dtrack, which writes a
headerless, tab-separated file of frame, x,
y and a flag column. A loader for it:
dtrack_loader <- function(x) {
# colClasses = "numeric" makes a malformed cell an error, not a silent NA
d <- utils::read.delim(x, header = FALSE, colClasses = "numeric")
if (ncol(d) < 3L)
stop("dtrack: expected at least 3 columns (frame, x, y) in ", basename(x))
data.frame(
id = tools::file_path_sans_ext(basename(x)), # one file, one track
frame = d[[1]],
x = d[[2]],
y = d[[3]]
)
}
register_loader_dialect("dtrack", dtrack_loader)After registration the dialect is used by name, like a built-in one:
track_file <- file.path(track_dir, "10_10_point02.txt")
ts_raw <- read_tracks(track_file, dialect = "dtrack")
head(ts_raw@data[, c("id", "frame", "x", "y")])
#> id frame x y
#> 1 10_10_point02 1 469.83 379.47
#> 2 10_10_point02 2 477.73 386.58
#> 3 10_10_point02 3 479.31 388.95
#> 4 10_10_point02 4 482.48 400.81
#> 5 10_10_point02 5 484.85 406.34
#> 6 10_10_point02 6 484.06 417.40A loader that reads files can take
read_tracks(read_opts = ) by declaring a
read_opts argument: it receives the non-NULL
entries (decimal, delim, sheet,
ext) and must honour them or stop. Without that argument,
any read_opts stop the load.
Given several files, the loader is called once per file and the
results are combined. Here pattern picks one session’s
trajectory files and leaves out the _point01 landmark
files, which share the same layout:
ts_session <- read_tracks_dir(track_dir, pattern = "^10_.*_point02\\.txt$",
dialect = "dtrack")
ts_session
#> Tracks: 29 trajectories, 4802 observations
#> Columns: id='id', time='frame', angle='angle' (radians), x='x', y='y'
#> id frame x y ..source_file angle
#> 1 10_1_point02 1 455.60 372.36 10_1_point02.txt 0.6851986
#> 2 10_1_point02 2 457.98 369.98 10_1_point02.txt 0.6795104
#> 3 10_1_point02 3 455.60 369.98 10_1_point02.txt 0.6820587
#> 4 10_1_point02 4 454.02 369.98 10_1_point02.txt 0.6837593
#> 5 10_1_point02 5 455.60 377.10 10_1_point02.txt 0.6914043
#> 6 10_1_point02 6 458.77 381.84 10_1_point02.txt 0.6941355Dialects live in-memory for the current session. To persist them
across projects, place the registration call in a package or in an
.Rprofile that you source before using
radiatR.