Skip to contents

YAML Parameters

Source: vignettes/instructions-yaml.qmd

What

WAVES uses a YAML file (_waves.yml) to define parameters needed for the main pipeline. For more information on what YAML is, reference vignette("yaml-2-minute-intro", package = "yaml12"). It is not a necessary read, but may help with troubleshooting if the pipeline is erroring at lst_yaml.

Why

By using a YAML file to define parameters, it allows end-users to see at a glance the hierarchy of parameters in one place. It also allows users to define different YAML files for different studies they may have access to (NOTE: this feature is still a work in progress).

Previous Step

Please make sure you have already ran the configuration pipeline as detailed in vignette("instructions-config") before proceeding.

Keys

The _waves.yml file has 7 top-level keys. Below are descriptions and examples for each top-level key:

study_timezone

A string scalar that corresponds to the timezone the study data was collected in. It should be a location in IANA “Olson” format. That is, in:

  • “[CONTINENT -or- OCEAN]/[CITY]” format

  • “Etc/GMT[INVERSE OFFSET]” format

  • “UTC” or “GMT”

See either:

  • The output from running Sys.timezone() within the R Console to see the timezone of your computer system in Olson format, which you can use if you are in the same timezone that the data was collected in. Or…

  • Reference “TZ identifier” column in the tz database to find the the correct timezone. Prefer to use the rows that are highlighted green.

Acceptable Unacceptable
America/Chicago, Etc/GMT+6 CST
America/New_York, Etc/GMT+5 EDT
America/Los_Angeles, Etc/GMT+8 PST
Asia/Hong_Kong Asia/Hong Kong
Asia/Katmandu Katmandu
Australia/Sydney NPT
Europe/London BST
Europe/Paris CET

sampling_frequency

A integer scalar corresponding to the sampling frequency of the wrist activity monitors. If sampling_frequency changed during data collection, then a sequence of integers should be entered.

Acceptable Unacceptable
100 100Hz
30 29.9

metadata_filepath

Filepath to csv file containing metadata for each participant, as described in Metadata Codebook

deidentify

Boolean where depending on the value:

  • TRUE: Pipeline will create a WAVES-specific ID for each participant listed within the metadata file specified by metadata_filepath. Option for sites that require additional anonymizing.

    • Each site will have the “key” that links the participant’s study ID’s with WAVES-specific ID’s, and will not be shared with the WAVES team.

    • The key will be located within the “reports” folder.

    • NOTE: This creation of a csv file that lists the key is a feature is still in progress. However, the merged output shared with the WAVES team does use WAVES-specific ID’s if deidentify is set to TRUE.

    • As the WAVES-specific ID is generated every time a change to the _waves.yml file is made, it is only applied to:

      • merged output that will be shared to the WAVES team
      • summary reports showing which files made it through the pipeline.

  • FALSE: Provided study and subject fields from metadata file will be used.

    • Not needed if it is okay to share site and original pid of each participant.

site_deid

Optional alphanumeric site identifier that will be used within the WAVES-specific ID. Usually an acronym of the university or lab where the study data is being held.

wrist_accelerometer

Sequence of mappings that define where raw wrist accelerometer files are in your computer system. Only the directories key or filepaths key should be defined, not both.

directories

One or more directories where the raw accelerometer files are being held. The directories should only hold raw files that are all the same extension (.gt3x, .cwa, .bin, .csv). Additionally, the directories:

  • can hold sub-directories or files that do not end in .gt3x, .cwa, .bin or .csv.

  • should not hold multiple formats of the same recording/session

    • For example, if MYFILE_001.gt3x is in the directory, there should not be a MYFILE_001.csv (raw csv export from ActiLife) in the directory as well.

  • should not hold raw files from monitors not worn at the wrist

If the directory contains multiple formats of the same recording/session or contains raw files from different wear locations, then the filepaths key should be defined instead.

filepaths

One or more filepaths that point directly to the raw files. Users will most likely define the filepaths by using the !expr tag to directly use R code. See the values as an Expression section for more information.

site_pid_observation_pattern

A string that represents the pattern in how raw accelerometer files were named, particularly to identify link the site string and the numeric identifier for each participant. This option is necessary in order to ensure all accelerometer files provided follow a consistent naming format and that the site and pid can be linked to the metadata provided.

For example, the wrist accelerometer files may be named like so:

MYSTUDY_9999_visit1

in which case, site_pid_observation_pattern should be:

(?<site>MYSTUDY)(?<pid>0000)visit(?<obs>0)

where:

  • (?<site>MYSTUDY) captures the character string meant to represent the site.

    • This is optional and should match the site variable from the provided metadata if provided.

    • The characters provided should be the same across ALL files.

      • Continuing from the above example, if a raw accelerometer file is named MYST_9999_visit1.csv, then the pipeline will throw an error as it is missing the MYSTUDY character specified as the site pattern.

    • The characters are case sensitive.

      • Continuing from the above example, if an activPAL file is named mystudy_9999_visit1.csv, then the pipeline will throw an error as the mystudy should be uppercase according to the provided pattern.

  • (?<pid>0000) captures the unique numeric identifier used per participant.

    • This is required.
    • Zeroes are used to represent numbers and should correspond with the pid variable from the metadata.
    • Note that in the example used, the pid is a 4 digits. In your specific case, the pid may be longer or shorter. Regardless, use the appropriate number of zeroes should be used and remain consistent across files.

  • (?<obs>0) captures the observation session.

    • This is optional.

    • If you are not supplying any direct observation data to the pipeline, then you can completely exclude this.

    • If you are supplying direct observation data, but you have one raw accelerometer file that includes all observation sessions, then you can also safely omit this as:

      1. The pipeline will only use site and pid to match the raw accelerometer file to direct observation data.
      2. The pipeline will know to “extract” the observation sessions from the provided raw accelerometer file.

  • Any characters not within a (? ) pattern will be expected to be the same across all files. If there is another variable within the filenames that do change between files (i.e. wrist location) such as below:

AL_LW_V3_102
AL_RW_V3_112

then place a . where the character string will change:

(?<site>AL)_.W_V3_(?<pid>000)

Finally, the pipeline assumes the provided string is at the very beginning of the filename.

reference

Sequence of mappings that define the reference measures being used and where they are located in your computer system.

direct_observation

filepath

Filepath that points directly to 3rd-person or 1st-person video-recorded direction observation labels formatted according to Reference Codebook Direct Observation section in .csv format.

still_image

filepath

Filepath that points directly to 1st-person still-image labels formatted according to Reference Codebook Wearable Camera section in csv format.

activpal

See Thigh-worn activPAL section for guidance on exporting 1-second epoch files and EventsEx files from PAL Batch.

1secEpochs_directories

One or more directories where 1-second epoch activPAL files are located. The directories:

  • can hold sub-directories or files that are neither 1-second or EventsEx files

  • can be the same directory as where the EventsEx files are being held

  • should not hold 1-second files exported with settings different than that specified within Reference Codebook Thigh-worn activPAL section.

1secEpochs_filepaths

One or more filepaths that point directly to 1-second epoch activPAL files. Users will most likely define the filepaths by using the !expr tag to directly use R code. See the Values as an Expression section for more information.

events_directories

One or more directories where 1-second epoch activPAL files are located. The directories:

  • can hold sub-directories or files that are neither 1-second or EventsEx files

  • can be the same directory as where the 1-second epoch files are being held

  • should not hold EventsEx files exported with settings different than that specified within Reference Codebook Thigh-worn activPAL section.

events_filepaths

One or more filepaths that point directly to EventsEx activPAL files. Users will most likely define the filepaths by using the !expr tag to directly use R code. See the Values as an Expression section for more information.

site_pid_observation_pattern

A string that represents the pattern in how activPAL files were named, particularly to identify the numeric identifier for each participant through zeroes. This option is necessary as, during initialization of an activPAL monitor, at most 9(?) characters are allowed.

For example, the wrist accelerometer files may be named like so:

MYSTUDY_9999_visit1

but the activPAL files like so:

MS9999v1

in which case, site_pid_observation_pattern should be:

(?<site>MS)(?<pid>0000)v(?<obs>0)

where:

  • (?<site>MS) captures the character string meant to represent the site.

    • This is optional if the site alphanumeric string was too long to put as the activPAL identifier AND is the same across all participants.

    • If provided, the characters do not need to exactly correspond with the site variable from the metadata. Since the site variable is assumed to be the same across all participants, the pipeline will know to match whatever is captured within (?<site> ) to the string provided in the metadata site variable.

    • The characters provided should be the same across ALL files.

      • Continuing from the above example, if an activPAL file is named 1234v1.csv, then the pipeline will throw an error as it is missing the MS characters.

    • The characters are case sensitive.

      • Continuing from the above example, if an activPAL file is named ms5678v1.csv, then the pipeline will throw an error as the ms should be uppercase according to the provided pattern.

  • (?<pid>0000) captures the unique numeric identifier used per participant.

    • This is required.
    • Zeroes are used to represent numbers and should correspond with the pid variable from the metadata.
    • Note that in the example used, the pid is a 4 digits. In your specific case, the pid may be longer or shorter. Regardless, use the appropriate number of zeroes should be used and remain consistent across files.

  • (?<obs>0) captures the observation session.

    • This is optional.

    • If you are not supplying any direct observation data to the pipeline, then you can completely exclude this.

    • If you are supplying direct observation data, but you either:

      1. Only have one observation session per participant so therefore don’t have anything within the activPAL filename that identifies the observation session.
      2. Have one activPAL file that corresponds

      then you can also safely omit this as:

      1. The pipeline will only use pid to match the activPAL file to direct observation data.
      2. The pipeline will know to “extract” the observation sessions from the provided activPAL file.

Finally, the pipeline assumes the provided string is at the very beginning of the filename.

actipass

TODO

directories
filepaths
site_pid_observation_pattern

Values as an Expression

YAML values can have a tag before defining a value, as noted in vignette("yaml-tags-and-advanced-features", package = "yaml12") . In WAVES, the !expr tag has been defined to evaluate text after the !expr tag as an R expression. For example, the following key: value pair

my_number: !expr 6 * 7

is equivalent to the following code in R

list(my_number = 6 * 7)

For any keys that have the word filepaths, it is recommended to use the !expr tag along with an R expression to easily define all filepaths instead of manually typing out every single filepath. This will typically be done using the list.files() function.

For example, if I wanted to list files that only had the .csv.gz extension within the WAVES repository, then the following would work in the YAML file:

filepaths: !expr list.files(path = "data/0_CONFIG/RAW", pattern = "$\\.csv\\.gz", full.names = TRUE)

Note that the code is all kept on the same line, and the full.names argument is set to TRUE.

If you were to write list.files() across multiple lines, it would have to be written like so:

filepaths:
  - !expr list.files(
      path = "data/0_CONFIG/RAW",
      pattern = "$\\.csv\\.gz",
      full.names = TRUE
    )

Note, in YAML language indentation matters, as it is essential for determining which sequences/mappings are within others. So the following would NOT work and return an error.

filepaths:
  - !expr list.files(
  path = "data/0_CONFIG/RAW",
  pattern = "$\\.csv\\.gz",
  full.names = TRUE
)

This would happen if you were to copy and paste the R code directly from a script to the YAML file. If you do so, make sure to appropriately indent the code such that the ) parenthesis is directly under the ! in !expr tag.

Next Step

Once the parameters in your YAML file have been entered, proceed to vignette("instructions-main") to run the main pipeline.