Tierpsy Tracker

Multi-Worm Behaviour Tracker

This project is maintained by ver228

Example Data

Example files can be found here. The zip file contains a multiworm video recorded using a high resolution fixed camera and a single worm video recorded using the WT2.0.

You can analyze the videos using the Batch Processing Multiple Files App. The videos require different analysis parameters since they belong to different setups, therefore they cannot be processed together.

For the multiworm video the Parameters File must be set to MULTIWORM_OPENWORM.json and the File Pattern to Include as *.mov as shown below:

screen shot 2018-06-11 at 12 47 16

For the multiworm video the Parameters File must be set to WT2_clockwise.json and the File Pattern to Include as *.avi as shown below:

screen shot 2018-04-25 at 09 13 19

The processing times for in MacBook Pro (15-inch, 2017) were 04:31 minutes for the multiworm video and 11:43 minutes for the singleworm video.

Detailed Instructions

Getting Started

Follow the installation instuctions and open a terminal or an Anaconda prompt (Windows) and type:

tierpsy_gui

The main widget should look like the one below:

TierpsyTrackerConsole

Set Parameters

The purpose of this widget is to setup the parameters used for Batch Processing Multiple Files. The interface is designed to help select the parameters that determine how videos are segmented and compressed. When imaging conditions such as lighting or magnification are changed, these parameters will need to be updated for good performance.

The most commonly adjusted parameter is the Threshold. If you have dark worms on a light background, Is Light Background? should be checked. In this case, pixels that are darker than the threshold value will be included in the mask. The selected value should be low enough to exclude as much background as possible without losing any part of the animals to be tracked. Below there is an example on how to do this.

If the objects to track are lighter than the background (e.g. if you are tracking fluorescent objects or using dark field illumination), un-check Is Light Background?. In this case, pixels that are above the threshold value will be included in the mask.

SetParameters

In some cases, even after adjusting the threshold there still remain large regions of background. If the tracked objects significatively change position during the movie you can enable the background subtraction as shown below. This method will consider anything that does not change within the specified frame range as background. However, if any of your animals are immobile during the entire frame range will be lost.

SetBgndSubt

Other important parameters to set are:

Extension Description
BASE* No features, only steps up to the skeleton orientation
TIERPSY* Add the steps for the tierpsy features calculation.
OPENWORM* Add the steps for the openworm features calculation.
*WT2 Add the necessary steps to analyze videos recorded using the WormTracker 2.0.
*SINGLE Same steps as BASE but the trajectories will be joined with the assumption that there is only a single worm in the video.
*AEX Add the steps to filter worms and obtain the food contour using deep learning models. This models might only work from data from the Behavioural Genomics Laboratory.

You can access further parameters by clicking Edit More Parameters. The explanation of each parameter can be found by using the contextual help. It is not always trivial to effectively adjust these other parameters, but if you believe you need too, I recommend using a small video (~100 frames) for testing.

When you are satisfied with the selected parameters select a file name and press Save Parameters. The parameters will be saved as a JSON file that can be used in Batch Processing Multiple Files. If you need to further modify a parameter you can either use a text editor to change the JSON file directly or reload the file by dragging it into the Set Parameters widget.

Batch Processing Multiple Files

BatchProcessing

This widget is used to execute all steps of tracking and feature extraction on each of the files on a given directory. The program allows a degree of parallelization by analyzing multiple files at the same time. The number of processes to run in parallel (Maximum Number of Processes) should not exceed the number of processor cores available on your machine to avoid slowing down the analysis.

Chosing the Files to be Analyzed

Tierpsy Tracker will analyse of the video files that are present in Original Video Dir including sub-directories. Particular files are included if their names match the File Pattern to Include, but do not match the File Pattern to Exclude.

Alternatively, one can create a text file with the list of files to be analysed (one file per line). The path to this file can then be set in Individual File List.

Parameters Files

Parameters files created using the Set Parameters widget can be select in the Parameter Files box. You can also select some previously created files using the drop-down list. If no file is selected the default values will be used.

Worm Tracker 2.0 Option

You can analyse videos created by the Worm Tracker 2.0 by selecting the parameters files WT2_clockwise.json or WT2_anticlockwise.json. Use the former if the ventral side in the videos is located in the clockwise direction from the worm head, and the later if it is in the anticlockwise direction. To select a subset of files with a particular orientation you can save each subset in a different root directory or include the orientation information in the file name and use use the Pattern include option . If you need to fine-tune the parameters you can edit the .json files either with a text editor or with Set Parameters.

Note that each of the video files .avi must have an additional pair of files with the extensions .info.xml and .log.csv. Additionally, keep in mind that if the stage aligment step fails, an error will be risen and the analysis of that video will be stopped. If you do not want to see the error messages untick the option Print debug information.

Analysis Progress

Tierpsy Tracker will determine which analysis steps have already been completed for the selected files and will only execute the analysis from the last completed step. Files that were completed or do not satisfy the next step requirements will be ignored.

Directory to Save the Output Files

The masked videos created in the compression step are stored in Masked Videos Dir. The rest of the tracker results are stored in Tracking Results Dir. In both cases the subdirectory tree structure in Original Videos Dir is recreated.

The reason for creating the parallel directory structure is to make it easy to delete the analysis outputs to re-run with different parameter values. It also makes it easy to delete the original videos to save space once you’ve arrived at good parameter values. If you prefer to have the output files in the same directory as the original videos you can set Masked Videos Dir and Tracking Results Dir to the same value.

Temporary directory

By default, Tierpsy Tracker creates files in the Temporary Dir and only moves them to the Masked Videos Dir or the Tracking Results Dir when the analysis has finished. The reasons to use a temporary directory are:

Some extra options:

Command Line Tool

The same functions are accesible using the command line. You can see the available option by typing in the main tierpsy directory:

python cmd_scripts/processMultipleFiles.py -h

Tierpsy Tracker Viewer

This widget is used to visualize the tracking results. You can move to a specific frame, zoom in/out, select specific trajectories, and visualize the skeletons overlayed on the compressed video, the trajectory paths or saved unmasked frames. See below for an example on how to use it.

MWTrackerViewer

Manually Joining Trajectories

You can manually correct the trajectories as shown below. Once you have finished open Batch Processing Multiple Files and re-run the analysis selecting FEAT_MANUAL_CREATE from the Analysis Start Point drop menu. This will execute the step FEAT_MANUAL_CREATE, and create a file with the extension basename_feat_manual.hdf5 with the same contents as basename_features.hdf5 but with the manually joined indexes.

TrackJoined

Viewer Shortcuts

W : label selected box as Single Worm.

C : label selected box as Worm Cluster.

B : label selected box as Bad.

U : label selected box as Undefined.

J : Join both trajectories in the zoomed windows.

S : Split the selected trajectory at the current time frame.

Up key : select the top zoomed window.

Down key : select the bottom zoomed window.

[ : Move the the begining of the selected trajectory.

] : Move the the end of the selected trajectory.

+ : Zoom out the main window.

- : Zoom in the main window.

> : Duplicated the frame step size.

< : Half the frame step size.

Left key : Increse the frame by step size.

Right key : Decrease the frame by step size.

Visualizing Analysis Results

The extracted features are store in the files that end with featuresN.hdf5 if the tierpsy feature route was selected or in features.hdf5 if the openworm route was selected. You can visualize the features in different ways as shown below:

features

From the plotting window can either save the plots or export the data of individual features/trajectories into csv files. If you would like to compare the data of multiple experiments we strongly recommed you to use the Features Summary app. If you would like to work directly with the timeseries data we recommend you to use read the data using a scripting language like python using the packages pandas and pytables, or MATLAB following the examples in the tierpsy_tools repository.

Features Summary

FeatSummary

The files will be located by doing a recursive search for matching the extension according to the table below.

Feature Type Is Manually Edited? File Extension
tierpsy Ticked featuresN.hdf5
tierpsy Unticked featuresN.hdf5
openworm Ticked feat_manual.hdf5
openworm Unticked features.hdf5

The results are saved into two separated .csv file located in the root directory. The first file, filenames_FEATURETYPE_SUMMARY_DATE.csv, contains the names of all the files found in the root directory. The is_good column is set to True if the file is valid and used in the summary. The second file, features_FEATURETYPE_SUMMARY_DATE.csv, contains the corresponding features summarized as described in the output files section. The two result files can be joined using the file_id column.