Behavior#

Audio#

class AudioInterface(file_paths: list[Annotated[pathlib._local.Path, PathType(path_type='file')]], *args, verbose: bool = False)[source]#

Bases: BaseTemporalAlignmentInterface

Data interface for writing .wav audio recordings to an NWB file.

Data interface for writing acoustic recordings to an NWB file.

Writes acoustic recordings as an AcousticWaveformSeries from the ndx_sound extension.

Parameters:

file_paths (list of FilePaths) – The file paths to the audio recordings in sorted, consecutive order. We recommend using natsort to ensure the files are in consecutive order:
```
from natsort import natsorted
natsorted(file_paths)
```
verbose (bool, default: False)

display_name: str | None = 'Wav Audio'#

keywords: tuple[str] = ('sound', 'microphone')#

associated_suffixes: tuple[str] = ('.wav',)#

info: str | None = 'Interface for writing audio recordings to an NWB file.'#

get_metadata_schema() → dict[source]#

Retrieve JSON schema for metadata.

Returns:: The JSON schema defining the metadata structure.
Return type:: dict

get_metadata() → DeepDict[source]#

Child DataInterface classes should override this to match their metadata.

Returns:: The metadata dictionary containing basic NWBFile metadata.
Return type:: DeepDict

get_original_timestamps() → ndarray[source]#

Retrieve the original unaltered timestamps for the data in this interface.

This function should retrieve the data on-demand by re-initializing the IO.

Returns:: timestamps – The timestamps for the data stream.
Return type:: numpy.ndarray

get_timestamps() → ndarray | None[source]#

Retrieve the timestamps for the data in this interface.

Returns:: timestamps – The timestamps for the data stream.
Return type:: numpy.ndarray

set_aligned_timestamps(aligned_timestamps: list[ndarray])[source]#

Replace all timestamps for this interface with those aligned to the common session start time.

Must be in units seconds relative to the common ‘session_start_time’.

Parameters:: aligned_timestamps (numpy.ndarray) – The synchronized timestamps for data in this interface.

set_aligned_starting_time(aligned_starting_time: float)[source]#

Align all starting times for all audio files in this interface relative to the common session start time.

Must be in units seconds relative to the common ‘session_start_time’.

Parameters:: aligned_starting_time (float) – The common starting time for all temporal data in this interface. Applies to all segments if there are multiple file paths used by the interface.

set_aligned_segment_starting_times(aligned_segment_starting_times: list[float])[source]#

Align the individual starting time for each audio file in this interface relative to the common session start time.

Must be in units seconds relative to the common ‘session_start_time’.

Parameters:: aligned_segment_starting_times (list of floats) – The relative starting times of each audio file (segment).

align_by_interpolation(unaligned_timestamps: ndarray, aligned_timestamps: ndarray)[source]#

Interpolate the timestamps of this interface using a mapping from some unaligned time basis to its aligned one.

Use this method if the unaligned timestamps of the data in this interface are not directly tracked by a primary system, but are known to occur between timestamps that are tracked, then align the timestamps of this interface by interpolating between the two.

An example could be a metronomic TTL pulse (e.g., every second) from a secondary data stream to the primary timing system; if the time references of this interface are recorded within the relative time of the secondary data stream, then their exact time in the primary system is inferred given the pulse times.

Must be in units seconds relative to the common ‘session_start_time’.

Parameters:

unaligned_timestamps (numpy.ndarray) – The timestamps of the unaligned secondary time basis.
aligned_timestamps (numpy.ndarray) – The timestamps aligned to the primary time basis.

add_to_nwbfile(nwbfile: NWBFile, metadata: dict | None = None, *args, stub_test: bool = False, stub_frames: int = 1000, parent_container: Literal['stimulus', 'acquisition'] = 'stimulus', iterator_options: dict | None = None, write_as: Literal['stimulus', 'acquisition'] | None = None)[source]#

Parameters:

nwbfile (NWBFile) – Append to this NWBFile object
metadata (dict, optional)
stub_test (bool, default: False)
stub_frames (int, default: 1000)
parent_container ({‘stimulus’, ‘acquisition’}) – The acoustic waveform series can be added to the NWB file either as “stimulus” or as “acquisition”.
iterator_options (dict, optional) – Dictionary of options for the SliceableDataChunkIterator.
write_as ({‘stimulus’, ‘acquisition’}, optional) – Deprecated. Use parent_container instead. Will be removed on or after February 2027.

Return type:

NWBFile

BORIS#

class BORISInterface(file_path: Annotated[pathlib._local.Path, PathType(path_type='file')], *, observation_name: str, metadata_key: str | None = None, verbose: bool = False)[source]#

Bases: BaseEventsInterface

Data Interface for the events of one observation of a BORIS project.

BORIS (Behavioural Observation Research Interactive Software) records time-constrained behavioral observations against video, audio, or a live session. A .boris file is one JSON document holding the whole record: the coding scheme, the subjects, and every observation with its events. An observation is the unit that corresponds to a session, so this interface takes one by name; use get_observation_names() to see what a file holds.

Every behavior the scheme declares becomes an event type, including one nothing was ever scored against, which is written as a zero-row contribution rather than dropped: the vocabulary is part of the record. A behavior’s declared type decides its extent. A point behavior occupies one row and has no extent; a state behavior occupies two, a start and a stop, which this interface pairs on subject plus code in order of appearance and writes as one row carrying the bout’s length. Pairing ignores the modifier string, since a bout can open with one modifier and close with another. A bout that opens and never closes keeps a NaN duration, which happens whenever a coder misses a stop in a live session and cannot be repaired afterwards.

Every behavior shares one duration column, so a NaN in it reads two ways: a point behavior, which has no extent to record, and a state bout nobody closed. The catalogue’s behavior_type is what tells them apart.

All of an observation’s behaviors are written into one table by default. A behavior may declare modifier slots, the qualifiers a coder answers whenever they score it (Walking asking for a speed and a direction), and each slot becomes a column named after it rather than surviving as the |-joined string BORIS records; a behavior that declares no such slot writes an empty cell there. The column is named after the behavior and the slot, so two behaviors declaring a slot of the same name get a column each and every column holds one vocabulary; a slot the scheme leaves unnamed falls back to its position. Which columns a behavior writes into is on the catalogue, in modifiers, along with the menu each offers in modifier_values.

The coding scheme is written alongside the events as an ndx-ethogram Ethogram catalogue in the behavior processing module, and the closed state bouts as an EthogramBouts table beside it where the observation has any. The catalogue is the durable half of a BORIS file, holding what is true of a behavior rather than of any occurrence, and the bouts table is the curated interval view that reads as an IntervalSet downstream. The events table remains the faithful record: it alone carries the point behaviors, the bouts that never closed, and the per-occurrence modifier answers, comments and subject attribution.

Initialize the BORISInterface.

Parameters:

file_path (FilePath) – Path to the .boris JSON document.
observation_name (str) – The observation to read, as get_observation_names() lists them.
metadata_key (str, optional) – The key this interface’s block sits under in metadata["Events"]. Defaults to the observation’s own name, boris_live_not_paired for an observation called live not paired, since a project holds many observations and a conversion running several of them would otherwise write them all under one key.
verbose (bool, default: False) – Whether to print progress.

keywords: tuple[str] = ('events', 'behavior', 'BORIS', 'ethogram', 'annotation')#

display_name: str | None = 'BORIS'#

info: str | None = 'Data Interface for one observation of a BORIS project.'#

associated_suffixes: tuple[str] = ('.boris',)#

static get_observation_names(file_path: Annotated[Path, PathType(path_type=file)]) → list[str][source]#

Return the names of the observations a .boris file holds, in file order.

Parameters:: file_path (FilePath) – Path to the .boris JSON document.
Returns:: The observation names, which are the handles observation_name takes. Empty where the project declares a coding scheme and was never coded against, which is a legal file.
Return type:: list of str

get_metadata() → DeepDict[source]#

Get metadata for this interface.

The observation’s date is reported as NWBFile/session_start_time and its description, where the coder wrote one, as NWBFile/session_description. No subject is reported: a BORIS subject is an animal being scored and one observation routinely carries several, so it cannot map onto the file’s single Subject and stays a per-event column instead.

One event_types entry is declared per behavior the scheme holds, its event_name seeded from the behavior’s code and its event_description from the behavior’s own description, which BORIS carries natively and most sources do not. All of them route to one table, declared in EventTables so the merged table carries the observation’s name rather than a derived one.

Returns:: The metadata dictionary for this interface.
Return type:: DeepDict

add_to_nwbfile(nwbfile: NWBFile, metadata: dict | None = None) → None[source]#

Write the observation’s events, its coding scheme and its state bouts.

Parameters:

nwbfile (NWBFile) – The NWB file to add the events to.
metadata (dict, optional) – Metadata dictionary; see get_metadata_schema(). If None, get_metadata() is used.

DeepLabCut#

class DeepLabCutInterface(file_path: Annotated[pathlib._local.Path, PathType(path_type='file')], *args, config_file_path: Annotated[pathlib._local.Path, PathType(path_type='file')] | None = None, subject_name: str = 'ind1', pose_estimation_metadata_key: str | None = None, verbose: bool = False, metadata_key: str | None = None, sampling_frequency: float | None = None)[source]#

Bases: BasePoseEstimationInterface

Data interface for DeepLabCut datasets.

Interface for writing DeepLabCut’s output files to NWB.

This interface reads DeepLabCut output files (.h5 or .csv) and converts them to NWB format using the ndx-pose extension. It extracts keypoints (bodyparts), their coordinates, and confidence values, and organizes them into a structured format within the NWB file.

Parameters:

file_path (FilePath) – Path to the file output by DeepLabCut (.h5 or .csv). The file should contain the pose estimation data with keypoints, coordinates, and confidence values.
config_file_path (FilePath, optional) – Path to the DeepLabCut .yml config file. If provided, additional metadata such as video dimensions, task description, and experimenter information will be extracted.
subject_name (str, default: “ind1”) – The subject name to be used in the metadata. For output files with multiple individuals, this must match the name of the individual for which the data will be added. This name is also used to link the skeleton to the subject in the NWB file.
pose_estimation_metadata_key (str, optional) – Deprecated. Renamed to metadata_key; passing it forwards the value to metadata_key and will be removed on or after February 2027. Passing both raises ValueError.
verbose (bool, default: False) – Controls verbosity of the conversion process.
metadata_key (str, optional) – The registry key under which this interface’s metadata is stored in the dict-based format. When None it resolves to a default ("deep_lab_cut_metadata_key"). The key is an internal handle and does not appear in the NWB file; rename NWB objects via their name fields in the metadata dict instead. To opt into the dict-based shape, call get_metadata(use_new_metadata_format=True).
sampling_frequency (float, optional) – The frame rate of the video the pose was estimated from, in Hz. A DeepLabCut output file’s rows are video frames and carry no times, and neither the file nor the project config records the rate, so one of sampling_frequency or set_aligned_timestamps is required before writing. Pass this for a constant frame rate; call set_aligned_timestamps when the frames have times of their own, from a hardware clock or an alignment against another stream.

Metadata Structure#

With use_new_metadata_format=True the metadata follows the unified, dict-based layout: a shared top-level Devices registry plus the pose registries under the top-level metadata["Pose"] modality, cross-referenced by key.

metadata = {
    "Devices": {
        "deep_lab_cut_metadata_key": {  # registry key (snake_case, never written to the file)
            "name": "CameraPoseEstimationDeepLabCut",
            "description": "Camera used for behavioral recording and pose estimation.",
        }
    },
    "Pose": {
        "Skeletons": {
            "deep_lab_cut_metadata_key": {
                "name": "SkeletonPoseEstimationDeepLabCut_SubjectName",
                "nodes": ["bodypart1", "bodypart2", ...],  # keypoints/bodyparts
                "edges": [[0, 1], [1, 2], ...],  # connections between nodes (optional)
                "subject": "subject_name",  # links the skeleton to the subject
            }
        },
        "PoseEstimations": {
            "deep_lab_cut_metadata_key": {  # keyed by metadata_key
                "name": "PoseEstimationDeepLabCut",
                "source_software": "DeepLabCut",
                "scorer": "...",
                "dimensions": [[height, width]],
                "original_videos": ["path/to/video.mp4"],
                "device_metadata_key": "deep_lab_cut_metadata_key",  # -> metadata["Devices"]
                "skeleton_metadata_key": "deep_lab_cut_metadata_key",  # -> Pose.Skeletons
                "PoseEstimationSeries": {
                    "bodypart1": {"name": "PoseEstimationSeriesBodypart1"},
                    "bodypart2": {"name": "PoseEstimationSeriesBodypart2"},
                    # one entry per bodypart; the dict key is the bodypart name
                },
            }
        },
    },
}

The registry key (deep_lab_cut_metadata_key above) is an internal handle that never appears in the NWB file; rename NWB objects through their name fields instead. get_metadata emits only values extracted from the DeepLabCut source plus the object name``s; per-series defaults (``description, unit, reference_frame, confidence_definition) are applied by the writer at write time, so set them here only to override.

The metadata can be customized by:

Calling get_metadata(use_new_metadata_format=True) to retrieve the default metadata
Modifying the returned dictionary as needed
Passing the modified metadata to add_to_nwbfile() or run_conversion()

See also our Conversion Gallery for more examples using DeepLabCut data.

Notes

When the subject_name matches a subject_id in the NWBFile, the skeleton will be automatically

linked to that subject.

display_name: str | None = 'DeepLabCut'#

keywords: tuple[str] = ('DLC', 'DeepLabCut', 'pose estimation', 'behavior')#

associated_suffixes: tuple[str] = ('.h5', '.csv')#

info: str | None = 'Interface for handling data from DeepLabCut.'#

classmethod get_source_schema() → dict[source]#

Infer the JSON schema for the source_data from the method signature (annotation typing).

Returns:: The JSON schema for the source_data.
Return type:: dict

get_available_subjects() → list[str][source]#

Extract available subjects from a DeepLabCut output file.

Parameters:

file_path (FilePath) – Path to the DeepLabCut output file (.h5 or .csv).

Returns:

List of subject names found in the file.

Return type:

list[str]

Raises:

IOError – If the file is not a valid DeepLabCut output file.
FileNotFoundError – If the file does not exist.

get_metadata_schema(*, use_new_metadata_format: bool = True) → dict[source]#

Retrieve JSON schema for metadata specific to the DeepLabCutInterface.

Returns:: The JSON schema defining the metadata structure.
Return type:: dict

get_metadata(*, use_new_metadata_format: bool = True) → DeepDict[source]#

Build the pose registries and their cross-references, with generic names and no free text.

An interface overrides this, calls it, and sets what its own format records. Nothing is invented here: no descriptions, no units, no reference frames, and no scorer, dimensions or video paths.

get_original_timestamps() → ndarray[source]#

Retrieve the original unaltered timestamps for the data in this interface.

This function should retrieve the data on-demand by re-initializing the IO.

Returns:: timestamps – The timestamps for the data stream.
Return type:: numpy.ndarray

get_timestamps() → ndarray[source]#

Retrieve the timestamps for the data in this interface.

Returns:: timestamps – The timestamps for the data stream.
Return type:: numpy.ndarray

set_aligned_timestamps(aligned_timestamps: list | ndarray)[source]#

Set aligned timestamps vector for DLC data with user defined timestamps

Parameters:: aligned_timestamps (list, np.ndarray) – A timestamps vector.

add_to_nwbfile(nwbfile: NWBFile, metadata: dict | None = None)[source]#

Conversion from DLC output files to nwb. Derived from dlc2nwb library.

Parameters:

nwbfile (NWBFile) – nwb file to which the recording information is to be added
metadata (dict) – metadata info for constructing the nwb file (optional).

FicTrac#

class FicTracDataInterface(file_path: Annotated[pathlib._local.Path, PathType(path_type='file')], *args, radius: float | None = None, configuration_file_path: Annotated[pathlib._local.Path, PathType(path_type='file')] | None = None, verbose: bool = False)[source]#

Bases: BaseTemporalAlignmentInterface

Data interface for FicTrac datasets.

Interface for writing FicTrac files to nwb.

Parameters:

file_path (FilePath) – Path to the .dat file (the output of fictrac)
radius (float, optional) – The radius of the ball in meters. If provided the radius is stored as a conversion factor and the units are set to meters. If not provided the units are set to radians.
configuration_file_path (FilePath, optional) – Path to the .txt file with the configuration metadata. Usually called config.txt
verbose (bool, default: False) – controls verbosity. True by default.

display_name: str | None = 'FicTrac'#

keywords: tuple[str] = ('fictrack', 'visual tracking', 'fictive path', 'spherical treadmill', 'visual fixation')#

associated_suffixes: tuple[str] = ('.dat',)#

info: str | None = 'Interface for FicTrac .dat files.'#

timestamps_column = 21#

columns_in_dat_file = ['frame_counter', 'rotation_delta_x_cam', 'rotation_delta_y_cam', 'rotation_delta_z_cam', 'rotation_delta_error', 'rotation_delta_x_lab', 'rotation_delta_y_lab', 'rotation_delta_z_lab', 'rotation_x_cam', 'rotation_y_cam', 'rotation_z_cam', 'rotation_x_lab', 'rotation_y_lab', 'rotation_z_lab', 'x_pos_radians_lab', 'y_pos_radians_lab', 'animal_heading', 'movement_direction', 'movement_speed', 'forward_motion_lab', 'side_motion_lab', 'timestamp', 'sequence_counter', 'delta_timestamp', 'alt_timestamp']#

column_to_nwb_mapping = {'animal_heading': {'column_in_dat_file': ['animal_heading'], 'description': "Animal's heading direction in radians from the lab's perspective.", 'reference_frame': 'lab', 'spatial_series_name': 'SpatialSeriesAnimalHeading'}, 'integrated_motion': {'column_in_dat_file': ['forward_motion_lab', 'side_motion_lab'], 'description': 'Integrated x/y position of the sphere in laboratory coordinates, neglecting heading.', 'reference_frame': 'lab', 'spatial_series_name': 'SpatialSeriesIntegratedMotion'}, 'movement_direction': {'column_in_dat_file': ['movement_direction'], 'description': "Instantaneous running direction of the animal in the lab coordinates. Direction inferred by the ball's rotation (roll and pitch)", 'reference_frame': 'lab', 'spatial_series_name': 'SpatialSeriesMovementDirection'}, 'movement_speed': {'column_in_dat_file': ['movement_speed'], 'description': 'Instantaneous running speed of the animal in radians per frame.', 'reference_frame': 'lab', 'spatial_series_name': 'SpatialSeriesMovementSpeed'}, 'position_lab': {'column_in_dat_file': ['x_pos_radians_lab', 'y_pos_radians_lab'], 'description': 'x and y positions in the lab frame in radians, inferred by integrating the rotation over time.', 'reference_frame': 'lab', 'spatial_series_name': 'SpatialSeriesPosition'}, 'rotation_cam': {'column_in_dat_file': ['rotation_x_cam', 'rotation_y_cam', 'rotation_z_cam'], 'description': "Orientation in radians from the camera's perspective. x: rotation to the sphere's right (pitch), y: rotation under the sphere (yaw), z: rotation behind the sphere (roll)", 'reference_frame': 'camera', 'spatial_series_name': 'SpatialSeriesRotationCameraFrame'}, 'rotation_delta_cam': {'column_in_dat_file': ['rotation_delta_x_cam', 'rotation_delta_y_cam', 'rotation_delta_z_cam'], 'description': "Change in orientation since last frame from the camera's perspective. x: rotation to the sphere's right (pitch), y: rotation under the sphere (yaw), z: rotation behind the sphere (roll)", 'reference_frame': 'camera', 'spatial_series_name': 'SpatialSeriesRotationDeltaCameraFrame'}, 'rotation_delta_error': {'column_in_dat_file': ['rotation_delta_error'], 'description': "Error in rotation delta in radians from the lab's perspective.", 'reference_frame': 'lab', 'spatial_series_name': 'SpatialSeriesRotationDeltaError'}, 'rotation_delta_lab': {'column_in_dat_file': ['rotation_delta_x_lab', 'rotation_delta_y_lab', 'rotation_delta_z_lab'], 'description': "Change in orientation since last frame from the lab's perspective. x: rotation in front of subject (roll), y: rotation to subject's right (pitch), z: rotation under the subject (yaw)", 'reference_frame': 'lab', 'spatial_series_name': 'SpatialSeriesRotationDeltaLabFrame'}, 'rotation_lab': {'column_in_dat_file': ['rotation_x_lab', 'rotation_y_lab', 'rotation_z_lab'], 'description': "Orientation in radians from the lab's perspective. x: rotation in front of subject (roll), y: rotation to subject's right (pitch), z: rotation under the subject (yaw)", 'reference_frame': 'lab', 'spatial_series_name': 'SpatialSeriesRotationLabFrame'}}#

spatial_series_descriptions = {'animal_heading': {'column_in_dat_file': ['animal_heading'], 'description': "Animal's heading direction in radians from the lab's perspective.", 'reference_frame': 'lab', 'spatial_series_name': 'SpatialSeriesAnimalHeading'}, 'integrated_motion': {'column_in_dat_file': ['forward_motion_lab', 'side_motion_lab'], 'description': 'Integrated x/y position of the sphere in laboratory coordinates, neglecting heading.', 'reference_frame': 'lab', 'spatial_series_name': 'SpatialSeriesIntegratedMotion'}, 'movement_direction': {'column_in_dat_file': ['movement_direction'], 'description': "Instantaneous running direction of the animal in the lab coordinates. Direction inferred by the ball's rotation (roll and pitch)", 'reference_frame': 'lab', 'spatial_series_name': 'SpatialSeriesMovementDirection'}, 'movement_speed': {'column_in_dat_file': ['movement_speed'], 'description': 'Instantaneous running speed of the animal in radians per frame.', 'reference_frame': 'lab', 'spatial_series_name': 'SpatialSeriesMovementSpeed'}, 'position_lab': {'column_in_dat_file': ['x_pos_radians_lab', 'y_pos_radians_lab'], 'description': 'x and y positions in the lab frame in radians, inferred by integrating the rotation over time.', 'reference_frame': 'lab', 'spatial_series_name': 'SpatialSeriesPosition'}, 'rotation_cam': {'column_in_dat_file': ['rotation_x_cam', 'rotation_y_cam', 'rotation_z_cam'], 'description': "Orientation in radians from the camera's perspective. x: rotation to the sphere's right (pitch), y: rotation under the sphere (yaw), z: rotation behind the sphere (roll)", 'reference_frame': 'camera', 'spatial_series_name': 'SpatialSeriesRotationCameraFrame'}, 'rotation_delta_cam': {'column_in_dat_file': ['rotation_delta_x_cam', 'rotation_delta_y_cam', 'rotation_delta_z_cam'], 'description': "Change in orientation since last frame from the camera's perspective. x: rotation to the sphere's right (pitch), y: rotation under the sphere (yaw), z: rotation behind the sphere (roll)", 'reference_frame': 'camera', 'spatial_series_name': 'SpatialSeriesRotationDeltaCameraFrame'}, 'rotation_delta_error': {'column_in_dat_file': ['rotation_delta_error'], 'description': "Error in rotation delta in radians from the lab's perspective.", 'reference_frame': 'lab', 'spatial_series_name': 'SpatialSeriesRotationDeltaError'}, 'rotation_delta_lab': {'column_in_dat_file': ['rotation_delta_x_lab', 'rotation_delta_y_lab', 'rotation_delta_z_lab'], 'description': "Change in orientation since last frame from the lab's perspective. x: rotation in front of subject (roll), y: rotation to subject's right (pitch), z: rotation under the subject (yaw)", 'reference_frame': 'lab', 'spatial_series_name': 'SpatialSeriesRotationDeltaLabFrame'}, 'rotation_lab': {'column_in_dat_file': ['rotation_x_lab', 'rotation_y_lab', 'rotation_z_lab'], 'description': "Orientation in radians from the lab's perspective. x: rotation in front of subject (roll), y: rotation to subject's right (pitch), z: rotation under the subject (yaw)", 'reference_frame': 'lab', 'spatial_series_name': 'SpatialSeriesRotationLabFrame'}}#

classmethod get_source_schema() → dict[source]#

Infer the JSON schema for the source_data from the method signature (annotation typing).

Returns:: The JSON schema for the source_data.
Return type:: dict

get_metadata() → DeepDict[source]#

Child DataInterface classes should override this to match their metadata.

Returns:: The metadata dictionary containing basic NWBFile metadata.
Return type:: DeepDict

add_to_nwbfile(nwbfile: NWBFile, metadata: dict | None = None)[source]#

Parameters:

nwbfile (NWBFile) – nwb file to which the recording information is to be added
metadata (dict, optional) – metadata info for constructing the nwb file.

get_original_timestamps()[source]#

Retrieve and correct timestamps from a FicTrac data file.

This function addresses two specific issues with timestamps in FicTrac data:

Resetting Initial Timestamp In some instances, FicTrac replaces the initial timestamp (0) with the system time. This commonly occurs when the data source is a video file, and OpenCV reports the first timestamp as 0. Since OpenCV also uses 0 as a marker for invalid values, FicTrac defaults to system time in that case. This leads to inconsistent timestamps like [system_time, t1, t2, t3, …]. The function corrects this by resetting the first timestamp back to 0 when a negative difference is detected between the first two timestamps.
Re-centering Unix Epoch Time If timestamps are in Unix epoch time format (time since 1970-01-01 00:00:00 UTC), this function re-centers the time series by subtracting the first timestamp. This adjustment ensures that timestamps represent the elapsed time since the start of the experiment rather than the Unix epoch. This case appears when one of the sources of data in FicTrac (such as PGR or Basler) lacks a timestamp extraction method. FicTrac then falls back to using the system time, which is in Unix epoch format.

Returns:: An array of corrected timestamps, in seconds.
Return type:: np.ndarray

Notes

The issue of the initial timestamp replacement appears in FicTrac 2.1.1 and earlier versions.
Re-centering is essential for timestamps in Unix epoch format as timestamps in an NWB file must be relative

to the start of the session. The heuristic here is to check if the first timestamp is larger than the length of a 10-year experiment in seconds. If so, it’s assumed that the timestamps are in Unix epoch format.

References

Issue discussion on FicTrac’s timestamp inconsistencies: rjdmoore/fictrac#29

get_timestamps()[source]#

Retrieve the timestamps for the data in this interface.

Returns:: timestamps – The timestamps for the data stream.
Return type:: numpy.ndarray

set_aligned_timestamps(aligned_timestamps)[source]#

Replace all timestamps for this interface with those aligned to the common session start time.

Must be in units seconds relative to the common ‘session_start_time’.

Parameters:: aligned_timestamps (numpy.ndarray) – The synchronized timestamps for data in this interface.

set_aligned_starting_time(aligned_starting_time)[source]#

Align the starting time for this interface relative to the common session start time.

Must be in units seconds relative to the common ‘session_start_time’.

Parameters:: aligned_starting_time (float) – The starting time for all temporal data in this interface.

extract_session_start_time(file_path: Annotated[Path, PathType(path_type=file)], configuration_file_path: Annotated[Path, PathType(path_type=file)] | None = None) → datetime | None[source]#

Extract the session start time from a FicTrac data file or its configuration file.

The session start time is determined from the data file if the timestamps are in Unix epoch format. If not, the function defaults to extracting the date from the configuration file and assuming that the start time is midnight. If neither of these methods works, the function returns None.

The session start time, has two different meanings depending on the source of the FicTrac data: - For video file sources (.avi, .mp4, etc.), the session start time corresponds to the time when the FicTrac analysis commenced. That is, the session start time reflects the analysis time rather than the actual start of the experiment. - For camera sources (such as PGR or Basler), the session start time is either the time reported by the camera or the system time if the camera’s SDK does not provide timestamps to Fictrac. In both cases, this time is the experiment start time, barring synchronization issues.

Parameters:

file_path (FilePath) – Path to the FicTrac data file.
configuration_file_path (FilePath, optional) – Path to the FicTrac configuration file. If omitted, the function defaults to searching for “fictrac_config.txt” in the same directory as the data file.

Returns:

The session start time of in UTC as a datetime object. None if the session start time cannot be extracted.

Return type:

datetime | None

parse_fictrac_config(file_path: Annotated[Path, PathType(path_type=file)]) → dict[source]#

Parse a FicTrac configuration file and return a dictionary of its parameters.

Parameters:: file_path (FilePath) – Path to the configuration file in txt format.
Returns:: A dictionary where the keys are the parameter names and the values are the parameter values.
Return type:: dict

LightningPose#

class LightningPoseConverter(file_path: Annotated[pathlib._local.Path, PathType(path_type='file')], original_video_file_path: Annotated[pathlib._local.Path, PathType(path_type='file')], labeled_video_file_path: Annotated[pathlib._local.Path, PathType(path_type='file')] | None = None, image_series_original_video_name: str | None = None, image_series_labeled_video_name: str | None = None, verbose: bool = False)[source]#

Bases: BaseDataInterface

Primary conversion class for handling Lightning Pose data streams.

The converter for Lightning Pose format to convert the pose estimation data along with the original and the optional labeled video added as ImageSeries to NWB.

Parameters:

file_path (FilePath) – Path to the .csv file that contains the predictions from Lightning Pose.
original_video_file_path (FilePath) – Path to the original video file (.mp4).
labeled_video_file_path (FilePath, optional) – Path to the labeled video file (.mp4).
image_series_original_video_name (string, optional) – The name of the ImageSeries to add for the original video.
image_series_labeled_video_name (string, optional) – The name of the ImageSeries to add for the labeled video.
verbose (bool, default: False) – controls verbosity. True by default.

display_name: str | None = 'Lightning Pose Converter'#

keywords: tuple[str] = ('pose estimation', 'video')#

associated_suffixes: tuple[str] = ('.csv', '.mp4')#

info: str | None = 'Interface for handling multiple streams of lightning pose data.'#

get_metadata(*, use_new_metadata_format: bool = True) → DeepDict[source]#

Child DataInterface classes should override this to match their metadata.

Returns:: The metadata dictionary containing basic NWBFile metadata.
Return type:: DeepDict

add_to_nwbfile(nwbfile: NWBFile, metadata: dict, *args, reference_frame: str | None = None, confidence_definition: str | None = None, starting_frames_original_videos: list[int] | None = None, starting_frames_labeled_videos: list[int] | None = None, stub_test: bool = False)[source]#

Add behavior and pose estimation data, including original and labeled videos, to the specified NWBFile.

Parameters:

nwbfile (NWBFile) – The NWBFile object to which the data will be added.
metadata (dict) – Metadata dictionary containing information about the behavior and videos.
reference_frame (str, optional) – Description of the reference frame for pose estimation, by default None.
confidence_definition (str, optional) – Definition for the confidence levels in pose estimation, by default None.
starting_frames_original_videos (list of int, optional) – List of starting frames for the original videos, by default None.
starting_frames_labeled_videos (list of int, optional) – List of starting frames for the labeled videos, by default None.
stub_test (bool, optional) – If True, only a subset of the data will be added for testing purposes, by default False.

Neuralynx NVT#

class NeuralynxNvtInterface(file_path: Annotated[pathlib._local.Path, PathType(path_type='file')], *args, verbose: bool = False)[source]#

Bases: BaseTemporalAlignmentInterface

Data interface for Neuralynx NVT files. NVT files store position tracking information.

Interface for writing Neuralynx .nvt files to nwb.

Parameters:

file_path (FilePath) – Path to the .nvt file
verbose (bool, default: False) – controls verbosity.

display_name: str | None = 'Neuralynx NVT'#

keywords: tuple[str] = ('position tracking',)#

associated_suffixes: tuple[str] = ('.nvt',)#

info: str | None = 'Interface for writing Neuralynx position tracking .nvt files to NWB.'#

get_original_timestamps() → ndarray[source]#

Retrieve the original unaltered timestamps for the data in this interface.

This function should retrieve the data on-demand by re-initializing the IO.

Returns:: timestamps – The timestamps for the data stream.
Return type:: numpy.ndarray

get_timestamps() → ndarray[source]#

Retrieve the timestamps for the data in this interface.

Returns:: timestamps – The timestamps for the data stream.
Return type:: numpy.ndarray

set_aligned_timestamps(aligned_timestamps: ndarray) → None[source]#

Replace all timestamps for this interface with those aligned to the common session start time.

Must be in units seconds relative to the common ‘session_start_time’.

Parameters:: aligned_timestamps (numpy.ndarray) – The synchronized timestamps for data in this interface.

get_metadata() → DeepDict[source]#

Child DataInterface classes should override this to match their metadata.

Returns:: The metadata dictionary containing basic NWBFile metadata.
Return type:: DeepDict

get_metadata_schema() → dict[source]#

Retrieve JSON schema for metadata.

Returns:: The JSON schema defining the metadata structure.
Return type:: dict

add_to_nwbfile(nwbfile: NWBFile, metadata: dict | None = None, *args, add_position: bool = True, add_angle: bool | None = None)[source]#

Add NVT data to a given in-memory NWB file

Parameters:

nwbfile (NWBFile) – nwb file to which the recording information is to be added
metadata (dict, optional) – metadata info for constructing the nwb file.
add_position (bool, default=True)
add_angle (bool, optional) – If None, write angle as long as it is not all 0s

SLEAP#

class SLEAPInterface(file_path: Annotated[pathlib._local.Path, PathType(path_type='file')], *args, video_file_path: Annotated[pathlib._local.Path, PathType(path_type='file')] | None = None, verbose: bool = False, frames_per_second: float | None = None, track_name: str | None = None, video_name: str | None = None, metadata_key: str | None = None)[source]#

Bases: BasePoseEstimationInterface

Data interface for SLEAP datasets.

Interface for writing sleap .slp files to nwb using the sleap-io library.

Parameters:

file_path (FilePath) – Path to the .slp file (the output of sleap)
verbose (bool, default: False) – controls verbosity. True by default.
video_file_path (FilePath, optional) – The file path of the video for extracting timestamps.
frames_per_second (float, optional) – The frames per second (fps) or sampling rate of the video.
track_name (str, optional) – Which tracked individual to write. An NWB file holds one subject, so a multi-animal .slp takes one interface per track. Call get_available_tracks to see them. Required when the file has more than one; the only track is used when it has one. A file where no instance carries a track, a labeling project or an untracked single-animal recording, is written as one individual and takes no track name.
video_name (str, optional) – Which recording to write, as the stem of its path. A .slp assembled in the SLEAP GUI can hold several recordings, and those are separate sessions rather than separate views, so they belong in separate NWB files. Call get_available_videos to see them. Required when the file holds more than one; the only recording is used when it holds one.
metadata_key (str, optional) – Key addressing this interface’s entries in the dict-based metadata. Derived from the track when not given ("sleap_track_0"), so one interface per track of the same file gets a distinct key and a converter can merge them.

display_name: str | None = 'SLEAP'#

keywords: tuple[str] = ('pose estimation', 'tracking', 'video')#

associated_suffixes: tuple[str] = ('.slp', '.mp4')#

info: str | None = 'Interface for SLEAP pose estimation datasets.'#

classmethod get_source_schema() → dict[source]#

Infer the JSON schema for the source_data from the method signature (annotation typing).

Returns:: The JSON schema for the source_data.
Return type:: dict

static get_available_tracks(file_path: Annotated[Path, PathType(path_type=file)]) → list[str][source]#

Return the names of the tracks that carry at least one instance, in the file’s own order.

Labels.tracks records the identities the tracking run created rather than the ones that survived it, so a file can declare a track that no frame holds an instance for. Those are not offered, since selecting one has nothing to write. Reading the frames costs nothing here, as load_slp has already read them.

A file holding several recordings is answered as a whole: a track offered here can still be empty in one of them, which is caught when the recording is selected.

static get_available_videos(file_path: Annotated[Path, PathType(path_type=file)]) → list[str][source]#

Return the video names in a .slp file, one per recording labeled in it.

The stems of the paths the file stores, since those are absolute paths from the machine that did the labeling and rarely resolve on the machine doing the conversion.

get_original_timestamps() → ndarray[source]#

Retrieve the original unaltered timestamps for the data in this interface.

This function should retrieve the data on-demand by re-initializing the IO.

Returns:: timestamps – The timestamps for the data stream.
Return type:: numpy.ndarray

get_timestamps() → ndarray[source]#

Retrieve the timestamps for the data in this interface.

Returns:: timestamps – The timestamps for the data stream.
Return type:: numpy.ndarray

set_aligned_timestamps(aligned_timestamps: ndarray)[source]#

Replace all timestamps for this interface with those aligned to the common session start time.

Must be in units seconds relative to the common ‘session_start_time’.

Parameters:: aligned_timestamps (numpy.ndarray) – The synchronized timestamps for data in this interface.

add_to_nwbfile(nwbfile: NWBFile, metadata: dict | None = None, **conversion_options) → None[source]#: Write the named track’s PoseEstimation container to the file’s behavior module.

get_metadata() → DeepDict[source]#

Build the pose registries and their cross-references, with generic names and no free text.

An interface overrides this, calls it, and sets what its own format records. Nothing is invented here: no descriptions, no units, no reference frames, and no scorer, dimensions or video paths.

Video#

MedPC#

class MedPCInterface(file_path: Annotated[pathlib._local.Path, PathType(path_type='file')], *args, session_conditions: dict, start_variable: str, metadata_medpc_name_to_info_dict: dict, aligned_timestamp_names: list[str] | None = None, verbose: bool = False)[source]#

Bases: BaseTemporalAlignmentInterface

Data Interface for MedPC output files.

The output files from MedPC are raw text files that contain behavioral data from the operant box sessions such as lever presses, reward port entries, nose pokes, etc. The output text files format this data into a series of colon-separated variables that are either single-line (for metadata) or multi-line (for arrays). The multi-line variables keep a colon-separated index of the array every 5 elements. For example, a single variable might look like:

Start Date: 11/09/18

while a multi-line variable might look like:

A:
     175.150      270.750      762.050      762.900     1042.600
    1567.800     1774.950     2448.450     2454.050     2552.800
    2620.550     2726.250

Different sessions are usually separated by a blank line or two.

This data is parsed by the MedPCInterface and added to the NWBFile as Events and IntervalSeries objects in the behavior module.

Deprecated since version Use: MedPCArrayEventsInterface, which writes the same events as native pynwb.event.EventsTable objects into nwbfile.events. This interface will be removed in v0.12.0, and ndx-events leaves NeuroConv with it.

Initialize MedpcInterface.

Parameters:

file_path (FilePath) – Path to the MedPC file.
session_conditions (dict) – The conditions that define the session. The keys are the names of the single-line variables (ex. ‘Start Date’) and the values are the values of those variables for the desired session (ex. ‘11/09/18’).
start_variable (str) – The name of the variable that starts the session (ex. ‘Start Date’).
metadata_medpc_name_to_info_dict (dict) – A dictionary mapping the names of the desired variables in the MedPC file to an info dictionary with the names of the variables in the metadata and whether or not they are arrays. ex. {“Start Date”: {“name”: “start_date”, “is_array”: False}}
aligned_timestamp_names (list, optional) – The names of the variables that are externally aligned timestamps, which should be retrieved from self.timestamps_dict instead of the MedPC output file.
verbose (bool, optional) – Whether to print verbose output, by default True

keywords: tuple[str] = ('behavior',)#

display_name: str | None = 'MedPC'#

info: str | None = 'Interface for handling MedPC output files.'#

associated_suffixes: tuple[str] = ('.txt',)#

get_metadata() → DeepDict[source]#

Child DataInterface classes should override this to match their metadata.

Returns:: The metadata dictionary containing basic NWBFile metadata.
Return type:: DeepDict

get_metadata_schema() → dict[source]#

Retrieve JSON schema for metadata.

Returns:: The JSON schema defining the metadata structure.
Return type:: dict

get_original_timestamps(medpc_name_to_info_dict: dict) → dict[str, ndarray][source]#

Retrieve the original unaltered timestamps dictionary for the data in this interface.

This function retrieves the data on-demand by re-reading the medpc file.

Parameters:: medpc_name_to_info_dict (dict) – A dictionary mapping the names of the desired variables in the MedPC file to an info dictionary with the names of the variables in the metadata and whether or not they are arrays. ex. {“A”: {“name”: “left_nose_poke_times”, “is_array”: True}}
Returns:: timestamps_dict – A dictionary mapping the names of the variables to the original medpc timestamps.
Return type:: dict

get_timestamps() → dict[str, ndarray][source]#

Retrieve the timestamps dictionary for the data in this interface.

Returns:: timestamps_dict – A dictionary mapping the names of the variables to the timestamps.
Return type:: dict

set_aligned_timestamps(aligned_timestamps_dict: dict[str, ndarray]) → None[source]#

Replace all timestamps for this interface with those aligned to the common session start time.

Must be in units seconds relative to the common ‘session_start_time’.

Parameters:: aligned_timestamps_dict (dict) – A dictionary mapping the names of the variables to the synchronized timestamps for data in this interface.

set_aligned_starting_time(aligned_starting_time: float, medpc_name_to_info_dict: dict) → None[source]#

Align the starting time for this interface relative to the common session start time.

Must be in units seconds relative to the common ‘session_start_time’.

Parameters:

aligned_starting_time (float) – The starting time for all temporal data in this interface.
medpc_name_to_info_dict (dict) – A dictionary mapping the names of the desired variables in the MedPC file to an info dictionary with the names of the variables in the metadata and whether or not they are arrays. ex. {“A”: {“name”: “left_nose_poke_times”, “is_array”: True}}

add_to_nwbfile(nwbfile: NWBFile, metadata: dict) → None[source]#

Define a protocol for mapping the data from this interface to NWB neurodata objects.

These neurodata objects should also be added to the in-memory pynwb.NWBFile object in this step.

Parameters:

nwbfile (pynwb.NWBFile) – The in-memory object to add the data to.
metadata (dict) – Metadata dictionary with information used to create the NWBFile.
**conversion_options – Additional keyword arguments to pass to the .add_to_nwbfile method.

VAME#

DataInterface for VAME behavioral segmentation data.

class VameInterface(file_path: Annotated[pathlib._local.Path, PathType(path_type='file')], *, session_name: str | None = None, latent_vectors_file_path: Annotated[pathlib._local.Path, PathType(path_type='file')] | None = None, motif_labels_file_paths: dict[str, Annotated[pathlib._local.Path, PathType(path_type='file')]] | None = None, community_labels_file_paths: dict[str, Annotated[pathlib._local.Path, PathType(path_type='file')]] | None = None, sampling_frequency_hz: float | None = None, metadata_key: str = 'VAMEProject', verbose: bool = False)[source]#

Bases: BaseTemporalAlignmentInterface

DataInterface for VAME behavioral segmentation data using the ndx-vame NWB extension.

VAME (Variational Animal Motion Encoding) segments animal behavior into discrete motifs by training a variational autoencoder on pose estimation time series.

This interface writes the per-frame motif labels, latent-space embeddings, and community labels to NWB using the ndx-vame extension.

The NWB container name and all series descriptions are controlled via the metadata dict under metadata["Behavior"]["Vame"].

Metadata Structure#

The container and its series live in separate, flat registries under metadata["Behavior"]["Vame"] (one registry per type), following the unified metadata pattern used elsewhere in NeuroConv: each series links up to its VAMEProject via vame_project_metadata_key rather than being nested inside it. Series keys are project-prefixed so several projects can share one NWB file.

metadata = {
    "Behavior": {
        "Vame": {
            "VameProjects": {
                "VAMEProject": {                       # keyed by metadata_key (default "VAMEProject")
                    "name": "VAMEProject",
                    "pose_estimation_metadata_key": "DLC",  # optional, -> Pose/PoseEstimations
                    "video_metadata_key": "video_0",   # optional, -> Behavior/InternalVideos|ExternalVideos
                }
            },
            "MotifSeries": {
                "VAMEProject_motif_kmeans": {
                    "name": "MotifSeriesKmeans",
                    "description": "VAME behavioral motif labels.",
                    "algorithm": "kmeans",
                    "vame_project_metadata_key": "VAMEProject",
                    "latent_space_metadata_key": "VAMEProject_latent_space",  # optional
                },
            },
            "CommunitySeries": {
                "VAMEProject_community_kmeans": {
                    "name": "CommunitySeriesKmeans",
                    "description": "VAME community labels ...",
                    "vame_project_metadata_key": "VAMEProject",
                    "motif_series_metadata_key": "VAMEProject_motif_kmeans",  # -> MotifSeries
                },
            },
            "LatentSpaceSeries": {                     # optional — one per project, shared across runs
                "VAMEProject_latent_space": {
                    "name": "LatentSpaceSeries",
                    "description": "VAME latent-space embeddings (30 dimensions per frame).",
                    "vame_project_metadata_key": "VAMEProject",
                },
            },
        }
    }
}

The metadata can be customized by:

Calling get_metadata() to retrieve the defaults.
Modifying the returned dictionary as needed.
Passing the modified metadata to add_to_nwbfile() or run_conversion().

Notes

motif_series_metadata_key in a CommunitySeries entry is automatically set to the matching run’s MotifSeries key when the same run exists in motif_labels_file_paths.
pose_estimation_metadata_key must name a PoseEstimation entry (in Pose/PoseEstimations) whose container is already present in the NWB file when add_to_nwbfile() is called (i.e. the pose estimation interface must run first).
video_metadata_key optionally names a video entry (in Behavior/InternalVideos or Behavior/ExternalVideos) whose ImageSeries is already present in the NWB file; each EthogramBouts links to it via source_video (the video interface must run first).
Everything inside a single VAME project links to the same pose estimation and the same video: pose_estimation_metadata_key and video_metadata_key live on the VameProjects entry (not on the per-run series), so every MotifSeries / CommunitySeries and every derived EthogramBouts in that project shares one PoseEstimation and one ImageSeries. These keys are single-valued by design: a VAME project is one aligned session with a single pose source. This assumes a single-camera PoseEstimation; a multi-camera project would point the same single key at an aggregate multi-camera pose container rather than needing multiple links.
To store two completely separate VAME trainings in the same file, create two VameInterface instances with different metadata_key values and wrap them in an NWBConverter.

Initialize VameInterface.

Parameters:

file_path (FilePath) – Path to the VAME config.yaml project file. The full configuration is serialized as JSON and stored in the VAMEProject.vame_config field.
session_name (str, optional) – Session name as it appears in config.yaml under session_names. When provided, motif labels, latent vectors, and community labels are auto-discovered from the standard VAME results layout under the config file’s parent directory:
```
<config_parent>/results/<session_name>/VAME/
    latent_vectors.npy
    <algorithm>-<n_clusters>/
        <n_clusters>_<algorithm>_label_<session_name>.npy
        community/
            cohort_community_label_<session_name>.npy
```
Algorithms and n_clusters are read from the config. Explicitly supplied motif_labels_file_paths, latent_vectors_file_path, or community_labels_file_paths arguments take precedence over auto-discovered paths.
latent_vectors_file_path (FilePath, optional) – Path to the .npy file containing VAME latent-space vectors (2D float32 array of shape (n_frames, n_latent_dims)). Shared across all algorithm runs — a single LatentSpaceSeries is written and each MotifSeries links to it.
motif_labels_file_paths (dict[str, FilePath], optional) – Dict mapping a run name to the .npy file containing VAME motif labels (1D int array, one cluster ID per video frame). Each entry becomes a separate MotifSeries in the NWB file. Example:
```
{"kmeans": "path/to/15_kmeans_label.npy", "hmm": "path/to/15_hmm_label.npy"}
```
community_labels_file_paths (dict[str, FilePath], optional) – Dict mapping a run name to the .npy file containing VAME community labels (1D int array, one community ID per video frame). When a run key matches one in motif_labels_file_paths, the CommunitySeries is automatically linked to that MotifSeries. Example:
```
{"kmeans": "path/to/cohort_community_label.npy"}
```
sampling_frequency_hz (float, optional) – Video acquisition rate in Hz (frames per second). Required when not providing aligned timestamps via set_aligned_timestamps().
metadata_key (str, default “VAMEProject”) – Key of this interface’s VAMEProject entry in metadata["Behavior"]["Vame"]["VameProjects"] (also the prefix of its series keys). Change this when storing results from multiple VAME projects in the same NWB file so each project has a unique metadata entry and VAMEProject container.
verbose (bool, default False) – Controls verbosity of the conversion process.

display_name: str | None = 'VAME'#

keywords: tuple[str] = ('VAME', 'behavioral motifs', 'pose segmentation')#

associated_suffixes: tuple[str] = ('.npy', '.yaml')#

info: str | None = 'Interface for adding data from VAME (Variational Animal Motion Encoding).'#

classmethod get_source_schema() → dict[source]#

Infer the JSON schema for the source_data from the method signature (annotation typing).

Returns:: The JSON schema for the source_data.
Return type:: dict

get_metadata_schema() → dict[source]#

Retrieve JSON schema for metadata.

Returns:: The JSON schema defining the metadata structure.
Return type:: dict

get_metadata() → DeepDict[source]#

Child DataInterface classes should override this to match their metadata.

Returns:: The metadata dictionary containing basic NWBFile metadata.
Return type:: DeepDict

get_original_timestamps() → ndarray[source]#

Retrieve the original unaltered timestamps for the data in this interface.

This function should retrieve the data on-demand by re-initializing the IO.

Returns:: timestamps – The timestamps for the data stream.
Return type:: numpy.ndarray

get_timestamps() → ndarray[source]#

Retrieve the timestamps for the data in this interface.

Returns:: timestamps – The timestamps for the data stream.
Return type:: numpy.ndarray

set_aligned_timestamps(aligned_timestamps: ndarray) → None[source]#

Replace all timestamps for this interface with those aligned to the common session start time.

Must be in units seconds relative to the common ‘session_start_time’.

Parameters:: aligned_timestamps (numpy.ndarray) – The synchronized timestamps for data in this interface.

static get_available_sessions(file_path: Annotated[Path, PathType(path_type=file)]) → list[str][source]#

Return the session names listed in a VAME config.yaml project file.

Parameters:: file_path (FilePath) – Path to the VAME config.yaml project file.
Returns:: Session names from the session_names field of the config, or an empty list if that field is absent.
Return type:: list[str]

add_to_nwbfile(nwbfile: NWBFile, metadata: dict | None = None, *, stub_test: bool = False, data_to_write: Literal['algorithm_output', 'ethogram', 'both'] = 'both') → None[source]#

Write VAME outputs to an NWBFile as a VAMEProject container.

The container name and all series descriptions are taken from the flat registries under metadata["Behavior"]["Vame"]. Call get_metadata() to inspect the defaults and override specific fields before conversion.

Parameters:

nwbfile (NWBFile) – Target NWB file.
metadata (dict, optional) – Metadata dictionary. VAME-specific fields live in flat registries under metadata["Behavior"]["Vame"]:
- "VameProjects" – dict of project key → {name, pose_estimation_metadata_key, video_metadata_key}. pose_estimation_metadata_key references a PoseEstimation entry whose container is already present in the NWB file (the pose interface must run first); video_metadata_key optionally references a video entry (in Behavior/InternalVideos or Behavior/ExternalVideos) whose ImageSeries the ethogram bouts link to.
- "MotifSeries" – dict of series key → {name, description, algorithm, vame_project_metadata_key, latent_space_metadata_key}.
- "CommunitySeries" – dict of series key → {name, description, algorithm, vame_project_metadata_key, motif_series_metadata_key}.
- "LatentSpaceSeries" – dict of series key → {name, description, vame_project_metadata_key}.
Each series links to its project via vame_project_metadata_key.
stub_test (bool, default False) – If True, only the first 100 frames of each data array are written.
data_to_write ({“algorithm_output”, “ethogram”, “both”}, default “both”) – Which of the two outputs to write. "algorithm_output" writes only the VAMEProject container and its faithful ndx-vame series (motif, community, latent space). "ethogram" writes only the derived, curated ndx-ethogram products. "both" (the default) writes both. With "ethogram" the faithful MotifSeries is absent from the file, so each bouts table’s source back-link to it is dropped; the source_pose and source_video links are external references and are kept.

Notes

For each motif run this also writes a curated ndx-ethogram product derived from the per-frame MotifSeries: an EthogramBouts table (the motif labels run-length-encoded into one row per bout) and its Ethogram catalogue. When both layers are written the faithful MotifSeries is kept and the bouts link back to it via source.

keypoint-MoSeq#

DataInterface for keypoint-MoSeq behavioral segmentation output.

class MoseqKeyPointsInterface(file_path: Annotated[pathlib._local.Path, PathType(path_type='file')], *, sampling_frequency_hz: float, recording_name: str | None = None, metadata_key: str | None = None, verbose: bool = False)[source]#

Bases: BaseDataInterface

DataInterface for keypoint-MoSeq output (results.h5).

Writes one recording of a results.h5: the syllables as an ndx-ethogram EthogramBouts table with its Ethogram catalogue, the centroid as a SpatialSeries in Position, the heading as a SpatialSeries in CompassDirection, and the latent-state trajectory as a TimeSeries.

Notes

There is no time base in the source. results.h5 carries neither timestamps nor a frame rate; the rate is a property of the recording the keypoints came from and lives only in the keypoint-MoSeq project config.yml, where it was typed by the user rather than measured, so sampling_frequency_hz has to be supplied.
The first frames of every recording are padding. The autoregressive model has no state for the first nlags frames (3 by default), so keypoint-MoSeq fills them by repeating the first real syllable. Those frames are written as they are, which makes the first bout start earlier than the first true syllable; the padding repeats a real value, so it cannot be detected from the file.
Syllable ids carry no meaning outside the run that produced them. They are renumbered at the end of a fit by how often each state is entered, and only the states the fit actually entered get an id, so the ids are neither stable across runs nor contiguous. The catalogue therefore enumerates the ids present in this recording rather than the configured state space, which results.h5 does not record.
A recording name maps to no session or subject. It comes from the input filename, so it carries a DeepLabCut scorer suffix when the pose came from DeepLabCut and is a bare name otherwise. Nothing in the file relates it to a session, so the links to an upstream PoseEstimation and to a behavioral video are supplied through metadata and never parsed from the name.

Initialize MoseqKeyPointsInterface.

Parameters:

file_path (FilePath) – Path to the keypoint-MoSeq results.h5.
sampling_frequency_hz (float) – Frame rate of the video the keypoints came from, in Hz. Required because keypoint-MoSeq records no time base of its own; take it from that video or from the fps field of the keypoint-MoSeq project’s config.yml.
recording_name (str, optional) – Name of the recording group to write. Recordings in one file are separate recordings with their own frame counts, so one interface writes one of them. Optional when the file holds a single recording; required otherwise. Use get_available_recordings() to list them.
metadata_key (str, optional) – Key of this interface’s entries in the metadata["Behavior"]["MoseqKeyPoints"] registries and in metadata["Behavior"]["Ethograms"]. Defaults to "keypoint_moseq". Change it when writing two recordings into one NWB file so their entries do not collide.
verbose (bool, default False) – Controls verbosity of the conversion process.

display_name: str | None = 'keypoint-MoSeq'#

keywords: tuple[str] = ('keypoint-MoSeq', 'MoSeq', 'behavioral syllables', 'pose segmentation')#

associated_suffixes: tuple[str] = ('.h5',)#

info: str | None = 'Interface for adding data from keypoint-MoSeq (Motion Sequencing on pose keypoints).'#

classmethod get_source_schema() → dict[source]#

Infer the JSON schema for the source_data from the method signature (annotation typing).

Returns:: The JSON schema for the source_data.
Return type:: dict

static get_available_recordings(file_path: Annotated[Path, PathType(path_type=file)]) → list[str][source]#

Return the names of the recordings held in a keypoint-MoSeq results.h5.

Parameters:: file_path (FilePath) – Path to the keypoint-MoSeq results.h5.
Returns:: One name per recording group, as keypoint-MoSeq derived it from the input filename.
Return type:: list of str

get_metadata_schema() → dict[source]#

Retrieve JSON schema for metadata.

Returns:: The JSON schema defining the metadata structure.
Return type:: dict

get_metadata() → DeepDict[source]#

Child DataInterface classes should override this to match their metadata.

Returns:: The metadata dictionary containing basic NWBFile metadata.
Return type:: DeepDict

add_to_nwbfile(nwbfile: NWBFile, metadata: dict | None = None, *, data_to_write: Literal['algorithm_output', 'ethogram', 'both'] = 'both') → None[source]#

Write one keypoint-MoSeq recording to an NWBFile.

Parameters:

nwbfile (NWBFile) – Target NWB file.
metadata (dict, optional) – Metadata dictionary. This interface’s fields live in flat registries under metadata["Behavior"]["MoseqKeyPoints"], each keyed by metadata_key:
- "Centroids" – {name, description, unit, reference_frame, container_name} for the centroid SpatialSeries and the Position container holding it.
- "Headings" – the same fields for the heading SpatialSeries and its CompassDirection container.
- "LatentStates" – {name, description, unit} for the latent-state TimeSeries.
- "Recordings" – {pose_estimation_metadata_key, video_metadata_key}, the optional links the curated bouts carry back to the pose and the video they were derived from. Both are registry addresses, resolved through metadata["Pose"]["PoseEstimations"] and metadata["Behavior"]["InternalVideos"] / ["ExternalVideos"], so the object named there must already be in the file.
The curated EthogramBouts and Ethogram names and descriptions live in the shared metadata["Behavior"]["Ethograms"][metadata_key] registry.
data_to_write ({“algorithm_output”, “ethogram”, “both”}, default “both”) – Which of the two outputs to write. "algorithm_output" writes only the per-frame arrays keypoint-MoSeq produced: the centroid, the heading and the latent-state trajectory. "ethogram" writes only the derived ndx-ethogram products, and since the latent-state series is then absent from the file the bouts’ source back-link to it is dropped; the source_pose and source_video links are external references and are kept. "both" (the default) writes both.

Behavior#

Audio#

BORIS#

DeepLabCut#

Metadata Structure#

FicTrac#

LightningPose#

Neuralynx NVT#

SLEAP#

Video#

MedPC#

VAME#

Metadata Structure#

keypoint-MoSeq#

This Page