VAME data conversion#
Install NeuroConv with the additional dependencies necessary for reading VAME data.
pip install "neuroconv[vame]"
Convert VAME behavioral segmentation
outputs to NWB using
VameInterface.
The interface reads per-session .npy output files—motif labels, latent-space
vectors and community labels—and writes them to NWB using the
ndx-vame NWB extension.
In addition to the faithful per-frame MotifSeries, the interface automatically derives a
curated ndx-ethogram product for each motif
run: an EthogramBouts table (the per-frame motif labels run-length-encoded into one row per
bout) plus its Ethogram catalogue. The faithful MotifSeries is kept alongside it, and each
bout links back to it via source.
A typical VAME project has the following output layout for each session:
<project>/
├── config.yaml
└── results/
└── <session_name>/
└── VAME/
├── latent_vectors.npy # (n_frames, zdims) float32
├── hmm-15/
│ ├── 15_hmm_label_<session>.npy
│ └── community/
│ └── cohort_community_label_<session>.npy
└── kmeans-15/
├── 15_kmeans_label_<session>.npy
└── community/
└── cohort_community_label_<session>.npy
Full conversion (auto-discover all runs from config)#
Pass session_name and the interface discovers all motif labels, latent vectors, and
community labels from the standard VAME results layout under the config file’s parent
directory. Algorithms and n_clusters are read directly from config.yaml, so both
kmeans and hmm runs end up in one VAMEProject group automatically.
When a key in community_labels_file_paths matches a key in motif_labels_file_paths,
get_metadata automatically sets motif_series_metadata_key so each CommunitySeries is
linked to its corresponding MotifSeries. No extra metadata wiring is needed.
Use get_available_sessions()
to list the session names recorded in config.yaml.
>>> from datetime import datetime
>>> from zoneinfo import ZoneInfo
>>> from pathlib import Path
>>> from neuroconv.datainterfaces import VameInterface
>>> project = BEHAVIOR_DATA_PATH / "vame" / "my_vame_project"
>>> # Discover available session names from config.yaml
>>> sessions = VameInterface.get_available_sessions(project / "config.yaml")
>>> session = sessions[0]
>>> interface = VameInterface(
... file_path=project / "config.yaml",
... session_name=session,
... sampling_frequency_hz=30.0,
... )
>>> metadata = interface.get_metadata()
>>> metadata["NWBFile"].update(
... session_start_time=datetime(2024, 1, 1, 12, 0, 0, tzinfo=ZoneInfo("US/Pacific")),
... session_description="Open-field behavioral recording segmented with VAME.",
... )
>>> # Add subject information (required for DANDI upload)
>>> metadata["Subject"] = dict(subject_id="subject1", species="Mus musculus", sex="M", age="P30D")
>>> interface.run_conversion(nwbfile_path=path_to_save_nwbfile, metadata=metadata)
Explicit file paths take precedence over auto-discovered ones, so you can override individual paths while still relying on auto-discovery for the rest:
>>> interface = VameInterface(
... file_path=project / "config.yaml",
... session_name=session,
... # override just the kmeans community labels; everything else is auto-discovered
... community_labels_file_paths={
... "kmeans": project / "results" / session / "VAME" / "kmeans-15" / "community" / f"cohort_community_label_{session}.npy",
... },
... sampling_frequency_hz=30.0,
... )
Specifying Metadata#
VAME metadata lives in flat registries under metadata["Behavior"]["Vame"]: a
VameProjects registry for the container, plus MotifSeries, CommunitySeries, and
LatentSpaceSeries registries for the series, each linking back to its project via
vame_project_metadata_key. metadata_key defaults to "VAMEProject" and sets both the
VameProjects entry key and the default VAMEProject group name written to the NWB file.
Series keys are project-prefixed and type-distinct (for example "VAMEProject_motif_kmeans")
so that several projects can share one NWB file without colliding.
Call get_metadata()
to retrieve the auto-populated defaults, then use
dict_deep_update() to update the defaults before conversion:
>>> from neuroconv.utils import dict_deep_update
>>> custom_metadata = {
... "NWBFile": {
... "session_start_time": datetime(2024, 1, 1, 12, 0, 0, tzinfo=ZoneInfo("US/Pacific")),
... "session_description": "Open-field behavioral recording segmented with VAME.",
... },
... "Subject": dict(subject_id="subject1", species="Mus musculus", sex="M", age="P30D"),
... "Behavior": {
... "Vame": {
... "VameProjects": {
... "VAMEProject": {"name": "VAMEKmeans15"},
... },
... "MotifSeries": {
... "VAMEProject_motif_kmeans": {
... "description": "k-means motif labels (15 clusters) for the open-field test session.",
... },
... },
... },
... },
... }
>>> metadata = dict_deep_update(interface.get_metadata(), custom_metadata)
>>> interface.run_conversion(nwbfile_path=path_to_save_nwbfile, metadata=metadata, overwrite=True)
Temporal alignment#
When the VAME outputs need to be aligned to another data stream, use
set_aligned_timestamps()
instead of sampling_frequency_hz:
>>> import numpy as np
>>> motif_labels_file_path = project / "results" / session / "VAME" / "kmeans-15" / f"15_kmeans_label_{session}.npy"
>>> aligned_timestamps = np.arange(len(np.load(motif_labels_file_path))) / 30.0
>>> interface = VameInterface(
... file_path=project / "config.yaml",
... motif_labels_file_paths={"kmeans": motif_labels_file_path},
... )
>>> interface.set_aligned_timestamps(aligned_timestamps)
>>> metadata = interface.get_metadata()
>>> metadata["NWBFile"].update(
... session_start_time=datetime(2024, 1, 1, 12, 0, 0, tzinfo=ZoneInfo("US/Pacific")),
... session_description="Open-field behavioral recording segmented with VAME.",
... )
>>> interface.run_conversion(nwbfile_path=path_to_save_nwbfile, metadata=metadata, overwrite=True)
Linking to pose estimation#
To record the upstream pose data used by VAME, set pose_estimation_metadata_key in the
metadata to the name of an existing PoseEstimation container. The container must already
be present in the NWB file when add_to_nwbfile is called—typically written by a preceding
DeepLabCutInterface
or SLEAPInterface.
>>> from neuroconv import NWBConverter
>>> from neuroconv.datainterfaces import DeepLabCutInterface
>>> class PoseAndVameConverter(NWBConverter):
... data_interface_classes = dict(
... DLC=DeepLabCutInterface,
... VAME=VameInterface,
... )
>>> converter = PoseAndVameConverter(
... source_data=dict(
... DLC=dict(file_path="/path/to/dlc_output.h5"),
... VAME=dict(
... file_path=str(project / "config.yaml"),
... motif_labels_file_paths={"kmeans": str(motif_labels_file_path)},
... sampling_frequency_hz=30.0,
... ),
... )
... )
>>> metadata = converter.get_metadata()
>>> metadata["Behavior"]["Vame"]["VameProjects"]["VAMEProject"]["pose_estimation_metadata_key"] = (
... "PoseEstimationDeepLabCut"
... )
>>> converter.run_conversion(nwbfile_path=path_to_save_nwbfile, metadata=metadata)
Multiple VAME projects in one file#
Use metadata_key to store results from multiple VAME projects in the same NWB file.
Each instance gets its own metadata entry and its own VAMEProject group:
>>> hmm_labels_file_path = project / "results" / session / "VAME" / "hmm-15" / f"15_hmm_label_{session}.npy"
>>> class MyConverter(NWBConverter):
... data_interface_classes = dict(
... VameKmeans=VameInterface,
... VameHmm=VameInterface,
... )
>>> converter = MyConverter(
... source_data=dict(
... VameKmeans=dict(
... file_path=str(project / "config.yaml"),
... motif_labels_file_paths={"kmeans": str(motif_labels_file_path)},
... sampling_frequency_hz=30.0,
... metadata_key="VAMEProjectA",
... ),
... VameHmm=dict(
... file_path=str(project / "config.yaml"),
... motif_labels_file_paths={"hmm": str(hmm_labels_file_path)},
... sampling_frequency_hz=30.0,
... metadata_key="VAMEProjectB",
... ),
... )
... )
>>> custom_metadata = {
... "NWBFile": {
... "session_start_time": datetime(2024, 1, 1, 12, 0, 0, tzinfo=ZoneInfo("US/Pacific")),
... "session_description": "Multi-project VAME behavioral segmentation.",
... },
... "Subject": dict(subject_id="subject1", species="Mus musculus", sex="M", age="P30D"),
... "Behavior": {
... "Vame": {
... "MotifSeries": {
... "VAMEProjectA_motif_kmeans": {"description": "k-means motif labels (15 clusters)."},
... "VAMEProjectB_motif_hmm": {"description": "HMM motif labels (15 states)."},
... },
... },
... },
... }
>>> metadata = dict_deep_update(converter.get_metadata(), custom_metadata)
>>> converter.run_conversion(
... nwbfile_path=path_to_save_nwbfile,
... metadata=metadata,
... overwrite=True,
... )