Fiber Photometry Metadata Structure#

This document describes the fiber photometry metadata system in NeuroConv, used by the single-series fiber photometry interfaces built on BaseFiberPhotometryInterface (currently TDTFiberPhotometryInterface and DoricFiberPhotometryInterface). It is intended as a reference for developers contributing new fiber photometry interfaces or modifying existing ones, and to document the design decisions behind the format.

A single-series interface writes exactly one FiberPhotometryResponseSeries (the ndx-fiber-photometry neurodata type), assembled from one or more input streams (atomic source signals — a TDT store, a Doric dataset). That series references a region of a FiberPhotometryTable, and each row of the table references the physical hardware (optical fiber, excitation source, photodetector, filters) and the indicator that produced the trace — much as an ecephys ElectricalSeries references a region of the electrodes table.

The typical conversion is a single interface writing one response series. That one series can still be multi-channel — a multi-fiber, dual-wavelength recording (a calcium signal and an isosbestic control per fiber) is usually written as one series whose data columns map to several table rows. NeuroConv also supports running several fiber photometry interfaces together, each writing its own series into one shared table (as several ecephys recording interfaces share one electrodes table), but that is the exception, used when you deliberately want distinct response series rather than channels of one series. The purpose of the fiber photometry metadata dict is to describe the hardware once and let each interface point its response series at the table rows it needs.

Design Principles#

The fiber photometry metadata system follows the same spirit as the ophys and ecephys systems (see Ophys Metadata Structure), specialized to the ndx-fiber-photometry data model:

  1. Its own top-level block, but shared devices. The fiber-photometry-specific metadata (indicators, viruses/injections, the FiberPhotometryTable, commanded voltage, and the response series) lives under metadata["FiberPhotometry"], not nested under metadata["Ophys"]. Fiber photometry is technically an optical method, but — like the ecephys/icephys split — it has its own interfaces, its own extension, and its own metadata block. The one exception is the hardware itself: device models and device instances live in the shared, cross-modality top-level registry metadata["Devices"] / metadata["DeviceModels"], because ndx-ophys-devices classes such as ExcitationSource are reused by other modalities and should not grow a fiber-photometry-only writing path.

    metadata["FiberPhotometry"]["FiberPhotometryIndicators"]["gcamp"]["label"] = "GCaMP7b"
metadata["DeviceModels"]["ofm"]["numerical_aperture"] = 0.48

  2. Dict-keyed containers, keyed by ``metadata_key``. Following the ophys metadata system, every container is a dict whose keys are metadata_key handles. The hardware and biology shared across a file (device models, devices, optical fibers, indicators, viral vectors and injections, commanded voltage, and the table’s rows) are keyed this way, and each interface’s response series is keyed by its own metadata_key. Keying every level lets several interfaces run in one conversion without clashing and merges cleanly across interfaces (dicts merge by key), and it decouples an object’s routing handle from its NWB name.

  3. Linking is by ``_metadata_key``, resolved at write time. Entries reference each other with <thing>_metadata_key fields holding the key of the referenced entry — a device names its model via device_model_metadata_key (into metadata["DeviceModels"]), a table row names its optical fiber via optical_fiber_metadata_key (into metadata["Devices"]), and a response series names the table rows it spans via fiber_photometry_table_region (a list of row keys). This matches the ophys convention. At write time each key is resolved through its container to the entry’s name and then to the actual NWB object; row references resolve to integer row indices (their position in the merged rows dict).

  4. One shared, keyed-row table. A file has a single FiberPhotometryTable. Every response series references a region of its rows by row key, so regions never depend on fragile integer indices.

  5. Runnable defaults with honest sentinels. get_metadata() returns a complete, editable scaffold so an interface runs on zero user metadata. Required fields the user must supply are pre-filled with sentinels — NaN for the required numeric wavelengths and the string "PLACEHOLDER" for required strings — and add_to_nwbfile warns about any that survive to write time (or raises, under strict=True).

Metadata Structure Overview#

The complete fiber photometry metadata structure (one interface with metadata_key="gcamp_dms"):

metadata = {
    "NWBFile": {...},   # Session-level metadata
    "Subject": {...},   # Subject information

    # ----- Shared, cross-modality device registry (top-level, from #1780) -----

    # Device models. Each entry carries a `type` naming its concrete ndx-ophys-devices class.
    "DeviceModels": {
        "ofm": {"type": "OpticalFiberModel", "name": "optical_fiber_model",
                "manufacturer": "Doric Lenses", "numerical_aperture": 0.48},
        "esm": {"type": "ExcitationSourceModel", "name": "excitation_source_model",
                "manufacturer": "Doric Lenses", "source_type": "LED", "excitation_mode": "one-photon"},
        "pdm": {"type": "PhotodetectorModel", "name": "photodetector_model",
                "manufacturer": "Doric Lenses", "detector_type": "photodiode"},
        # (also BandOpticalFilterModel, EdgeOpticalFilterModel, DichroicMirrorModel)
    },

    # Device instances. Each carries a `type` and links its model by `device_model_metadata_key`.
    "Devices": {
        "fiber_dms": {"type": "OpticalFiber", "name": "optical_fiber_dms", "device_model_metadata_key": "ofm",
                      "fiber_insertion": {"depth_in_mm": 4.2, "insertion_position_ap_in_mm": 0.8}},
        "led_465": {"type": "ExcitationSource", "name": "excitation_source_465",
                    "device_model_metadata_key": "esm"},
        "pd": {"type": "Photodetector", "name": "photodetector", "device_model_metadata_key": "pdm"},
        # (also BandOpticalFilter, EdgeOpticalFilter, DichroicMirror)
    },

    "FiberPhotometry": {

        # ----- Shared containers: dicts keyed by metadata_key (built once per file) -----

        # Indicators, and optionally the viruses/injections that delivered them
        "FiberPhotometryViruses": {
            "aav_gcamp": {"name": "aav_gcamp", "construct_name": "AAV-GCaMP7b"},
        },
        "FiberPhotometryVirusInjections": {
            "inj_dms": {"name": "injection_dms", "viral_vector_metadata_key": "aav_gcamp"},
        },
        "FiberPhotometryIndicators": {
            "gcamp": {"name": "gcamp", "label": "GCaMP7b",
                      "viral_vector_injection_metadata_key": "inj_dms"},
        },

        # Optional commanded-voltage drive signals (data-bearing; read from a stream)
        "CommandedVoltageSeries": {
            "cv_dms": {"name": "commanded_voltage_dms", "stream_name": "Fi1r", "index": 0,
                       "unit": "volts", "frequency": 211.0},
        },

        # The single shared table, rows keyed by metadata_key. Each row = one fiber x one channel.
        "FiberPhotometryTable": {
            "name": "fiber_photometry_table",
            "description": "Fiber photometry setup.",
            "rows": {
                "dms_465": {
                    "location": "DMS",
                    "excitation_wavelength_in_nm": 465.0,
                    "emission_wavelength_in_nm": 520.0,
                    "indicator_metadata_key": "gcamp",
                    "optical_fiber_metadata_key": "fiber_dms",
                    "excitation_source_metadata_key": "led_465",
                    "photodetector_metadata_key": "pd",
                    "commanded_voltage_series_metadata_key": "cv_dms",   # optional
                },
            },
        },

        # ----- Per-interface response series, keyed by metadata_key -----
        "gcamp_dms": {
            "name": "FiberPhotometryResponseSeries",
            "description": "GCaMP7b calcium signal in DMS.",
            "unit": "a.u.",
            "fiber_photometry_table_region": ["dms_465"],     # list of ROW metadata_keys
            "fiber_photometry_table_region_description": "DMS calcium signal.",
        },
    },
}

The metadata_key Parameter#

Every single-series interface accepts a keyword-only metadata_key parameter, following the same pattern as the ophys interfaces (see Ophys Metadata Structure). When None (the default), the interface generates a key from the parameters that make it unique — for fiber photometry, the stream_names — so two interfaces over different streams get distinct keys automatically:

interface = TDTFiberPhotometryInterface(folder_path=..., stream_names="_405R")
interface.metadata_key   # -> "fiber_photometry_405r"

metadata_key scopes only this interface’s response series entry. The shared containers are keyed independently: devices, indicators, and table rows each have their own metadata_key handles, so if several interfaces are combined they can reference the same optical fiber or table row by key regardless of their own response-series keys. This split is deliberate — the response series is the interface’s own output (one per interface), while the hardware is shared file-wide.

Note that the response series’ NWB name ("FiberPhotometryResponseSeries" by default) is distinct from its metadata_key. The key identifies the interface’s entry; the name names the object written to the file. Combine multiple interfaces in a converter and give each a distinct metadata_key (and typically a distinct series name) to write several series into one file.

Shared Containers and Metadata-Key Linking#

Following the ophys system, entries reference each other with <thing>_metadata_key fields holding the key of the referenced entry (not its NWB name). The reference chain is:

  • A device instance (in metadata["Devices"]) references its model (in metadata["DeviceModels"]): "device_model_metadata_key": "ofm".

  • A virus injection references its viral vector: "viral_vector_metadata_key": "aav_gcamp".

  • An indicator optionally references its injection: "viral_vector_injection_metadata_key": "inj_dms".

  • A table row references its devices and indicator via optical_fiber_metadata_key, excitation_source_metadata_key, photodetector_metadata_key, indicator_metadata_key, and optionally dichroic_mirror_metadata_key, excitation_filter_metadata_key, emission_filter_metadata_key, and commanded_voltage_series_metadata_key. The device keys point into the shared metadata["Devices"] registry.

  • A response series references the table rows it spans: "fiber_photometry_table_region": ["dms_465", "dms_405"] (a list of row keys).

At write time each _metadata_key is resolved through its container to the entry’s name and then to the actual NWB object (e.g. an optical_fiber_metadata_key resolves to Devices[key]["name"] and then nwbfile.devices[name]). A device instance and its model are separate entries linked by device_model_metadata_key, written through the shared device helpers (_add_device_to_nwbfile / _add_device_model_to_nwbfile); an entry’s type selects the concrete ndx-ophys-devices class. Optical fibers are the one special case: each carries a nested fiber_insertion block (stereotaxic coordinates and insertion geometry), a contained component specified inline rather than a separately-keyed object.

The FiberPhotometryTable and Regions#

A file has exactly one FiberPhotometryTable. Each row describes one fiber × one excitation channel — its brain location, its excitation/emission wavelengths, the indicator, and the hardware that recorded it. A multi-fiber, dual-wavelength setup therefore has several rows (e.g. dms_465, dms_405, dls_465, dls_405).

Each FiberPhotometryResponseSeries references a region of the table — the subset of rows whose channels make up that series’ data columns — via fiber_photometry_table_region, a list of row metadata keys. At write time these keys are resolved to integer row indices (their position in the merged rows dict), so the region ordering matches the data’s channel ordering and no interface needs to know the absolute index a row landed at. Referencing a row key that is not present in the table is a loud error. This key-based resolution is what makes multi-interface conversions tractable: interface B can point its region at rows that interface A contributed, purely by key.

Object Creation and Idempotency#

Like the ophys system, objects are not created when the metadata dict is assembled — they are created during add_to_nwbfile. The shared containers (devices, indicators, and the table) are built once per file, which matters in the less-common case where several interfaces contribute to one table:

  1. The device/indicator/table helpers are idempotent by NWB name. The interface assembles the whole FiberPhotometry lab-metadata (devices, indicators, viruses, injections, and every table row) from its metadata; if a later interface finds it already present, it reuses it and adds only its own response series.

  2. When several interfaces run together, their metadata is merged before writing (by the converter’s dict_deep_update, which merges the keyed containers by key), so every interface sees the full set of shared rows and devices and whichever runs first can build the complete table.

  3. Declaring the same-named shared object with different contents in two interfaces is currently a silent merge for devices and device models: the shared device registry reuses whichever object was written first and ignores the conflicting metadata. Restoring the loud-error-on-conflict behavior at the registry level is tracked in #1782.

Commanded voltage is a special case among the shared containers: it is data-bearing (it reads samples from a stream, not just static metadata). Each CommandedVoltageSeries entry names the input stream_name (and optional channel index) to read; a table row then references the resulting series through its commanded_voltage_series_metadata_key. This is how frequency-multiplexed setups associate each demodulated signal with the sinusoidal drive that produced it.

Default Scaffold and Placeholder Sentinels#

get_metadata() returns the NWBFile basics merged with the default DeviceModels, Devices, and FiberPhotometry blocks produced by get_default_fiber_photometry_metadata(). The scaffold is a complete, runnable structure — one placeholder device (and its model) of each kind in the top-level registry, one indicator, one table row, and one response series entry under the interface’s metadata_key — with required fields pre-filled with sentinels:

  • Required numeric fields (excitation_wavelength_in_nm, emission_wavelength_in_nm, numerical_aperture) default to NaN. The ndx spec forbids null here, so there is no honest numeric sentinel; NaN is used and flagged.

  • Required string fields (location, an indicator label, descriptions) default to the distinct sentinel "PLACEHOLDER" — distinct from a deliberate "unknown" so that intentionally declaring a field unknown silences the warning.

add_to_nwbfile scans for surviving sentinels and emits a UserWarning naming each unfilled field (or raises, under strict=True). This lets an interface run end-to-end for a quick test while nagging about anything that must be set before archiving.