Conversion Gallery#
This section provides an in-depth look at how nwb2bids extracts metadata from NWB files and organizes it into BIDS-compliant sidecar files.
For each type of metadata, we’ll show:
How to express the intended data fields of the NWB neurodata type using PyNWB.
What the corresponding extracted BIDS sidecar file contains.
This should help clarify the mapping between NWB and BIDS fields which nwb2bids handles.
Note
We’ll use the same example data from the Tutorials section to demonstrate the fine-grain file contents, so be sure to have those files generated before proceeding.
Dataset Description#
The dataset_description.json file at the root of the BIDS dataset contains high-level metadata
about the entire dataset. Unless you are converting a dataset that has already been uploaded to DANDI (and thus
may contain several high-level metadata fields not found in any individual files), this information
must be provided via the additional metadata feature (see Tutorial 5 - Additional metadata).
A typical dataset_description.json file might look like:
{
"Name": "`nwb2bids` Conversion Gallery Example",
"BIDSVersion": "1.10.1",
"HEDVersion": "8.3.0",
"Description": "An example dataset description!",
"DatasetType": "raw",
"Authors": ["Baker, Cody"],
"License": "CC-BY-4.0",
"GeneratedBy": [
{
"Name": "nwb2bids",
"Version": "0.9.1",
"Description": "Tool to reorganize NWB files into a BIDS directory layout.",
"CodeURL": "https://github.com/con/nwb2bids"
}
]
}
Hint
If you are using the nwb2bids.DatasetConverter.from_remote_dandiset() method, this file should be autopopulated for you with as much metadata as could be inferred from the DANDI API.
Subjects & Sessions#
In a single NWB file (which can typically represent a single ‘session’), subject-level metadata is attached as a
singular Subject object. In BIDS, information about all subjects in the dataset is collected in the top-level
participants.tsv and participants.json files, while session-level metadata goes into per-subject
sessions.tsv and sessions.json sidecar files.
NWB Session:
nwbfile = pynwb.file.NWBFile(
session_id="A",
session_start_time=datetime.datetime(1970, 1, 1, tzinfo=dateutil.tz.tzutc()),
session_description="An example NWB file containing ecephys neurodata types - for use in the nwb2bids tutorials.",
identifier=uuid.uuid4().hex,
)
NWB Subject:
subject = pynwb.file.Subject(
subject_id="001",
sex="M",
species="Mus musculus",
)
nwbfile.subject = subject
BIDS Sessions:
The sessions.tsv file contains a row for each session of a subject:
| session_id |
|---|
| ses-A |
BIDS Participants:
The participants.tsv file contains a row for each subject:
| participant_id | species | sex |
|---|---|---|
| sub-001 | Mus musculus | M |
Note
The column order in all TSV files is strictly enforced by BIDS validation, and nwb2bids will make every effort to produce valid files based on input data.
And the corresponding participants.json file provides detailed descriptions of each column:
{
"participant_id": "A unique identifier for this participant.",
"species": "The species should be the proper Latin binomial species name from the NCBI Taxonomy (for example, Mus musculus).",
"sex": "String value indicating phenotypical sex, one of \"male\", \"female\", \"other\".\n\tFor \"male\", use one of these values: male, m, M, MALE, Male.\n\tFor \"female\", use one of these values: female, f, F, FEMALE, Female.\n\tFor \"other\", use one of these values: other, o, O, OTHER, Other."
}
Mapping:
NWB Field |
BIDS Field |
|---|---|
|
|
|
|
|
|
|
|
|
|
General Metadata#
NWB files can contain a number of high-level metadata fields that describe the overall experiment, acquisition parameters, and other details that don’t neatly belong to specific neurodata types.
These fields are typically set on the top-level pynwb.NWBFile object, though there are a few that belong to
particular subfields such as pynwb.DeviceModel.
NWB file metadata:
general_metadata_nwbfile = pynwb.NWBFile(
session_id="B",
session_start_time=datetime.datetime(1970, 1, 2, tzinfo=dateutil.tz.tzutc()),
session_description="An example NWB file used for demonstration of general metadata mapping.",
identifier=uuid.uuid4().hex,
institution="My Institution",
pharmacology="carprofen, 5 mg/kg, given peri-operatively and maintained during recording sessions to manage pain from the craniotomy/implant without sedation.",
protocol="Mice were placed in a running wheel and allowed to run freely while neural activity was recorded. Mice were trained to stop running in response to air puffs.",
slices="After the experiment, brains were extracted and sliced into 5 micron sections from primary motor cortex were fixed in epoxy resin for subsequent icephys study.",
)
general_metadata_device = pynwb.file.DeviceModel(
name="GeneralMetadataDevice",
description="This is an example device used for demonstration of general metadata mapping.",
manufacturer="imec",
model_number="NP2014",
)
general_metadata_nwbfile.add_device_model(general_metadata_device)
BIDS general metadata:
Depending on the modality, the ecephys.json or icephys.json file contains metadata such as:
{
"InstitutionName": "My Institution",
"InstitutionAddress": "123 Institution Rd, My City, My State, My Country",
"InstitutionalDepartmentName": "My Department",
"PowerLineFrequency": "60", // In Hz
"Manufacturer": "imec",
"ManufacturersModelName": "NP",
"ManufacturersModelVersion": "2014",
"RecordingSetupName": "MyLabsRig",
"SamplingFrequency": 30000.0, // In Hz
"DeviceSerialNumber": "ABC123",
"SoftwareName": "RecordingSoftware",
"SoftwareVersions": "1.0.0",
"RecordingDuration": "7200", // In seconds
"RecordingType": "continuous",
"SoftwareFilters": {"Anti-aliasing filter": {"half-amplitude cutoff (Hz)": 500, "Roll-off": "6dB/Octave"}},
"HardwareFilters": {"Highpass RC filter": {"Half amplitude cutoff (Hz)": 0.0159, "Roll-off": "6dB/Octave"}},
"BodyPart": "BRAIN",
"BodyPartDetails": "primary motor cortex",
"BodyPartDetailsOntology": "http://purl.obolibrary.org/obo/UBERON_0001384"
}
{
"PharmaceuticalName": "carprofen",
"PharmaceuticalDoseAmount": 5,
"PharmaceuticalDoseUnits": "mg/kg",
"PharmaceuticalDoseRegimen": "Given peri-operatively and maintained during recording sessions to manage pain from the craniotomy/implant without sedation.",
"PharmaceuticalDoseTime": "0", // In seconds
"SampleEnvironment": "in-vivo",
"SupplementarySignals": "Running wheel velocity.",
"TaskName": "My Running Task",
"TaskDescription": "Mice were placed in a running wheel and allowed to run freely while neural activity was recorded.",
"Instructions": "Mice were trained to stop running in response to air puffs."
}
{
"SampleEnvironment": "ex-vivo",
"SampleEmbedding": "Epoxy resin",
"SliceThickness": "5", // In microns
"SampleExtractionProtocol": "Aliquot of primary motor cortex extracted from brain and embedded in epoxy resin."
}
Mapping:
Currently, only one electrical series per NWB file is supported for automated metadata extraction. Additional sidecar files should be copied and modified to correspond to additional electrical series within the same NWB files. The use of acq-[label] suffixes in the other sidecar filenames is recommended to avoid confusion between series.
NWB Field |
BIDS Field |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Ecephys Probes#
In NWB, recording hardware is represented as Device objects. In the BIDS ecephys specification,
these become entries in the probes.tsv and probes.json sidecar files.
NWB Device:
probe = pynwb.file.Device(
name="ExampleProbe",
description="This is an example ecephys probe used for demonstration purposes.",
manufacturer="`nwb2bids` test suite",
)
BIDS Probes:
The probes.tsv file contains a row for each probe:
| probe_name | type | manufacturer | description |
|---|---|---|---|
| ExampleProbe | n/a | `nwb2bids` test suite | This is an example ecephys probe used for demonstration purposes. |
Note
The column order in all TSV files is strictly enforced by BIDS validation, and nwb2bids will make every effort to produce valid files based on input data.
And the corresponding probes.json file provides detailed descriptions of each column:
# TODO: fix this
{}
Mapping:
NWB Field |
BIDS Field |
|---|---|
|
|
|
|
|
|
|
|
Ecephys Electrodes#
Electrodes represent the physical recording sites on a probe. In classic NWB, these may be stored in the
electrodes table, which requires each electrode to also link to an ElectrodeGroup, commonly interpreted as a
traditional ‘shank’ in ecephys. In BIDS, they appear in the electrodes.tsv and electrodes.json sidecar files.
Note
Certain special probes, such as Neuropixels, may overload the electrodes table to actually store information about each recording channel. In this case, a special column electrode.contact_id is used to distinguish physical contacts from recording channels.
NWB Electrode Table:
shank = pynwb.ecephys.ElectrodeGroup(
name="ExampleShank",
description="This is an example electrode group (shank) used for demonstration purposes.",
location="hippocampus",
device=probe,
)
nwbfile.add_electrode_group(shank)
for _ in range(8):
nwbfile.add_electrode(
imp=150_000.0,
location="hippocampus",
group=shank,
filtering="HighpassFilter"
)
BIDS Electrodes:
The electrodes.tsv file contains a row for each electrode:
| name | probe_name | x | y | z | hemisphere | impedance | shank_id | location |
|---|---|---|---|---|---|---|---|---|
| e000 | ExampleProbe | n/a | n/a | n/a | n/a | 150.0 | ExampleShank | hippocampus |
| e001 | ExampleProbe | n/a | n/a | n/a | n/a | 150.0 | ExampleShank | hippocampus |
| e002 | ExampleProbe | n/a | n/a | n/a | n/a | 150.0 | ExampleShank | hippocampus |
| e003 | ExampleProbe | n/a | n/a | n/a | n/a | 150.0 | ExampleShank | hippocampus |
| e004 | ExampleProbe | n/a | n/a | n/a | n/a | 150.0 | ExampleShank | hippocampus |
| e005 | ExampleProbe | n/a | n/a | n/a | n/a | 150.0 | ExampleShank | hippocampus |
| e006 | ExampleProbe | n/a | n/a | n/a | n/a | 150.0 | ExampleShank | hippocampus |
| e007 | ExampleProbe | n/a | n/a | n/a | n/a | 150.0 | ExampleShank | hippocampus |
You may notice many differences between the classic NWB electrode fields and the electrodes.tsv file. BIDS
requires several fields that NWB does not, but their values may be set to n/a if they are not known. Additionally,
NWB stores impedance values in units of Ohms, while BIDS expects kOhms - nwb2bids handles this conversion automatically.
And the corresponding electrodes.json file provides detailed descriptions of each column:
# TODO: fix this
{}
Mapping:
NWB Field (Ecephys) |
BIDS Field |
|---|---|
“e{ |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Ecephys Channels#
Channels represent separate data streams recorded from the physical electrodes. In NWB, this information is often
combined with the previously shown electrodes table and additional columns may be used to disambiguate physical
contacts from recording channels. In BIDS, channels are described separately from electrodes via the channels.tsv
and channels.json sidecar files.
BIDS Channels:
The channels.tsv file contains a row for each channel:
| name | electrode_name | type | units | sampling_frequency | stream_id | gain |
|---|---|---|---|---|---|---|
| ch000 | e000 | n/a | V | 30000.0 | ExampleElectricalSeries | 3.02734375e-06 |
| ch001 | e001 | n/a | V | 30000.0 | ExampleElectricalSeries | 3.02734375e-06 |
| ch002 | e002 | n/a | V | 30000.0 | ExampleElectricalSeries | 3.02734375e-06 |
| ch003 | e003 | n/a | V | 30000.0 | ExampleElectricalSeries | 3.02734375e-06 |
| ch004 | e004 | n/a | V | 30000.0 | ExampleElectricalSeries | 3.02734375e-06 |
| ch005 | e005 | n/a | V | 30000.0 | ExampleElectricalSeries | 3.02734375e-06 |
| ch006 | e006 | n/a | V | 30000.0 | ExampleElectricalSeries | 3.02734375e-06 |
| ch007 | e007 | n/a | V | 30000.0 | ExampleElectricalSeries | 3.02734375e-06 |
You may notice many differences between the classic NWB electrode fields and the channels.tsv file. In particular,
a number of these values are not specified in the NWB electrodes table, but are instead set on any data-containing
ElectricalSeries objects that link to those electrodes. In these cases, nwb2bids will attempt to find
and extract the relevant values.
And the corresponding channels.json file provides detailed descriptions of each column:
# TODO: fix this
{}
Mapping:
NWB Field (Ecephys) |
BIDS Field |
|---|---|
ch{ |
|
e{ |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|