Metadata feedback

[alexpiet]

• acquisition.stimulus_epochs.performance metrics have null values:
  "reward_consumed_during_epoch": null,
  "reward_consumed_unit": null,
  "trials_finished": null,
  "trials_rewarded": null,
  "trials_total": null
• acquisition.stimulus_epochs.training_protocol_name is null
• acquisition.subject_details have null values
  "animal_weight_post": null, # the post is weighed after the session
  "animal_weight_prior": null,
  "anaesthesia": null, # Maybe this is the schema format, but "None" seems like it would make more sense that null which implies unknown.
• instrument.calibrations is empty. I'm not sure why the calibrations were in the acquisition file and not in the instrument file.
• instrument.components.camera has missing info, like frame rate
• harp boards dont have version info:
  "core_version": null,
  "data_interface": "USB",
  "firmware_version": null,
  "hardware_version": null,
• instrument.location is null, and should describe the room number?
• instrument.modification_date is always the same as the acquisition date, which is not true
• subject.light_cycle is null, but should be "reverse" or something

[micahwoodard]

• acquisition.stimulus_epochs.training_protocol_name is null

Is there a specific string for this? Is this the task variant or just dynamic foraging?

• acquisition.subject_details have null values
  "animal_weight_post": null, # the post is weighed after the session
  "animal_weight_prior": null,

Getting the weight might be tricky since this is done through a separate waterlog process. @bruno-f-cruz does vr map weights? Looking at your metadata mapper, I didn't see anything but I could have missed it.

• instrument.calibrations is empty. I'm not sure why the calibrations were in the acquisition file and not in the instrument file.

idk the acquisition schema has a place for instrument calibration in 2.6.0 here . But will add to instrument as well

• harp boards dont have version info:
  "core_version": null,
  "firmware_version": null,
  "hardware_version": null,

I can get the hardware and firmware but am unsure about core_version. Is this the version of the device itself?

• subject.light_cycle is null, but should be "reverse" or something

I don't see subject.light_cycle in the acquisition or instrument. Is this with another model that sci comp handles mapping?

[micahwoodard]

instrument.modification_date is always the same as the acquisition date, which is not true

this is a little tricky since this would require historical data. @bruno-f-cruz it looks like vr sets modification date to acquisition date. Any insight on how to achieve this? I know @dbirman had mentioned something about pinging an endpoint to check against previous instrument.jsons. Is there any additional info you could provide on that front?

[bruno-f-cruz]

We do what you describe and always change the modification date. The metadata requires the coordination of several different people and teams that the current infrastructure does not facilitate. We decided that this heuristic would at least not provide wrong metadata, which in my point of view is better than serializing the wrong metadata.

If this is a requirement put this data somewhere in ficus and ask the team to draw up an SOP on how that should be updated.

[bruno-f-cruz]

Re: weight: As I mentioned before, this is data that SciComp can already grab from waterlog at metadata gather time. I don't think we should have two parallel interfaces for the same thing and risk mismatches.

[dbirman]

Re: weight: As I mentioned before, this is data that SciComp can already grab from waterlog at metadata gather time. I don't think we should have two parallel interfaces for the same thing and risk mismatches.

We don't have direct access to waterlog right now from the metadata-service. I'll check whether it shows up as a dataverse table, but I don't think those are in production yet.

For the instrument.json, we did add support for uploading instrument.json files to the metadata-service so that they can be pulled automatically by the gather metadata job. If you used that workflow you could check against the endpoint to see if a change occurred. I'm not sure this is worth fixing right now though because when we get to the future "Data Registry" backend for the schema this will be fixed by that change (in the registry making a change will be controlled by an API, so it becomes trivial for us to track when someone makes a change and control the modification date field ourselves)

[dbirman]

@micahwoodard separately, training_protocol_name is supposed to be a foreign key that points to a TrainingProtocol in the procedures. Bruno and I have discussed how to fix this for "Aind.Behavior" behavior data and I have a plan to implement this purely in the GatherMetadataJob. As long as a the trainer state json files are coming through with these assets you don't have to worry about this.

[alexpiet]

this is a little tricky since this would require historical data.

@micahwoodard The solution we used on the current DF implementation is to save the most recent rig.json (metadata v1) on the rig. We generate a new rig.json and if anything has changed we bump the modification date, otherwise use the old rig.json and its corresponding older modification date. We have been doing this for 2+ years with no problems and no maintenance issues. I don't see why that couldn't be immediately implemented. Implemented here: https://github.com/AllenNeuralDynamics/dynamic-foraging-task/blob/a934aa09b0946ba6388a8fca6d965c87362e0af4/src/foraging_gui/RigJsonBuilder.py#L18

The metadata requires the coordination of several different people and teams that the current infrastructure does not facilitate. We decided that this heuristic would at least not provide wrong metadata, which in my point of view is better than serializing the wrong metadata.

I don't understand this point. All information about the rig is available at the rig. I don't know why we would need, or much less want, to track this via centralized infrastructure instead of local information. I disagree that updating the modification date to the acquisition date is not "wrong metadata". It incorrectly tells the user that something was modified, even though nothing was modified.

SIPE is going to be responsible for maintaining this, so its up to you @micahwoodard. Just sharing my opinion.