dandi_compute_code.aind_ephys_pipeline#

dandi_compute_code.aind_ephys_pipeline.prepare_aind_ephys_job(
pipeline_version,
content_id=None,
dandiset_id=None,
dandiset_path=None,
config_key='default',
parameters_key='default',
pipeline_directory=None,
silent=False,
)[source]#

Prepares an AIND ephys job by generating a submission script and returning the script file path.

Parameters:

  • pipeline_version (str) – The version of the pipeline to use, which will be used to checkout a branch of the pipeline repository.

  • content_id (str | None) – The content ID for the data to be processed.

  • dandiset_id (str | None) – The Dandiset ID for the data to be processed. Required if content_id is not provided and will be used to look up the content ID if content_id is not provided.

  • dandiset_path (str | None) – The local path to the Dandiset data to be processed. Required if content_id is not provided and will be used to look up the content ID if `content_id is not provided.

  • config_key (str) – The short name of the configuration to use. Must be a key registered in registries/registered_configs.json.

  • parameters_key (str) – The short name of the parameters to use. Must be a key registered in registries/registered_params.json.

  • pipeline_directory (Path | None) – Local path to the AIND pipeline repository.

  • silent (bool) – Whether to suppress output messages from the DANDI client. Default is False.

Returns:

script_file_path – The path to the generated submission script.

Return type:

Path

Raises:

ValueError – Raised in any of the following situations. - Neither content_id nor both dandiset_id and dandiset_path are provided. - pipeline_version is an empty string. - pipeline_version equals the unsupported v1.0.0. - config_key is not registered in registered_configs.json. - parameters_key is not registered in registered_params.json. - The resolved parameters file is missing a pipeline_version field. - The parameters file pipeline_version is in a different major series than the requested pipeline_version. - The parameters file pipeline_version is newer than the requested pipeline_version. - The MD5 checksum of the resolved config or parameters file does not match its registry entry. - content_id is not present in the content-id-to-Dandiset mapping. - content_id maps to the retired sandbox dandiset. - The sub BIDS entity cannot be extracted from the resolved Dandiset path. - A resolved git commit hash does not match the expected 40-character hexadecimal format.

dandi_compute_code.aind_ephys_pipeline.submit_job(script_file_path)[source]#

Submit a pipeline script via sbatch.

Raises:

RuntimeError – If the DANDI_API_KEY environment variable is not set, if the sbatch submission returns a non-zero exit code, or if the subsequent dandi upload invocation returns a non-zero exit code.

Return type:

None

Parameters:

script_file_path (Path)

dandi_compute_code.aind_ephys_pipeline.generate_aind_ephys_submission_script(
script_file_path,
log_directory,
nwb_file_path,
results_directory,
work_directory,
apptainer_cache_directory,
environment_directory,
config_file_path,
pipeline_file_path,
pipeline_repo_directory,
pipeline_version,
temp_name,
done_tracker_file_path,
params_file_path,
)[source]#

Generate AIND Ephys submission script from template.

Arguments are ordered as they occur in the submission template.

Parameters:

  • script_file_path (Path) – Where to write the submission script.

  • log_directory (str) – The log directory.

  • nwb_file_path (str) – The input NWB file path.

  • results_directory (str) – The results directory.

  • work_directory (str) – The work directory.

  • apptainer_cache_directory (str) – The Apptainer cache directory.

  • environment_directory (str) – The conda environment to activate.

  • config_file_path (str) – The configuration file path.

  • pipeline_file_path (str) – The pipeline file path.

  • pipeline_repo_directory (str) – Path to the base pipeline repository intended to be used.

  • pipeline_version (str) – The pipeline version, which is used to checkout a branch of the pipeline repository.

  • temp_name (str) – The name of the temporary processing directory.

  • done_tracker_file_path (str) – The path to the ‘done’ tracker file.

  • params_file_path (str) – The parameters file path.

Return type:

None