# Ephys/Imaging Automation Pipeline
# Initial configuration
- Note: These steps are only needed if a label in red with "Configuration needed" is shown on the top right corner of the GUI.
- Switch to "System Configuration" tab.
- Click "Start Configuration" button.
- In "System Name" dropdown, select corresponding system (normally named room#-Recording#).
- In "Associated Behavior Rig" dropdown, select corresponding rig paired with recording system.
- Click "Add Rig to System" button.
- Repeat steps 4 & 5 until all rigs associated to recording system are added.
- In "Recording Modality" dropdown select corresponsing modality of the recording system.
- In "Recording Root Folder" area, click folder button to search for Parent path where all recordings will be stored.
- Click "Configure System" button.
If something was configured incorrectly, you can repeat the whole process.
# Automation GUI "regular use" Manual
- The automation GUI is located on the desktop in all microscope/ephys recording system across BRAINCoGS. You can identify it by a desktop BRAINCoGS icon called "Recording_Automation_GUI".
- It is encouraged to register & process every recording with the Automation GUI just after recording has ended.
# Default use case
- Open the GUI just after recording has ended.
- Select the recording directory from dropdown list in section 1.
- If no behavior associated with this recording uncheck ****
- Check "Add surgery & insertion device if missing" checkbox if you want to add electrode/microscope/optic fiber insertion coordinates.
- Select behavior session corresponding to recording from dropdown list in section 2.
- Click Register Recording button.
- If no surgery data is found for the subject, a small form to insert surgery and device insertion coordinates will be shown.
- Common coordinates for previous recordings:
Area | ml(mm) | ap(mm) | depth(mm) | theta(°) | phi(°) | rho(°) |
---|---|---|---|---|---|---|
mPFC L HEMIS | -0.6 | 1.8 | 3.8 | -45 | -9 | 180 |
mPFC R HEMIS | 0.6 | 1.8 | 3.8 | 45 | 9 | 0 |
HPC L HEMIS | -1.5 | 2 | 1.8 | -45 | -9 | 180 |
HPC R HEMIS | 1.5 | 2 | 1.8 | 45 | 9 | 0 |
- Wait until recording has been copied to PNI storage resources (cup).
- After recording has been transferred a small dialog with recording id will be shown.
- User can check basic info of the new recording in the "Recording Table" tab. This tab will be active after recording is transferred.
# Notes
- This use case assumes that default parameters will be used to process the recording.
# "Advanced" (select parameters) use case
- Follow 1-4 from Default use case
- Uncheck "Use default processing parameters for recording?" checkbox
- Click "Select parameters >>>" button
- On the "Select parameters" tab (assume that all probes/fovs are going to be processed with same parameters)
# On section 1:
- In Preprocessing Params Lists dropdown Select a Preprocessing Param list
- You can check which preprocessing steps are defined for that list on the listbox to the right)
- If you select a preprocessing step from the listbox you can check the specific parameters for that preprocessing step on the text area to the right.
# On section 2:
- In Processing Params" dropdown select a Processing parameter suited for recording
- If you select a processing parameter from the dropdown you can check the specific parameters for it on the text area to the right.
- Click Register Recording button.
- Next actions are idential to the "Default use" case
# Monitor jobs
- After a recording is submitted to processing the user is able to track, reprocess and visualize all jobs corresponsing to that recording.
# Track status
- Switch to "Manage Processing Jobs" tab.
- Use "User", "Subject" & "Date" dropdwons / edits to find desired jobs
- Select row corresponding to the corresponding job:
- Corresponding Job Status history is shown for selected job.
# If status of job is in error status (-1, -2; "Error in recording process"):
- You can open the Error log file on the bottom right corner by clicking "Open Error Log File" button.
- If you consider that what cause the error is solved now, you can rerun job by clicking "Rerun job" button.
# If status of job is in finished status (7, 8; "Dat in element"):
- You can open the Output log file on the bottom right corner by clicking "Open Output Log File" button.
- You can visualize results from processing by clicking the bottom rightmost buttons: (Open Phy, Open IBL Atlas for ephys Open Suite2p-GUI for imaging)
# Rerun jobs
- A job can be rerun with different parameters by clicking New Job With different Parameters button.
- Follow "Advanced" (select parameters) use case
# Create new sets of processing parameters for recordings
- In order to process recordings with parameters different than default you need at least one of the following:
- A new ser or processing parameters (parameters for the algorithm to process, e.g. kilosort, suite2p).
- A list of preprocessing steps formed by a set of preprocessing params (e.g. perform catgt + tprime before kilosort).
# Create new Processing parameters (algorithm pararms)
- Note: You need a json file with all parameters ready before starting this process.
- Here is an example of a kilosort parameter json file. It can serve you as an example for creating your own:
- Switch to "Create Parameters" tab.
# On section 0:
- In "recordig modality" dropdown select to which modality the parameters correspond.
- In "user" dropdown Select your NETID username (select general-user if this parameter will be shared by many users).
# On section 1a:
- In "proc. Param Method" dropdown, select which method (algorithm) these parameters refer to.
# If the algorithm is not found on "proc. Param Method" dropdown:
- Check "define new proc. param method ?" checkbox.
- Write new processing method name in "New Proc. Param. method" edit
- Check automation pipeline developer guide to add corresponding processing code. (LINK MISSING)
- In "proc. Param Set Description" edit, write a small description for the new parameters.
- Click "Upload Proc-Parm Set json file" button and search/load for your json file with parameters. You can check your parameters on the text area on the right.
- Click "Register Proc. Param Set" button.
# Create new preprocessing parameters (a single step on preprocessing list)
- Note: You need a json file with all parameters ready before starting this process.
- Here is an example of a catgt parameter json file. It can serve you as an example for creating your own:
- Switch to "Create Parameters" tab.
# On section 0:
- In "recordig modality" dropdown select to which modality the parameters correspond.
- In "user" dropdown Select your NETID username (select general-user if this parameter will be shared by many users).
# On section 1b:
- In "Preproc.-Param Method" dropdown, select which method (tool) these parameters refer to.
# If the tool is not found on "Preproc.-Param Method" dropdown:
- Check "define new prerproc.-param method ?" checkbox.
- Write new preprocessing method name in "New Preproc.-Param method" edit.
- Check automation pipeline developer guide to add corresponding preprocessing code. (LINK MISSING).
- In "Preproc.-Param Set Description" edit, write a small description for the new parameters.
- Click "Upload Preproc.-Parm Set json file" button and search/load for your json file with parameters. You can check your parameters on the text area on the right.
- Click "Register PreProc. Param Set" button.
# Create new preprocessing parameters list (a set of preprocessing paramaters)
- Note: If you just created a preprocessing parameter, restart GUI.
- Switch to "Create Parameters" tab.
# On section 0:
- In "recordig modality" dropdown select to which modality the parameters correspond.
- In "user" dropdown Select your NETID username (select general-user if this parameter will be shared by many users).
# On section 1c:
- In "Pre-Params List Name" dropdown, write a suitable name for the list.
- In "Pre.-Param List Description" edit, write a small description for the new list.
- From "Pre.-Params steps" dropdown select the desired preprocessing step as part of the list.
- Click "Add Preparam Steps" button
- Repeat steps 6 & 7 to add all desired preprocessing steps for the list.
- Use "v" & "^" buttons to set order of preprocessing steps.
- Use "x" button to delete an unwanted preprocessing step.
- Click "Register pre param list" button.