Outline of Data Processing Pipeline (in Progress)

General workflow

  1. Each experiment is 15 trays @ 20 plants each = 300 plants
  2. Experiments start out in the Experiment Tracking System (ETF)
    1. The tracking system will be a MySQL DB and web front end that contains the meta info for each trial/project
      • Start Date
      • End Date
      • Accessions / genotypes in this trial
      • Barcode for each plant
      • Map of how plants are randomized in the 15 trays
      • Trayscan Protocol description and file
      • See full overview of Project DB here
  3. At each time point (might be once a day or once an hour) the TrayScan is run and the following raw data is generated
    1. IR/Temp image from FLIRcam in RAW format (must be reprocessed manually)
      • RAW files can be auto-processed with imageJ or similar
      • We use the software Fiji which is based on ImageJ: LINK
      • Fiji script to process IR RAW files into PNG file: LINK
    2. Three RGB images (top and two sides – we’ll probably only use top image for now)
      • RGB images are created in RAW and then the TrayScan autoprocesses them into PNG files (no user input needed)
    3. Fluorcam fluorescence data: many layers, wrapped in TAR file
      • This data is not currently available without manual processing
  4. Trayscan data is autosaved to the two TrayScan PC’s
  5. Automated task uses SyncTool to copy all files to the ANU Instrument share where it is avaialble to all labs
  6. Automated (user initiated?) task copies Borevitz Lab files to the Borevitz Lab server.
  7. Automated processes monitor the Lab share incoming data folders to process new images using the full phenomics pipeline
    • See “Data Processing Steps” page (to be created)
  8. Results of processed data re available in near realtime on the Borevitz Lab web site.


Required components to get pipeline working

(Things with link are either completed or in progress, no link = probably not done yet)