TTM daily processing


Every day at  01:55  the TT analysis machine kauri starts the shell script /ncc/ttpro/cron/collect-and-process-data

This page describes the various steps, their output and solutions for occasional problems. Also see the pages on filesystem structure, tape handling and the data processing FAQ.

processing steps

(text in fixed width font refers to commands in the script)
  1. free space on the filesystem used for raw data storage:
    cleanup -rawdata
  2. retrieve new data from remote boxes:
    collect_data
  3. store the retrieved data on tape:
    store-data -raw
  4. process routing vector (RVEC) files and update the traceroute database:
    updatedb $raw_datalist
  5. free space on the filesystem used for processed data (PCKB) storage:
    cleanup -pckb
  6. merge send packets with received packets and find matching routing vectors:
    merge_data
  7. create ROOT files from the processed data:|
    make_root
  8. store the newly created ROOT files on tape:
    store-data -root
  9. store tape from drive to jukebox magazine:
    empty-drive
  10. process the ROOT files for the last day:
    delayStats

Error conditions

The above programs/scripts each have many possible error conditions; the most likely will be discussed on the pages documenting these subprocesses. The error conditions for the overall controlling job are:
  1. ABORT: index file /ncc/ttpro/collected_data already exists!

    This indicates a previously started collect_data is either still running, or the output of it has not been further processed. The operator will have to find the particular reason and when necessary take corrective action.

  2. ABORT: /ncc/ttpro/collected_data: empty list??

    The collect_data process finished but did not transfer any new files. Obviously, there's no more work left for the data processing cron job, which is why the job aborts. However, it is a strange situation which certainly needs checking by the operator. In any case, the file /ncc/ttpro/tapes/collected_data must be removed to prevent problems with the next run of data collection.

  3. ABORT: RVEC analysis failed !!!

    The traceroute Database could not be updated. Maintainer(s) of this software will have to correct this problem ASAP. Once it is fixed, the RVEC analysis for this day will have to be redone and all other remaining steps of collect-and-process-data will have to be redone by hand.

  4. index file /ncc/ttpro/tapes/processed_data already exists.
    ABORTING data processing!

    This indicates a previously started merge_data is either still running, or the output of it has not been further processed. The operator will have to find the particular reason and when necessary take corrective action. (for example: once the active job has finished, restart merge_data and continue with ROOT file creation and storage)

  5. index file /ncc/ttpro/tapes/root_data already exists.
    ABORTING data processing!

    This indicates a previously started make_root is either still running, or the output of it has not been further processed. The operator will have to find the particular reason and when necessary take corrective action. (for example: create & store ROOT files from the processed_data list).

  6. ABORT: /ncc/ttpro/tapes/root_data: empty list?

    The make_root process finished but did not create any new files. Obviously, there's no more work left for the data processing cron job, which is why the job aborts. However, it is a strange situation which certainly needs checking by the operator (did merge_data complete succesfully?). In any case, the files /ncc/ttpro/tapes/collected_data, /ncc/ttpro/tapes/processed_data and /ncc/ttpro/tapes/root_data must be renamed to prevent problems with the next run of collect-and-process-data.


$Id: index.html,v 1.3 2001/11/23 14:59:03 wilhelm Exp $