Every day at 02:00 the TTM production machine (i.e. kauri)
starts the shell script
/ncc/ttpro/cron/collect-and-process-data. The various steps of the
job are described on a (currently somewhat outdated)
separate page. This page focusses on recovering
from a (partial) failure of the daily processing job. As it is impossible
to document every little step that potentially can go wrong, only the
larger components are discussed.
The most frequent reasons for this job failing is a non-existig or out of date /ncc/ttpro/config/LIST_OF_TESTBOXES file. If there's another problem, either look in the source /ncc/ttpro/src/tracerouteDB/ or ask the softies to fix it (Manuel wrote the program).
Once fixed, restart the data analysis by submitting /ncc/ttpro/cron/reprocess-todays-data:
kauri$ su ttrafficYou will get confirmation of job submission:
kauri$ batch
at> /ncc/ttpro/cron/reprocess-todays-data
at> <ctrl-d>
commands will be executed using /bin/bash
job 996844789.b at Fri Aug 3 15:19:49 2001
Now you can continue with your other tasks, the result of the at job will be mailed to ttraffic
NOTE: If the job's failure is not detected until the day after, submit the similar but tuned for older data /ncc/ttpro/cron/reprocess-yesterday-data script.
This could be for a number of reasons, ranging from something as simple as a missing (or already existing file) to a core dumping program. Use your own judgement to fix the problem, than submit /ncc/ttpro/cron/merge-and-plot script to restart the analysis where it left off.
kauri$ su ttrafficAgain, if you are one day late, submit /ncc/ttpro/cron/merge-and-plot-yesterday-data instead
kauri$ batch
at> /ncc/ttpro/cron/merge-and-plot
at> <ctrl-d>
This happens for example when the ttraffic account has no access to the WWW server. Verify that all plots have been updated in /ncc/ttpro/data/plots/day , .../week and .../month than push them to the webserver with:
kauri$ /ncc/www/bin/ppush ripencc/mem-services/ttm/PlotsAs this process is much more time consuming than the TTM way of publishing plots, you can also opt to copy and paste the appropriate commands from the main collect-and-process-data script.
The most likely cause of this is a problem with NFS, either the filer itself is unavailable or kauri has been left with stale NFS file handles. Once the problem has been fixed, submit the make-plots script to create and publish all daily (24 hr, week & month) plots.