N^2 REQUIREMENTS and IMPLEMENTATION GUIDE LINES:
*** Current Status(14.06) ***
- ttreg: 100% finished
- ttregconfig: 100% finished
- ttconfig: 100% finished
- configuration_targets.cgi: 100% finished
- createconfigfile.pl: 100% finished
- analyse_tb_configdata.pl: 100% finished
- collect_tb_configdata.pl: 100% finished
- createrelationgroupconfigfile.pl : 100% finished
*******************
Quick View:
Please refer to diagram for a nice view of the system data flow. In order to make things easier I will aproach this explanation on a step-by-step metodology. If you have more questions you might find an asnwer on the sectionn bellow.
- 1. /ncc/ttpro/bin/process_relationgroups.sh starts the process;
- 2. /ncc/ttpro/bin/collect_tb_configdata ssh to all ACTIVE test boxes of type C and D. It then collects ttxy:/usr/home/ttraffic/config/relationgroup.config and stores it at NCC:/ncc/ttpro/data/config/new/ttxy.relationgroup.config;
- 3. /ncc/ttpro/bin/analyse_tb_configdata extracts changes on Relation Groups analysing /ncc/ttpro/data/config/new/ttxy.relationgroup.config. The tool ttregconfig is used to add and/or delete ttreg entries(u-delay-t, r-delay-t);
- 4. /ncc/ttpro/bin/createrelationgroupconfigfile extracts relation group information from ttreg entries and creates relationgroup.config files which are stored on .../files/ttxy_specific/usr/home/ttraffic/config/relationgroup.config;
- 5. /ncc/ttpro/bin/createconfigfile extracts relation group information from ttreg entries and creates configfile files which are stored on .../files/ttxy_specific/usr/home/ttraffic/config/relationgroup.config;
- 6. /ncc/ttpro/bin/upload_tb_configdata upload relationgroup.config files to ttxx: /data/apache/htdocs/cgi-bin/relationgroup.config
- 7. At 05:30 CFengine tt_sync files to test boxes.
OBS:
- tt-ops can add or delete measurement relations using the ttregconfig tool
- Hosts can add or delete Local-Host measurement relations using the configuration_targets.cgi script.
1 - General Considerations:
1.a - Goal:
Currently TTM setup supports a full mesh of connections where each test box sends measurement packets to every active test box in the system. The goal is setup a non-full mesh scenario.
1.b - Definitions:
- non-full mesh: each test box has a group of targets. The measurement packets are sent to/received from all these targets. This group of targets is called 'relation group'.
- relation group: a test box must have a 'relation group' with at least one and a maximal number of MAX_NUMBER_OF_TARGETS targets. Each relation group is composed by many 'measurement relations'.
- measurement relation: is composed by a test box identifier and one or more measurements parameters. From a Host perspective there are three measurement relation types:
- 1 - TTM-Operator: defined by TTM operators;
- 2 - Local-Host: defined by the local host;
- 3 - Remote-Host: defined by another host;
- measurement restrictions: a host has some restrictions when setting new measurement relations. They basically are: maximal(minimal) packet size in bytes and maximal(minimal) rate in PacketNumber/hour.
- reciprocity rule: the measurement is one-way but bi-lateral. That means if test box A chooses box B as target box B must have box A as target too.
1.c - System elements:
This section describes elements(scripts, applications, configuration files and etc) which must be changed or created. Some of these elements have its requirements(and implementation details) described on the sections bellow.
On the Test Box:
- configuration_targets.cgi : it generates HTML page from where host can maintain the relation group. In case of no User Interface availability(Groups A & B) this service is NOT provided.
- relationgroup.config: it has current-state data for configuration_targets.cgi
On @NCC:
- process_relationgroups.sh: it's the starting script which executes all the necessary application on the right order. It must be created.
- collect_tb_configdata script: it collects relationgroup.config files from test boxes. It must be created.
- analyse_tb_configdata Tool: it uses relationgroup.config files and ttreg entries to update relation group data on ttreg entries.
- ttreg database: it is the main storage place for configuration information. It must be changed to handle the non-full mesh scenario. New attribute values must be added to its structure;
- ttconfig: it is the "ttreg <-> any other application" interface. It must be changed to handle the non-full mesh scenario. A new view must be added;
- createconfigfile script: it generates configuration files from a ttconfig's ttreg view. In order to handle non-full mesh it must be re-written;
- CFengine: it is the system maintenance tool. In order to handle non-full mesh some new rules must be added;
- TTM WWW: it is the main web page with plots showing measurement data. In order to handle non-full mesh some changes are required.
- createrelationgroupconfigfile: it generates relationgroup.config files with current relation group set for all test boxes.
- upload_tb_configdata: copy relationgroup.config files to all TB's
- process_relationgroups.sh script: it executes collect_tb_configdata, analyse_tb_configdata, create_relationgroupconfigfile, create_configfile and upload_tb_configdata in the right order.
1.d - System general description:
Happens on test box:
- Each test boxes has a starting relation group defined by TTM operators. Test boxes may have distinct relation groups.
- A test box User Interface shows always the current relation group. As there are distinct measurement relations types, distinct colors are used to help the Host to distinguish between them. The measurement relations are stored locally on relationgroup.config.
- The Host can change the relation group from the User Interface. Before storing the data the CGI script do some consistency checks.
Happens on @NCC:
- at least once a day process_relationgroups.sh is executed by CFE.
- the configfile and relationgroup.config files which result from the processes executed by process_relationgroups.sh are stored on the test boxes specific directories.
- At 05:30 the files are uploaded to the testboxes by CFengine::tt_sync.
2 - TTREG
2.a - New attributes:
- o-delay-t
- description: target delays defined by tt-ops.
- format: id portnumber rate packetsize.
- u-delay-t:
- description: user defined delay target.
- format: id portnumber rate packetsize.
- r-delay-t:
- description: remoted generated delay target.
- format: id portnumber rate packetsize.
- o-tracer-t
- description: trace route targets defined by tt-ops.
- format: id rate
- u-tracer-t
- description: user defined trace route targets.
- format: id rate
- r-tracer-t
- description: remote generated trace route targets.
- format: id rate
- port_number
- description: UDP port number for TTM measurement traffic
- auxmr
- description: stores Measurement Relations to be restore in future
2.b - Important remarks:
- attributes for bandwidth measurement or any other service to be implemented by TTM in the future can be added without big troubles.
- The fields above are all public and non-mandatory and they might occur multiple times per entry(i.e. they are non-single attributes).
- format description:
- id: test box id number. Not related to our own name definition( ttxy ).
- portnumber: UDP port number for TTM traffic.
- rate: number of measurement packets per hour.
- poacketsize: size of the measurement packet in bytes. Obs: Each byte has 11 bit(8 data, 2 start amd 1 stop)
As a example ttreg for tt25 would be:
o_delay_t: tt03 6000 20 100
o_delay_t: tt57 4000 15 100
o_delay_t: tt23 2000 20 150
u_delay_t: tt28 1200 14 120
u_delay_t: tt27 1010 24 120
o_tracer_t: tt03 10
o_tracer_t: tt57 5
u_tracer_t: tt28 4
u_tracer_t: tt27 4
2.c - ttregconfig:
- It's a ttreg edit tool. It can add, delete and modify ttreg entries.
- Command Line Syntax: ttregconfig [-v] command attribute_name target attribute_value
- Arguments description:
[-c] -> also creates information which will be used afterwards by analyse_tb_configdata when updating relationgroup.config files
[-v] -> verbose mode
command -> ADD: add new value(s) to attribute_name
DEL: delete value(s) from attribute_name
MOD: modify value(s) in attribute_name
attribute_name -> attribute name as defined in the table structure
target -> CHAIN: command affect all OFF, SETUP, WATCH and ON TB's
TTLIST: command affect all WATCH and ON TB's
filename.txt: command affect all text boxes which names are listed on filename.txt
ttxy: command affect only text box ttxy
attribute_value -> new value to be used by the ADD|MOD|DEL commands
- Examples:
ttregconfig ADD monitor CHAIN 193.0.0.0/22
ttregconfig DEL dnsserv tt23 193.23.45.101
ttregconfig MOD geoloc tt04 "52.2237 +/- 0.0000, 4.5327 +/- 0.0000"
- composed attribute values must be delimited by quotation marks(i.e. "Singel 258" or "52.2237 +/- 0.0000, 4.5327 +/- 0.0000")
2.c.a -ttregconfig FAQ:
- If you're DELeting an attribute, do you still have to specify it's value?
It depends on which attribute one wants to delete. If it's a single attribute like "id" which can't repeat on a table it's NOT necessary to specify any value. But if one wants to delete a non-single attribute like "monitor" then it's real value must be specified.
- In case of DELeting a non-single attribute is the 'attribute_value' checked against the real value?
Yes and if there's no match nothing will be deleted.
- What happen if one executes a DEL command for a non-single 'attribute_name' with no 'attribute_value' on the command line?
Unfortunately all occurrences of the specified 'attribute_name' will be deleted from the selected entries. Example : 'ttregconfig DEL o_tracer_t tt73' will delete all acurrences of "o_tracer_t" on tt73 entry.
- What happens if one executes a MOD command with no 'attribute_value' on the command line?
It has no sense. The Print_Usage() function will be executed and the execution aborted. If that doesn't happen please tell me because it's a bug.
- What happens if one tries to ADD or MOD an attribute value with wrong syntactic rules?
ttreg config does NOT make syntaxs authentication. So the wrong value will be stored.
- What happens if one tries to ADD an 'attribute_value' which already exists on the entry?
This value will be duplicated. Thus ttregconfig does NOT do any syntax or consistency check.It's a simple tool that believes on its users intteligence and knowledge about ttreg structure.
3 - ttconfig
Add more two views:
- TARGETS: print out the new target attributes.
- CITIES: print out a list composed by ttxy - city
4 - Poissonfiles creation:
- might be coded in Perl now(old version is a bourne shell script)
- Care for transition smoothness: if o_delay_t = all_chain it generates config files as it currently does. Otherwise generates config files with selected target information on it.
5 - configuration_targets.cgi:
- It generates HTML code for Test Box User Interface
- Host can:
- ADD: new measurement relation ( target , packet length, measurement rate)
- DEL: measurement relation
- Configuration data is saved to ../assets/relationgroup.config
-
- TT-Operator: BLACK
- Local-Host(approved): BLUE
- Remote-Host: PURPLE
- On-line Consistency check:
- 60 > packet length < 1500
- measurement rate < 1800.
- relationgroup.config format:
StandardPacketLenght [0-9]*
MaxPacketLenght [0-9]*
StandardRate [0-9]*
MinPacketLenght[0-9]*
PortNumber [0-9]*
MaxRate [0-9]*
RED target packetsize rate
BLUE target packetsize rate
PURPLE target packetsize rate
BLACK target packetsize rate
6 - analyse_tb_configdata program:
- It analyses test box configuration data, run sanity rule checking and store information on ttreg entries
- Sanity rules:
- ttxx can't have ttxx as target;
- reciprocity: if ttxx has ttyy as target, ttyy must have ttxx as target;
- portnumber: if ttxx host chooses port number N all measurement ttraffic addressed to ttxx must come on port N. This functionality is not officialy supported but it works if necessary;
- Guidelines for measurement relations(m.r.) handling:
- A TT-Operator m.r. is stored on o-delay-t(o-tracer-t) attribute from ttreg entries.
- A Local-Host m.r. is stored on u-delay-t(u-tracer-t) attribute from ttreg entries.
- A Remote-Host m.r. is stored on r-delay-t(r-tracer-t) attribute from ttreg entries.
- TT-Operator one has high priority. Thus it promptly becomes active.
- An test box having a relation group where one target is present in more than one m.r. is NOT a problem. In this case Host will have two distinct measurements for one path.
- One Measurement Relation is defined by Target and Packet Lenght. Measurement Relation with higher Rates always replace lower Rate ones. The lower rates ones are stored in ttreg auxmr attribute and are restored after the higher Rate one are deleted.
- If a "big change" on the relation group is noticed tt-ops must be informed.
7 - tt-ops:
- tt-ops is only required to maintain the TT-Operator set of measurement relations.
- It's possible to do it by hand editing ttreg entries one by one or one can use ttregconfig tool.