Traffic analysis and infrastructure monitoring in CESNET2 Network

Tom Kosnar, CESNET

There are several areas of interest in monitoring CESNET2 - the Czech NREN. This demonstration aims to introduce two locally developed software packages covering IPv4 traffic analysis and infrastructure monitoring areas. They will be demonstrated as real applications at the workshop. More information about these projects can be found at https://jerry.ten.cz/pam2001/.

IPv4 traffic analysis in CESNET2 Network

IP traffic analysis system aims to provide full, long time accounting and partial traffic analysis for any NREN member and other organizations connected to it. It processes accounting data generated by our backbone border routers in Cisco NetFLow(TM) format at this time, but is capable to process any "IP to IP" based accounting data. Our idea was to design an accounting system with generic data processing as much as possible independent on network infrastructure physical and logical topology changes. The motivation was to avoid accounting system reconfigurations while reconfiguring network hardware components or changing NREN architecture. This effort resulted in solution based on IP address space distribution and its binding to NREN border points and NREN customers.

Main goal of this project was to provide long time aggregated customer accounting, but the system was designed to satisfy other requirements which we consider to be useful for network administrators, application and service developers and network capacity planning as for instance are: (1) customer to customer traffic, (2) peering traffic, (3) customer's top data sources, (4) customer's top sessions, (5) top sources and sessions globally, (6) customer's portion of total volume traffic transferred through NREN and others.

The accounting data are processed in the following steps: (1) short time primary accounting data collecting and partial aggregation, (2) complete IP address to IP address ranges resolution, (3) per customer aggregated statistics computing, (4) global statistics computing.

As was mentioned above the system provides generic data processing, which means in this case, that statistics are not computed on demand, but all results for all known IP address ranges are computed per each atomic time step. In other words this means, that the functionality of the system relies on the efficiency of algorithms and data model used as well as the database engine speed to compute all requested statistics within base pooling interval.

The system is accessible through interactive interface distributed by WWW. Authorization is hierarchical following the user and connected organization relationship.

Infrastructure monitoring in CESNET2 Network

Network infrastructure monitoring project in the Czech NREN was set up to develop a monitoring system, providing network infrastructure measurement, middle time data storage and on demand results presentation. The idea was to develop automated monitoring system capable to follow changes in real infrastructure and providing measurement of monitored objects (systems, interfaces, lines, ATM channels, frame relay channels, multicasts and others) in more complex way as usual.

The measuring part of the system is mainly based on SNMP. There is a convenient set (extensible) of MIB-OIDs selected for each possibly monitored object type. Measurement and self configuration are network node based and running in never ending loops according to parameters set up by system administrator. Special care was taken of internal warning system, which is able to notify unexpected measuring states (time-outs, objects removal) as well as unexpected measured values (outside limits, changes). To optimize measurement system behavior a wide set of parameters (data expiration, measurement processes management, old data aggregation schemes and many others) was implemented. To make easier "line accounting" requested by many operators, special mechanism tracking the "jumping" SNMP interface IDs was incorporated.

User access is provided through interactive interface distributed by WWW. Active network devices and its components (measured objects) are presented in hierarchical form. Personal and global profiles to store frequently requested sets of objects are available. Authorization is hierarchical and in general is defined by combination of accessible item types (from basic informations to error reporting ones) and measured objects (network device or its component). User interface is designed to work in point to multi-point architecture (one interface delivery, more measurement instances). To enable periodical tasks to be run the application can be controlled on background by agent to produce static HTML result pages.

Administration interface is delivered by Web server too. It enables to set up the measurement engine parameters as well as the set of measured network devices including separate measurement strategy for each of them.