How to use the Interactive Testbox Map generator

By Andrew Marcus, RIPE NCC intern from January 2002 to April 2002

Structure

The map program is broken down into three groups of files, each of which resides in its own directory within the program directory:

When map_gen.pl is run, the files in the latter two directories are automatically copied to the appropriate places in the webserver directory tree. In the current implementation, both sets are files are located in the same directory, although the reason for the separation is that the CGI files could be put in the /cgi-bin/ directory, while the WWW files can not.

Naming conventions

Although this might not hold everywhere, the standard I ended up using is the following:
- map_xxxx: a program, script or CGI which, when run, produces something as output. Includes: map_gen.pl map_import.pl map_index.cgi etc.
- maps.xxxx: a helper or include file to a script, something which is compiled perhaps, but not run on its own. Includes: map images, maps.config, maps.include, maps.functions.js etc.


The Pieces

gen/map_gen.pl

This program, which should be run at regular intervals, like once a day, generates the maps using the GMT mapping tools. It converts them from ps to gif formats using the ps2gif script. It also creates a file, named maps.include, in the CGI directory, which contains all the html needed to display the maps on different layers, and the javascript to manipulate them (show, hide, etc...).

Information about the locations of files and directories it needs to run is located in maps.locations.

The map_gen.pl program is very flexible in what it outputs, and it can be configured using the maps.config file. This file lets you specify what regions you would like your maps to cover, and what you would like the outputs (maps and html) to look like.

Command Line: The program provides several options when called from the command line. If called with an '-m' flag, it will accept as its remaining inputs a list of the maps you would like to produce. This is useful when you're defining a new map and would like to keep generating that one until it aligns correctly. Another useful option, which is unimplemented, might allow you to tell map_gen.pl NOT to sync the files, probably preferable when you're specifying a new map. See map_edit.

gen/maps.config

This file defines the way the output of map_gen.pl should look, and what it should contain. It defines GMT styles for creating maps, and also defines the appearance of clusters of testboxes and which statuses of boxes to include. Read the comments in the file for more information.

To add a new map definition, append it to the end of this file, following the format of the other defined maps. Before running map_gen.pl again, make sure the file with the new changes you've made still compiles successfully.

gen/maps.locations

This file specifies the locations of all directories, and is used by many different scripts. However, there are currently two copies, one in gen/ and the other in cgi-bin/. As the NOTE in gen/maps.locations explains, this is due to a variation in the mount structure of the web directory tree. Therefore, in its current implementation, gen/maps.locations is used by the generation scripts, while cgi-bin/maps.locations is uploaded to the server and is used by all CGI functions.

Many of the folders listed here need to be accessed both from an NFS context, for instance, for use with shell commands, and from a web context, for instance, in a link from a CGI page. Therefore, all directory paths in this file have been divided into two pieces: for instance, $WWWpre and $WWWdir.

$WWWdir represents the part of this directory tree which is shared by both the web server and the NFS server, such as /ripencc/... $WWWpre represents the prefix used by the NFS server, to provide a full path. The current problem with the two different maps.locations versions only occurs in prefix paths, because the NFS mountings are different for the two machines.

The sync directories, such as $CGIsync, provide the local location (i.e., in this directory) of all of the files. A copy of everything is kept here locally, and then anything which has changed is uploaded to the server when map_gen.pl is run. Perhaps a future map_gen command-line option could be to supress the sync process, for debugging purposes.

gen/map_edit

The original intent for this script was to produce a web interface useful for defining new maps; For instance, it is very easy to determine pixel coordinates of an image if something equivalent to a server-side imagemap were used by the image. Simply feed the coordinates of your mouse click directly back to the script, and store it somewhere.

Unfortunately, I did not get to fully implement this feature. What I have completed of it so far is in the gen/ directory, but its completion, perhaps, remains as a future project for another student like myself.

cgi-bin/maps.include

The html output of map_gen.pl, which includes: Javascript definitions, Style definitions, Imagemap area definitions, and Layer definitions. It also includes some additional html elements associated with the maps, such as the bar above the map which displays the location of a selected testbox.

To achieve the interactivity of the map program in as many browsers as possible, style sheets are used to create layers, which are then manipulated by Javascript. If you are unfamiliar with style sheets, I would recommend reading more about them somewhere like here. It took me a lot of research on sites like this to come up with a cross-browser implementation. For information about how the interface actually works, read more about the interface.

cgi-bin/map_import.pl

Clicking on a testbox on the map invokes a link stored as an imagemap area. However, these links need to be dynamic, since they will need to store information about which testboxes have been selected. The map_import.pl script serves as an interface between map_index.cgi and maps.include. Dynamic links in maps.include are denoted by special symbols such as #link#?tt23. These symbols are easily located by map_import.pl, and replaced with a real HTML link, which includes whichever CGI parameters are necessary. A regular expression search-and-replace is used to accomplish this.

The map_import.pl program also serves as one of the two ways to harness the power of the maps interface from an external CGI. Read more about how to do that.

cgi-bin/map_index.cgi

This script provides the main client interface to the maps program. It keeps track of user input, and whether the user has selected a source or destination testbox. When a user has selected both source and destination, it redirects the user to a plots page or to an external script.

Because source and destination must be set one at a time, the interface provides a means for the user to choose to select a destination first, so as to see summaries of all packets sent to that destination, as well as from it. In the future, a feature could be added to support certain users, who only have access to their box and all packets sent to or from it.

cgi-bin/plots.cgi

This script serves as a wrapper for the plots pages. Given a source and a destination, it will fetch the appropriate plots page, and add links to it to redirect the user back the map page, allowing them to change none, one, or both of the testboxes they've selected. In the future, it could be expanded as part of the feature to support users, to allow them options that relate specifically to their testbox.

cgi-bin/map_index_alternate.cgi

This is one of two scripts which provide the Click Interface. It reads data from map_import.pl and parses it using regular expressions to locate relavent pieces of the code, such as imagemap definitions. It provides the user with a single map, (in a separate window), from which they can make their next selection.

cgi-bin/map_index_cluster.cgi

This is one of two scripts which provide the Click Interface. It reads data from map_import.pl and parses it using regular expressions to locate relavent pieces of the code, such as layer definitions. It provides the user with a list of testboxes contained within a certain cluster.


Implementation Details

The map interface

There are two versions of the interface, the MouseOver version, and the Click-through version. Which one a client uses depends on what features his or her browser can support. Unfortunately, some browsers implement layers and style sheets very differently than others, so it's hard to find features and implementations all will support.

The MouseOver interface

This interface is created by the use of overlapping layers. To support the maximum number of browsers, these layers are defined using style sheets. The layers are also defined within each other in the HTML document, creating the effect of them being stacked, or nested. The outermost layer, containing the world map, is positioned relatively on the page, which prevents it from overlapping anything else, because it is automatically placed below. However, every layer nested inside is positioned absolutely with respect to the base layer. This setup makes if very easy to define the locations of each map in a semi-browser-independant way, while still preventing keeping the maps confined to their area of the page.

Javascript is used to show and hide the maps as the user moves his or her mouse over the image. Functions are defined for the Internet Explorer 4+, Netscape 4+ and Netscape 6 document object models, so theoretically each will treat the layers in the appropriate manner. Sometimes, however, the solution had to be hacked a bit, for instance, to make maps disappear when the user moves the mouse over a map below it. Netscape 6 supports MouseOver events, even on images under the influence of imagemaps, which provides a nice, easy solution. Unfortunately, in Netscape 4 and Internet Explorer, imagemap areas override MouseOver properties, so I had to compensate by adding an imagemap area at the back, which covers the whole map. Netscape 6, however, does not support that, so I ended up with two standards, and lots of browser checking.

The maps are structures in a sort of a tree, such that the base map is considered the "root", and all maps and cluster layers that it opens directly are it's "children", and it is the "parent" of those maps. The nesting of the maps in the html code exactly reflects this tree.

The Click-through interface

This interface is provided for use by people whose browsers do not support the MouseOver interface. If a user bring his or her mouse over Europe on the world map, for instance, if the map of Europe does not pop up by itself, the user can click on that spot on the imagemap anyway. If they have javascript enabled, it will pop up another window containing only the map of Europe, from which they can continue. (using map_index_alternate.cgi) They can repeat the process in this window, as necessary, bringing up more windows containing maps or listings of clusters, until they eventually select a testbox. At this point, all the intermediate windows will close and they'll be redirected to the original page, which will then process their request. In essence, the interface tries to simulate the functionality of the MouseOver interface as cleanly as possible, using windows instead of layers. And if a browser does not support Javascript, or it is turned off, this will still work, but bring up all the additional maps in the same window, which makes navigation a little more challenging.

Clusters

Any time two or more testboxes appear to overlap on a map, they are called a cluster. One can't simply separate the boxes is clusters by zooming the map in farther, because, for instance, two boxes could reside in the same building. Therefore, a list of all boxes within a cluster is displayed when the user selects its location on the map. map_gen.pl defines a rather complicated algorithm to determine the locations of clusters, because, just because box A is within a certain radius from box B, and box B is within a certain radius from box C, doesn't necessarily mean that box A is within that radius from box C, yet A, B, and C should all be considered members of the same cluster. The algorithm itself is documented in the comments of map_gen.pl. You can set what radius to choose when grouping boxes into clusters, however, by changing a variable defined in maps.config.

Statuses

Testboxes exist in several status, such as OFF, ON, SETUP, etc. The maps program supports displaying boxes in several different status, and can change the appearance of a box based on its status. Statuses and their appearances are defined in maps.config. There are settings for defining both the appearance of the dot on the map and the appearance of the testbox name, when it appears in a cluster. These should be similar colors, but not necessarily the same, because certain colors show up different on a white versus a blue background.


How to use the map interface from an external program

There are two methods for connecting the map interface to an external program, such as plots-on-demand. They are as follows:

Method 1: Use map_index.cgi as a standalone tool, which can redirect its output to the p-o-d page, along with any other variables you'd like to send it as well. Format the link to the page in the form http://www.ripe.net/ripencc/mem-services/ttm/Plots/maps/map_index.cgi?srcvar=SRC&dstvar=DST&suffix=.ripe.net&redirect=/cgi-bin/ttm/plots-on-demand?date_start=20020301&date_end=20020302&mindelay=10, where you can pass it as many variables as you like (assuming they don't conflict with map_index.cgi's own variables), and you specify srcvar, dstvar and suffix to get the variables in the right format for your script. The default values for these variables are: 'src', 'dst', and no suffix.

Method 2: Load the map_import.pl script, and run the function import_maps ( url, boxvar [, redirect] ), where url is the URL of your script including query string (to set all the links to point to) and boxvar is the name of the variable in which you'd like the most recently-selected testbox to be stored. (You can also optionally feed it another url to redirect its final output to, as in Method 1, but you can probably leave this blank for Method 2.)

Most likely, the calling script will need to copy large parts of the map_index.cgi code, in order to process the input. In map_index.cgi, the most recently selected box comes in as a variable called 'box', and the code needs to work out by itself whether it's in fact a source or a destination. However, one could probably simplify this by calling import_maps() from within an IF statement, thereby automatically putting the variable in the right place. You will probably have to add the .ripe.net suffix yourself though, unless you modify maps_import.pl to handle an extra input. I didn't bother. Perhaps this is a development for the future.

By default, (if no redirect url is specified), map_index.cgi redirects the user to plots.cgi, which is a convenient wrapper for the plots pages.


Possible additions for the future

- You probably want to implement support for users with limited access to data, for instance, data to and from their own testbox only. I believe that map_index.cgi and plots.cgi can be easily modified to support this functionality. Instead of allowing the user two choices of testbox, for instance, only allow one, but keep the links allowing the user to choose between source and destination. This would allow them to be able to select any box, and see traffic to and from that box to their own box.

- You may want to clean up the maps.locations issue, and standardize things to only require one locations file. However, this won't make the code any more robust, so it's probably not worth too much fretting over.

- You may want to update the import_maps() to allow the addition of a suffix field, to facilitate its use for embedding maps in other programs, such as plots-on-demand

I'm sure there are a million other enhancements that could be made to the interface that I haven't thought of. If you have any questions about implementing any of them, or any other questions about the interface and how to use it, please feel free to contact at the email address listed at the top of this page.


$Id: README.html,v 1.2 2006/11/04 14:49:59 ruben Exp $