Archive for July 2003
Some comments on the current implementation of autoprovisioning for the SNMP poller.
First of all, from a developers perspective it is easiest by far if the information necessary is all centralized into one spot. This, in fact, is how you created the current implementation. The Dependency Viewer (still no new name…) just writes the provisioning information to the polling provisioner (layer 2 provisioner). The polling provisioner stores the information then in “measurements.conf”.
This creates a tightly coupled integration, where the contents of measurements.conf need to be completely determined by the dependency viewer tool. In order to have this working properly, nobody else should be allowed to touch measurements.conf, since that would break the sync between measurements.conf and the sql database that stores the component and datastream information. One of the problems is that the current implementation does not try to sync measurements.conf with the sql database, so if somebody or something modifies measurements.conf, that change won’t be reflected in SQL.
We could try to fix this, as I said, by forbidding anyone to change measurements.conf manually, or other than through the dependency viewer, but this would lead us along the wrong path.
Instead, we should try to keep the provisioning information at layer 2 and 3 independent. This gives us a more flexible implementation, that admittedly, is a bit more work to implement. Keeping the information independent means loose coupling, which has a lot of benefits. For instance, it means that people are explicitly allowed to manually change measurements.conf. Making the parts more independent makes it easier for people to provision the parts, and usually makes the system as a whole more robust.
So what does this all mean?
1. We need to identify which layer is authoritative (has the final say over) which information
2. We need to explicitly sync the information for which a system is not authoritative.
The Poller provisioner is authoritative for:
– Poll interval
– Mibname and other poller specific information
– Activation status of a datastream
Layer 3 is authoritative for
– Which datastreams belong to a component, as identified by measurement type
From the perspective of the Dependency viewer, we need to get the component and datastream (measurement type) information from the SQL database, but we need to get poll interval, mibname, and whether or not data is actually collected for a datastream (activation status) from the poller provisioner. Whenever the Dependency viewer shows us the information on a graph (datastream), that is where the informaiton should come from. If the poller provisioner does not contain a record for the datastream/measurement, the datastream/measurement is not active at that time.
From the perspective of poller provisioning, it does not matter how the information gets into the measurements.conf. Since measurements.conf is the authoritative source on the information even in Dependency viewer, if someone enters it manually, it will show up in the dependency viewer. Obviously, since the polling provisioner is not authoritative on components and datastreams / measurement types, new components and datastreams cannot be created from it.
There are a few consequences for the implementation:
1. The dependency viewer should get poll interval, mibname and activation status info from the poller provisioner. As mentioned, activation status just means the presence or absence of a record for the datastream
2. The polling provisioner should read the measurements.conf file upon every request (or monitor the contents of this file in another way), since this file can be modified outside of the provisioner process
3. The polling provisioner should have a method to retrieve the record for a single datastream
There were a couple of other points, independent of these, that I mentioned to Chou-san before:
1. The poller should be sent a HUP signal after an update, in order to force it to read new information. Otherwise, the measurements.conf would be changed, but the poller wouldn’t know it.
2. Optimistic locking is not implemented.
3. Change of measurement type should be allowed. As the Dependency viewer allows it too, this change would (silently) not be propagated to the poller, which would be a bug.
4. The measurments.conf file should be left intact, including comments. The current implementation just writes the comments at the start of the file, and rearranges the order of the records. Implementation hint: just read the file into a perl array, which each element representing a line. Keep a mapping from the hash to this array, and keep that up to date in case elements are deleted (easiest is to not actually delete an array element, but just mark it deleted). Additions should just go to the end of the array.