Overview of histological features for identifying the level of hormone treatment. The number of deviating features (group IV) is the sum of features in group III and either group I (male) or group II (female) that have a state differing from “none”. Feature 2 is excluded from this sum since it only applies to female animals.
Expert systems, or more precise decision support systems, are valuable tools for structuring the results of scientific research and to translate this to knowledge. The decision support system Determinator is now used for several years as a platform for models to identify subjects [1, 2]. The system is based on the two main different procedures for identification [3, 4]; a single access key (tree) and a free access key (matrix). The latter option provides the possibility to calculate the match between the subject as chosen by the user and the objects as included in the data model, based on a range of characteristics. In addition, a matrix allows to make selections, to filter the set of available objects and to compare two objects for their variability.
Datamodels for Determinator can be constructed using a Developer, which is part of the entire Determinator platform. Besides defining the objects (descriptions, illustrations and labels), the characteristics, and the connection between them (the matrix), the Developer also allows to evaluate the structure of the data model. Several parameters and metadata for the evaluation of a data model are part of the Developer.
This chapter provides the logic basis for the Determinator platform and introduces the background and calculation of four different parameters for the evaluation of data models: the coverage of variability space of the total data model or of a single object, the redundancy in a data model, and the capability to distinguish between different objects. The way in which these parameters are developed and applied will be demonstrated using a real case concerning the diagnosis of illegal hormone treatment of veal calves [5, 6]. The applicability of the parameters will be discussed and the development of a specific case (histological diagnosis) in a general platform (Determinator) will be evaluated.
2. Material and methods
A datamodel developed in the framework of the DSS Determinator includes the following tables:
List of features, with image file names and descriptions,
Groups of features, with names and descriptions,
List of targets, with image file names, descriptions and labels,
Match table, with the feature on the rows and the targets on the columns,
Tree information per node, with descriptions and image file names.
A data model consists of
2.1. Logic basis
Every cell in the matrix
The use of different states of a feature can add to the separation capability of that feature. Assuming three feature states:
In this logic distribution, feature state
The DSS Determinator allows the user to choose a subject for identification and to answer a range of questions denoting the
A typical dichotomous tree consists of nodes (lemmas), which can point to either two targets (leaves), two nodes (branches) or combination of the two. Basically, every lemma in a tree is based on the decision rule:
The combined feature state
2.2. Quality parameters
The following parameters for validation of data models are being developed and evaluated in the framework of this paper.
Overlap between the areas of two targets exists when a variability range for target
Overlap per feature:
Average overlap for all feature differences between two targets
The average redundancy of the total data model is the averaged overlap of every combination between two targets
The smaller the average redundancy, the smaller the chance that a certain range of feature states of a chosen subject will result in two or more match percentages of 100 % (according to (4); see object 3 in Figure 2). Redundancy is related to the correlation coefficients among features.
The capability to distinguish between two targets
This can be rewritten as:
If the distinction between two targets is based on only one feature
A data model can identify uniquely every target if and only if every combination between two targets
A datamodel can be indicated as suboptimal or not valid when the differentiation coefficient
Every target possesses a part of the
Coverage of space of a single target
In the situation of
The larger the coverage of the total variability space, the smaller the chance that a certain range of values of a subject will result in no match percentage of 100 % (according to (4)). In the situation that
The diagnosis for illegal growth hormone use in veal calves will be used as illustration of model development and performance testing.
3. Model development and application
The use of illegal growth promoters is, although prohibited in the European Union, still part of current practice in animal farming. Reasonable monitoring of the hormones is hampered by the fact that the hormone or hormone cocktail is metabolised or excreted within a period of a few weeks. The effects of the use of hormones, however, can be seen in histological stained sections of either the prostate (male calves) or gland of Bartholin (female calves) with different staining techniques. The monitoring by means of histological examinations appears to be an important instrument in maintaining legislation for food safety and animal health [5, 6]. The interpretation of histological disorders needs a high level of expertise. An expert model has been developed in the framework of the DSS Determinator, in order to support the user to identify the extent of hormone treatment of veal calves. The different quality parameters will be illustrated after a further presentation of the model.
The data model consists of 13 features to identify a treatment level indicated as “normal”, “suspect” or “positive”. The features are presented in Table 1, and some of them are illustrated in Figure 3.
|I||1||Presence of metaplasia (male)||[none,mild,severe]|
|II||2||Ratio between ducts and glandular tissue||[normal,more_ducts,mainly_ducts]|
|II||3||Presence of metaplasia (female)||[none,mild,severe]|
|II||4||Combined presence of metaplasia and an elevated duct ratio||[no,yes]|
|III||5||Presence of hyperplasia||[none,mild,severe]|
|III||6||Presence of cysts||[none,mild,severe]|
|III||7||Presence of hypersecretion||[none,mild,severe]|
|III||8||Presence of vacuolisation||[none,mild,severe]|
|III||9||Presence of muceus cells||[none,moderate,severe]|
|III||10||Presence of inflammation||[none,moderate,severe]|
|III||11||Presence of folding in the urethra||[none,moderate,severe]|
|III||12||Presence of thickening in the urethra||[none,moderate,severe]|
|IV||13||Number of deviating features||[0,....,9]|
There are two strategies to reach a diagnosis:
A quick, general diagnosis. Depending on the sex of the calf, selecting either feature groups I and IV (male) or groups II and IV (female) is sufficient.
An extended diagnosis. In addition to the feature groups as indicated in strategy A group III is necessary.
The kernel of the data model consists of the groups I, II and IV to give a diagnosis of the treatment level. The diagnosis for possible hormone treatment in female calves is more complicated than for male calves. This is caused by the natural production of oestrogen hormones, which is lacking in male calves. The simple diagnosis <IF metaplasia=present THEN target positive> needs further support in female calves. A second diagnostic feature is used based on a larger share of ducts in the glandular tissue. The basic rule is then expanded to <IF metaplasia=present AND duct_ratio=elevated THEN target positive>. For both male and female calves the diagnosis “suspect” is supported by the number of deviating features. The duct ratio is excluded from this feature since it applies only to female calves. The logic tables to diagnose the level of treatment are presented in Table 2.
|[more,mainly]|| “positive”||#=[0,1,2] → “normal”|
|[normal]|| #=[1,2] → “normal”|
#=[3,...,9] → “suspect”
|#=[3,...,8] → “suspect”|
| “positive”|| #=[0,1,2] → “normal”|
#=[3,...,8] → “suspect”
The diagnoses as illustrated in Table 2 can be extended further by including the individual features of group III (Table 1). The number of deviating features (feature 13) needs to be adjusted accordingly. The basic rules are translated in a formal decision tree, as shown in Figure 4.
Finally, the decision tree is used as basis for a free access key. The importance and position of the feature indicating the presence of metaplasia is different for male and female diagnosis. For the latter only the combination of metaplasia and elevated duct ratio is decisive for the diagnosis “positive”. As a consequence, the presence of metaplasia is included twice in the free access key as feature 1 (group I for male animals) and feature 3 (group II for female animals). The free access key was optimised by giving all features a suitable weighting factor. All features of group III got the factor one.
The performance of the model is tested in eight runs following the two strategies. The continuous feature 13 is varied between 0 and 9 in every run in combination with the appropriate choices for the other features, as follows:
A1 (male): groups I and IV are used. Choice for feature 1 is [none].
A2 (male): groups I and IV are used. Choice for feature 1 is [mild] unless:
F 13,k =0 → F 1,k =[none].
A3 (female): groups II and IV are used. Choices for features 2 and 3 are [normal] and [none].
A4 (female): groups II and IV are used. Choices for features 2 and 3 are [more_ducts] and [none].
A5 (female): groups II and IV are used. Choices for features 2 and 3 are [more_ducts] and [mild] unless:
F 13,k =0 → F 3,k =[none].
B1 (male): groups I, III and IV are used. Choice for features 1 is [none].
B2 (male): groups I, III and IV are used. Choice for features 1 is [mild] unless:
F 13,k =0 → F 1,k =[none].
F 13,k =>1 → the appropriate number of features of group B and C get the state [mild] or [moderate].
B3 (female): groups II, III and IV are used. Choice for feature 3 is [mild] unless:
F 13,k =0 → F 3,k =[none].
F 13,k =>1 → the appropriate number of features of group B and C get the state [mild] or [moderate].
The choice for [severe] instead of [mild] will give identical results except for the presence of hyperplasia (feature 5).
In every run the matches between the simulated subject and all three targets (treatment classes) “normal”, “suspect” or “positive” were calculated according to equation (4). The results for the eight runs are shown in Figures 5 and 6.
The model after adjusting the appropriate weighting factors shows the highest match percentage for the same target (class) as indicated by the tree (Figure 4) in all cases. The percentage for a diagnosis “positive” of a male animal (Figure 5) is 0% when no deviating feature is found, in contrast to a diagnosis of a female animal (Figure 6) where an elevated duct ratio can be found in combination with # deviating features = 0. For the same reason is the difference between the diagnoses “normal” and “positive” smaller for male animals (Figure 5d) than for female animals (Figure 6d) in the case that # deviating features = 1. In general, the comparable situations as illustrated in Figures 5a and 6a/b, in Figures 5c and 6c, and in Figures 5d and 6d respectively, shows highly comparable results. The addition of the features of group III (Figures 5b, 5d, 6d) modifies the outcome of the model in the sense that in a lot of cases not 100% score can be reached. This reflects the situation that the finding of metaplasia (male) or the combination of metaplasia and an elevated duct ratio (female) accompanied with only a few or even no other deviations is unlikely or highly unlikely.
The large coverage of the targets indicated as “positive” (Table 3) is caused by the situation that the model is focusing on the correct diagnosis of possible treatment minimising the possibility of having false negative results. In both cases for male and female calves the final diagnosis is based on one feature (see Table 2 and Figure 4), whereas the states of the other features are overruled.
|001. Normal (prostate)||0.03||(11)|
|002. Suspect (prostate)||2.89||(11)|
|003. Positive (prostate)||34.68||(11)|
|004. Normal (Bartholins gland)||0.01||(11)|
|005. Suspect (Bartholins gland)||13.49||(11)|
|006. Positive (Bartholins gland)||34.68||(11)|
The correlation between the features is shown in Table 4. Only a full correlation is found between the two features indicating the presence of metaplasia. This feature is included twice since different weighting factors appeared to be needed for the different animal types. Another reasonable high correlation factor was found between the duct ratio and the combined presence of metaplasia and elevated duct ratio. The presented level of correlation coefficients is in line with the calculated average redundancy: 0.405 (equation (8)).
The match table (Table 5) shows the relative resemblance between the targets based on equation (7). Except for the diagonal, the green colour, based on the calculations using equation (9), indicates that every target can be diagnosed uniquely compared to any other target. Hence, the separation capability is 100% (equation (10)).
The process of identifying the level of treatment with growth hormones of veal calves is a rather specific situation for diagnosing in the broader framework of application of DSS in medicine [8-10]. Only one feature matters, all other features will only modify the probability that a diagnosis belongs to the correct class. Besides that, a constraint dependency rule existsbetween feature 13 (number of deviating features; Table 1) and the totalof features from group III plus either from group I or group II which show a state other than normal. The importance of the main features is visible in Table 2 and Figure 4. The two main features (male: presence of metaplasia, female: combined presence of metaplasia and an elevated duct ratio) both got a weighting factor of 9 in order to outnumber the features in group III for reaching a correct diagnosis (number of features in group III plus 1). Since the presence of metaplasia in the diagnosis of a female calf does not form the exclusive indicator for treatment in contrast to the position of that feature in the diagnosis of the male calf, it got a weighting factor of only 1. The weight factors in the current model are fixed instead of being input sensitive .
There is no generic method for validation of data models in expert systems . In the current study a top down modelling approach was chosen: logic tables lead to a decision tree, which was the basis for the full matrix of the free access key. This approach does not provide a tool for handling constraint dependency rules , which was solved here by optimising the weighting factors. Rass et al.  listed a number of requirements for valid expert systems. Of these, the requirements for minimising the redundancy and for avoiding unintended synonyms are now supported bymeasures to calculate the extent of these parameters: redundancy (equation (8)) and separation capability (equation (10)), respectively.
The position of the features of group III (Table 1: indicating the individual deviating characteristics) in an extended diagnosis (Figures 4b, 4d, 5d) can be discussed in terms of fuzzy logic principles. In several experiments with fuzzy logic comparable results have been found [9, 13]. Here, probability or uncertainty is the basic aspect causing patterns in the model outcomes that can be explained as membership functions . As an example, the presence of metaplasia in a prostate is a definite diagnosis for treatment with growth hormones (n = 1 in Figure 5c in concordance with the tree in Figure 4), but it is highly unlikely that with such a diagnosis none of the other features of group III (Table 1) would show a state deviating from normal. The probability that an animal with the sole presence of metaplasia belongs to membership class “positive” is only slightly higher than its membership to the class “suspect” (n=1 in Figure 5d). The kernel model without using the individual features of group III (strategy A) seems sufficient to reach a diagnosis. All the features underlying the depending feature 13 (group IV) are nevertheless included in the model in order to improve the performance of the user by supporting his or her examinations, and to provide the possibility of an iterative process of optimising the diagnosis .
Existing results of optimising a datamodel for reaching a diagnosis reveal that lower numbers of features appeared to be optimal . In those cases that a model consists of only a few features, expressing them in terms of space dimensions (e.g. a two-dimensional space in Figure 2), a major part of the variation space might be covered. Increasing numbers of features (i.e. dimensions) result in an exponentially growing number of theoretically existing feature combinations that are not linked to a target. In the present study a total of approx. 14 % of the variability space was not covered by any target (Table 3). In order to evaluate this non-assigned part of the variability space, let us assume a variable number of features
The resulting multidimensional spaces for a number of features ranging from 2 to 8, the number of targets accommodated and the resulting coverage are shown in Table 6. If more than one state of a feature can identify a target a larger coverage can be expected. This is the case in the here presented datamodel for the diagnosis of hormone treatment, since the probability to correctly classify all situations of hormone treatment was maximised. This is illustrated in Table 3. The high coverage of approx. 85.8% of the current model can be explained by the situation that the model was optimised to find all occasions of illegal use of hormones, i.e. the coverage of the classes “positive” was maximised.
The development of a specific model for reaching a histological diagnosis in a general platform provides several constraints, such as the lack of automatically calculating the number of deviating features (feature 13) from the number of individually selected features of group III. The advantage of the current procedure is the strict framework which forces to analyse the information structure in detail, and generic tools are available for testing and evaluation.
The presented parameters for redundancy, uniqueness, separation capability and coverage of variability space provide useful tools for the validation of a datamodel. The Developer as part of the Determinator system implements these parameters in an ordered manner, as exemplified in Table 5. The development and performance of the datamodel for reaching a diagnosis of the treatment of veal calves with hormones in the framework of Determinator reveals that a specific model can be developed and applied successfully in a generic framework.