Since there is not a gold standard dataset nor a benchmark, we design a strategy for the evaluation of our algorithm. The key concept of our strategy is to align a networks with respect to itself and with respect to an altered network obtained by adding/removing nodes and edges to the initial one, i.e. adding noise. We perform the evaluation of L-HetNetAligner by analysing results obtained when we have aligned the initial network to those obtained using various levels of alterations considering 5%, 10%, 15%, 20% and 25% of added noise. Our aim is to demonstrate the ability of our algorithm to build high-quality alignments and to demonstrate that the quality of the alignment increases when considering network colours.
Consequently we have built the noisy counterparts 1) for each of the heterogeneous synthetic network versions with one, two, three, and four colours and 2) for the Hetionet network with one two, three, and four colours.
Then, we apply L-HetNetAligner to build the alignment of each synthetic network with its counterparts. To build the alignment graph, we set ∆ threshold = 2. Then, we apply MCL on the alignment graphs to mine subnetworks. We perform different experiments and we have obtain best results setting the Inflation Parameter of MCL equal to 2.8 (we here present only these results). The output consists of local alignments as relevant modules.
Then, we evaluate alignment quality. For this aim, we computed the NCV-GS3 measure on each extracted module.
Our goal was to demonstrate that the extracted modules present high-quality alignments in terms of reconstruction of the true node mapping and conservation of as many edges as possible. We have computed the NCV-GS3 for local alignments of synthetic networks and Hetionet network.
We should note that the NCV-GS3 values increase when increasing the number of colours, showing the best results in 4 coloured versions for both synthetic networks and Hetionet network.
FIGURE 1
FIGURE 3
Functional Quality Evaluation
Also we have evaluated the quality of results by assessing the biological relevance of extracted modules from Hetionet network.
Our aim is to determine if the modules extracted by aligning the Hetionet network with more node colours are better in terms of semantic similarity than those extracted from aligning the Hetionet networks with a single node colour. Therefore, we computed the semantic similarity values of extracted modules using the Resnik’s semantic similarity measure32 with the Best-Match Average (BMA) approach. Then, we estimated the average semantic similarity for all the discovered modules from aligned networks. Figure 3 reports an overview of average semantic similarity values of modules obtained by align Hetionet network with its noisy counterparts for each coloured version. As seen, the 4 coloured version presents the highest semantic similarity compared than other coloured versions.