OpenModeller SDM Results

Ok, so I deconstructed the habitat environment file into 10 separate layers using the QGIS raster calculator. In each file 0 indicates no feature and 1 indicates the presence of the habitat feature. I decided to stick with the default SVM algorithm and no tuning. The results were to say the least underwhelming, all the models except the common pipistrelle gave absolutely meaningless results. After many hours of investigation it appears that if you don't have a presence sample in any one of the particular environment layer, it causes the model to fall over with no errors or warnings... So, I set up a subset of layers applicable to each species so that at least the model would run, and it is these results presented in the second column in the table below.

Some points to note:

• The algorithm definitely does not cope with a single categorical raster, you have to split the individual layers out

• If you have a redundant habitat feature/layer then the algorithm does not cope with this (this must be a bug)

• Some of the output has been "swiss-cheesed" where habitat data is not available due to the problem above

• The default regularisation factor appears to be more broad than the default MaxEnt model judging by the spread in the results

• There is now a significant level of correlation between the MaxEnt and these SVM results, the main differences appearing to be down to the regularisation factor

There is certainly enough correlation to suggest that this particular algorithm has potential, but having to separate the habitat categories into different binary (0,1) rasters is a major additional piece of work, and having ten times the number of files to keep under configuration control, is, well, ten times the problem of controlling one! I'm also not confident in the robustness of the SVM implementation in this case, it should handle environment layers where there is not a corresponding presence, or at least inform the user why it's producing garbage. The other main algorithm of interest to me was GARP, but the results are pretty well useless and more work is needed to understand what is going wrong. The OpenModeller MaxEnt algorithm is not much better either, and if you want a to use a MaxEnt model, avoid it and stick with the Princeton Team's package.

If you are looking for a robust SDM package that has been optimised for general use and has years of validation to back it up, I would strongly advise you to stick with the original MaxEnt package from Princeton University. This statement in no way is intended to marginalise the OpenModeller project team, it's just down to the maturity and extensive published data using MaxEnt.

If you're prepared to work on your own algorithms and optimise/validate these, then OpenModeller is the choice for you. Without doubt, OpenModeller has the potential to overtake MaxEnt in the longer term and you need to keep a watching brief and get involved if you have the time!

Barbastella Barbastellus

 Eptesicus Serotinus

Myotis Alcathoe

 Myotis Mystacinus

Myotis Nattereri

 Nyctalus Leisleri

Nyctalus Noctula

 Pipistrellus Pipistrellus

Pipistrellus Pygmaeus

 So, how different is different? I've not undertaken any tuning of the SVM model so perhaps what follows is bit of worst case, however the figure below shows the magitude of the difference between the SVM model and the equivalent MaxEnt one on a pixel by pixel basis calculated using the QGIS raster calculator.

 The scale is such that any blue areas indicate regions where the models tie up reasonably well. The green areas indicate a difference in presence probability of approximately 0.4. At first glance this does not look that spectacular a result, but considering that the SVM result is just a single training and test set "result" and not an average of many trials like the MaxEnt ones, this shows a surprisingly high degree of similarity. The main differences can be put down to the difference in regularisation (tuning) and the presence sample allocation between training and testing for the SVM model (the SVM model basically uses just one of the 50-100 trials run in the MaxEnt model).