FoCal Bilinear
Tools for detector fusion and calibration, with use of side-information.
This is a modification of some of the tools of the FoCal Toolkit to be able to incorporate side-info or quality measures to try to produce more discriminative and better-calibrated detection scores.
The primary tool of FoCal Bilinear performs supervised training of a fusion of m detector scores, which can be modulated by n side-info parameters. The fusion is bilinear to allow scores to be modulated (multiplied) by the side-info. Bilinear means that if the side-info were constant, then it would be a linear fusion of scores, or if the scores were constant, it would be linear fusion of side-info. The fusion is designed for a situation where:
The scores have discriminative power (more positive for targets and more negative for non-targets), while
the side-info gives little or no discrimination between targets and non-targets, but has the potential to improve scores by shifting them or by modulating their amplitudes.
This function is designed to train side-info assisted fusions, where each side-info parameter is confined to the range 0 to 1. One possibility is for the side-info to indicate a few discrete categories of trials. For example:
If there are three kinds of trials, the side-info vector could indicate these categories via hard labels of the form: [1,0,0] or [0,1,0] or [0,0,1]. In this case, this function would effectively train three separate fusers, one for each of the three categories. The side-info vector effectively chooses which fuser to use for a given trial.
If categories cannot be recognized with certainty, the side-info vectors could be soft labels in the form of probability distributions, such as: [0.9, 0.05, 0.05] or [0.1, 0.2, 0.7] etc. In this case, the side-info vector gives the weighting for a trial-dependent, convex combination of the three separate fusers.
This tool may be used as is, if a side-info extractor already exists and can supply its output in the probability distribution form suggested above. However it is possible to incorporate this tool in a more complex iteration to also optimize the extractor of side-info. One possible way of doing this is via an EM-like algorithm:
Use some method to assign one of a small number (say N) of discrete labels to every detection trial of a development database.
Use supervised training of your favourite pattern recognizer to recognize these labels as accurately as possible. Output the recognition result as a calibrated posterior probability distribution. FoCal Multi-class may be useful here to optimize the multiclass calibration. These posterior distributions form your side-info.
Train a bilinear fuser (this toolkit) to fuse your favourite detection scores with above side-info. The result gives N different affine score fusers, each for a different trial category.
For every trial, select the fuser which gives the best score (most positive for target trial, or most negative for non-target trial) and re-label your development data accordingly.
If not converged, go back to 2.
Download MATLAB code: focal_bilinear.zip
Update: replace train_side_info_fusion.m with this version: train_sideinfo_fusion.m (This update adds a compensation for category-dependent target/non-target proportion mismatches in training data.)
Place all MATLAB files in your path and type at the MATLAB prompt:
>> demo_sideinfo_fusion
The code is documented. You will mostly need to use the functions train_sideinfo_fusion() and apply_bilinear_fusion().
Ideas for this tool came from collaboration with Daniel Ramos, discussions with Patrick Kenny and David van Leeuwen and an ICASSP 2008 paper by Luciana Ferrer entitled "System combination using auxiliary information for speaker verification".