Bot's PhD work Diary

Post date: Aug 17, 2011 8:48:12 PM

The version 4.0.0 -- ITSBN with evidence in every scale

/home/student1/MATLABcodes/ITSBN_4.0.0_package_testing

The folder for the MLSP paper's results are

/home/student1/MATLABcodes/MLSP2011_ITSBN_3.0.2

which uses the ITSBN version 3.0.2. The results are finalized already.

Also we developed the segmentation evaluation using the Cauchy-Schwarz divergence. The code is here

/home/student1/Dropbox/random_MATLAB_codes/Image_Segmentation_Evaluation

Experiment of using GMM and VBGMM on the pixel-level image segmentation

/home/student1/Dropbox/random_MATLAB_codes/GMM_multiple_features_image_segmentation

I overlay the superpixel-boundary on the segmentation result, and see how well it fits the groundtruth boundary. [done]

The majority vote scheme with superpixel (a.k.a 3S) has been developed, and the algorithm is explained in detail here. [done]

Additionally, BIC+GMM and VBGMM for image segmentation are implemented in order to figure out the optimal number of classes. Somehow, it seems to me that GMM+BIC outperform VBGMM. [done]

[Aug 22, 2011] GMM+BIC is then implemented with ITSBN version 4.5.0 as the initial clustering for GMM. [done] The results look good, and the code is pretty clean. Here are some explanation for the algorithm.

1. Let's assume we the segmentation from GMM majority vote already, which means that each superpixel is assigned with a particular winner label already.
2. We use the labels as initial points for GMM when calculating mean and covariance matrix for ITSBN. The CPTs for the ITSBN are calculated according to the mean and covariance matrices.

-------------------------------------------------------------

[Aug 24, 2011] Some more results have been produced.

• ITSBN 3.0.2 (3 3 4) -- evidence at only the leaf level. results showing in MLSP2011 paper
  • /home/student1/MATLABcodes/MLSP2011_ITSBN_3.0.2/Figures_result
• ITSBN 3.0.2 (3 5 7) -- evidence at only the leaf level. results to compare with version 4.0.0
  • smb://bot@128.227.119.24/homes/bot/research/ITSBN_3.0.2_package_testing/Figures_result
• ITSBN 4.0.0 -- evidence in all scale.
  • smb://bot@128.227.119.24/homes/bot/research/ITSBN_4.0.0_package_testing/Figures_result
• ITSBN 4.5.0 -- evidence in all scale and initialized with 3S(GMM+BIC)
  • /home/student1/MATLABcodes/ITSBN_4.5.0_package_testing/Figures_result
• ITSBN 5.0.0 -- will be skipped to 6.0.0
• ITSBN 6.0.0 -- working on it
• 3S(GMM+BIC) alone
  • Berkeley BSDS500 dataset
    • /home/student1/MATLABcodes/result_samp_maj_vote_BIC_GMM/Berkeley
    • The number of classes are chosen using BIC criteria with the cluster number = 4-7
  • MSRL_146_run1 The number of classes are chosen using BIC criteria with the cluster number = 4-7
    • /home/student1/MATLABcodes/result_samp_maj_vote_BIC_GMM/MSRL_146_run1
  • MSRL_146_run2 The number of classes are chosen using BIC criteria with the cluster number = 2-5
    • /home/student1/MATLABcodes/result_samp_maj_vote_BIC_GMM/MSRL_146_run2

[Aug 25, 2011] Anand suggested using the evaluation code from Berkeley's. [not yet]

We should use several criteria to compare:

• Berkeley's PR
• Bot's novel generalized PR
• Randindex

Now I'm working on coding ITSBN version 6.0 which combine pixel-level features with superpixel-level features like I mentioned earlier in the recent idea depicted below.

I also would like to change the name from ITSBN to DDT because the former one is not self-explainable. DDT stands for Data-Driven Tree Bayesian Network.

Anand also suggested to make another experiment on the scale of the segmentation:

Originally, there is no clear idea on what does it mean by unsupervised image segmenting because the information of an object

significantly depends on the scale it is observed. Hence, an image segment of an image may or may not be meaningful in a particular

scale. This gives a new conclusion for a future of making an image dataset for unsupervised image segmentation. That is, a scene or

objects should be obtained collectively across multiple scale (or distance) so that we can use objects' common appearance across the

image or scale to validate the meaning of an image segment. This concept might invalidate the Berkeley dataset.

The procedure is as follows:

1. Obtain multiscale photos of a scene or objects. For instance, 3 pictures for 3 different distances to the object with cluttered background.
2. Run DDT on each of the picture
3. Spot common image segments/objects shared among all 3 images' the segmentation results. The shared objects/segments indicate that the segments are meaningful because they are presented/detected across the images. That also means the segment is scale invariant which can be one good property of "object".

I think we have to carefully define what makes object "an object" in order to claim the meaningness of a segment.

<Do it quick!>

[Aug 30, 2011]

For a few days, I tried to finish the ITSBN version 6.0. The obstacle is the bug I have when using pixel-level segmentation. At least today I can make ITSBN version 6.0.0 run smoothly in the case where there is no missing pixels/patches. However, with missing patches, the results look very weird. This is under investigation! Some preliminary results are shown in here.

[Aug 31, 2011]

This morning, the bugs are fixed already! And the code are cleaned and works pretty well so far. I am running the code on the Berkeley's dataset.

1) For now, I should look at the evaluation method and code from Berkeley.

I got the evaluation code of BSDS300 working already!

2) There is an approach that I should compare against [MDL]. The website not only provides the code, but also the results and the benchmark code. So, I should look at this website seriously.

3) Xiaofeng Ren's website contains a lot of empirical study about superpixel, hence is a very good resource to write about superpixel. Moreover, there is a discussion on the completeness of superpixel map which can answer the question about why I pick N_sup = 200. Refer to URL for more details.

4) Also read the internal report about Berkeley's dataset pdf.

[Sep 1, 2011]

1. Run TSBN on the Berkeley dataset
   • [done] we have the code ready already for TSBN
2. Experiment on the number of iterations
   • [done] We finally found that 15 iterations is sufficiently good.
3. Experiment on run-time vs # of nodes
   • I have this in the previous post.
4. Update the code fn_EM_ITSBN.m so that we can control the number of maximum iterations and all the parameters for each iteration
   • [done] We have done this in the code fn_EM_ITSBN.m in version 4.5.0 but haven't updated the one in the dropbox folder.

[Sep 4, 2011]

1. I am able to run the evaluation code for BSDS30 successfully! However, I don't know how to run it on BSDS500?? Therefore, I have to test my algorithm on BSDS300 instead of BSDS500. We are finishing some of them

1. The evaluation code is located at
   1. /home/student1/MATLABcodes/ITSBN_6.0_dev_dir/demo_evaluation1.m
   2. At this point, we can run the boundary evaluation and produce a report webpage.
2. I explore the results from UCM2, it looks good, but I have to ask the author if this is the correct result or not?
   1. The UCM2 results posted in the website is the same thing as presented in the paper?
   2. How to plot our PR curve compared to those algorithms presented in the paper?
   3. Code to evaluate the segmentation like PRI, etc as appeared in the paper [Use code from Yang's website instead]

[Sep 6, 2011]

1. Discuss with Anand
   1. superGMM is very interesting should aim for CVPR
      1. Need to improve the result to be competitive with the state of the art: change the features, do whatever!
      2. Should think about how to pick the number of components: using variational Bayes, Figueiredo (whose MATLAB code is available here), BIC. [comment] I use Figueiredo' work, but it's over-segments the data--at least in colorspace case.
   2. For unsupervised image segmentation, we need to
      1. What is the state of the art?
      2. How are we doing with respect to state of the art?
      3. Why the unsup img segm make sense? it is actually nonsense, why does it make sense to you?
   3. Here might be the state of the art
      1. Saliency driven unsupervised segmentation [link] ICCV 2009: There are table containing several algorithms
      2. Segmentation of Natural Images by Texture and Boundary Compression [link] IJCV 2011: The code to benchmark is provided
      3. This website contains links to a couple of main segmentation algorithm [link]
      4. Unsupervised Segmentation of Natural Images via Lossy Data Compression [link] PAMI2007: This webpage contains code for benchmarking with 4 main indices: PRI, VoI, GCE, BDE, which are standard for many papers.
2. What we can improve on the existing DDTs' results
   1. We can play with the features: sparse-SIFT, HOG, k-mean-Texton, color consistency [link]
   2. Build a tree structure from UCM2. I will have to implement region-merge algorithm
   3. Change the number of levels and labels
   4. Next, we should consider using PRI, VoI, GCE, BDE for image segmentation evaluation!!!. Visit this link for more details on the evaluation method. [done] The code works well.
   5. Change the weights for each boundary level. [doing] I'm now experimenting on this issue.
   6. Use UCM boundary as our superpixel
   7. Using graph-based region merging to build a tree

[Sep 7, 2011]

1. Today I'm running more experiments on DDT4.5 with several settings in weights for each level of boundaries. I hope to see the changes in the best F-measure.
2. Also I run the mevTSBN on BSDS300.
3. I'm thinking about having a directory to keep all the segmentation results from those algorithm I tried.

[Sep 8, 2011]

1. Review Figueiredo's EM algorithm
2. Work on the UCM's boundaries
   1. Convert UCM --> segments
   2. Make tree out of the UCM
   3. Run and see the results. Perhaps I should run it on the training set
3. I should think about running multiscale superGMM.--> Sep 8!
   1. Run superGMM for each level, the user needs to specify the number of model-class
   2. At the end, combine the boundaries from each level together before evaluation
4. Implement Figuerado's EM to the superGMM so that it can figure out the number of the model-label automatically.