Second Approach: Probabilistic Model



The objetive of this second approach is to evaluate the performance of a segmentation method where depth data is considered in GrabCut's energy function in analogous way to color. Here we propouse a seeding strategy based in depth data.



Energy Function Definition

The energy function to be considered is,

equation1.png

where WC represents the Color Model and WD the Depth Model, both defined as Gaussian Mixture Models.

The Data Term is given by the expression,

equation2.png
where parameters aC and aD are control parameters for Color and Depth Data.

Finally the Smoothness Term is given by ,
equation3.png
equation4.png
where parameters gC and gD are control parameters for Color Smoothness and Depth Smoothness.

Seeding

  1. Construct a Background Depth Model  from the pixels in the border of the Selection Rectangle.
  2. Construct a Foreground Depth Model from  the pixels in the interior of the Central Rectangle, this is, the rectangle of sides 1/2 of the Selection Rectangle and located in its centre.
  3. Discard the Gaussian components of the Foreground Depth Model whose mean value have the highest probability according to the Background Depth Model. 
  4. Adjust the participation coeficients of the remainig components of the Foreground Depth Model to sum up to one.
  5. Evaluate all pixels in the image respect to these models, and mark them as Foreground Seeds or Background Seeds according to the obtain likelihood.
  6. From the Foreground Seeds build the definite Foreground Depth and Color Models. Do the same with the Background Seeds.


Results

RGB Seeds Result
CaballoInt-0.png CaballoInt-1.png CaballoInt-2.png
Fabian2Int-0.png Fabian2Int-1.png Fabian2Int-2.png
PumbaInt-0.png PumbaInt-1.png PumbaInt-2.png

Remarks

  • Seeding was not accurate and consequently most of results were unsatisfactory.
  • Try to build good models for depth data distribution from Gaussian Mixtures is a difficult task. You must deal with the fact that BG depth-variance is (in general) much larger than FG depth-variance.
  • There is a lot of work to do in parameter calibration for the propoused energy function.