Raghudeep Gadde* Varun Jampani* Martin Kiefel Daniel Kappler Peter V. Gehler
Abstract
In this paper we propose a CNN architecture for semantic image segmentation. We introduce a new “bilateral inception” module that can be inserted in existing CNN architectures and performs bilateral filtering, at multiple feature-scales, between superpixels in an image. The feature spaces for bilateral filtering and other parameters of the module are learned end-to-end using standard backpropagation techniques. The bilateral inception module addresses two issues that arise with general CNN segmentation architectures. First, this module propagates information between (super) pixels while respecting image edges, thus using the structured information of the problem for improved results. Second, the layer recovers a full resolution segmentation result from the lower resolution solution of a CNN. In the experiments, we modify several existing CNN architectures by inserting our inception modules between the last CNN (1 × 1 convolution) layers. Empirical results on three different datasets show reliable improvements not only in comparison to the baseline networks, but also in comparison to several dense-pixel prediction techniques such as CRFs, while being competitive in time.
Illustration of a bilateral inception module.
Paper
Please consider citing the following paper if you make use of this work and/or the corresponding code:
@inproceedings{gadde16bilateralinception,
title = {Superpixel Convolutional Networks using Bilateral Inceptions},
author = {Gadde, Raghudeep and Jampani, Varun and Kiefel, Martin and Kappler, Daniel and Gehler, Peter},
booktitle = {Computer Vision -- ECCV 2016},
series = {Lecture Notes in Computer Science},
publisher = {Springer International Publishing},
month = oct,
year = {2016}
}
Code
We integrated bilateral inception module codes into Caffe neural network framework. Code is available at this GitHub page.
Caffe Layers for Bilateral Filtering
Although bilateral filtering can be implemented with a single neural network layer, to be computationally efficient, we implemented bilteral filtering with different computation blocks. This allows to re-use feature transformations and pairwise distance computations across different bilateral inception modules. The following figure shows the computational blocks in the bilateral filtering operation.
Computation flow of the Gausian bilateral filtering.
The bilateral filtering operation is implemented with following computations (layers) in Caffe neural network framework: Feature computation (SpixelFeature layer) and transformation (standard Caffe Convolution layer); Pairwise similarity computation (Pdist layer); Scaling of the kernels (standard Caffe Scale layer); Kernel normalization (standard Softmax layer) and Matrix multiplication of kernel with input data (MatMult layer). We also implemented a layer (Smear layer) to go from superpixels back to pixels. Below, we will briefly describe the non-standard Caffe layers that we used for bilateral filtering computation.
Superpixel Feature Computation and Transformation:
The following Caffe prototxt layers illustrate the computation of color and position features for superpixels; feature transformation and; smearing the superpixel features to all the image pixels.
# Extract Superpixel features
layer {
type: "SpixelFeature"
name: "spixel_feature"
bottom: "img" # Image data: blob dimensions - N x C x H x W
bottom: "spixel_index" # Superpixel indices for image pixels: blob dimensions - N x 1 x H x W
top: "flatten_xyrgbxy_features" # Output superpixel features - N x F x 1 x max_spixels; where 'F' is feature dimensionality and 'max_spixels' (see below) is the maximum number of super pixels
spixel_feature_param {
type: AVGXYRGBXY # Feature type: : At present 'AVGXY', 'AVGRGBXY', 'AVGRGB' and 'AVGXYRGBXY' are supported
max_spixels: 1000 # Maximum number of superpixels
ignore_feature_value: 255 # Output feature value for the non-populated output elements
ignore_idx_value: 100000 # This input superpixel ID value will be ignored
rgbxy_rgb_scale: 0.1 # Color feature scale in RGBXY features
rgbxy_xy_scale: 0.01 # Position feature scale in RGBXY features
xy_scale: 0.1 # Position feature scale in XY features
}
}
# Transform the features (standard 1x1 convolution)
layer {
type: "Convolution"
name: "transform_features"
bottom: "flatten_xyrgbxy_features"
top: "flatten_xyrgbxy_features2"
convolution_param {
num_output: 7
kernel_size: 1
}
param {
lr_mult: 1
decay_mult: 0
}
param {
lr_mult: 0
decay_mult: 0
}
}
# Smear the features to image size
layer {
type: "Smear"
name: "feature_smear"
bottom: "flatten_xyrgbxy_features2" # Superpixel features: blob dimensions - N x F x 1 x max_spixels
bottom: "spixel_index" # Superpixel indices for all pixels - N x 1 x H x W
top: "xyrgbxy_features" # Image features - N x F x H x W
smear_param{
ignore_feature_value: 255 # Output feature value for the ignored pixels
ignore_idx_value: 100000 # This input superpixel ID value will be ignored
}
}
Pairwise Similarity Computation:
Once the pixel and superpixel features are computed, the pariwise similarity distance kernel between the input/output pixels or superpixels can be computed using ‘Pdist’ layer. The following prototxt samples show computation of pairwise similarity kernels with different features and between pixels/superpixels and superpixels.
# RGBXY kernel from pixels -> spixels
layer {
type: "Pdist"
name: "pixel_rgbxy_dist"
bottom: "pooled_rgbxy_features" # Input blob-1: blob dimensions - N x C x H1 x W1
bottom: "flatten_rgbxy_features" # Input blob-2 - N x C x H2 x W2; H2 = 1 for superpixels
top: "pixel_rgbxy_distkernel" # Output blob - N x C x H2W2 x H1W1; Note the top size is second_bottom_size x first_bottom_size
pdist_param {
ignore_value: 255 # assigns 1e10 (large number) for distance to the elements with this value.
scale_value: -1.0 # Scales the computed kernel with this value.
}
}
# XY kernel from pixels -> spixels
layer {
type: "Pdist"
name: "pixel_xy_dist"
bottom: "pooled_xy_features"
bottom: "flatten_xy_features"
top: "pixel_xy_distkernel"
pdist_param {
ignore_value: 255
scale_value: -1.0
}
}
# RGBXY kernel from spixels -> spixels
layer {
type: "Pdist"
name: "spixel_rgbxy_dist"
bottom: "flatten_rgbxy_features"
bottom: "flatten_rgbxy_features"
top: "spixel_rgbxy_distkernel"
pdist_param {
ignore_value: 255
scale_value: -1.0
}
}
Matrix-Vector Multiplication:
The matrix-vector multiplication between the bilateral filter kernel and the input CNN data can be performed using ‘MatMul’ layer as follows.
# Matrix-vector multiplication between the input data and the kernel
layer {
type: "MatMul"
name: "apply_fc6_kernels"
bottom: "fc6_reshape" # Input data: blob dimensions - N x C x T x D; T = 1 for our case (e.g., flattened FC6 signal with C channels and D points; or signal with D superpixel pixels)
bottom: "fc6_exp_kernel_pool" # Input bilateral filter kernel with K kernels - N x K x L x D
top: "fc6_kernels_out" # Filtered signal: N x CK x T x L
}
Adding Bilateral Inceptions into CNNs
A bilateral inception (BI) module is a weighted combination of bilateral filters at different scales (see Figure at the top of this page). Due to modular computation of bilateral filtering (see above), adding BI modules into existing CNN architectures involves adding several new layers. In this work, we add BI modules between standard 1x1 convolution layers found in many state-of-the-art segmentation architectures. This is illustrated in the following figure.
Illustration of CNN layouts with and without bilateral inception (BI) modules.
Below is a modified Caffe prototxt of DeepLab-LargeFOV with two inception modules BI6(2) and BI7(6). BI6(2) is added after FC6 and has two kernels, wherease BI7(6) is added after FC7 and has 6 kernels.
name: "VOC12"
input: "img"
input: "spixel_index"
input_shape {
dim: 1
dim: 3
dim: 513
dim: 513
}
input_shape {
dim: 1
dim: 1
dim: 513
dim: 513
}
layer {
name: "dummy1"
type: "DummyData"
dummy_data_param {
num: 1
channels: 1
height: 1
width: 1000
data_filler {
type: "constant"
value: 1
}
}
top: "data1"
}
#############################
# Compute superpixel features
#############################
layer {
type: "SpixelFeature"
name: "spixel_feature"
bottom: "img"
bottom: "spixel_index"
top: "flatten_xyrgbxy_features"
#top: "xyrgbxy_features"
spixel_feature_param {
type: AVGXYRGBXY
max_spixels: 1000
ignore_feature_value: 255
ignore_idx_value: 100000
rgbxy_rgb_scale: 0.1
rgbxy_xy_scale: 0.01
xy_scale: 0.1
}
}
###################################
# Superpixel feature transformation
###################################
layer {
bottom: "flatten_xyrgbxy_features"
top: "flatten_xyrgbxy_features2"
name: "transform_features"
type: "Convolution"
convolution_param {
num_output: 7
kernel_size: 1
}
param {
lr_mult: 1
decay_mult: 0
}
param {
lr_mult: 0
decay_mult: 0
}
}
#########################################
# Smear superpixel features to image size
#########################################
layer {
type: "Smear"
name: "feature_smear"
top: "xyrgbxy_features"
bottom: "flatten_xyrgbxy_features2"
bottom: "spixel_index"
smear_param{
ignore_feature_value: 255
ignore_idx_value: 100000
}
}
#######################################################
# Slice Superpixel and Pixel Features into XY and RGBXY
#######################################################
layer {
name: "slice_features"
type: "Slice"
bottom: "flatten_xyrgbxy_features2"
top: "flatten_xy_features"
top: "flatten_rgbxy_features"
slice_param {
axis: 1
slice_point: 2
}
}
layer {
name: "slice_features2"
type: "Slice"
bottom: "xyrgbxy_features"
top: "xy_features"
top: "rgbxy_features"
slice_param {
axis: 1
slice_point: 2
}
}
###############################################################
# Convert 513x513 features into 65x65 (same as FC6 output size)
###############################################################
layer {
name: "_layer_0"
type: "Interp"
bottom: "rgbxy_features"
top: "pooled_rgbxy_features"
interp_param {
shrink_factor: 8
}
}
layer {
name: "_layer_1"
type: "Interp"
bottom: "xy_features"
top: "pooled_xy_features"
interp_param {
shrink_factor: 8
}
}
############################################################################
# RGBXY (transformed features) similarity kernel from pixels -> superpixels
############################################################################
layer {
type: "Pdist"
name: "pixel_rgbxy_dist"
top: "pixel_rgbxy_distkernel"
bottom: "pooled_rgbxy_features"
bottom: "flatten_rgbxy_features"
pdist_param {
ignore_value: 255
scale_value: -1.0
}
}
############################################################################
# XY (transformed features) similarity kernel from pixels -> superpixels
############################################################################
layer {
type: "Pdist"
name: "pixel_xy_dist"
top: "pixel_xy_distkernel"
bottom: "pooled_xy_features"
bottom: "flatten_xy_features"
pdist_param {
ignore_value: 255
scale_value: -1.0
}
}
#################################################################################
# RGBXY (transformed features) similarity kernel from superpixels -> superpixels
#################################################################################
layer {
type: "Pdist"
name: "spixel_rgbxy_dist"
top: "spixel_rgbxy_distkernel"
bottom: "flatten_rgbxy_features"
bottom: "flatten_rgbxy_features"
pdist_param {
ignore_value: 255
scale_value: -1.0
}
}
###############################################################################
# XY (transformed features) similarity kernel from superpixels -> superpixels
###############################################################################
layer {
type: "Pdist"
name: "spixel_xy_dist"
top: "spixel_xy_distkernel"
bottom: "flatten_xy_features"
bottom: "flatten_xy_features"
pdist_param {
ignore_value: 255
scale_value: -1.0
}
}
########################################################################
# Kernel pool for BI6(2) bilateral inception module to be used after FC6
########################################################################
layer {
name: "concat"
bottom: "pixel_rgbxy_distkernel"
bottom: "pixel_xy_distkernel"
top: "fc6_dist_kernel_pool"
type: "Concat"
concat_param {
axis: 1
}
}
########################################################################
# Kernel pool for BI7(6) bilateral inception module to be used after FC7
########################################################################
layer {
name: "concat"
bottom: "spixel_rgbxy_distkernel"
bottom: "spixel_xy_distkernel"
bottom: "spixel_rgbxy_distkernel"
bottom: "spixel_xy_distkernel"
bottom: "spixel_rgbxy_distkernel"
bottom: "spixel_xy_distkernel"
top: "fc7_dist_kernel_pool"
type: "Concat"
concat_param {
axis: 1
}
}
#######################################################
# Scale, exponentiate and normalize the BI6(2) kernels
#######################################################
layer {
name: "dummy"
type: "DummyData"
dummy_data_param {
num: 1
channels: 1
height: 1
width: 1
data_filler {
type: "constant"
value: 1
}
}
top: "data2"
}
layer {
name: "scale_fc6"
type: "Convolution"
bottom: "data2"
top: "scale_fc6"
param {
lr_mult: 1
decay_mult: 0
}
param {
lr_mult: 0
decay_mult: 0
}
convolution_param {
num_output: 2
kernel_size: 1
weight_filler {
type: "constant"
value: 1
}
bias_filler {
type: "constant"
value: 0
}
}
}
layer {
bottom: "scale_fc6"
top: "scale_fc6"
name: "relu_scale_fc6"
type: "ReLU"
}
layer {
name: "flattendata"
type: "Flatten"
bottom: "scale_fc6"
top: "scale_fc6_flat"
flatten_param {
axis: 0
}
}
layer {
name: "scale_fc6_kernels"
type: "Scale"
bottom: "fc6_dist_kernel_pool"
bottom: "scale_fc6_flat"
top: "fc6_scale_kernel_pool"
scale_param {
axis: 1
}
}
layer {
type: "Softmax"
name: "exp_fc6_kernels"
bottom: "fc6_scale_kernel_pool"
top: "fc6_exp_kernel_pool"
softmax_param {
axis: 3
}
}
#######################################################
# Scale, exponentiate and normalize the KI7(6) kernels
#######################################################
layer {
name: "scale_fc7"
type: "Convolution"
bottom: "data2"
top: "scale_fc7"
param {
lr_mult: 10
decay_mult: 0
}
param {
lr_mult: 0
decay_mult: 0
}
convolution_param {
num_output: 6
kernel_size: 1
weight_filler {
type: "constant"
value: 1
}
bias_filler {
type: "constant"
value: 0
}
}
}
layer {
bottom: "scale_fc7"
top: "scale_fc7"
name: "relu_scale_fc7"
type: "ReLU"
}
layer {
name: "flattendata"
type: "Flatten"
bottom: "scale_fc7"
top: "scale_fc7_flat"
flatten_param {
axis: 0
}
}
layer {
name: "scale_fc7_kernels"
type: "Scale"
bottom: "fc7_dist_kernel_pool"
bottom: "scale_fc7_flat"
top: "fc7_scale_kernel_pool"
scale_param {
axis: 1
}
}
layer {
type: "Softmax"
name: "exp_fc7_kernels"
bottom: "fc7_scale_kernel_pool"
top: "fc7_exp_kernel_pool"
softmax_param {
axis: 3
}
}
###########################################
# Start of standard DeepLab-LargeFOV model
###########################################
layer {
name: "conv1_1"
type: "Convolution"
bottom: "img"
top: "conv1_1"
convolution_param {
num_output: 64
pad: 1
kernel_size: 3
#engine: CAFFE
}
}
layer {
name: "relu1_1"
type: "ReLU"
bottom: "conv1_1"
top: "conv1_1"
}
layer {
name: "conv1_2"
type: "Convolution"
bottom: "conv1_1"
top: "conv1_2"
convolution_param {
num_output: 64
pad: 1
kernel_size: 3
#engine: CAFFE
}
}
layer {
name: "relu1_2"
type: "ReLU"
bottom: "conv1_2"
top: "conv1_2"
}
layer {
name: "pool1"
type: "Pooling"
bottom: "conv1_2"
top: "pool1"
pooling_param {
pool: MAX
kernel_size: 3
stride: 2
pad: 1
}
}
layer {
name: "conv2_1"
type: "Convolution"
bottom: "pool1"
top: "conv2_1"
convolution_param {
num_output: 128
pad: 1
kernel_size: 3
#engine: CAFFE
}
}
layer {
name: "relu2_1"
type: "ReLU"
bottom: "conv2_1"
top: "conv2_1"
}
layer {
name: "conv2_2"
type: "Convolution"
bottom: "conv2_1"
top: "conv2_2"
convolution_param {
num_output: 128
pad: 1
kernel_size: 3
#engine: CAFFE
}
}
layer {
name: "relu2_2"
type: "ReLU"
bottom: "conv2_2"
top: "conv2_2"
}
layer {
name: "pool2"
type: "Pooling"
bottom: "conv2_2"
top: "pool2"
pooling_param {
pool: MAX
kernel_size: 3
stride: 2
pad: 1
}
}
layer {
name: "conv3_1"
type: "Convolution"
bottom: "pool2"
top: "conv3_1"
convolution_param {
num_output: 256
pad: 1
kernel_size: 3
#engine: CAFFE
}
}
layer {
name: "relu3_1"
type: "ReLU"
bottom: "conv3_1"
top: "conv3_1"
}
layer {
name: "conv3_2"
type: "Convolution"
bottom: "conv3_1"
top: "conv3_2"
convolution_param {
num_output: 256
pad: 1
kernel_size: 3
#engine: CAFFE
}
}
layer {
name: "relu3_2"
type: "ReLU"
bottom: "conv3_2"
top: "conv3_2"
}
layer {
name: "conv3_3"
type: "Convolution"
bottom: "conv3_2"
top: "conv3_3"
convolution_param {
num_output: 256
pad: 1
kernel_size: 3
#engine: CAFFE
}
}
layer {
name: "relu3_3"
type: "ReLU"
bottom: "conv3_3"
top: "conv3_3"
}
layer {
name: "pool3"
type: "Pooling"
bottom: "conv3_3"
top: "pool3"
pooling_param {
pool: MAX
kernel_size: 3
stride: 2
pad: 1
}
}
layer {
name: "conv4_1"
type: "Convolution"
bottom: "pool3"
top: "conv4_1"
convolution_param {
num_output: 512
pad: 1
kernel_size: 3
#engine: CAFFE
}
}
layer {
name: "relu4_1"
type: "ReLU"
bottom: "conv4_1"
top: "conv4_1"
}
layer {
name: "conv4_2"
type: "Convolution"
bottom: "conv4_1"
top: "conv4_2"
convolution_param {
num_output: 512
pad: 1
kernel_size: 3
#engine: CAFFE
}
}
layer {
name: "relu4_2"
type: "ReLU"
bottom: "conv4_2"
top: "conv4_2"
}
layer {
name: "conv4_3"
type: "Convolution"
bottom: "conv4_2"
top: "conv4_3"
convolution_param {
num_output: 512
pad: 1
kernel_size: 3
#engine: CAFFE
}
}
layer {
name: "relu4_3"
type: "ReLU"
bottom: "conv4_3"
top: "conv4_3"
}
layer {
name: "pool4"
type: "Pooling"
bottom: "conv4_3"
top: "pool4"
pooling_param {
pool: MAX
kernel_size: 3
stride: 1
pad: 1
}
}
layer {
name: "conv5_1"
type: "Convolution"
bottom: "pool4"
top: "conv5_1"
convolution_param {
num_output: 512
pad: 2
kernel_size: 3
#engine: CAFFE
dilation: 2
}
}
layer {
name: "relu5_1"
type: "ReLU"
bottom: "conv5_1"
top: "conv5_1"
}
layer {
name: "conv5_2"
type: "Convolution"
bottom: "conv5_1"
top: "conv5_2"
convolution_param {
num_output: 512
pad: 2
kernel_size: 3
#engine: CAFFE
dilation: 2
}
}
layer {
name: "relu5_2"
type: "ReLU"
bottom: "conv5_2"
top: "conv5_2"
}
layer {
name: "conv5_3"
type: "Convolution"
bottom: "conv5_2"
top: "conv5_3"
convolution_param {
num_output: 512
pad: 2
kernel_size: 3
#engine: CAFFE
dilation: 2
}
}
layer {
name: "relu5_3"
type: "ReLU"
bottom: "conv5_3"
top: "conv5_3"
}
layer {
name: "pool5"
type: "Pooling"
bottom: "conv5_3"
top: "pool5"
pooling_param {
pool: MAX
kernel_size: 3
stride: 1
pad: 1
}
}
layer {
name: "pool5a"
type: "Pooling"
bottom: "pool5"
top: "pool5a"
pooling_param {
pool: AVE
kernel_size: 3
stride: 1
pad: 1
}
}
layer {
name: "fc6"
type: "Convolution"
bottom: "pool5a"
top: "fc6"
convolution_param {
num_output: 1024
pad: 12
kernel_size: 3
#engine: CAFFE
dilation: 12
}
}
# DeepLab model until FC6
#####################################################
# Flatten FC6 result and multiply with BI6(2) kernels
#####################################################
layer {
name: "reshape"
type: "Reshape"
bottom: "fc6"
top: "fc6_reshape"
reshape_param {
shape {
dim: 0
dim: 0
dim: 1
dim: -1
}
}
}
layer {
type: "MatMul"
name: "apply_fc6_kernels"
top: "fc6_kernels_out"
bottom: "fc6_reshape"
bottom: "fc6_exp_kernel_pool"
}
###################################################
# Linearly combine the bilateral filtering outputs
###################################################
layer {
name: "fc6_w"
type: "Convolution"
bottom: "data1"
top: "fc6_w"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 0
decay_mult: 0
}
convolution_param {
num_output: 2048
kernel_size: 1
weight_filler {
type: "constant"
value: 1
}
bias_filler {
type: "constant"
value: 0
}
}
}
layer {
name: "fc6_w_1"
type: "Eltwise"
bottom: "fc6_kernels_out"
bottom: "fc6_w"
top: "fc6_w_1"
eltwise_param {
operation: PROD
}
}
layer {
name: "img_split"
type: "Slice"
bottom: "fc6_w_1"
top: "fc6_1"
top: "fc6_2"
slice_param {
slice_dim: 1
slice_point: 1024
}
}
layer {
name: "fc6_eltwise"
type: "Eltwise"
bottom: "fc6_1"
bottom: "fc6_2"
top: "combine_fc6_result"
eltwise_param {
operation: SUM
}
}
layer {
bottom: "combine_fc6_result"
top: "combine_fc6_result"
name: "relu6"
type: "ReLU"
}
layer {
bottom: "combine_fc6_result"
top: "combine_fc6_result"
name: "drop6"
type: "Dropout"
dropout_param {
dropout_ratio: 0.5
}
}
######################################
# Standard DeepLab-LargeFOV FC7 layer
######################################
layer {
bottom: "combine_fc6_result"
top: "fc7"
name: "fc7"
type: "Convolution"
convolution_param {
num_output: 1024
kernel_size: 1
}
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
}
#########################################
# Multiply FC7 result with BI7(6) kernels
#########################################
layer {
type: "MatMul"
name: "apply_fc7_kernels"
top: "fc7_kernels_out"
bottom: "fc7"
bottom: "fc7_exp_kernel_pool"
}
#####################################################
# Linearly combine BI7(6) bilateral filtering results
#####################################################
layer {
name: "fc7_w"
type: "Convolution"
bottom: "data1"
top: "fc7_w"
param {
lr_mult: 10
decay_mult: 1
}
param {
lr_mult: 0
decay_mult: 0
}
convolution_param {
num_output: 6144
kernel_size: 1
weight_filler {
type: "gaussian"
std: 0.001
}
bias_filler {
type: "constant"
value: 0
}
}
}
layer {
name: "fc7_w_1"
type: "Eltwise"
bottom: "fc7_kernels_out"
bottom: "fc7_w"
top: "fc7_w_1"
eltwise_param {
operation: PROD
}
}
layer {
name: "split7"
type: "Slice"
bottom: "fc7_w_1"
top: "fc7_1"
top: "fc7_2"
top: "fc7_3"
top: "fc7_4"
top: "fc7_5"
top: "fc7_6"
slice_param {
slice_dim: 1
slice_point: 1024
slice_point: 2048
slice_point: 3072
slice_point: 4096
slice_point: 5120
}
}
layer {
name: "fc7_eltwise"
type: "Eltwise"
bottom: "fc7_1"
bottom: "fc7_2"
bottom: "fc7_3"
bottom: "fc7_4"
bottom: "fc7_5"
bottom: "fc7_6"
top: "combine_fc7_result"
eltwise_param {
operation: SUM
}
}
layer {
bottom: "combine_fc7_result"
top: "combine_fc7_result"
name: "relu7"
type: "ReLU"
}
layer {
bottom: "combine_fc7_result"
top: "combine_fc7_result"
name: "drop7"
type: "Dropout"
dropout_param {
dropout_ratio: 0.5
}
}
######################################
# Standard DeepLab-LargeFOV FC8 layer
######################################
layer {
bottom: "combine_fc7_result"
top: "prob"
name: "fc8_voc12"
type: "Convolution"
convolution_param {
num_output: 21
kernel_size: 1
}
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
}
#######################################
# Smear the superpixel output to pixels
#######################################
layer {
type: "Smear"
name: "outputsmear"
top: "prob_image"
bottom: "prob"
bottom: "spixel_index"
smear_param{
ignore_feature_value: 255
ignore_idx_value: 100000
}
}
# End of the model