Welcome to CS With James
GoogLeNet was the first network with the modular design. The module is called inception module and it looks like this.
Conv2D_reduce = Conv2D(16, (1,1), strides=(2,2), activation='relu', padding='same')(input)
Conv2D_1_1 = Conv2D(16, (1,1), activation='relu', padding='same')(input)
Conv2D_3_3 = Conv2D(16, (3,3),strides=(2,2), activation='relu', padding='same')(Conv2D_1_1)
Conv2D_5_5 = Conv2D(16, (5,5),strides=(2,2), activation='relu', padding='same')(Conv2D_1_1)
MaxPool2D_3_3 = MaxPooling2D(pool_size=(2,2), strides=(2,2))(input)
Cov2D_Pool = Conv2D(16, (1,1), activation='relu', padding='same')(MaxPool2D_3_3)
concat = Concatenate(axis=-1)([Conv2D_reduce,Conv2D_3_3,Conv2D_5_5,Cov2D_Pool])
This is how I made the inception module this is not exactly how it is designed in the paper, but the dataset is too different so I tried so many different designs but this one works, doesn’t mean this is the best.
Actually there is not so many to talk about, there is no new layers used. The difference from the other Network is the module itself.
1×1 Convolution is for reduce the dimension, it is also called bottle neck layer, so they can reduce the overall computation complexity and reduce dimension of the output layer.
The accuracy of this network wasn’t amazing in CIFAR 10. It was 67.38%. Consider it is using different dataset from its original paper. However, you realized that even there were some improvement on the network design but there is not so much improvement on the accuracy. The Image-net is very big size and there are so many classes, so the network has to be very complex and deep to do well in the image-net dataset. However, in the CIFAR10 or MNIST the dataset is small and it is not so complex, so it doesn’t achieve so much improvement. That is reason why people still tend to use AlexNet based design.
However, in the real world. The image size is huge. So, this kind of network architecture might work well, but we just started to learn Deep Learning, so we are going to focus only on the small network designs.
Unfortunately this is it for this tutorial. I believe there is going to be so many thing to discuss in DenseNet.