Convolutional neural networks (CNNs) are a class of Deep Artificial Neural Networks that are commonly applied to image processesng tasks, such as object detection and identification.

CNNs share the underlying structure with the MLPs seen previously, in fact:

1. Feed Forward Structure: they generally organize their nodes (or neurons) into layers, with the output of each layer fed forward to the next layer for further processesng.
2. Learning through backpropagation: during training, a loss function is used to calculate the error margin between the actual output and the desired output. This error margin is then used to update the weights through a process known as backpropagation, with the goal of reducing the loss function and thus the error margin.

MLPs can only work with images if they are first converted into a vector of pixel values. The problem with this transformation is the potential loss of spatial integrity, as information about how pixels combine with each other could be lost.

Convolutional neural networks maintain the spatial integrity of input images because they can extract information from a data matrix through convolutional filters; moreover, they can analyze color images (thus in three dimensions) by processesng each channel individually, but keeping them grouped as the same input. In practice, convolutional filters are a set of weights that are applied to the pixel values in our input image. These weights are learned and refined through backpropagation during the training phase.

The use of convolutional filters in CNNs is inspired by the work of Hubel and Wiesel, two neuroscientists who studied how images trigger neuronal activation in the visual cortex. In their study, they suggested that visual processesng is a hierarchical process that begins with the identification of basic features that are then combined into more complex constructs. Similarly, filters in CNNs allow the neural network to identify basic features and highlight them; these can then be forwarded for further processesng, but must first be corrected by an activation function.

The filtering process, as can be seen in the example in Figure 2.16, consists of sliding a convolutional filter over an image and generating a filtered version of the image.

Example of convolutional filter application

Convolutional filters can enhance and suppress specific features present in input images (for example curves, edges, or colors). Filters within a CNN can mainly differ in 4 characteristics:

• The number of convolutional filters: CNNs can have dozens or even hundreds of filters. Each convolutional filter generates its own feature map that is forwarded for further processesng.
• Filter size: Convolutional filters must be large enough to account for features that span multiple pixels, but small enough to be reusable across an image.
• Stride: this parameter indicates the number of pixels constituting the unit of movement for processing. Longer strides will result in greater computational speed but in smaller matrices with a potential loss of important features.
• Padding: allows, as exemplified in Figure 2.17, to modify the input image by filling it with empty pixels to ensure that the filtered image maintains the same dimensions as the original input image.

Application of a convolutional filter with padding

Application of a convolutional filter without padding

In a convolutional neural network, the inputs \(I\) are images, thus matrixs in three dimensions. Formally \(I\) has dimension \(H \times W \times C\) where $H W $ are the pixels while \(C\) are the channels, therefore:

\[I \in \mathbb{R}^{H \times W \times C}\]

Assuming a filter \(K \in \mathbb{R}^{k_1 \times k_2 \times C \times D}\) and a bias \(b \in \mathbb{R}^{D}\) the output of the convolutional procedure is:

\[(I \ast K)_{ij} = \sum_{m = 0}^{k_1 - 1} \sum_{n = 0}^{k_2 - 1} \sum_{c = 1}^{C} K_{m,n,c} \cdot I_{i+m, j+n, c} + b \]

The weights and bias are updated through a forward-propagation and back-propagation procedure, generally inserting a fully connected MLP at the end of the filters.

In CNNs, the most common activation function is ReLU, because it ensures that only nodes with a positive activation send their values forward, which guarantees that:

• With fewer activated nodes, there is less processing to do.
• Positively activated nodes indicate significant aspects. Focusing on these can lead to better models with higher accuracy scores.
• There is less noise within the network, reducing the dangers of overfitting or correlation errors between input and desired output.

A common attribute in convolutional neural networks is Pooling, exemplified in Figure 2.18. Pooling normally occurs after the feature maps have been passed through the ReLU activation function. The goal of pooling is to reduce the dimensions of the image matrix without loss of information. In turn, this operation reduces the amount of processing required, saving time and resources in the training phase. The most common varieties of pooling are:

1. Max Pooling: Accepts the maximum pixel value within the filter. \[ y_{ijk} = \max \{ y_{i'j'k} : the \leq i' < i+p, \space \space j \leq j' < j + p \} \] \[ \]
2. Average pooling: Takes the average pixel value within the filter. \[ y_{ijk} = mean \{ y_{i'j'k} : the \leq i' < i+p, \space \space j \leq j' < j + p \} \] \[ \]
3. Sum Pooling: Sums the pixel values within the filter. \[ y_{ijk} = sum \{ y_{i'j'k} : the \leq i' < i+p, \space \space j \leq j' < j + p \} \] \[ \]

Pooling example

Another important building block of CNNs is batch normalization. To increase the stability of a neural network, the batch mean is subtracted from the input values and then divided by the standard deviation. Normalization is typically applied before an activation layer and is used to speed up the process of minimizing the loss function.

\[ \]

\[ \mu_B = \frac{1}{m} \sum_{i=1}^m x_i \]

\[ \sigma_B^2 = \frac{1}{m} \sum_{i=1}^m (x_i-\mu_B)^2\]

\[ \hat{x_i} = \frac{x_i - \mu_B}{\sqrt{\sigma^2_B + \epsilon}} \]

\[ y_i = \gamma \hat x_i + \beta == BN_{\gamma,\beta}(x_i) \]

\[ \]

The offset \(\beta\) and the scale factor \(\gamma\) are learnable parameters that are updated during back-propagation as if they were normal weights. If we set \(\gamma\) to 1 and \(\beta\) to 0, the entire process is just standardization.

import numpy as np
from scipy.signal import convolve2d

# Create a simple 28x28 binary image (hollow square)
image = np.zeros((28, 28))
image[5:23, 5:23] = 1
image[8:20, 8:20] = 0

# Apply Sobel filter for edge detection
sobel_x = np.array([[-1, 0, 1],
                    [-2, 0, 2],
                    [-1, 0, 1]])
edges = convolve2d(image, sobel_x, mode='same')
print(f"Input shape: {image.shape}")
print(f"Output shape: {edges.shape}")
print(f"Non-zero pixels (edges detected): {np.count_nonzero(edges)}")

# Output:
# Input shape: (28, 28)
# Output shape: (28, 28)
# Non-zero pixels (edges detected): 136

def max_pool_2d(x, pool_size=2):
    h, w = x.shape
    h_out, w_out = h // pool_size, w // pool_size
    output = np.zeros((h_out, w_out))
    for the in range(h_out):
        for j in range(w_out):
            patch = x[i*pool_size:(i+1)*pool_size, j*pool_size:(j+1)*pool_size]
            output[i, j] = np.max(patch)
    return output

pooled = max_pool_2d(np.abs(edges), pool_size=2)
print(f"After 2x2 max pooling: {pooled.shape}")
print(f"Reduction: {image.size} -> {pooled.size} pixels ({pooled.size/image.size*100:.0f}%)")

# Output:
# After 2x2 max pooling: (14, 14)
# Reduction: 784 -> 196 pixels (25%)

def relu(x):
    return np.maximum(0, x)

def conv2d_forward(image, filters):
    """Apply multiple 3x3 filters to an image."""
    n_filters = filters.shape[0]
    h, w = image.shape
    output = np.zeros((n_filters, h-2, w-2))
    for f in range(n_filters):
        output[f] = convolve2d(image, filters[f], mode='valid')
    return output

# 4 random 3x3 filters (like a conv layer with 4 output channels)
np.random.seed(42)
filters = np.random.randn(4, 3, 3) * 0.1

# Forward pass: Conv -> ReLU -> Pool
conv_out = conv2d_forward(image, filters)
relu_out = relu(conv_out)
pool_out = np.array([max_pool_2d(relu_out[i]) for the in range(4)])

print(f"Input:      {image.shape}")
print(f"After Conv: {conv_out.shape} (4 filters)")
print(f"After ReLU: {relu_out.shape}")
print(f"After Pool: {pool_out.shape}")
print(f"Flattened:  {pool_out.flatten().shape[0]} features")

# Output:
# Input:      (28, 28)
# After Conv: (4, 26, 26) (4 filters)
# After ReLU: (4, 26, 26)
# After Pool: (4, 13, 13)
# Flattened:  676 features