Anomaly Detection in ImagesClassify and Individuate anomalies with ConvolutionalMarco CerlianiBlockedUnblockFollowFollowingJul 11In Machine Learning is normal to deal with Anomaly Detection tasks.
Data Science frequently are engaged in problem where they have to show, explain and predict anomalies.
I also made a post about Anomaly Detection with Time Series, where I studied an internal system behaviour and I provided anomaly forecasts in the future.
In this post I try to solve a different challenge.
I change domain of interest: swapping from Time Series to Images.
Given an image, we want to achive a dual purpose: predict the presence of anomalies and individuate them, giving a colourful representation of the results.
THE DATASETI got the data from internet: The crack dataset contains images (url format) of wall cracks.
1428 images are provided: half of them show new and uncorrupted pieces of wall; the remaining part show cracks of various dimensions and types.
The first step consist in making a get request to read the images, resize and transform them in array format.
images = []for url in tqdm.
tqdm(df['content']): response = requests.
get(url) img = Image.
open(BytesIO(response.
content)) img = img.
resize((224, 224)) numpy_img = img_to_array(img) img_batch = np.
expand_dims(numpy_img, axis=0) images.
append(img_batch.
astype('float16'))images = np.
vstack(images)As you can see from the samples below, in our data are present different types of wall cracks, some of them aren’t so easy to identify also for me.
Exemples of Crack and No CrackTHE MODELWe want to build a machine learning model which is able to classify wall images and detect at the same time where anomalies are located.
To achieve this dual purpose, the most efficient method consists in building a strong classifier.
It will be able to read and classify our input images as ‘damaged’ or ‘not damaged’.
At last step, we’ll make use of knowledge learned by our classifier to extract useful information which will help us to detect also where are anomalies.
But let’s proceed with order and start to assemble our Neural Net…For this kind of task I’ve chosen a silver bullet of computer vision, the loyalty VGG16.
We load and remake train of VGG16.
This is very easy to do in Keras with only few lines of code.
vgg_conv = vgg16.
VGG16(weights='imagenet', include_top=False, input_shape = (224, 224, 3))for layer in vgg_conv.
layers[:-8]: layer.
trainable = FalseIn details, we imported the VGG architecture allowing training of the last two convolutional blocks.
This will permit our model to specialize itself for our classification task.
For this purpose we’ve also excluded the top layers of the original model replaicing them with another structure.
x = vgg_conv.
outputx = GlobalAveragePooling2D()(x)x = Dense(2, activation="softmax")(x)model = Model(vgg_conv.
input, x)model.
compile(loss = "categorical_crossentropy", optimizer = optimizers.
SGD(lr=0.
0001, momentum=0.
9), metrics=["accuracy"])At classification stage the GlobalAveragePooling layer reduces the size of the preceding layer by taking the average of each feature map.
This choice, plus the omitted usage of intermediate dense layer, permits to avoid overfitting.
The training is simple and easy if you have at disposal a GPU.
COLAB gave us the weapons we needed to speed up this process.
We also used a simple data generator provided by Keras for image aumentation.
At the end we were able to achive an overall accuracy of 0.
90, not bad!From sklearn documentationLOCALIZE ANOMALIESNow, with our model trained, we play with it in order to extract all the useful information which we’ll permit us to show cracks in our wall images.
We try to make this process easy and nice to see at the end with heat map representation.
The useful information we need are located at the top.
Particularly we access to:Convolutional layers: upper we go in our VGG structure and more important features the network has created.
We’ve selected the last convolutional layer (‘block5_conv3’) and cut here our classification model.
We’ve recreated an intermediate model which, given the original image as input, output the related activation map.
Thinking about dimensionality, our intermediate model augments the channels (new features) and reduces dimensions (height and width) of our initial image.
Final Dense layer: for each class of interest we need these weigths, which are responsable to provide the final results of the classification.
With these compressed objects in our hands we have all the knowledge to localize cracks.
We want to ‘paint’ them on an original image in order to make the results easy to understand and nice to see.
‘Unzip’ these informations is easy in python: we only have to make bilinear upsampling to resize each activation map and compute a dot product.
This magic is accesible executing a simple function:def plot_activation(img): pred = model.
predict(img[np.
newaxis,:,:,:]) pred_class = np.
argmax(pred) weights = model.
layers[-1].
get_weights()[0] class_weights = weights[:, pred_class] intermediate = Model(model.
input, model.
get_layer("block5_conv3").
output) conv_output = intermediate.
predict(img[np.
newaxis,:,:,:]) conv_output = np.
squeeze(conv_output) h = int(img.
shape[0]/conv_output.
shape[0]) w = int(img.
shape[1]/conv_output.
shape[1]) act_maps = sp.
ndimage.
zoom(conv_output, (h, w, 1), order=1) out = np.
dot(act_maps.
reshape((img.
shape[0]*img.
shape[1],512)), class_weights).
reshape(img.
shape[0],img.
shape[1]) plt.
imshow(img.
astype('float32').
reshape(img.
shape[0], img.
shape[1],3)) plt.
imshow(out, cmap='jet', alpha=0.
35) plt.
title('Crack' if pred_class == 1 else 'No Crack')I display the results in the image below, where I’ve plotted the crack heat maps on test images classified as crack.
We can see that the heat map is able to generalize well and point pieces of wall containing cracks.
Show anomalies in Crack imagesSUMMARYIn this post we produce a machine learning solution for anomalies identification and localization.
All these functionalities are accesibile implementing a single classification model.
During training our Neural Network acquires all the relevant information which permit it to operate classification.
After this phase, we’ve assembled the final pieces which have told us where the crack is in the image, without additional work!CHECK MY GITHUB REPOKeep in touch: Linkedin.