Detecting Ships in Satellite ImageryDaniel MoraiteBlockedUnblockFollowFollowingApr 11Classifying ships in San Franciso Bay using Planet satellite imagerySatellite imagery is one of the most abundant data sources that a data scientist can play with.
This is part for which I have chosen to look into it in the first place, since it reduce my work of collecting data or even collateral research for personal projects.
There is a downside to it as well: my personal computer storage size and computational power(.
a MacBook Air).
I will soon look into AWS Amazon Web Services to compensate for it.
Meanwhile I have discovered a data set that is quite small and I can run it on my machine: uses one of my favorite providers, Planet satellite imagery.
About the data:includes 4000 80×80 RGB images labeled with either a “ship” or “no-ship” classification, valued 1 or 0.
image chips are orthorectified to a 3 meter pixel sizedataset as .
png image chips, image filename follows a specific format: {label} __ {scene id} __ {longitude} _ {latitude}.
pnglongitude_latitude: longitude and latitude coordinates of the image center pointdataset is also distributed as a JSON formatted text file, contains: data, label, scene_ids, and location listspixel value data for individual images is stored as a list of 19200 integers: first 6400 contain the red channel, next 6400 the green, and last 6400 the blue.
list values at index i in labels, scene_ids, and locations each correspond to the i-th image in the data listclass labels: “ship” class includes 1000 images, near-centered on the body of a single ship.
“no-ship” class includes 3000 images, 1/3 are a random sampling of different landcover features.
— that do not include any portion of an ship.
next 1/3 are “partial ships”, and 1/3 are images that have previously been mislabeled by machine learning models(because of strong linear features).
What we want to achieve: Detect the location of ships in satellite images, which can be applied to solving problems as: monitoring port activity and supply chain analysis.
First PartReading and Preparing DataWe make sure we import all the libraries and modules that we need, apart from regular Keras: Sequential, Dense, Flatten, Activation and Dropout will also use Conv2D and MaxPooling2D (see full notebook end of article).
Now let`s download and study the data set:f = open(r'.
/ships-in-satellite-imagery/shipsnet.
json') dataset = json.
load(f) f.
close() input_data = np.
array(dataset['data']).
astype('uint8') output_data = np.
array(dataset['labels']).
astype('uint8') input_data.
shape (4000, 19200) # and since I was currios to see how the tupple of arrays of arrays look like: input_data array([[ 82, 89, 91, .
, 86, 88, 89], [ 76, 75, 67, .
, 54, 57, 58], [125, 127, 129, .
, 111, 109, 115], .
, [171, 135, 118, .
, 95, 95, 85], [ 85, 90, 94, .
, 96, 95, 89], [122, 122, 126, .
, 51, 46, 69]], dtype=uint8) # now we realize that this is not a photo format that we can visualize, in order to be able to read an image we need to reshape the array/input_data: n_spectrum = 3 # the number of color chanels: RGB weight = 80 height = 80 X = input_data.
reshape([-1, n_spectrum, weight, height]) X[0].
shape # let`s pick one channel pic = X[3] red_spectrum = pic[0] green_spectrum = pic[1] blue_spectrum = pic[2]and the fun part: plotting the photo on all 3 channels:plt.
figure(2, figsize = (5*3, 5*1))plt.
set_cmap('jet')#show each channelplt.
subplot(1, 3, 1)plt.
imshow(red_spectrum)plt.
subplot(1, 3, 2)plt.
imshow(green_spectrum)plt.
subplot(1, 3, 3)plt.
imshow(blue_spectrum)plt.
show()Just don`t get discouraged if some of the photos as X[0] might have all the 3 bands RGB the same, just try another one X[3].
A vector of 4000 elements is our output:output_data array([1, 1, 1, .
, 0, 0, 0], dtype=uint8) np.
bincount(output_data) array([3000, 1000])Vector contains of 3000 zeros and 1000 units = 1000 images are tagged with “ship” and 3000 images with “not ship”.
Preparing the data for kerasFirst categorically encoding the labels:# output encoding y = np_utils.
to_categorical(output_data, 2)Second shuffle all indexes:indexes = np.
arange(4000) np.
random.
shuffle(indexes)Pick X_train, y_train:X_train = X[indexes].
transpose([0,2,3,1]) y_train = y[indexes]And of course normalization:X_train = X_train / 255 # images are type uint8 with values in the [0, 255] interval and we would like to contain values between 0 and 1Second PartTraining the model/ neural networknp.
random.
seed(42) # network design model = Sequential() model.
add(Conv2D(32, (3, 3), padding='same', input_shape=(80, 80, 3), activation='relu')) model.
add(MaxPooling2D(pool_size=(2, 2))) #40×40 model.
add(Dropout(0.
25)) model.
add(Conv2D(32, (3, 3), padding='same', activation='relu')) model.
add(MaxPooling2D(pool_size=(2, 2))) #20×20 model.
add(Dropout(0.
25)) model.
add(Conv2D(32, (3, 3), padding='same', activation='relu')) model.
add(MaxPooling2D(pool_size=(2, 2))) #10×10 model.
add(Dropout(0.
25)) model.
add(Conv2D(32, (10, 10), padding='same', activation='relu')) model.
add(MaxPooling2D(pool_size=(2, 2))) #5×5 model.
add(Dropout(0.
25)) model.
add(Flatten()) model.
add(Dense(512, activation='relu')) model.
add(Dropout(0.
5)) model.
add(Dense(2, activation='softmax'))for details on: relu, softmax and dropout, please see my previous Github blog article on Tensorflow Keras# optimization setup sgd = SGD(lr=0.
01, momentum=0.
9, nesterov=True) model.
compile( loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy']) # training model.
fit( X_train, y_train, batch_size=32, # 32 photos at once epochs=18, validation_split=0.
2, shuffle=True, verbose=2)Get and grab a nice cup of tea or matcha cause this might take a few minutes (or at least it took a few on my machine as you can see bellow):Train on 3200 samples, validate on 800 samples Epoch 1/18 – 67s – loss: 0.
4076 – acc: 0.
8219 – val_loss: 0.
2387 – val_acc: 0.
9025 Epoch 2/18 – 89s – loss: 0.
2227 – acc: 0.
9034 – val_loss: 0.
1767 – val_acc: 0.
9150 Epoch 3/18 – 74s – loss: 0.
1809 – acc: 0.
9278 – val_loss: 0.
1481 – val_acc: 0.
9425 Epoch 4/18 – 72s – loss: 0.
1444 – acc: 0.
9428 – val_loss: 0.
1201 – val_acc: 0.
9600 Epoch 5/18 – 48s – loss: 0.
1334 – acc: 0.
9522 – val_loss: 0.
1126 – val_acc: 0.
9513 Epoch 6/18 – 42s – loss: 0.
1221 – acc: 0.
9591 – val_loss: 0.
0879 – val_acc: 0.
9637 Epoch 7/18 – 40s – loss: 0.
1068 – acc: 0.
9625 – val_loss: 0.
0846 – val_acc: 0.
9738 Epoch 8/18 – 45s – loss: 0.
0820 – acc: 0.
9716 – val_loss: 0.
0808 – val_acc: 0.
9675 Epoch 9/18 – 41s – loss: 0.
0851 – acc: 0.
9728 – val_loss: 0.
0626 – val_acc: 0.
9838 Epoch 10/18 – 40s – loss: 0.
0799 – acc: 0.
9709 – val_loss: 0.
0662 – val_acc: 0.
9762 Epoch 11/18 – 42s – loss: 0.
0672 – acc: 0.
9800 – val_loss: 0.
0599 – val_acc: 0.
9812 Epoch 12/18 – 41s – loss: 0.
0500 – acc: 0.
9813 – val_loss: 0.
0729 – val_acc: 0.
9738 Epoch 13/18 – 42s – loss: 0.
0570 – acc: 0.
9784 – val_loss: 0.
0625 – val_acc: 0.
9788 Epoch 14/18 – 43s – loss: 0.
0482 – acc: 0.
9828 – val_loss: 0.
0526 – val_acc: 0.
9775 Epoch 15/18 – 42s – loss: 0.
0510 – acc: 0.
9822 – val_loss: 0.
0847 – val_acc: 0.
9762 Epoch 16/18 – 44s – loss: 0.
0440 – acc: 0.
9841 – val_loss: 0.
0615 – val_acc: 0.
9800 Epoch 17/18 – 41s – loss: 0.
0411 – acc: 0.
9862 – val_loss: 0.
0559 – val_acc: 0.
9775 Epoch 18/18 – 42s – loss: 0.
0515 – acc: 0.
9834 – val_loss: 0.
0597 – val_acc: 0.
9775 Out[31]: <keras.
callbacks.
History at 0xb47f7bfd0>for details on: categorical_crossentropy, ‘accuracy’, as well the Loss Function please see my previous Github blog article on NN.
Third PartApply the model and Search on the image# download image image = Image.
open(r'.
/ships-in-satellite-imagery/scenes/sfbay_1.
png') pix = image.
load()‘plt.
imshow(image)’ if you want to have a quick look, though in order to be able to properly make use of it, we need to create a vector:n_spectrum = 3 width = image.
size[0] height = image.
size[1] # creat vector picture_vector = [] for chanel in range(n_spectrum): for y in range(height): for x in range(width): picture_vector.
append(pix[x, y][chanel]) picture_vector = np.
array(picture_vector).
astype('uint8') picture_tensor = picture_vector.
reshape([n_spectrum, height, width]).
transpose(1, 2, 0) plt.
figure(1, figsize = (15, 30)) plt.
subplot(3, 1, 1) plt.
imshow(picture_tensor) plt.
show()Now let’s search for ships on the imagepicture_tensor = picture_tensor.
transpose(2,0,1) # Search on the image def cutting(x, y): area_study = np.
arange(3*80*80).
reshape(3, 80, 80) for i in range(80): for j in range(80): area_study[0][i][j] = picture_tensor[0][y+i][x+j] area_study[1][i][j] = picture_tensor[1][y+i][x+j] area_study[2][i][j] = picture_tensor[2][y+i][x+j] area_study = area_study.
reshape([-1, 3, 80, 80]) area_study = area_study.
transpose([0,2,3,1]) area_study = area_study / 255 sys.
stdout.
write('
X:{0} Y:{1} '.
format(x, y)) return area_study def not_near(x, y, s, coordinates): result = True for e in coordinates: if x+s > e[0][0] and x-s < e[0][0] and y+s > e[0][1] and y-s < e[0][1]: result = False return result def show_ship(x, y, acc, thickness=5): for i in range(80): for ch in range(3): for th in range(thickness): picture_tensor[ch][y+i][x-th] = -1 for i in range(80): for ch in range(3): for th in range(thickness): picture_tensor[ch][y+i][x+th+80] = -1 for i in range(80): for ch in range(3): for th in range(thickness): picture_tensor[ch][y-th][x+i] = -1 for i in range(80): for ch in range(3): for th in range(thickness): picture_tensor[ch][y+th+80][x+i] = -1Of course you can pick more steps instead of 10 or maybe less: just make sure you have patience cause this might take as well a while.
step = 10; coordinates = [] for y in range(int((height-(80-step))/step)): for x in range(int((width-(80-step))/step) ): area = cutting(x*step, y*step) result = model.
predict(area) if result[0][1] > 0.
90 and not_near(x*step,y*step, 88, coordinates): coordinates.
append([[x*step, y*step], result]) print(result) plt.
imshow(area[0]) plt.
show()As you can see: It did classify as ship images that have straight lines and bright pixelswhich I guess is the next step in finding a way to polish the model — though that’s for another time.
Or if you give it a second run:Now let’s make sense of tags and find them on the image:for e in coordinates: show_ship(e[0][0], e[0][1], e[1][0][1])picture_tensor = picture_tensor.
transpose(1,2,0)picture_tensor.
shape(1777, 2825, 3)plt.
figure(1, figsize = (15, 30))plt.
subplot(3,1,1)plt.
imshow(picture_tensor)plt.
show()Of course you can re-train the model and give it another run or with the current model give a second search and see what you might get.
Good Luck and maybe next time will spot some cars .
if we find some nice labeled data.
Sources:Planet data you can check how to extract images from my previous Medium articlemy Tensorflow article on KerasKaggle competition data downloadfind the full Jupyter notebook on GitHub.. More details