Technion-bury

http://visl.technion.ac.il/projects/2005s01/

Работа сделана по заданию геофизика А.Ягодина.

Analysis of behavior of fish in aquarium

by Daphna Karnin and Nataly Slutsky

Supervised by Johanan Erez

Abstract

Animal behavior can be researched and classified to detect future events of nature. The most common example is of animal behaving different than usual before an earthquake. This project proposes an automatic system of tracking of fish, as a first stage to classify its behavior. When receiving videos of fish in an aquarium, the system can find the location of fish in each frame and compose its path. Image processing tools are used in order to recognize fish in the movie frames. After objects are recognized,the system performs data collection about all objects: each fish, along the time axis for every frame. This data includes the area of the fish (as seen by the system), the bounding box of the fish and centroid, representing the fish location. After collecting this data, it is processed and displayed on the original movie.

Original Frame

The Result

The algorithm

Pre- step: every indexed picture from the movie is converted into a grayscale image. First, we produce a background image of the aquarium. This image is obtained as an average of frames in two short clips (every clip containing about 200-300 frames). We used one clip with as few fish as possible, in order for the frames to contain maximum information of the image of the aquarium. This gives us a good proximity of the aquarium background.

Background Image

At this point we start an "online" algorithm, meaning that all actions described below are performed on a single frame, and only when finished, we move on to the next one.

Grayscale Image

The background is subtracted from the single frame image. In the following image one can see: the brighter pixels are the fish, since only they are moving along the background.

Single frame minus background

To obtain better results the new image is scaled. The histogram is stretched between zero and one, to exploit all the dynamic range.

Scaled frame minus background

For binarization we used the K-means algorithm. This is a common solution for segmentation of objects. It partitions the pixels in the image into k=2 clusters, in several iterations.

Binary Image

This binary image is "toothed", and every object is broken into several parts. Therefore, we want to make it smoother, to connect different parts of the same fish object to one another. To do this we perform some morphological operations. First we perform dilation on the image (using a structuring element object for neighborhood). We chose to use a disk structuring element, since the fish have some of a round shape. Its size was chosen by observation, comparing the results

Binary image after morphologic operation

Then we "cut" off small objects. We assume that the size of a fish is at least 300 pixels. This number was obtained by measuring of fish objects in different frames. We measure the size of each object in the image. If it is smaller than the minimum fish size, it is cut,meaning that its pixels are marked as black.

Binary image,removing small objects.

This is the image that we analyze. Now we have a number of distinct objects, each one regarded as a single fish. Each object is numbered, and its centroid and bounding box are calculated. This data is stored into a temporary data structure (""struct""). In order to save data per fish, we defined a struct that contains properties of each object. In this struct, we save the centroid and the bounding box. The data of fish in the movie is stored into our data structure: a matrix composed of the described fish structs. Each column in the matrix represents a frame in the movie, and each row represents a fish object. For each object in the current frame, we look for overlap of its bounding box with every bounding box in the previous frame.

Results

To display the results of our algorithm and analysis we create a demo movie. Each frame of this movie is composed of the original frame, in color, with the fishs bounding boxes, as found by the system, drawn on it. Single fish bounding boxes are in green, and a group of fish is bounded by a blue box.

Result image

Note the three fish at the bottom right of the image. Here they are seen as three separate fish.

Result image

On this next frame fish number two and three become one group.

Result image

In the next image the paths of fish in one movie are drawn on the background image. The beginning of each route is marked with the fish number and a symbol which is different for each fish. The small arrow marks the end of the route, pointing at the general direction of the fish.

Fish Paths, movie fish4m11s, frames 45-65

Tools

This project was implemented entirely in Matlab, version 6.5 and 7.1. The movies of the fish were all filmed at one aquarium in the private laboratory of Geophysics Engineer Yagodin Alexandr, in cooperation with the staff of the VISL lab. Adobe Premier was used to cut short clips of the aquarium movies, and to convert them into ".avi" format.

Acknowledgments

The initiative of this project came from Yagodin Alexandr (predict.y@gmail.com). The idea that stands behind this project was submitted as patent: «Procedeu de prognozare a cutremurilor dupa comportamentul pestilor in acvariu» Yagodin Alexandr, Moldova, 19/07/2005.

We are grateful to our project supervisor Johanan Erez for his help and guidance throughout this work. We would also like to thank the VISL lab staff for their assitance in the technical aspect.

We are also grateful to the Ollendorf Minerva Center for supporting this project.

Related documentation

Images

Example Movies

Full documentation