Particle-Filter Multi-Target Tracking Algorithm Based on Dynamic Salient Features

In order to address the problem of tracking different moving targets in image sequence against a complicated background, this paper presents a particle-filter multi-target tracking algorithm based on their dynamic salient features. By making use of the research findings on visual attention, the algorithm adopts the robust dynamic salient features as a result of combining the gray-scale and details with the motion characteristics of such targets as the state vector of particle filter. The algorithm is highly robust as it contains salient features originating from the low-level features of the targets. Meanwhile the particle filter allows optimized estimation of non-linear and non-Gaussian models. As a consequence, the algorithm is capable of managing traces in tracking different targets and dealing with their appearance, disappearance, mergence, splitting and sheltering by obstacles. Experiments show that this new algorithm enables tracking of multiple targets in complicated image sequence.


Introduction
The research in multi-target tracking plays an important role in a number of military and civil areas. Compared with single-target tracking, it is challenged by state measurement, linkage between different targets and model estimation for each moving target. Since the 1970s, researchers have successively proposed many multi-target tracking algorithms, including nearest-neighbor filter algorithm, joint probability data association (JPDA) filter algorithm, multi-hypothesis tracking filter algorithm and multi-target tracking algorithm based on the theory of random sets [1][2][3][4] . Some of the algorithms, however, only select the measurement closest to the predicted position of the targets tracked in respect of statistics as the candidate through which their traces are updated, which is subject to false or lost tracking [1]. Some algorithms suppose that the quantity of targets remain constant during tracking, which is often unrealistic [5,6]. Some may have some limitation in actual applications due to the number of calculations that increases rapidly with the quantity of targets and observations. [7] Others are difficult to offer optimal state solutions in analytic form though they provide a relatively self-contained theoretical system for eliminating multi-target tracking problems by integrating the theory of random sets with Bayes's theory. In such a situation, therefore, someone advanced a multi-goal tracking algorithm based on particle filter [5,8,9] , which renders optimal state solutions in analytic form using a group of weighted particles approaching probability hypothesis density (PHD). However, the algorithm has the disadvantage of lack of data relevant modules such that the states of targets are described in a collective form and it is unclear which state corresponds to a specific target. Therefore this algorithm cannot reveal the motion trace of a single target definitely. It is for this reason that the academic community is in great need of new multi-target tracking algorithms.
As we know, particle filter is virtually an open system whose state vector may be set depending on the actual problems to be solved. Many researchers have done a lot in this respect. For example, Yao Jianmin et al solved target tracking to some extent by choosing Gabor wavelet features as the state vector of particle filter [10]; Wang Jian et al achieved single-target tracking of color images against a complicated background [11]; and Yang Tao et al conducted real-time detection and tracking of human head by combining grads and geometric information with particle filter [12] . Even more researchers consider the position information of targets as the state vector of particle filter, and determine their positions through iterative filtering.
Based on the above algorithms, the paper presents an algorithm that uses the salient features of targets as the state features in particle filtering process and predicts their positions though iterative filtering while detecti ng moving targets in single frame using visual saliency maps for data association between predicted and detected positions of targets, managing the trace of each moving target according to the result of such association and thereby tracking several targets. It has overcome the disadvantages of common algorithms of multi-target tracking such as uncertain motion trajectory of single target, unclear target state, incorrect estimation of target quantity, and uncontrollable computation amount. In addition, because of dynamic salient features resulting from the fusion of diverse low-level target features, the algorithm has high robustness and low image SNR requirement so that it is fully capable of tracking a number of targets and addressing the issues like target appearance, disappearance, mergence, splitting, sheltering by obstacles and multi-target track management.
The rest of the paper is organized as follows. Section 2 contains the fundamental principles and description of the multi-target particle-filter tracking algorithm based on salient features. Section 3 describes the experimental results. A conclusion is drawn in Section 4.

Definition of salient features
Visual attention is a process of human vision system in choosing knowledge and filtering visual information based on scenery image salience as well as the relationship between target and scene. Besides the interested targets in a real scenery image, it includes remarkable interference information. Visual attention mechanism can help brain filter out such interference information, and pay attention to interested objects such that the vision perception process is selective. The ability of the visual attention mechanism in filtering and selecting visual information has greatly aroused the researchers' enthusiasm of research. They have presented a number of visual attention models in the field of machine vision. Visual attention model based on salience was first proposed by Koch and Ullman [6] at first and thereafter a corresponding computation model by Itti and Koch [7]. 18 18 The latter has three basic information processing modules. several parallel and separable feature maps recording the properties in a number of feature dimensions in different positions in visual field and thereby calculating single feature salience of different feature dimensions in each position; a salient map integrating the salience of different feature maps into a total salient measure and guiding the visual attention process; and a WTA (winner-take-all) network selecting the most salient position dynamically as the attention focus from the salient map [5].
Therefore, from the theoretical model of Itti and Koch, the so-called salient feature may be defined as a total

Extraction of salient features
Visual salience is a local contrast resulting from multiple visual sensitivity features. The more obvious the difference is, the stronger the salience will be, and vice versa. For achromatic infrared or visible image, the features of visual sensitivity include grayscale, details, motion, etc. Visual sensitivity is generated under the following two premises [5]. According to the theory of mechanic vision, the local contrast of the positions in specific regions with certain features can be obtained by means of Center-Surround operator [5,6,7,8,9,18,19] and convolution of the feature map. In shape of DOG function, the Center-Surround operator simulates the manner in which human vision system senses stimulus. See formula (1). However, the convolution is difficult to be performed in engineering practice. By consulting references [5] and replacing the convolution process with the process of "difference between scales" , the algorithm enhances the central excitation region, restraining surrounding regions, strengthens the contrast of the features and achieve transformation from the feature of visual sensitivity to vision stimulation . Where, 4 means the following process: Interpolate F s to the same size of image region as F c , then subtract pixel by pixel and obtain its absolute value. Thus, a salient map of grayscale and detail features may be obtained by following the above definition: Secondly, in order to obtain a salient map of motion features, the grayscale of image pixels can be projected in x and y directions respectively so that 2 dimensional images can be transformed into one dimensional curve. After that perform operations related to the characteristic curves of adjacent frames to determine the translation estimation between two continuous frames. Based on this estimation, the motion salient map ˆk M can be 21 21 determined as follows:

Theory of particle filtering
The particle-filter algorithm is an optimal algorithm based on Monte Karlo and Bayes estimation theory. It can be expressed as: prior probability current measurements posterior probability. Note the following nonlinear and non-Gaussian model: Where, ( ) G x is a Dirac function. Then a discrete weighted approximate formula may be used to approach the true posterior probability 0, 1 Furthermore, State k X 's estimation ˆk X base on 1 k Z may be approached by the following formula: According to the Law of great number, formula (11) is convergent, i.e.
In addition, when the posterior variance of k X is bounded (i.e. var( ) k X f ), the central limit theorem holds true, Where, p N of o means convergence depending on probability distribution [10] .
Generally, the flow chart of standard particle filter comprises initialization, time updating, observation updating, re-sampling and other steps. The current filters such as assistant particle filter, regular particle filter and improved particle filter based on genetic algorithm are the transmutations from standard particle filters. The state vector 0, i k X may be defined as required depending on the different problems to be solved. It may be defined as the position of a target, certain feature attributes or feature vector characterizing the target, etc.

Particle-filter multi-target tracking algorithm based on dynamic salient features
To sum up, integrating the salience formed by grayscale, detail and motion features, the salient feature is characterized by higher robustness and interference resistance than unitary image grayscale feature. If this feature is combined with particle filter and used as the state feature vector of a particle filter, the algorithm is Therefore, this algorithm may be used to address the issues including target appearance, disappearance, 23 23 mergence, splitting, sheltering by obstacles. Based on the concept, the process can be shown as follows, Step Step 2: On the basis of the first frame salient feature figure 1 ( , ) S x y , establish the initial track 1 ( ) T l and initial particle group 1 ( ) P l for each detected moving target , ( , ) k l Md x y , extract the salient feature for each target and use it as the state value for particle filter. When the target number reaches L, the numbers of initial tracks and initial particle groups are both L. Let p N be the particle number for each particle group and then initiate each particle group. After the steps of time updating, observation updating and re -sampling independently for every particle group ( ) k P l , the state estimation of each particle group can be obtained.
Step Step 4: Reliably track all moving targets appearing in the scenery image.

Experimental results
According to algorithm principles described above, we have verified the feasibility and effectiveness of the particle-filter multi-target tracking algorithm based on salient features proposed herein through the following experiments:

Experiment 1:
By following the algorithm proposed herein, static salient features are extracted from a group of traffic surveillance image sequence diagrams, which are in size of 768 576 u and available from ftp://pets.rdg.ac.uk.
As shown in Fig. 3.1, Fig. 3.1 (a) represents the 5 th , 32 nd , 45 th , 89 th and 147 th frames in the sequence diagrams.
The group of sequence diagrams contains a number of moving targets in difference sizes. z In the 32 nd , 45 th and 89 th frames, thanks to its relatively high distinctiveness of change in margin details, Target 4 is always visible in the saliency maps in the cause of partial obstruction -complete obstruction -partial obstruction by a sign board. Fig. 3.1 (c) is the resultant motion saliency map of the 5 th , 32 nd , 45 th , 89 th and 147 th frames obtained using corrected gray projection algorithm. In Fig. 3.1 (c), it can be seen that: z Target 1 and 2 that were highly significant in the static salient pictures are not very notable in the motion saliency maps as they are not motion-characteristic.
z Owing to their motion characteristics, Target 3, 4, 5, 6 and 7 are relatively distinct in the motion saliency maps.
z All the new targets appearing in the 32 nd , 89 th and 147 th frames including Target 8, 9, 10 and 11 are relatively notable in the motion saliency maps. tracking algorithm based on dynamic salient features as proposed in this article. In Fig. 3.2, it can be seen that the algorithm may be used to track multiple targets against complicated background and to effectively deal with the problems of new target appearance, disappearance, mergence, splitting and obstruction by obstacles.
For example, new targets including Target 6, 7 and 8 appear in Fig 3.2 b, d and e respectively. In addition to tracking the existing targets reliably, therefore, the proposed algorithm establishes motion traces and particle swarms concerning the new targets, and tracks them. In Fig 3.2 d and e, Target 3 and target 4 no longer move, and tracking four frames, the algorithm removes the motion traces of the two targets and the corresponding particle swarm without further tracking. Target 4 is hidden by obstruction in Fig 3.2 b, c and d, and split up into two targets in Fig. 3.2c, one small and the other big. However, the algorithm still well tracks Target 4. In Fig. 3.2e, Target 7 is split up into two moving targets. The algorithm can not only reliably track Target 7 before splitting, but also properly trace Target 11 originating from Target 7 depending on the mergence and splitting of the target.

It indicates that:
z Target 4 is obstructed by obstacles during movement and therefore its trace is temporarily interrupted.
z Target 7 is split up into two targets midway. One of the targets is numbered as "7" and continues to move while the other new target starts its new trace.

27
Target 9, 10 and 11 appear later while Target 3 and 4 disappear early, which is obviously shown by their traces.

Conclusion
In conclusion, the dynamic salient feature presented in the paper simulates the features of human visual system for observation with superior capability in detecting a number of moving targets. Such dynamic salient feature is combined with open particle filter framework as the state vector of particle filtering in order to achieve filtering iteration and target tracking. The experimental results show that the proposed algorithm can make full use of the robust performance of salient feature and take advantage the capability of particle filter in solving nonlinear and nor-Gaussian problems. In addition, it can be used to address the problems of target appearance, disappearance, mergence, splitting, obstruction by obstacles and thereby track several targets reliably.