Signal Processing with Regularized Multistep Support Vector Method

A new method is proposed for processing the signal distorted by random noise. The processing model is based on a statistical regularization method, and the obtained system of linear equations and inequalities is solved using a multistep support vector method. An advantage of this approach is that the iterative nature of the algorithm makes it possible to take into account the a priori information on the solution represented by the inequalities. The results of numerical experiments showing the efficiency of the algorithm are given.


Formulation of the Problem
The problem of experimental data processing is the ill-posed one and the success of its solution depends of the set of a'priori information included in the algorithm. Usually two types of a'priori information are used for image reconstruction. The first kind is the information about smoothness of the function and the level of experimental noise. At that case one can make an attempt to find the solution by minimization some function (for example the function of Bayesian type). The second kind is the concrete information about the lever of some parts of the signal. It can be the same characteristics of apparatus function, information about signal components etc. In combination with limited nature and no-negativity of any image the use of such information leads to the problem of looking for the extreme of the function with limitations [1,2].
More simply both of that approaches can be combined when the signal forming can be described by the linear operators. In that case the signal processing problem can be represented as the system of m -equations in n-measured space: where ψ is the vector looking for with n dimension, f is the vector of measured data with m dimension, K is the reformative matrix. K can be the unit matrix in the case of image smoothing or the operator describing the apparatus destructions or the data interpolation matrix etc. The problems of such types are the ill-posed ones and the method of statistic regularization is widely used for their solution.

Description of the Method
The general regularized statistical estimation of the vector ψ can be represented in the form: or ψ =β 2 Ω p -1 K T (β 2 KΩ p -1 K T +V) -1 f, where V is the covariance error matrix, W=V -1 , D is the matrix of numeric differentiation, D p is the p-power of the matrix, Ω p is the p order stabilizer determining the smoothness of ψ,Ω p =(D p ) T D p , α is the regularization parameter : When the general statistical regularized estimation is constructed a priori information of the first kind is taken into account but the information of the second kind represented in the inequality form cannot be included. So in [3,4] some another way of smoothness accounting was used. Equation (1) was multiplied by no-degenerating matrix H and supplemented with the equation representing the smoothness of model function.
If matrix H is such that H T H=W then the solution of the system (3),(4) with the minimal sum ||η|| 2 + ||ϕ|| 2 is equal to the regularized statistical estimation. In order to take into 2 Signal Processing with Regularized Multistep Support Vector Method account a priori information represented in the inequality form it is necessary to construct some iteration process. In such case the model function limitations can be taking into account on any step of the iterations. Let us to exclude ψ from (4) by (5): The system (6) is joint one. It is evidence because for example 0, For H is undegenerated matrix of n-order, the matrix A of nx2n-order is undegenerated too, and (6) is joint undefined system of linear equations. In [1] it was found the connection of the minimal norm solution of system (6) with regularized statistical estimation (3). The regularized statistical estimation can be get from (5): The consecutive projection method is one of the methods widely used for linear equations systems solving. For the linear equations systems in the R N space <ai,x>=bi , i= 1…M (8) that method has the form: where x (0) is the arbitrary point from R N , i k = mod M (k)+1.
The method (8) generates the point consecution which is coverage to the nearest to x 0 solution of system (7). The only criterion of process (8) stop is the achievement the enough small maximum value of the discrepancy. Let us notify any solution of inequality system: as the ε-approximation solution of system (7). Here ε is the small enough positive number. In [5,6] the multistep support vector method is constructed for the ε-approximation solution of system (7) search. That method is the realization of consecutive projection method with circular advancing of the equations and passing the few-breaked equations. If system (7) is such that the solution is exist then the algorithm based on the support vectors method will stop after finite number of steps and the last point x (k) will be the ε-approximation solution of system (7).

The Solution of the Systems of the Linear Equations with a Multistep Support Method
Let x 0 be the arbitrary point from R N . Let us assume k = 0, i=1, l=0, select small enough positive number ε. If l < M, then go to 1, otherwise stop; x (k) is εapproximate solution of the system (4), (5).
For example let us suppose that vector ψ cannot be negative.
Then system of linear equations (3),(4) must be supplemented by the system of linear inequalities: In [7,8] it is shown that if the set of the inequalities system solutions <с i ,x> ≥ d i (i=1…L) contains the sphere with the radius equal to ε then the method of support vectors with a composite step gives it possible to get the solution after finite number of iterations.

The Solution of the Systems of the Linear Inequalities with a Multistep Support Vector Method.
Let x0 be the arbitrary point from RN. Let us assume k = 0, i=1, l=0, select small enough positive number ε. If l < L, go to 1, otherwise stop; x(k) is εapproximate solution of the system (7). For getting the ε-approximation solution of system (7) satisfying to the inequalities of (11) type the algorithm using the multistep support vector method for solving jointly the linear equations and inequality systems is constructed.

The solution of the systems of the linear equations with a multistep support vector method. No-negativity is taken into account.
Let x0 be the arbitrary point from RN. Let us assume k = 0, p2 = 1, select small enough positive number ε. Let us assume i = 1, l = 0, p1 = 0.

Numerical Experiments and Conclusions
Mathematical experiments indicated that the quality of data processing increased when a priori information represented in the form of the inequalities was taken into account. Figure 1. The results of "experimental" data smoothing (matrix K=E). The continuous curve corresponds to the model function, * corresponds to "experimental" function, + corresponds to smoothing without taking into account non-negativity, stroke and dotted line corresponds to smoothing with taking into account non-negativity. So in the case of random noise on t 10 percent from the maximum of modal function the error of reconstruction changes from 0.07 (without accounting of the modal function no-negativity) to 0.03 (with accounting of the modal function no-negativity). The results of smoothing the "experimental" data (matrix K=E) with and without taking into account no-negativity are represented on Fig.1 and Fig.2.
The algorithm based both on the method of statistical regularization and the multistep support vector method can be used for many kinds of experimental data processing problems (computed tomography, the increasing the photo pictures quality, the apparatus distortion removing, the complex signal decomposition on elementary components, the mixtures analysis etc.). Also it can be used for the multi-dimensional data processing.