Sobol' Sequences Application in Dynamic Stochastic Systems Optimization

The paper overviews modern methods of optimization of systems parameters at design stage, based on application of τ LP sequences or Sobol’ sequences with uniform distribution density and best property of evenness among modern uniform grids. Very often projected systems are complicated and bad formalized or even non-formalized. For such case, accurate mathematical methods for searching of solution of multi-parameter multi-criteria optimization problem in modern computational mathematics are absent. If indices of quality are non-formalized and haven’t strict expressions for derivatives of functions, it is useful the using of uniform distributed sequences for complicated functions testing in the procedures of searching of approximate “rational” solution of optimization problems. Usually “rational” solution represents some improvement of criteria values without application of accurate procedures for searching of extremum. The best property of grids evenness may give a significant acceleration of searching procedures. In the paper different procedures, demanded for examination of space of systems parameters, are considered and compared. An application of classical I.M. Sobol’ and R. B. Statnikov procedure, PLP-search and τ LP -search with averaging algorithms for optimization of dynamic stochastic systems is discussed. The latest algorithm is the most suitable for optimization of non-formalized systems, adequately describing only by means of simulation models. Such variant of approximate optimization algorithm is named optimization-simulation. It is the most convenient for design of complicated modern devices and only for them optimization problem may be formulated and solved for case of criterion, given in form of continuous curve. Examples of optimization problems decisions for complex technical systems are shown.


Introduction
There are many problems in design of modern dynamic stochastic systems. Detailed analysis [1,2] of different optimization problems allowed formulating a set of optimization problems with various numbers of indices of quality and with various kinds of restrictions. The results of analysis are showed on Figure.1. It is a hierarchic scheme of optimization problems with different problem definition.  [1,2]. One-criterion optimizations problem for dynamic stochastic system with vector of input parameters α and index of quality ) (α F may be given as follows statement demonstrated earlier in [1,2].
To define such parameters values and systems structure, which provide Advisedly a set of restrictions are separated at two groups, because of in first group restrictions may be mathematically formulated and in second group restrictions may be defined as non-formalized affirmations about systems properties. A set of restrictions (1) is formulated after analytic investigation but these restrictions don't exhaust all features of system under investigation. A set of restrictions (2) has algorithmic form, besides of mathematical expressions it may contain logic procedures and numerical procedures for calculating and checking some combinations of values of parameters.
Multi-criteria optimizations problem contains a set of indices of quality. It demands prior classification and division of this set on importance and unessential factors. After that a kind of criteria defines peculiarities and a level of complexity of optimization problem. The problem with maximal complexity contains criterion given in form of continuous curve. On the second level problems with vectors of different criteria may be formulated. Sometimes there are single general criterion exists. Such problem is solved for the general criterion and for vectors of attendant criteria only restrictions must be fulfilled.
Multi-criteria optimizations problem may be given in general case as follows statement showed earlier in [1,2].
To define such parameters values and systems structure, which provide joint extremum for the set of criteria crit l L , l ), Such statement means that strict mathematical solution of optimization problem is impossible. Only special kind of solution named as "rational" solution, i.e. complex of recommendations for choosing of systems parameters and conditions for controller's functioning, may be created after realization of searching procedures. For dynamic stochastic systems solution of optimization problem is sufficient if particular improvement of characteristics or structure of system under consideration is realized. Separate single variant of global solution of optimization problem for dynamic stochastic systems usually is inaccessible. Mostly the Pareto set of variants is defined and then group of researchers and experts choose final solution. Sometimes another method of expert's judgment is applied. This may be the region of compromise decisions or the set of compromise decisions. Separate decision in form of vector of systems parameters may be created according to integral criterion or additional empirical condition. Usually complicated special procedures are needed for multi-criteria problems solving. This paper is overview of application of uniform grids for investigation and search of "rational" solution in process of optimization of dynamic stochastic systems.
For investigation of non-formalized or very complicated functions as indices of quality of designed technical systems it is needed to have special tool for fast examine of space of parameters. It may be quasi-random points for objects having strict mathematical description. For more complicated objects uniformly distributed sequences or grids are used. The values of chosen criteria are calculated in these points with computer aid. After that optimal solutions are determined according to different procedures, connected with optimization theory or artificial intelligence methods.
A numerical sequence with good properties of uniformity in multidimensional space of parameters, named τ LPsequence or Sobol' sequence, proposed by Sobol' [3], will be very useful for such investigation. Now it is the best among sequences with good properties of uniformity [4][5][6].
Sobol' sequence [3,7] has many useful peculiarities providing fast and detailed oversight a space of parameters. After doubling quantity of points in testing procedure researcher may save all results from first series of experiments. New points place between old points in a space of parameters and new results supplement old results increasing information about object. Consequently density of oversight increases for invariable space of parameters. This property allows discovering essential distinctions in behavior of indices of quality. The survey of modern meshes was done in [8].

Methods Review
There are many results devoted to grids method in modern research publication. Among them three global lines of investigation with Sobol' sequence application may be determined.

Optimization Problems with Several Criteria Decision
I.M. Sobol worked with different special problems, in particular, with calculation of multidimensional integrals. He introduced sequence with uniform distribution density and created many theorems proofs for investigation of τ LPsequence and its remarkable properties [3,7].
Let's consider, how to form points of the τ LP -sequence in the unit n-dimensional hypercube n K . Previously, it is need to introduce the so-called binary parallelepipeds k P that are the set of points with the coordinates ) , . . . , , ( At present could say that the   This method has got further evolution in [10][11][12] and other publications. R.B. Statnikov and coauthors devote a big attention to adjustment of the initial problem statement and detailed multicriteria analysis with expert's participation and special program packet using. At the stage of exploration the Pareto set of points of space of parameters, suspicious at extremum, is created. Then group of experts in a continuing dialogue with special computer's program check the constraint set and create Pareto optimal solutions which are suitable for object under investigation. Authors affirm that realized such procedure in "designing the space shuttle, nuclear reactors, unmanned vehicles, aviation, cars, pumping units, ships, metal tools, bridges, wind power system, wireless battlefield networks, energy efficient sensor networks, and robots." The procedure of decision-making relies on Parameter Space Investigation method (PSI method). A competent expert may define the price of changing constraints when create acceptable condition for values of all contradictory indices of quality. The procedure of more precise definition of space of parameter for solution searching is described in full in paper [12].
Such analysis R.B. Statnikov and coauthors named as the search of the feasible solution set for improvement of functioning of the object under consideration by all concerned criteria. They check the cost of optimal solution and recommend using various optimization procedures for improving the preliminary results.

Sobol' Sequences Using in Experiments Planning for Mechanical Systems Design
will be vector of indices of quality.
Matrix for experiments planning is chose with Sobol' sequences using. For every series of experiments h levels of values of parameters α are calculated with table of directing numerator [7]. Authors suppose different variants of levels quantities: constant, alternating. Boundaries may be coincided with boundaries of region G and may be bigger or smaller than boundaries of region G.
After investigation by means of PLP-search Pareto set must be defined and then region of compromise and regression equations must be created: A set of system were optimized by means of proposed method. Among them Turning separating buffet with hydro mechanical drive was described as system involving 3 nonlinear differential equations with second order. Three indices of quality and 9 parameters entered in this model. After investigation the region of compromise decisions was defined. It had volume equal 0.2 % from initial region of system's parameters changing.
Another example of investigation connects with functioning Pneumatic controller of pressure with increased accuracy. It was modeled as system consisted from 4 nonlinear differential equations of second order for 1 index of quality and for 4 system's parameters. Because of simulation problem has single criteria the region of best values of index of quality was found. It had volume equal 0.5 % from initial region of system's parameters changing.
Pneumatic knocker-up machine was described as system involving 4 nonlinear differential equations with second order. This mathematical model was used in optimization problem for one index of quality and 8 parameters in system's space of parameters. After investigation 4 parameters, defining index of quality, were determined. The region of best values of index of quality for this model had volume equal 5 % from initial region of system's parameters changing.
Investigation of Multi loop planetary gearing was fulfilled with mathematical model consists of 23 linear non heterogeneous differential equations of second order. Sixth indices of quality and 25 parameters were involved in this model. After experiments according to PLP-search method 8 parameters, defining all indices of quality simultaneously, were chose. The region of compromise decisions was found.
The mathematical model of sewing machine contains 5 linear non heterogeneous differential equations of second order. It has 5 indices of quality and 6 parameters. In the region of compromise decisions regression equations was created for owner system's frequencies and parameters from system's space of parameters.
Resonance transformer for ship's propeller shaft has mathematical model as a system from 2 nonlinear differential equations of second order for 6 parameters and one index of quality. After investigation according PLP-search method 2 main parameters was found and index of quality was increased in 5.2 times.
Transmission of main drive of the working stand rolling mill was described by means of 5 nonlinear differential equations of second order. Optimization problem was solved for 5 indices of quality in the space of 5 parameters. The region of compromise decisions has volume equal 3.5 % from initial region of system's parameters changing.
Heat exchanging apparatus was described by means of single partial differential equation in space of parameters since 8 until 18, i.e. 11 equations were checked in the process of investigation. There were 4 indices of quality in the optimization problem. Main parameters were defined for all equations and regions of compromise decisions were created.
The application of method of PLP-search, which already recommended itself positively according to criterion of the rational conducting of computational experiments, is examined on the example of problems of the selection of quantity and places of installation of the lattices distance piece (partitions).
The main purpose of PLP-search method coincides with purpose of theory of experiments planning: decreasing of quantity of experiments and increasing of precision of object's definition. It is very difficult problem and there are no any proofs of an accuracy of PLP-search method in author's publications.

LP -search with Averaging Using
Simulation of dynamic stochastic systems demands prolonged time and considerable computational resources because of collection of results from imitation experiments for given statistics low reproduction is difficult problem even for modern computing system. Preliminary operations involve searching of analytic descriptions of modeling system or its parts, specification of space of parameter for investigation, functional and parametric constraints for indices of quality, checking of model adequacy region, creating of simulation algorithm and fast-acting program code, permitting to fulfill imitation experiments in necessary quantity and in given time. Modern computing systems have unlimited abilities but adequate simulation model constructing is both science and art simultaneously as it was at the earliest stage of the computer age.
Optimization-Simulation, i.e. application of approximate methods of optimization with calculation of values of indices of quality after imitation experiments with simulation model of system under investigation, must be used when an adequate mathematical model for system under investigation and design is absent. Researchers are interested in total investigation of object's behavior. So methods of total oversight of space of parameters are needed. The space of parameters would be maximal and the stage of exploitation is absent. All remarkable properties of Sobol' sequences are necessary in this case. One of such method is Consequently approximate solution for the problem of definition of the region of efficiency in input parameters space, in which values of indices of quality are better, then in other regions, may be found.
Such statement of the optimization problem opens big possibility for solving of different problem in dynamic stochastic systems design. It describes many non-formalized problems. For its solving grid's methods for investigation of complex systems, represented in form of simulation model, will be very useful. It fulfils a search of improved values of system's parameters or system's structure by means of imitation experiments in points of sequence with good properties of uniformity in multidimensional space of parameters [1,2,8,[18][19][20][21][22][23].
Values of indices of quality may be examined in τ LP -sequences or Sobol' sequences points. This procedure was named τ LP -search with averaging. Because of layout of "rational" solution is unknown at the initial moment of investigation and τ LP -sequence has the best quality of uniformity in multidimensional space of parameters an examination spreads on wide area of space of parameters and allows overlooking a set of point with various values of coordinates, i.e. with various values of system's parameters.
As a result of this approach the search of solution is accelerated. Subsequently different variants of system with various values of parameters may be tested. After investigation Pareto set will involve no comparable variants of solution. Significant quantity of imitation experiments in proposed search of "rational" solution will give significant quantity of variants of solution.
The most complicated variant of optimization problem involves continuous criterion as index of quality. In this case a continuous criterion must be replaced by means of a set of separate points, i.e. by means of vector criterion. The procedure of discretization of continuous curve allows to reform continuous set of points into vector, composed of separate points with common procedure of calculation of values of criterion in every point. A continuous set of points of curve will be replaced by means of a finite set of criteria. Preliminary established set of limitations will be formulated. If there are several continuous criteria, instead of every continuous criterion special vector criterion will be formulated. Dimensionality of space of criteria will increase noticeably. It is known that space of coordinates of τ LPsequence was defined for dimensionality, equal 50.The procedures of calculation may be parallelized and an increased processing speed will permit to realize necessary volume of imitation experiments. As a result components of all vector criteria will be described and will have a view: Statement of the problem will involve a few stages of qualitative analysis for checking values of continues indices of quality. Addition actions for creating of simulation model for stochastic dynamic system must be fulfilled preliminary and all algorithms for indices of quality calculation must be verified and coded. Finally the statement of the problem will be done in follows form [2,8].
To define such parameters values and systems structure, which provide joint extremum for the set of criteria  In the process of solving the problem the estimate of the region of the Euclidean space or the set of estimates from the space of estimates with given metric [2,8,20] are determined. For these estimates the points' quality indices constitute the Pareto set. Components of vectors' criteria, found after reforming of continuous characteristics, alternate in preset bounds and give discrete image of these continuous characteristics. It has shape of curve in hole with all its needed peculiarities. In the region of efficiency joint extremum of indices of quality is detected in the sense of definition of multi criteria optimization problem decision.
Only optimization-simulation method may give approximate solution for such optimization problem with great success. For complicated dynamic stochastic systems even small improvement may give significant effect. As a result appropriate values of systems' parameters may be defined or systems' structure with improved characteristics may be chose and new possibilities to construct suitable dynamic stochastic system will be realized. The procedure for approximate solution combines the methods of statistical modeling, the Monte Carlo method and τ LP -search with averaging [2,8,20]. In the region of efficiency the averaged values of indices of quality will be better than in other regions. Theorems of existence and convergence of solution are proved and discussed in [8,20]. After the region of efficiency definition the new problem of adequate and convenient description of the region of efficiency for future application appears. It must be solved by using modern methods of computer science [8,20].
The first example of τ LP -search with averaging application concerns of the region of efficiency definition for data communication system (DCS) with adaptive control of transmitter power through short-wave (SW) radio channels with fading. This system is well known and successfully worked at analog communication's times. This example was considered in detail [18,21,24].
Imitation model was created as a number of algorithms which describing all DCS functional blocks [8,18,20,21]. DCS worked in condition of slow interference fading. The four-parameter distribution density or Nakagamy law of the signal-to noise ratio at the input of the receiver was chose as a model of the generalized Gauss channel.
Imitation experiments according to  Another application of τ LP -search with averaging connects with defining skip checking block coefficients in Net Satellite Radio-Navigation System [20].
An example of extension of method into systems with continuous and discrete parameters is realized in procedures for choosing of the noise-resistant correction codes. It is described in [22]. For problem solution τ LP -search with averaging repeats several times for fixed values of discrete parameters and variable values of continuous parameters. If there is common region of efficiency after complete investigation final common region of efficiency is determined. An example of system with continuous criterion is considered in [2,8].

Conclusions
All above listed methods have empirical character. So they haven't general appointment and usually connect with optimization problems of special types. All existence directions of Sobol' sequences using and all new developments created with application of modern information technologies and parallel computers will be very fruitful for dynamic stochastic systems investigation.
PSI method pays big attention to the questions of correct choice of space of solution of optimization problems but it demands adequate analytical model of object under investigation and very skilled experts for decision-making.
Both PLP-search and τ LP -search with averaging will be over after a special decided rule is constructed. Usually it has a view of procedures of choosing of suitable combinations of parameters in the process of imitation experiments. At these points indices of quality have improved values and continuous criterion has necessary shape and all restrictions on its vectors components are satisfied. It is proved that τ LP -search with averaging reflects stable regularity [8,20] in stochastic system functioning and allows formulating "rational" decision of optimization problem. The main shortage of this method is a fact that time of "rational" solution search may be very long. It is profitable for systems described by means of imitation statistical models. It gives effect when analytical model is absent but procedures of optimization are necessary.
A few remarks about the possibility of proving the fairness of supposed procedures. Now existence proofs have statistical character. It is possible to lay a foundation for such algorithms attributes as an existence, a convergence, and a uniqueness of solution in conditions when quantity of experiments tends to infinity. Because of quantity of experiments in real investigations is limited, term "rational" solution is supposed and any improvement of indices of quality is saluted. For limited quantity of experiments obtained solution isn't final. For another quantity of experiments it may have another value, but conception of statistical stability saves its significance. If there is no statistical stability any statistical solution hasn't real meaning. All new variants of existence algorithms and new approaches for problems solving are welcomed.
All above listed methods aren't free from defects, but all above listed shortage of optimization methods with Sobol' sequences using don't cross out that remarkable fact that these methods helped solving many weighty problems and unfortunately another methods for considered optimization problems solving now are absent. There are very few perspectives to find strict mathematical methods for solving of such problems in the nearest future. So all aforesaid methods are useful and may be improved and developed. Some directions of improvement of optimization procedures involve application of new methods of making -decision, investigation of statistical regularities in various data domains, looking for proofs for different stages of existence algorithms and so on.
The main purpose of this paper was a demonstration both advantages and disadvantages of existence methods of approximate solution of optimization problems in investigation of dynamic stochastic systems. It allows to give a jog to creating of new design methods and will give more effective devices.