Review of the Literature on Multicriteria Methods Applied in the Field of Public Security

Public security is a topic with increasing relevance over the years. In this context, the growing demand requires decisions that consider multiple points of view, some of them subjective. This work is based on the assumption that few studies have been conducted in the public security sector employing multiple criteria techniques to support the decision making procedure. In this context, this research had the objective of mapping the application of MCDA methods within the scope of public security. A literature review in the journals of the Scopus and ISI Web of Science databases was carried out in October 2016, accessed through Capes portal, integrating the following themes: public security and multicriteria decision aid. A systematic search was conducted by applying the Webibliomining method. After applying specific keywords and filters 965 records were returned from Scopus and 788 records from the ISI Web of Science. The review of findings summary resulted in a set of 19 articles validated as compliant for inclusion in the revision, where 19 were from Scopus and 6 were shared with the Web of Science base. The analysis of the articles showed that there is no predominant multicriteria method in the context of the search. The recurrent applications refer to the resolution of the policing distribution problem, and classification of crime rate by region. The key contribution of this work is to provide the reader with a map of public safety problems already addressed in the context of MCDAs approaches and to highlight aspects of this problem that have not yet explored in the context of the MCDA.


Introduction
Making a decision is an important part of human activities. Some decisions are relatively simple, others complex [1]. As they assert [2,3] that some decisions may be relatively simple, especially if the consequences of a poor decision are small, while others can be very complex and have significant consequences.
Real-life problem-solving in general involves several conflicting points of view, which must be taken into account together in order to arrive at a reasonable decision [4]. In practice, Multiple Criteria Decision Analysis (MCDA) is concerned with evaluating a collection of possible courses of action or options and this evaluation could be in the form of selecting a more preferred option, classifying the options from the best to the worst [5]. In daily practices, the application of the MCDA is crucial in allocating finite resources between competing alternatives and interests [6]. Researchers refer to multicriteria methods in different ways. They often refer to multi-criteria decision making on multiple criteria decision making (MCDM). Some authors prefer the name multiple criteria decision aid on aiding (MCDA), according Roy [7]. Others use the designation of multiple criteria decision analysis.
MCDA is a sub-field of operational research or management science [2]. Multi-criteria decision analysis (MCDA) is an umbrella approach that has been applied to a wide range of natural resource management situations [8,9]. More than half a century has attracted researchers' attention to solving a diversity of problems in various fields of knowledge. Numerous publications have reported their applications in the most diverse methods that comprise their universe of study, for example: [10][11][12][13]. According Govindan et al. [2] and Mendoza et al. [9] several classifications of MCDA methods have been proposed in the literature. Basically, MCDA methods are categorized into two groups based on the size of the set of alternatives under consideration. The multi-decision decision making (MADM) methods are designed for problems with a pre-defined set of alternatives, whereas multi-objective decision making (MODM) methods are for problems where the set of alternatives are not predefined [14]. It should be noted that the terms MADM and MCDA (or MCDM) are sometimes used interchangeably in the literature, which may lead to some confusion as asserted [2,15]. Another classification for the MCDA methods was proposed by Belton et al. [8] which subdivided into three types: (1) Value measurement models: Numerical scores are constructed to represent the degree to which one decision option may be preferred to another. These scores are initially developed for each individual criterion, and are then synthesized to aggregate into higher-level preference models; (2) Objective, aspiration or reference level models: Desirable or satisfactory levels of achievement are established for each criterion. The process then seeks to discover options that are closest to achieving these desirable goals or aspirations; and (3) Outranking models: alternative courses of action are compared in parity, initially in terms of each criterion, in order to identify to what extent a preference for one over the other can be affirmed. By aggregating this preference information into all relevant criteria, the model seeks to establish the strength of evidence favoring the selection of one alternative over another [9].
As you say Govindan et al. [2], the number of literature reviews on the most popular MCDA methods has grown. These articles can usually be classified into three categories: 1. Categorization and description of methods applications. These reviews focus on a specific method and provide a categorization of the documents according to the application areas in which the method was considered. As examples we have [1] (TOPSIS) and [16] (AHP). 2. Methodological development of the method. Here, the focus is the review of articles related to the methodological developments of a given method. As examples we have [17] (MAUT) and [18] (ELECTRE). 3. Articles that focus on both items described above.
Decision within public safety is part of a peculiar context that involves analyzing the problem in the light of multiple perspectives or multiple evaluation criteria. As recorded in the reviews made by Behzadian et al. [19], Govindan et al. [2], and Zyoud et al. [11] it is noted that there were no specific citations in the area of public security, which is a gap to be explored. However, despite the successes already achieved in the Multiple Criteria Decision Aid (MCDA) matter, there is a perception that there are few decisions making related works in the public safety sphere whose modeling is supported by the concepts of this matter (MCDA).
In this context, the following question arises: how has the application of MCDA methods in the public security sector evolved over the years? As main objective the present work sought to identify the applications of MCDA in the field of public security identifying the criteria used in each case.
This work is based on the assumption that few studies have been conducted in the public security sector employing multiple criteria techniques to support the decision making procedure. Thus, it aims to map the intellectual production as published in scientific journals found in Scopus and Web of Science bases, accessed through the CAPES portal, which considers the use of the MCDA to model decisions in public security, identifying mainly, the methods employed, the contexts in which such modeling was done, the criteria used and the findings reached in such previous works.

Methodology
The methodology applied in this work is based on the Webibliomining works proposed by Costa [56], and also explored by Barros et al. [57] and Pereira et al. [58]. More specifically, six steps are performed: 1. Defining the research sample 2. Research in the sample, with keywords 3. Identification of journals having the highest number of published articles on the topic 4. Identification of those authors with the highest number of publications 5. Chronological production survey, determining the cycles with highest production 6. Selection of articles for bibliographic research, which should consider:


The most relevant articles  Identification of the first and last authors writing on the topic  Identification of the most relevant texts in each major production cycle.
These steps were performed and the results thereof are detailed in the next section.

Sample Definition
The sample searched corresponds to such articles indexed in the Web of Science and Scopus Bases, which indexes a vast and recognized collection of scientific journals. Access to the bases was through Capes Portal. Such bases were chosen because to it was available via the Capes newspaper's portal and, mainly, due to its representativeness and comprehensiveness. These databases index the contents of journals, covering all areas of knowledge. As for the time frame, the survey was conducted in October 2016, covering all years available in the database.

Search in the Sample
The search was performed by using public safety and multi-criteria related words. In relation to public security, there is a divergence in the meaning of the terms, for example, the word "Security" is related to property security, personal security, at a military level, or even national sovereignty/stability. While "Safety," also reflects security but is rather related to health, physical integrity, hazard protection, or absence of risk caused by unsafe conditions. Thus, both keywords were used for the sake of comprehensiveness. For selecting the articles, an advanced search was performed employing Boolean expressions ("AND" and "OR"), which allows the combination of keywords in the search for a better approach to the specific term. The search process was based on the use of the most common expressions associated with the term "multicriteria decision aid" and also based on such public safety inherent terms, such as safety and security. In this way, the following keywords were used: (secur* OR safe*) AND (multicriteria OR "multicriteria" OR "multi-criteria" OR "multiplecriteria" OR mcda OR mcdm)) then Scopus returned 965 records, while Web of Science database returned 788 records; regarding the types of documents, the search was filtered for article, review, and, as for the type of outlet, for journals. At each base, filters were applied to these collections of records in order to identify articles addressing the use of multi-criteria methods in the context of public safety, as illustrated in Table 1.
The application of keywords and search filter allowed reducing the collection of entries to those potentially matching the search theme. At a later time, the summary of such records was analyzed, which resulted in a collection of 19 papers validated as matching for inclusion in the review, where 19 were from Scopus database and 6 were from the Web of Science database (6 were common to both database), as shown in Figure 1.
It should be noted that the search phrase and the filters were used as decision aids so as to provide a starting point for the papers for further refinement. According to Neves et al. [59], this is a usual strategy employed in researches involving a systematic literature review, and is employed to eliminate homonymy-related noise. Neves et al. [59] illustrated this when quoted ANP term. It is the acronym both for the multicriteria method "Analytic Network Processes", and for the National Agency of Petroleum (the Brazilian Agency that deal with the regulation of relationship between Petroleum industry and Market), which justifies refining from the initial base of articles potentially matching the search. Table 1 summarizes the main results from the initial search. The second column records the initial search terms, including the results from Scopus and Web of Science database. Scopus returned 965 records for papers published in Journals, while the Web of Science recorded 788 papers in the database. In the second phase 23 filters associated with multicriteria methods were applied, as shown in Figure 1. All 19 selected papers potentially matching the public safety theme that will become the starting point for literature review are shown in Table 2.
Following is an analysis of the data in this Figure, and also a summary of each of the papers listed therein.

552
Review of the Literature on Multicriteria Methods Applied in the Field of Public Security

Scopus
Web Of Science Initial search phrase, restricting the search to paper, Review and Journals only ((secur* OR safe*) AND (multicriteria OR "multi criteria" OR "multi-criteria" OR "multiple criteria" OR mcda OR mcdm)) 965 788 Filters associated to the term multicriteria  The framework of intelligent decision support system for Indian police Scopus Adler et al. [70] The Traffic Police Location and Schedule Assignment Problem Scopus Mignelli et al. [71] Use of multi-criteria model to compare devices for the protection of roads against Rockfall Scopus Manning et al. [72] Overview of: "Valuing developmental crime prevention" Scopus Chen et al. [73] Application of GIS for the determination of hazard hotspots after direct transportation linkages between Taiwan and China Scopus Gurgel et al. [74] the multicriteria prioritization model to support public safety planning Scopus Amendola et al. [75] An experimental study of compressed work schedules in policing:

Advantages and Disadvantages of various shift lengths Scopus
Karvetski et al. [76] Scenario and multiple criteria decision analysis for energy and environmental security of military and industrial installations Scopus Lau et al. [77] Optimizing patrol force deployment using a genetic algorithm Scopus Nutt et al. [78] Drug harms in the UK: A multicriteria decision analysis Scopus Source: Prepared by the authors

Identification of Outlets with the Highest Number of Papers Published
Regarding the identification of the outlets matching the specific theme of the research, 18 outlets were identified, 17 out of which would contain one paper per outlet, and only one, Mathematical Problems in Engineering, published two papers, as shown in Table 3. This note shows a diversity of outlets, with no authors' preference as for any specific outlet.

Identification of Authors with the Highest Number of Publications
As regards the identification of those authors with the most publications, this research made no difference between authors and co-authors. Thus, 58 researchers were identified, but only Camacho-Collados, M. and Liberatore, F. outstood with three publications each, as shown in Table  4. The remaining 56 authors had only one publication each, which allows inferring that the topic in question is a matter of concern for many researchers, which reinforces the relevance of the topic.

Survey of Bibliographic Production
The publications chronology for the starting nucleus of the bibliographic review spanned the years from 2010 to 2016, where the years from 2013 to 2015 outstand, as shown in Table 5. Source: Prepared by the authors

Topic Included in the Survey
This section provides a brief description of the 19 papers selected to be included in the research, identifying the methods employed, the range of the application, the countries where the research was conducted, and the findings and contributions. For didactic purposes the papers were classified into five subdivisions according to the theme, which the authors called: Public Security; Road Safety; Crime Prevention; Navigation safety; and Security of Military. Table 6 provides a description of the information below, as well as the criteria used in each of the applications.

Public Security
In this subdivision, nine papers were included whose applications were related to the operational and administrative activities carried out by police institutions. Thus, Liberatore et al. [61] study aimed to guarantee the efficiency and a workload balance among a subset of patrolling sectors by setting up an optimization process, including the development of a model referred to as Multicriteria Police Districting Problem on Graph (MC-PDP), which relied on the support provided by the Spanish National Police Corps (SNPC). The novelty of this model is that evaluates the workload associated with a specific patrol sector according to various criteria such as the area, the risk of crime, diameter and insulation, and to find a balance between the overall efficiency and workload distribution among the agents, according to the preferences of a decision maker. The MC-PDP was solved by a heuristic algorithm can quickly generate patrolling settings that are more efficient than those adopted by the SNPC. The methodology employed included comparing three local search algorithms capable to generating good patrol configurations in a short period of time: the Simple Hill Climbing (SHC); The Steepest Descent Hill Climbing (SDHC); And the Tabu Search (TS). The results of computational experiments showed that the TS, tested with actual crime data from the Madrid district generated solutions that are on average better than those identified by the SDHC algorithm proposed in previous research. As a contribution, the author offers new lines to be pursued in terms of modeling. The author also suggests including model demographics such as racial composition in a census tract. Future work may explore the impact of minimizing racial discrimination on the performance of the resulting patrolling configurations.
Figueiredo et al.
[60] discussed the use of a GIS-MCDM approach to obtain a simple and intuitive explanation of the results. They integrated the DRSA method with the GIS tool in order to assess the level of security in the city of Recife, Pernambuco, Brazil. The SIG-MCDM connection was motivated because spatial information is available. In addition, the authors report that the construction of the model was carried out in two steps: (1) selection and evaluation of reference examples analyzed in the spatial form using DRSA method and (2) the application of the GIS-MCDM tool decision rules to generate the final recommendation. According to the authors, contributions expected as study objectives were achieved. Firstly, the possibility of adapting the GIS-MCDM system thereby creating a classification problem in a georeferenced environment for final decision was set. According to the understanding of the authors, the results are helpful in for decision-making and planning procedure to solve public safety problems through an appropriate application of the results, which may provide a final recommendation for the current decision problem. In addition, they assessed the impacts caused by the chosen reference examples and concluded that they are helpful in security policies. Finally, in view of the junction of Geographic Information System (GIS) with MCDM, the work will not exhaust the possibilities of studies in the field of public security.
Camacho-Collados et al. [64] proposed a decision aid system (DSS) designed to implementing a new paradigm for efficient allocation of human resources in an area under the jurisdiction of a police department targeting to reduce the likelihood of criminal acts. This DSS, referred to as Predictive Police Patrolling DSS (P3-DSS) was developed in collaboration with the Spanish National Police Corps (SNPC). The DSS consists of three key elements that identify the policing strategy: Data Pre-Processing Unit (DPPU), Crime Risk Forecasting Unit (CRFU), and Patrol Sector Optimization Unit (PSOU). The model was tested in the city of Madrid and the results, according to the authors, were satisfactory because, in addition to predicting where crimes were likely to occur, the system optimized the efficient allocation of human and material resources for the enforcement of patrol service. In this regard, the authors point to the need to include new variables to improve the system.
The research developed by Bouranta et al. [65] aimed to learn the perception of the men in the street regarding the work of the police and the relationship with the satisfaction of the police in relation to their own work. The authors used methodology of multicriteria satisfaction analysis (MUSA). The authors reported that, according to the results, police officers are dissatisfied with their jobs, whilst citizens are quite satisfied with police performance. Specifically, the ability of the police to be available to the patrol is currently hampered by paperwork and other typical obligations. The tangible criteria, which refer to buildings, offices and surveillance equipment, have been characterized as suitable by both police officers and citizens. It was also remarked that police officers are not very satisfied with their workload, which may be the reason why citizens reported medium levels of median in terms of police response capacity. This research contributed to adding to the management literature as regards job and service satisfaction.
In his study, Gupta et al. [69] aimed to present the architecture of an intelligent Decision aid system designed to fulfill the emerging responsibilities of law enforcement agencies. The proposed Intelligent Police System (IPS) was designed to meet emerging requirements and provide information at all levels about decision making by introducing a multi-level interface framework between the user and the crime analysis model. The proposed IPS framework is based on data mining and performance measurement techniques to extract useful information, such as crime hot spots, to predict crime evolution, and police station management units based on crime prevention measures. IPS has been implemented in actual Indian crime data supplied by the National Crime Records Bureau (NCRB), which provides an example of the effectiveness and utility of the proposed system. IPS can play a vital role in improving the outcome of crime investigation, crime detection and other key operation areas in the police organization by analyzing crime data and sharing information.
In the research work developed by Adler et al. [70] the states that the daily mission of a traffic police patrol is to provide a service to the public primarily by enforcing laws and assisting road users following any accidents or other traffic issue calls. A traffic police location and schedule assignment (TPLSAP) location through a road network ensures that the traffic police will carry out their mission effectively. In the search for effective road network coverage solutions, a linear multi-objective program was developed in two phases: the first phase includes three distinct objectives. The functions target to maximize the following: (1) the presence of traffic police; (2) police presence in black spots where frequent traffic violations occur; and (3) the time available for proactive work. In the second TPLSAP formulation phase, distance and time halo effect integer linear programs produce a detailed, a daily displacement scheduling detailed across the planning horizon. According to the authors, the formulation of a case study from the long-distance road network in northern Israel was applied, which highlights potential improvements over current schedules.
The research conducted by Chen et al. [73] was motivated by the need to develop a risk assessment procedure that was both robust and reliable in determining transport risks between Taiwan and China. These risks, which are related to mixed types of transport, including air and sea routes, play a key role in public safety, and the effects thereof must be taken into account. This study employs the Geographic Information System (GIS) technology to analyze the spatial locations of such maritime cases that emerged in Taichung Harbor area from 2000 to 2010. Characteristics related to spatial characteristics were surveyed, estimates were made by using the gathering and scattering model, hotspots were explored, and the planned patrol and monitoring points designed to provide duty information to coastal patrol institutions.
According to Amendola et al. [75] the objectives of this paper include testing the impacts of three work shifts (8, 10 and 12 hours) on performance, health, safety, quality of life, sleep, fatigue, watchfulness, off-duty employment, and overtime among police officers. The analysis model employed was the block randomized ANCOVA in which pre-test dependent measures served as co-variable. The results indicated that those working in 10-hour shifts had significantly higher quality of life at work (f = 0.16) and on average significantly more sleep (f = 0.19) than those working in 8-hour shifts. Moreover, those working in 8-hour shifts on average significantly did more overtime work (f = 0.42) than those working in 10-hour and 12-hour shifts. Besides, officers who work 12-hour shifts experienced greater sleepiness levels (f = 0.20) and reported lower alertness levels (f = 0.21) in the work than officers working in 8-hour shifts. The study conducted by Lau et al. [77] created a patrol service optimization model designed to provide security for critical areas in the city, with the objective of guaranteeing property security, since Mainland China has suffered with this type of occurrence. A linear programming mathematical model was developed that fits the practical situation. A case study on the proposed methodology was applied. Compared to the current model, the proposed method achieved a 27.5% improvement.

Road Safety
In this subdivision two works were included whose applications were related to the safety of urban roads or taxiways. Thus, the study conducted by Chen et al. [62] aimed to generate a method for developing a road safety risk index (RSRi), in view of the increasing number of deaths and injuries caused by vehicle crash, which proved to be a serious public health and safety problem according to the study. As a methodology, the authors used the entropy method, which is a statistical parameter used to measure, in a sense, how much information is generated on average for each indicator. The measurement resulting from the method will generate the input weights in the Technique for Order Preference by Similarity to an Ideal Solution (TOPSIS), which, combined with the rank-sum ratio (RSR), will generate the RSRi. The authors applied the methodology to a set of 31 provinces in China. The results indicate that the proposed model will provide a broader picture as compared to traditional approach as it only considers a few factors such as mortality rate or accidents rate road safety. Comparing the robustness of the findings pointed to the feasibility of applying the method to solve such performance evaluation problems comprising multiple alternatives and multiple criteria, as well as several decision making activities in several other fields.
Mignelli et al. [71] aimed to determine the best protection techniques for rock sliding on mountain roads. This topic has a direct correlation with the activity of the road police, whose mission is the safety of road users. In this regard, the methodology employed by the authors included the choice of a multicriteria method, in this case the Analytic Hierarchy Process (AHP). This method was chosen due to an understanding by which AHP is an appropriate tool since it considers the various aspects involved in the evaluation and design of rock slide protection devices. The model discussed in this paper took into consideration all five key aspects -economic, environmental, design, transportation, and social -included in the decision-making process on the choice of a rock fall protection device. As a result, the study states that including more criteria improved the quality of the choice because it is no longer done only by the economic criterion. The study also points to increased transparency in the decision-making process and the inclusion of stakeholders in the process.

Crime Prevention
In this subdivision were included six studies whose subject of the applications were related to the activities of crime prevention, be it by combating drugs or mapping areas of criminal incidence. In this way, the research conducted by De Paula Silva et al. [63] aimed to analyze the spatial pattern of maxillofacial injuries caused by interpersonal violence based on the location of the victim's residence, and to investigate the existence of socio-spatial vulnerability conditions in such areas. The authors employed data from those victims treated in three emergency hospitals in Belo Horizonte, Brazil, between January 2008 and December 2010. The method employed combined the statistical analysis of the variables with the use of Multi-criteria analysis -MCDA. As a result, the research states that the victim's place of residence showed a spatial aggregation pattern, thereby pointing to the existence of a cases polarization in such areas with socio-economic disadvantages and worse infrastructure. A high density of interpersonal violence victims was detected in areas with high sociospatial vulnerability. Such areas were limited to a number of slums with a history of violence linked to drug trafficking. Understanding this dynamics can direct the efforts made by the managers to reduce violence and the impact of such events on public health through public policies focused on more vulnerable groups.
The work done by Camacho-Collados et al. [67] develops the optimization model for patrol sectors in the National Body of the Spanish police, which is part of the Decision Aid System (DSS) for the implementation of predictive policing patrols in Madrid.
In the research developed by di Bella et al. [68] provided a comparative study including traditional crime indicators by employing three multivariate data analysis techniques: (Principal Components Analysis, Multidimensional Scaling and Partial Order Scalogram Analysis by (order-preserving) Coordinates). The original dataset included all crime incident reports collected by local Carabinieri at three stations during the time period from January 1, 2009 to July 27, 2010 in the Marassi neighborhood in the city of Genoa, Italy. As a result, the authors proposed a multi-indicator approach for ranking and prioritizing urban safety issues based on the Partial Order Scalogram Analysis by Coordinates, which showed substantial advantages as an operational tool for Public Administrations in a Smart Cities framework.
Manning et al. [72] claim that crime prevention programs generate positive returns on investment. However, previous studies on such returns have not properly quantified and weighted impacts on various domains of quality of life, e.g., social and emotional development and family well-being or have provided a protocol for deciding between programs that recognizes these various domains. Thus, the purpose of this study was to propose a method for classifying the program alternatives. The authors methodologically used a multi-criteria decision-making technique (MCDM). Both subjective decisions and objective evidence were incorporated through the method that allowed building a common metric to make structured choices among the various options for the development of crime prevention programs. As a result, the study shows that a structured preschool program that incorporates family intervention and support was the preferred option to reduce juvenile crime.
Gurgel et al. [74] study states that solving operational problems is a frequent task in public security decision-making. However, the tactics and strategy pillars are usually disregarded. Thus, this work focuses on a strategic issue, namely, the prioritization of some areas of the city in which there is a degree of occurrences contributing to increase the crime rate. A multi-criteria approach is taken. The reason for this is that such a situation is usually considered from the standpoint of the degree of police reports. The proposed model is based on a multi-criteria SMARTS method and was applied in a Brazilian city. It combines a multi-criteria method and a Monte Carlo simulation to support a robustness analysis. As a result, a few differences between the model developed and the current police reports model are highlighted. As a contribution, the authors believe that this method can support different policies for areas in the city, thereby indicating where there strong actions should be enforced, including infrastructure investments, monitoring procedures and other public security policies.
The research work developed by the Nutt et al. [78] sought to develop a methodology to classify drugs according to the damage caused to users. Understanding such problem can be helpful in the decision making regarding the set-up of policies related to health, public safety and social assistance. The methodology applied the multi-criteria decision analysis modeling (MCDA) for a range of drugs. The authors reported that members of the Independent Scientific Committee on Drugs, including two invited experts, met in a 1-day interactive workshop to mark 20 drugs under 16 criteria: nine of them related to the damage a drug may cause to the individual, and seven to the damage to third-parties. Drugs were scored 100 points and criteria were weighted to indicate their relative importance. The MCDA modeling showed that heroin, crack and methamphetamine were the most harmful drugs to the individual, whilst alcohol, heroin and crack were the most harmful to third-parties. Overall, alcohol was the most harmful drugs, with heroin and crack second and third places respectively. The results give support to the work done to assess the damage caused by drugs, and shows how the ranking improves with the MCDA weighting approach by increasing the differentiation between more and less harmful drugs. However, the findings correlate poorly with the present UK drug classification, which is not based solely on harm considerations 3.6.4. Navigation Safety In this subdivision was included a work whose subject of the application was related to the safety of the traffic of vessels. Thus, Özdemir et al. [66] state in their study that it is commonly accepted that most maritime causalities are caused by human error. The role of human factor in the maritime accident and the potential reasons for such assumption can be quantitatively assessed according to (MCDM) methodology of specialized knowledge and various decision criteria. To investigate what makes "human factor" to be the first in ship accidents, a hybrid approach was applied in this study. Two methods were proposed: the decision-making trial and evaluation laboratory (DEMATEL) and the analytical network process (ANP) designed to evaluate the significance of human factors in maritime accidents. Quantitative assessments of human errors in maritime operations can greatly improve the decision-making process and reduce potential risks. As a result of this study, the three top priorities in the assessment systems were assumed as: 'capacity, competences, expertise' (8.94%), 'physical condition' (8.77%), '-Sea conditions" (8.21%) and the least significant criterion was "cargo characteristics" (2.21%). This study was selected and included in the starting nucleus due to the possibility of adapting such methods to assess the risk of accidents within the scope of public safety activities.

Security of Military
In this subdivision was included a work whose subject of the application related to the safety of military installations. Thus, Karvetski et al. [76] study addresses the issue inherent to the physical security of military installations, as regards the supply of energy, which may jeopardize the safety of such facilities. The authors employed a scenario analysis and multiple criteria decision analysis (MCDA) methodology to choose the best investment to be made in the energy matrix project. This paper was chosen due to the similarity by which police installations and systems must remain constantly in operation.

Concluding Remarks
Revisiting the question that motivated the present research, which sought to learn the evolution of the application of MCDA methods in the scope public security, it can be stated that the target was reached and the hypothesis is few studies available in the scope of public security employing multicriteria techniques for decision support was confirmed. Upon the use of keywords specific to the topic and with the restrictions on the type of documents, such as article and review, and the type of publication outlets restricted to journals, the application to the databases returned 965 entries in Scopus and 788 entries in the ISI Web of Science base. Filters were applied in each base to such collections of entries, aiming to identify such papers that contemplated the use of multicriteria methods in the context of the public security area. The application of keywords and search filter allowed reducing the collection of entries to those potentially matching the search theme. Reviewing the summary of such entries resulted in a collection of 19 papers validated as matching for inclusion in the review, 19 from the Scopus database and 6 shared with the Web of Science database. The papers were reviewed and reported in the body of work, but no predominant method was identified.
This study was limited to applying the methodology to the analysis of outlets indexed in two indexation bases available on the CAPES outlets portal. In general, it can be assumed that the main limitation for this research is associated with the fact that the sample includes only those papers indexed in the Scopus or in the ISI Web of Science; despite the quality of the indexers in both databases, important contributions, not yet indexed in them, may not have been considered in the search. Based on the study developed, on the preparation of the proposed methodology and on the expertise gained during its application, the following suggestions are proposed for future development: search other databases, for example, Scielo, and EBSCO.
This research also determined that the use of multicriteria methods is still little explored in such topics such as: patrolling optimization, identification of areas prone to crime, selection of police officers, selection of public safety projects, police service performance evaluation, and crime indicators. This research classified the articles into five subcategories as: public security; road safety; crime prevention; navigation safety; and security of military. As well as identifying the criteria used in eleven of the nineteen related works. As a contribution, the research sought to begin filling a gap found in the literature.