The Effect of Design Based Learning on Pre-service Science Teachers' Decision Making Skills

In this research, it is aimed to examine in depth the effect of design based learning (DBL) program on pre-service science teachers’ on development of the decision making skills and determine the opinions of pre-service science teachers regarding the effect of the DBL program on decision making skills. For this purpose, embedded integrated design of a mixed-methods approach was employed in this study. In the quantitative part, one-group pre-test and post-test experimental design was used. Multiple-case (embedded) design was used in the qualitative part. The study group for the quantitative part of the research is 36 pre-service science teachers determined by convenience sampling method. The qualitative study group are 6 pre-service science teachers determined by maximum variety sampling. The quantitative data of the study were collected via decision-making skills test. DBL activities worksheet and semi-structured interviews were used as qualitative data. Findings indicated that there was a significant difference between participants’ pre-test and post-test decision skills measures in favor of the post-tests. In this study, it was determined that the decision process related to the design challenge revealed from the real life situations were identification of the problem, evaluation of the alternatives, decision applications, decision and


Introduction
Interest of educators in STEM education based on relevant, focused and integrated teaching of the disciplines of science, technology, engineering and mathematics [1,2] is increasing each day. One of the most important reasons for this is to increase the interest of individuals in science, technology, engineering and mathematics disciplines and to encourage them to pursue careers in these disciplines [3][4][5]. In addition, STEM education is also earning the interest it deserves by developing 21st century skills. According to the results of the research, STEM education develops both the interests, career awareness and attitude of the individuals as well as 21 st century skills such as creative thinking, entrepreneurship, communication, decision making [1][2][3][4][6][7][8][9][10][11]. STEM education also fits well with real-life problem situations [2][3][4][12][13][14].
The interdisciplinary teaching approach is based on teaching multiple disciplines by relating them to each other. How STEM education based on the interdisciplinary teaching approach can be realized in the context of disciplinary instructional programs has been the subject of many studies [12,[15][16][17][18][19]. The engineering design problems and process can provide an important context as to how STEM education can be implemented in disciplinary instructional programs. In the most general sense, engineering design, a problem solving approach of engineers, is a process starting with the definition of the problem and proceeding till it reaches the solution to meet the determined criteria and the limits for the specified performance [1,4,20]. Engineering design may include disciplines of science, technology and mathematics as required by nature. So, engineering design problems constitute an important context as a way to implement integrated STEM education in science courses [7,12,[21][22][23][24]. Focusing on engineering in science lessons has an important role at the point of students' understanding developing towards science and associating science with everyday life [25]. It may also have the opportunity to motivate students to learn science concepts and mathematics, including engineering technology [12].
Engineering design problems become a tool for learning in science courses and provide an environment for both the development of science knowledge and skills [25,26]. In other words, during design activities, students need to learn new information and used much skills, design a successful product, and are thus motivated to learn that way [26].
Students are confronted with meaningful design problems that involve teaching content (such as science, mathematics knowledge), and the systematic engineering design process is used to solve the engineering design problem [18,22,27]. In this approach, it is aimed that learners acquire the knowledge of science or mathematics that exists in the problem during solution process of the engineering design problem. In addition to, learners acquire skills as critical thinking, decision making, co-operation working, and entrepreneurship in design process. Therefore, the engineering design problem can be considered a tool that provides a real-life context for both science and mathematics learning, motivates learners as well as improves engineering knowledge by having the student experience the design process [7,12,[22][23][24]. For this reason, the engineering design process provides an important context for STEM education due to the interdisciplinary teaching of science and mathematics disciplines.
Designing the school science/math course in the context of engineering problems can be answered with a question "How do engineers design?". Hynes [28] describes the design process of engineers with a dynamic and iterative structure rather than a rigid and linear one, with nine steps: Defining the problem, determining the needs for the problem, developing the possible solutions, choosing the best solution, making the prototype, testing and evaluating the solution, presenting the solution, revising and completing the decision [21].
The teaching methods structured in the context of engineering design problems are called "design based learning" in the literature [19,24,29,30]. How to use engineering design problems in science lessons is the answer to the question "how can an engineer adapt design process to a science lesson?", with many approaches as a response to the question on the integration of engineering into science teaching. These approaches, which address engineering design problems as a context for science education, are included in related literature as "Design Based Science Education". From the aforementioned approaches, the suggestion by Wendell et al. [19] that the integration of the engineering design process into science courses is important in that it is a synthesized nature as revealed by an examination of field research. In addition, Wendell et al. [19] structured the method they proposed with the question "How does an engineer design?". This approach presents a engineering design problem (Wendell et al. [19] refers to "grand engineering design challenge") involving knowledge and skills for these achievements by choosing 8-10 achievements targeted to students, and students are asked to define the problem. For example, considering the highest energy efficiency, longevity and optimum cost, building a building's heat insulation can be a grand design challenge. The grand design challenge involves a lot of science achievements such as the importance of heat insulation, contribution of heat insulation to family and country acquisition, recognition of insulation materials, selection criteria of insulation materials in buildings. Students will need some knowledge and skills to perform the grand design challenge. In this direction, it is planned to acquire the necessary knowledge and skills through mini-investigations or mini-challenge before starting the grand design challenge. Through mini investigations and mini-challenge, students will gain the knowledge and skills necessary for the grand design challenge and will develop the suggestions for the grand design and will determine the most appropriate for the grand design challenge as expected. At the next step, students will create a prototype for the solution they suggest for the grand design challenge which they think is the best fit. Then students will test the prototypes for design solutions and will improve or redesign them in line with the deficiencies and share their designs.
In the design based learning process where engineering design process is used as a pedagogical tool to carry out science education [31], students carry on a process like the one that professional engineers do [25]. They learn necessary scientific concepts in this step to solve engineering design problems [32]. Many studies point out that "decision making" skills are effectively used in engineering design process [1,31,[33][34][35][36]. Fila and Purzer [34] described engineering design process as an approach that provides implicit learning for decision making owing to this very relationship.

Decision Making Skills
"Decision making" skills, which are common among 21st century skills defined by many international organizations. In the 21st Century skills described by various organizations, there are differentiated skills areas, and decision making skills are emphasized by all of these organizations. Also, decision making skills are included among the skills that aim to improve the science curriculum of Turkey. Decision making, in a broader sense, can be described as identifying a choice among many other alternatives [37,38]. In addition, the identified choice should be fulfilled under the guidance of some valid arguments [39]. Therefore, the definition of decision making reflects a process rather than an instant action [40,41]. Two major theories to decision making have been identified. Decision theories can be classified into two categories: descriptive theories and normative theories [42]. Normative theories concerned with how decisions should be made rather than how they have been made [41,43,44]. From the normative perspective, various researchers have proposed different models to formulate decision making processes [40,41,[45][46][47][48][49][50]. In these models it is seen that the process is evaluated in three to six steps according to the detailed handling of the steps of decision making. In this study, decision making skills are dealt with by identifying the problem, generating alternatives, evaluating 2890 The Effect of Design Based Learning on Pre-service Science Teachers' Decision Making Skills alternatives, choosing an alternative, implementing the decision, evaluating decision effectiveness [48]. A six-step process was used to examine the progress of the decision making process in detail.
Decision making skills has an extensive structure with quite wide borders by nature. It has been used in the context of many disciplines, such as economics, management, engineering, public administration and sports branches. For this reason, the decision theory explaining decision making has been the subject of scientists working in different disciplines to work on their own perspective, despite that decision making as a mutual concept and structure in the frame of two theories normative and descriptive [43,44,51]. Operational definition of "decision making" in this research is based on normative theory. Normative decision theory is based on the concept of how a decision must be made, whereas descriptive decision theory is based on how we have made decisions in reality [41,52]. Normative decision theory is based on the idea that decision-makers are rational people who choose the best decision option among all possible options and it focuses on how people must make a decision within specific rules [41,43,47,49]. Since normative decision theory approaches decision making as an analytical process, it allows decision making to be tested objectively. However, it is also criticized because individuals are not always rational decision makers [41].
In daily life, people make many decisions. When making these decisions, they compare the options they are met with according to selection criteria, order them and choose, and this selection process is not always simple for people [48]. As Çınar [53] emphasizes, they can only be convinced that they have chosen the right solution, in very simple cases, if there is only one selection criterion. In everyday life, there are usually several criteria for selection, and evaluating each criterion makes the process more difficult. This choice, which is made based on the evaluation of the various alternatives that often conflict with each other (eg fuel economy and performance for the automobile engine), is called multi-criteria decision making [54]. For example, if a person wants to buy a new car, he or she may have the aim of it being the cheapest, most economical, and most comfortable, and having some sporting features, and this person must make this selection according to criteria such as price, economy, comfort and sporting features. However, when deciding, they will see that these criteria conflict with each other. As a matter of fact, it is obvious that a sporty car will not be the cheapest, and a spacious and comfortable vehicle will not be the most economical. Therefore, since none of these criteria can be realized at once, "compromises" will need to be made for some [53]. Problems involving multiple criteria and conflicts between criteria can be resolved by multi-criteria decision making processes [53]. In everyday life, sub-criteria can also be found in more complex decision making situations. The main problem in such a decision making process is which is the best choice from the set of options that are evaluated according to the conflicting measures. Chankong and Haimes [55] describe the multi-criteria decision making process with the steps of identifying the problem, determining the criteria and how to measure them, defining the problem, creating a logical, graphical or physical model, analyzing, evaluating and interpreting data. Multi-criteria decision making criteria require that the most appropriate option for the problem context should be determined, taking into account the level of importance of each criteria [48]. The conceptual framework of decision making in this research was planned based on multi-criteria decision making and normative theory (explained previous paragraph).

Decision Making Skills and Design Based Learning
National Research Council (in USA) [4] specifies how engineers work in the context of engineering design process. Engineers also make decisions about their designs, and developing designs by trial and error is a very rare condition [4]. Engineers analyze models and prototypes they build, and collect data on whether they work by testing their designs [4,56]. It can be said that the process of intensive decision-making in this process is the subject [4,6]. Such an arrangement of engineering design process and decision-making process is reflected in the definition of engineering design process [6]. For example, the engineering design process is defined by ITEA [International Technology Education Association] [20] as a recurring decision-making process. In the report of National Academy of Engineering and NRC [1], engineering design names are defined as something to produce, to design and plan, while as an action it is referred to as a recurring decision making process by producing plans aimed at human needs or problems. Literature review in the decision-making process, information gathering and interviews with experts show that the step of identifying the problem is similar to the step of identifying the problem in the engineering design process. Again in the decision making process, the determination of alternatives, their measurement, the creation of decision matrices and decision trees will help in decision making, and decision making steps, the similarity with selecting the most appropriate solution using decision matrices in the engineering design process in order to facilitate the selection in accordance with the criteria and its limitations is quite clear [57,58]. This relationship between the engineering design process and the decision-making process led to the belief that the development of decision-making skills could be supported by science education through engineering applications [6]. Denson [33], in a study where a synthesis of experience in engineering design approach in STEM education was carried out, emphasizes that future researchers who want to conduct research on engineering design approach should focus on decision-making skills. Jonansen [35] draws attention to the relationship between decision-making process and engineering design process, explaining the relation between these two processes by drawing a three-dimensional model. It is stated in the model that the spiral is related to the engineering design process, and that the decision process in each step supports the design phases. Although Denson [33] and Jonansen [35] have not conducted an empirical study of the decision-making skills of the engineering design process, they theoretically pointed to the similarity between these two processes. It is thought that this research will contribute to literature in the context of revealing the effect of engineering design activities on decision making skills in a practical manner. Fila and Purzer [34] described engineering design as an approach that provides implicit learning for decision-making owing to this very relationship. Accordingly, it can be stated that engineering experience are considered as a very important area for science education and an important acquisition for the 21st century skills [31], so, these skills should be used to support the decision-making skills depending on the design activity.
In this study, it is aimed to investigate in depth how the DBL process influences the decision making ability of the pre-service science teachers and the opinions of pre-service science teachers regarding the effect of the DBL process on decision making ability. Therefore, the following research questions were addressed: 1.
How do (DBL) processes impact pre-service science teachers' decision-making skills? 2.
What are the views of pre-service science teachers on the impact of DBL on the decision-making process?

Design of Research
In this research, a mixed-methods approach was employed [59,60]. In the quantitative part, one-group pre and post-test experimental design [61][62][63] was used to investigate the effects of DBL program on pre-service teachers' decision making skills. In the qualitative part, it has been investigated in depth what kind of behaviors pre-service teachers' exhibit regarding decision-making skills in the DBL program process. Besides, pre-service teachers' views examined about DBL programs' effect of decision making skills. In the qualitative part was used multiple-case (embedded) design. The multiple cases examined in the research are behaviors of pre-service teachers regarding the decision-making skills in each activities in the DBL process. The unit of analysis is pre-service teachers.

Context of Research
The setting for this research was a fourteen-week (four lessons per week) conducted at the course of Science Instruction Laboratory Applications I, which takes place in undergraduate program for science teachers. In the research, the course was conducted DBL process was carried out in the frame of the unit pattern roughly approximates one cycle through the engineering design process (as illustrated in figure 1) developed by Wendell et al. [19] and used by many researcher in Turkey. The engineering design process in the middle of the model proposed by Wendell et al. [19] was included in 5 steps for the first and middle school level. A more detailed discussion of the engineering design process in the context of the study group of this study was deemed appropriate. Hynes et al.' (2011) 9-steps engineering design process cycle was used to study the engineering design process more closely in the context of the study group of this research. Thus, proposed by Wendell et al. (2010) the unit design pattern roughly approximates one cycle through the engineering design process is arranged as in Figure 1.
In the first step of the DBL process, "identify need or problem" and "research need or problem", participants were presented a grand engineering design challenge. Then, participants discuss what they knew about the problem and what kind of information they needed in order to perform grand design challenge. They describe the problem better they determine the success criteria and constraints about the grand design challenge. In the step of "develop possible solutions", participants worked in groups of four or five to perform mini-design challenges and mini science investigations to structure the knowledge and skills that they will enable success on the grand design challenge. At the end of the step of developing possible solutions, participants were asked to evaluate first solution offers related to grand design challenge. In the step of "select best possible solution", participants working in groups were asked to determine the group design decision by researching individual solution offers that group members have developed. In this step, participants can accept one of the suggestions offered by any of the group members, they can put strong points of various solution offers together, or they can develop a new solution. The "construct a prototype" and "test and evaluate solutions" steps take place: in the last two to three lessons, participants build, test their solution to the grand design challenge. And then, in the step of "communicate the solution" present to their classmates an explanation of how it works. Finally, last steps of "redesign" and "completion decision", if necessary, the design is revised and completed.
Throughout the process, four grand engineering design challenges [Solar / Vehicle Design, Aquarium Design, Perfume Production and Horse Antler Design] and numerous mini-designs and mini-researches have been carried out to acquire knowledge and skills to perform the grand design challenges were conducted with prospective teachers [65]. 2892 The Effect of Design Based Learning on Pre-service Science Teachers' Decision Making Skills

Participants of Research
The study group for the quantitative part of the research is 36 pre-service science teachers. Study group determined by convenience sampling method. For the qualitative part of the research, the study group consisted of 6 pre-service science teachers selected from the quantitative study group with maximum diversity sampling.
The study group consisted of 26 woman and 10 men. Pre-service science teachers are graduated from high school (f=26), technical high school (f=4), vocational high school (f=3), Anatolian high school (f=3). When we look at the order of preference of the science teacher education program that they are currently attending in university preferences, 24 of the pre-service teachers are in the first five preferences, 4 of 6-10, 5 of 11-15 and 3 of 16-20. Nine of the pre-service teachers have engineering departments at university preferences while 27 of them have no engineering departments. The 6 participants who constituted the qualitative study group of the research were selected from the sample in such a way as to provide maximum variety in terms of gender, high school type, preference order, availability of engineering in their preferences and design / project implementation status during their education life. The nick names of the participants are as follows: Nature, Sea, Sun, Soil, Rock, and River.

Data Collection Tools
The data of the study were collected Decision Making Skills Test (DMST), DBL activities worksheet and semi-structured interviews.

Decision Making Skills Test (DMST)
DMST was prepared for middle school students by Ercan and Bozkurt [6]. In this research, DMST was adapted (conducting validity and reliability) to pre-service science teachers with the permission of the researcher.
DMST was developed to determine whether the choice of individuals among many other alternatives is correct or not, especially when they face a problem of normative decision theory [6]. DMST was developed based on Lunenburg's [48] decision making steps. These steps are listed as follows: identifying the problem, generating alternatives, evaluating alternatives, choosing an alternative, implementing the decision, and evaluating decision effectiveness. It is expected that students will understand the problem, determine the alternatives taking into account the relative importance of the criteria, and select the best ones or determine the effective criteria for selecting this alternative by giving a selected alternative to the real-life situation presented in the problem. The questions prepared for DMST were multiple-choice and there were six options for each question Thirteen questions were prepared by the researchers by organizing Ercan & Bozkurt's [6] test in such a way that all of the questions (10 questions) would be adapted to the level and adding 3 more questions. The questions were first repeatedly reviewed by the researcher at different times. After that, the questions were sent to 3 experts. One of these experts had studies in engineering, two of them in science education. These experts were expected to carry out the evaluation in terms of criteria (accuracy, appropriateness or relevance to test specifications, technical questions-construction flaws, grammar, offensiveness or appearance of bias, level of readability) developed by Crocker & Algina [66].
The experts' evaluations were discussed with the researcher and a science education specialist who was involved in decision making skills, and necessary corrections were made. Experts have assessed that two items should be removed from the test. They say that two items (one of very difficult, one of very easy) are not suitable for the level of pre-service teachers. The researcher performed a small grub (fourteen pre-service science teachers) test before removing these two substances from the test. It has been decided to remove these two questions from the test because the pre-service science teachers have opinions like experts.
The test was applied to 368 pre-service science teacher (from five university) to conduct their validity and reliability studies. The measurement reliability (KR-20 internal consistency) was . 71. In this context, it can be said that the measurements of the test are reliable. It was determined that the measurements had a normal distribution and then the values of the item difficulty and discriminative indices of the test were calculated. It was determined that the discriminative indices of all 11 items in the trial form is suitable for use in the test (mean 0.44). Then the difficulty values of the items in the test are calculated. The items in the test were found to be 3 difficult, 3 easy and 5 moderately difficult. The mean difficulty of the test was .52. The indices of discrimination of the items are suitable as test items and the test contains both difficult, easy and moderately difficult questions in terms of difficulty. For this reason, it was decided that all eleven items should be included in the test.
In the test consisting of 11 multiple-choice questions, each question has 1 correct answer, each correct answer is 1 point, wrong answer, blank, or multiple choice items are scored with 0 points. The minimum score of the test is 0 and the highest score is 11. The reliability of the pretest and posttest measurements obtained with DMST was calculated as .72 in this study.

DBL Activities Worksheet
DBL activities worksheets developed by the researcher to us the basic lesson material in this study. DBL The worksheets were used as a supporting data source for quantitative data in order to determine how the pre-service science teachers' decision making skills changed during the research. Participants made some drawings on these documents, filled out decision matrixes, and expressed their reflective opinions during DBL activities. Hence, the researchers were able to use these documents as the qualitative data source.
Participants in the DBL program performed activities involving group work consisting of 3 or 4 participants and 4 different grand design challenges. The activity worksheets were used to examine the development of behaviors exhibited by pre-service science teachers in decision making process.
DBL activities are described in Wendell, et al. (2010) on the basis of a design-based science education model (detailed in the context of research). Each of the activity consists of a grand design challenge, mini-challenge and mini-investigations to support the acquisition of the knowledge and skills needed to perform this grand design challenge.
Science Teaching Laboratory Applications I course consists of four course hours per week. Each activity took at least two weeks and a maximum of four weeks. For example, in a four-week activity, the first week grand design challenge is explained and steps are taken to identify the problem. Then mini-challenge and mini-investigations are carried out for two weeks and grand design challenge are done in the laboratory during the last week. It was thought that it would be useful to explain one of the activities in detail to understand how the activities are being used to determine the change in decision making skills.
The "Vehicle / Toy Design" event starts with the following problem: "When Ali plays with the toy his brother Ahmet bought, the battery runs out within half an hour. Ahmet puts a new battery in, but this runs out in half an hour too. This situation frustrates Ahmet a little." The design challenges for pre-service science teachers are to implement a design to solve the problem. Before passing through mini challenge and mini investigations, what criteria and constraints should be given to successful solution/solutions that will enable Ahmet to get rid of this situation in the event plan are queried first. Subsequently, pre-service science teachers are directed to conduct research through computers connected to the internet in each group in the classroom environment before proposing solutions for the problem. In the continuation of the activity, pre-service teachers propose more than one solution for the problem in accordance with their research. There are decision matrices in the event plan to evaluate the situation of meeting the criteria and constraints of the problem for each solution suggestion.
The best solution suggestion for the problem is decided upon in line with the matrices created for the criteria and constraints, and the selected solution suggestion is dawn in the relevant part of the activity. Each group presents their decision matrices and why they selected a solution suggestion in terms of solution suggestions regarding criteria and constraints and shares these with their classmates. Decision matrices are included in each of the four grand design activities.
After the above mentioned steps related to the grand design challenge in the activity, mini challenge and mini investigations are started to prepare the pre-service teachers to acquire the necessary knowledge and skills for the realization of the grand design challenge.
There are three mini challenge and mini investigations for the "Vehicle / Toy Design" event. The first of these, "Energy Transformations: Solar Energy to Heat Energy". In this mini challenge/investigation pre-service science teachers are asked to design a cooker to heat / cook food using solar energy. The activity for mini-challenge questioned what the pre-service science teachers know and what they should know for their design, the constraints for the mini-challenge, and the criteria for the design's success.
The second mini-challenge / investigation for "Vehicle / Toy Design" event is "A Way to Benefit from Solar Energy: What is a Solar Battery? How does it work? "In this mini-challenge/investigation, the pre-service science teachers answer the question: "How is a solar battery made?". They discussed the working mechanism of the solar battery by designing a solar battery with the materials given to them. The final mini-challenge / investigation is "Circuit Creation Using a Solar Battery". Here, the pre-service science teachers used the solar batteries and bulbs given to them to create circuits by making serial and parallel connections and discussed the variables such as the brightness of the bulbs and how long the batteries lasted. After the mini-challenge / investigation in the activity, the pre-service science teachers were asked if they wanted to make any changes to their solutions aimed at the grand design challenge, and if so what the changes were. After mini-challenge / investigation and changes that pre-service science teachers thought to make, they returned to the grand design challenges and repeated the design drawing in the direction of the changes they thought of making, discussing their drawings in groups and then making their designs and testing them.

Semi-structured interviews
Two semi-structured interviews were held with each of the pre-service science teachers (qualitative study group, f=6) in the middle of the application period (first interview) and at the end (last interview). Semi-structured interviews were aimed to determine the opinions of the pre-service science teachers on the impact of decision making skills of DBL program.
In semi-structured interviews, pre-service science teachers were asked what DBL's skills they developed. Subsequently, it was questioned whether this process contributed to the development of decision making skills. Pre-service science teachers with positive statements were asked how this development was.

Data Analysis
Before deciding on the statistical methods to be used in the analysis of quantitative data, histogram graphs and skewness coefficients of each measurement were examined and the scores were checked for normality with the Kolmogorov-Smirnov test (KS) to determine whether the data showed normal distribution. It has been found that each of the measurements made according to these graphs and values has a normal distribution.
Accordingly, the paired sample t test was used to compare the pre-test and post-test of the change of the pre-service science teachers' decision making skills. The effect size (eta square) was calculated to provide a more comprehensive assessment of the statistically significant differences in the t-test results. The effect size indicates how much of the total variance on the variable or factor dependent variable is explained [67]. The effect size varies from zero to one. Effect sizes classified as small (.01), medium (.06) and large (.14) [67].
When the qualitative data of the study were analyzed and the presentation of the findings was planned, the multiple-case (embedded) design mentioned in the model of the study was taken as basis. The elements of bullying decision making are analyzed and presented in the context of the analysis unit (pre-service science teachers).
Activity documents were used as a supporting data source in order to examine the development of pre-service science teachers' decision making skills in depth. The data obtained from these sources were analyzed using the constant comparative and descriptive method [68].
The research on decision making skills and engineering design process [1,28,31,33,34,35,40,41,[45][46][47][48][49][50] has been examined in order to determine what behaviors pre-service science teachers can reflect on their ability to make decisions on event documents. The steps of the decision making process [48] and the engineering design process [28] were considered together, and elements that could be included in the event documents for this phase were identified. So, the "partial framework" presented in Table 1 was created for data analysis. The activity sheets of the participants were analyzed in accordance with Table 1.
There are four activities in the DBL program. The activity document for each group was analyzed by constant comparative analysis method from the partial framework presented Table 1. Activity worksheets of four different activities were analyzed in order of do of the activities. The activity worksheets were analyzed by taking into account the factors involved in each step of the decision making process.
Activities were conducted with group work. For this reason, findings from the activity documents will be presented in the context of the activity sheets of the group of each of the six pre-service science teachers constituting the study group. In other words, each of the activity worksheets of the groups of participants was analyzed in order to examine in depth the development for each step of the decision making process, such as identifying the problem, creating alternatives.
The pre-service science teachers participating in the case study are from the same group of Nature and Sea, the two pre-service science teachers have only one activity document. River, who was a participant of the study, did not attend the same group for 2 weeks. For this reason, it was decided not to include the data related to River's activity documents in the findings.
Data obtained with semi-structured interviews were analyzed using a combination of descriptive analysis and constant comparative analysis techniques. The steps of decision making have formed the framework of descriptive analysis. The researchers started in the open coding process in the first step and when they encountered each new data they asked "What does this mean" or "How does it fit" and they have examined each category carefully and formed a detailed category.

Results from Decision Making Skill Test
The results of the comparison of the decision making skills pre-test post-test measurements with the paired samples t-test are presented in Table 2. As indicated in Table 2, pre-post-test measures of the pre-service science teachers decision making skills showed a statistically significant difference in favor of post-test measures (t(35)=-3;37, p<.01). The mean of the pre-service science teachers for DMST pre-test measurements was 6.64, while the mean for the post-test measurements were 7.50. This finding can be interpreted as the large effect of DBL in Science Teaching Laboratory Applications I to the development of pre-service science teachers' decision making skills (µ2=.25).

Findings Related to the Decision Making Process of the Nature and Sea Groups Activity Documents
It has been found that the steps of the Nature and Sea' group during the grand design challenge in the four activities was more successful in the last two activities in the "identification the problem" step. It is expected that in this step corresponding to the identification of the problem in the engineering design process and the identification of the problem needs that the pre-service teachers will determine the success criteria and constraints of the design challenge and what they know / should know about the problem situation. In Table 3, the success criteria and constraints of the pre-service teachers regarding the grand design challenge from the first activity towards the last activity findings about whether these determinations are correct or not are included.
As seen in Table 3, Nature and Sea's group have confused or misjudged the criteria and constraints in the 2896 The Effect of Design Based Learning on Pre-service Science Teachers' Decision Making Skills firstly and secondly activities. For example, the "cost" they set as criteria in the first activity is not a criterion but a constraint. In the second event, "aquarium size" is not a criterion or constraint for this design challenge.
Participants were hesitant to determine what they know / should know in order to perform the design in the first activity. In the second activity, they could determine what they should know, but not what they know. They were able to make more appropriate and detailed determinations in the third and fourth activities.
"We need to investigate what tools will keep the temperature constant, what filters there are for oxygen..." (2nd Activity) "One has to know how all simple machines work? We know simple machines, but we still have to review them. In fact, how can a mechanism be established with more than one simple machine? We need to learn how to build a unified system in detail. "(4th Activity) Nature and Sea's group did not have any problem in four activities during the "generating alternatives" of the decision making steps. It is expected that participants will be able to demonstrate solution proposals for the grand design challenge and support the solution proposals with scientific information and evidence at this step of development of possible solutions in engineering design process.
The Nature and Sea group did not elaborate on scientific proof for the solutions proposed in the first activity. In the second activity they have explained their solutions in more detail in comparison with the previous design challenge. In the third and fourth activities, they explained the solution proposals with scientific reasons. In addition, they have developed suggestions for solutions for each activity in the direction of knowledge and skills gained with mini-design and research.
The decision making steps of "evaluating alternatives and choosing an alternative" overlaps with the "choosing the best solution" step in the engineering design process. At this step, it is expected that the pre-service science teachers should form a decision matrix about the success criteria and evaluate the solution proposals in terms of criteria and constraints using these matrices. Nature and Sea's group have made evaluations considering multiple criteria from the second activity. The evaluation made by the group for the 3rd activity is as follows: "We are closer to realizing our first suggestion. It is easier to extract the fruit essence and the scent will be longer lasting than floral scents. We decided not to use ready-made essences because that will increase the cost. We can also make it more certain that we are obtaining organic produce by using fruits. Ultimately, if this is a preliminary model, then we thought it might not be good to use the essence to make production. " In the "implementing the decisions" step of the decision making process, engineering design process "test and evaluate solution; completion decision" steps, the pre-service science teachers are expected to explain which solution they have decided upon in terms of both criteria and constraints, and with scientific reasoning. They chose the solution that they thought was the most accurate in the decision making matrices in the first activity, but they could not explain with scientific reasons in the context of criteria and constraints how they decided in final design challenge. In the second activity, they explained their final solutions in terms of matrices' evaluations. Evaluations at the third and fourth activities seem to reflect the multi-criteria decision making process.
"The solution we choose is appropriate in terms of criteria and constraints. We used a filter to clean the water in the aquarium, the elodea plant to increase the amount of oxygen in the water, the scavenger fish for bottom cleaning ... We did not use the heater because we learned that the room temperature is suitable... "(2nd Activity) "We decided on the first solution suggestion. It meets all our criteria. We've done theoretical calculations for efficiency. According to these calculations, if the radius of the bigger pulley is 4cm for example, the small pulley rotates 4 times when we turn it. As a result, strength provides efficiency" (4th Activity) It is expected that, in the "evaluating decision effectiveness" step of the decision making process, the "communicate the solution, redesign, and completion decision" steps of engineering design process, participants test the prototype they have created and evaluate the results, make improvements when required and share the decision. In the first activity, the group has prototyped the final design that they decided as the best solution and did not succeed in the application. Final drawings were questioned with researcher support. The deficiencies were identified, the improvements were made, and the results were re-tested and were successful. Other activities did not need to be improved after the testing phase.

Findings Related to the Decision Making Process of the Sun' Group Activity Documents
During the grand design challenge in the four activities of the Sun' group, there are some shortcomings in the criterion and constraints set out in the first two grand design challenge, in "identifying the problem", which is the identify need or problem and research need or problem in the engineering design process and they seem to be more successful in the design challenge in the last two grand design challenge. For example, they expressed the constraints that they set for the first grand design challenge in the form of "no night-time use and no electricity", neither of these constraints is against the problem. In the second grand design challenge, it was determined that only one of the criteria they determined was not related to the problem, while the criteria and constraints they determined in the other grand design challenge were completely suited to the problem.
Participants in the Sun' group were hesitant to determine what they know / should know in order to perform the design in the first activity. In the second activity, they could determine what they should know, but not what they know. They were able to make more appropriate and detailed determinations in the third and fourth activities.
While the decision making cycle of "generating alternatives" produced two proposals for the design challenge in the first grand design challenge of the Sun' group in the corresponding step of the development of possible solutions in the engineering design process, three suggestions in the second grand design challenge, two in the third grand design challenge, and one suggestion could be generated in the grand design challenge. This finding suggests that the Sun group has problems or difficulties in generating alternatives from the decision making steps. However, it has been determined that participants can explain the possible solutions from the second activity in detail with scientific evidence. For example, although only a single possible solution has been put forward in the last activity, they have discussed this proposal in detail with scientific reasons.
In the steps of "evaluating alternatives and choosing an alternative" during the engineering design process of "choosing the best solution" of the decision making process, they did not have any problems in four activities. However, it can be said that they are more successful at the "implementing the decision" step in evaluating the proposals because they better reflect the multi-criteria decision making process from the first grand design challenge to the last grand design challenge.
However, the assessment made by the group in the "implementing the decision" step for the last grand design challenge, for example, is successful because the "implementing the decision" step group is influenced by alternative solutions to the design challenge, but it does not reflect "decision making" among multiple alternatives as it is focused on a single solution. It can be argued that they analyze the problem in the last activity better, and therefore they may think that they can reach the result by putting forward only a single solution. As a matter of fact, their success in the application phase suggests this.
In the "evaluating decision effectiveness" steps of the decision making process, the "test and evaluation solutions, communicate the solution, redesigning and completion decision" steps of engineering design process, it has been determined that no problems were encountered during the process.

Findings Related to the Decision Making Process of
the Soil' Group Activity Documents When the Soil' group was examined from the process documents for four activities, it was determined that all of the criteria and constraints determined for the problem in the last activity were acceptable, although they experienced similar problems such as misrepresentation of one or several of the criterion and constraints in the first three activities during the "identifying problem" step. In this respect, it can be considered that the decision making process during the process has evolved in the "identifying the problem" step. For example, the "economic suitability" referred to as the criterion in the first grand design challenge is a constraints, not a criterion for the grand design challenge. Likewise, "ensuring that the pH value remains constant", which is expressed as a constraint in the second activity, is not a constraint but a criterion. Participants in the Soil' group were hesitant to determine what they know / should know in order to perform the design in the first two activities. They were able to make more appropriate and detailed determinations in the last two activities.
In the steps of "develop possible solutions" during the engineering design process of "generating alternatives" of the decision making process, they were able to produce more alternatives in the last activity. For example, when presenting two solution proposals (rechargeable battery, solar cell use) in the first activity, it has been determined that this recommendation cannot be expressed in detail with scientific evidence. In the second activity, 3, 6 in the third activity and 2 in the last activity, solution suggestions were able to be made and they could be expressed clearly with scientific reasoning.
In the steps of "evaluating alternatives" during the engineering design process of "select best possible solution" of the decision making process. The Soil' group did not have a problem at first in the activity, but it was thought that the successful results in the first activity were due to presentation of the matrix tables ready to reflect the purpose because they could not properly form the decision matrices in the next two activities. Even though they cannot form decision matrices in the second and third activities, it is determined that they tried to carry out the evaluation by considering more than one criterion in the decision process.
In the last activity, the decision matrices and evaluation that they put forward reflects the multi-criteria decision making process and they have explained why they decided on this design decision. For this reason, they are thought to have developed on this step. Indeed, they have expressed in a way that reflects the multi criteria decision making process, which one of the solutions proposed in the "implementing the decision" phase has been chosen.
It has been determined that the decision making process of the "evaluating decision effectiveness" engineering design process is more successful after the first grand design challenge in the steps of "test and evaluation solutions, communicate the solution, redesigning and completion decision".

Findings Related to the Decision Making Process of the Rock Group Activity Documents
When the Rock' group was examined from the process documents for four activities, it was determined that they experienced similar problems such as misrepresentation of one or several of the criterion and constraints in the first two activities during the steps of "identify need or problem" during the engineering design process of "identifying the problem" of the decision making process. However, in the last two activities, it has been determined that all of the criteria and constraints they set for the problem are appropriate. For example, the "being economical" criteria they set as criteria in the first activity is not a criterion but a constraint. As another example, the "aquarium size", expressed as a constraint in the second activity, is not a criterion or constraint for the design challenge in this activity.
It has been determined that the decision making process has progressed in comparison with the first activity in the second and third activity at the step of "generating alternatives", but not in the last grand design challenge. For the first design challenge, they proposed 2 different solution suggestions, for the second and third activities they proposed 3 different solution suggestions. In the last activity, it was determined that they provided a single solution suggestion. As a matter of fact, this situation can suggest that they group has not developed in the "generating alternatives" step, while it may also suggest that they did not create alternative solution because the only solution made by the group was found to be strong for the final design challenge. It is seen that the participants explain each of the solution suggestions from the second activity with scientific reasoning.
The Rock' group had problems in the first activity in the "evaluating alternatives and choosing an alternative" steps, but they did not have any problems at these steps after the second activity. This finding may be indicative of the group's progress towards the two steps. Indeed, they have explained the reasons for choosing which of the solution proposals they have selected so as to reflect the multi-criteria decision making process.
The group has not experienced any problems since the first grand design challenge in the "implementing the decision and evaluating decision effectiveness" steps of the decision making process

Findings Related to Pre-service Science Teacher Nature
In the first interview carried out in the middle of the DBL Program, Nature described the problem of decision making as identifying the problem, generating alternatives, evaluating alternatives, and choosing an alternative steps by associating it with the course process, expressing that the course being run with DBL had helped develop decision making skills. Nature also stated in the first interview that every day decision making abilities were also affected by the applications developed in the class. In the interview at the end of the DBL program, Nature expressed that, together with the positive impact on decision making skills of DBL, all of the activities in the engineering design process are aimed at decision making skills and that they have gained speed in the decision making process. Another point that Nature has emphasized during the last interview is that by developing the definition of the problem, creating alternatives, evaluating alternatives, and decision steps in the decision making process in the activities in the course, it has enabled them to be more questioning of events faced in daily life.
It has been seen that Nature has expressed the views that decision making skills have been developed in the second interview similar to the first interview, however that in the second interview, more clear expressions were used to describe the development of decision making skills. The reason for this can be considered to be the activities carried out throughout the process leading to the ability to express the impact of the engineering design process and decision making skills with more scientific reasoning.

Findings Related to Pre-service Science Teacher Sun
In the first interview carried out in the middle of the DBL program, Sun described the problem of decision making as identifying the problem, generating alternatives, evaluating alternatives, and implementing the decisions steps by associating it with the course process, expressing that the Science Teaching Laboratory Applications I course being run with DBL had helped develop decision making skills. In the second interview, Sun explained that DBL program had impacted decision making skills and gave examples from everyday life. The example given by the Sun contains all the steps of the decision making process. Sun also emphasized in the second interview that the phase of determining the criteria and constraints of the process, is an important factor contributing to the development of decision making.

Findings Related to Pre-service Science Teacher Soil
In the first interview, Soil described the impact of the DBL program on the decision making process, mentioning all steps of the decision making cycle. Soil also stated in the first interview that every day decision making skills were also affected by the applications developed in the class but he had yet to notice this. Soil stated that carrying out laboratory lessons with DBL in the second interview affects decision making skills in both class and daily life and that the decision making process is accelerated.
Soil has expressed an opinion that it has improved his decision making skills both at the first interview and the last interview. However, unlike the first interview, in the last interview, it is emphasized that the decision making process has not only developed but become accelerated. From this finding, it can be said that the decision making ability of Soil started to develop in the middle of the process, and at the end of the process, it progressed to him becoming an individual who can make better decisions.

Findings Related to Pre-service Science Teacher Sea
Sea explained that conducting the Science Teaching Laboratory Applications I course with DBL has improved decision making skills, and explained that the reasons for this are the processes carried out. Sea has not provided an example, but has made general statements about the expectation that it will improve the ability to make decisions in daily life at the first interview. At the last interview, he expressed himself with specific examples.
It can be said that Sea has used more clear expressions of how the process has developed decision making skills in the last interview.

Findings Related to Pre-service Science Teacher Rock
Rock explained DBL program has developed decision making skills, and explained that the reasons for this are using all of the steps in the decision making cycle. Rock stated that decision making skills have improved in the last interview and explained this situation by associating them with the decision making skill test applied as preliminary and final test.
Rock explained the development of his decision making skills using all steps of the decision making cycle in the first interview. He was also found that to use reinforcing expressions in the last interview. Indeed, it can be said that Rock's awareness of the development of decision making skills has developed throughout the process.

Findings Related to Pre-service Science Teacher River
River explained DBL program has developed decision making skills, and has associated this with the identifying the problem, evaluating alternatives and choosing an alternative the decision steps of the decision making process, and has expressed the steps taken in the design challenge. River stated in the first interview that her decision making skills had developed, that they had not experienced problems in the determination of criteria and constraints, which is the identifying the problem step, but had difficulties in developing possible solutions, which is the generating alternatives step of the decision making process. River stated that the decision making skills had improved in the last interview, highlighting the importance of finding alternative solutions for design challenge, which are the generating and evaluating alternatives steps of the decision making process. River expressed that the decision making ability was reflected in her daily life and she gave an example from her daily life in the last interview.

Discussion
Decision skills measures of pre-service science teachers after DBL program increased and a statistically significant difference was found between pretest and final test measures. From this finding, it was concluded that the Science Teaching Laboratory Applications I conducted with DBL has the effect on the pre-service science teacher's development of decision making skills. This endorses research that points out the similarities between the decision making process and the steps of the engineering design process, indicating that engineering design activities will have an impact on decision making. The decision making process, presented as a cycle by Lunenburg [48], which begins with the identifying the problem, generating alternatives, evaluating alternatives, choosing an alternative, implementing the decision, evaluating decision effectiveness steps, has many similarities with the engineering design process. The engineering design process also starts with the determination of the problem, and includes the steps of developing the possible solutions, choosing the best solution, constructing the prototype for the selected solution, evaluating the solution, presenting and redesigning if necessary, and completing the decision [28]. When these two processes are considered together, the engineering design process can be thought of as a recurring decision making process by producing plans for human needs or problems [69] while decision making involves choosing between several alternatives [1]. Inasmuch that, during the engineering design process, various resolutions must be made towards a solution [35]. It can be said that the similarity between the decision making process and the engineering design process is also evident from the results of this study. In research by Dym, Wood & Scott [69] it is argued that the engineering design development process is supporting of effective decision making.
The development of the decision making skills of pre-service science teachers has been presented with quantitative data. The quantitative data relates to the selection of the best solution by considering the multiple choice and alternative solutions of real life problems and the relative importance of the criteria in the decision making process or the determination of the criterion for the preferred solution by going through the selected solution for a real life problem. In this direction, the quantitative data obtained by DMST have given results regarding the decision making skills of pre-service science teachers in the "identifying the problem", "evaluating alternatives" and the "choosing an alternative" steps. However, the decision making process is a process with steps as mentioned above. Activity documents of groups of pre-service science teachers in the qualitative part of the study were used as a supporting data source in order to examine in detail the progress of the decision making skills of the participants throughout the DBL program and the relationship between the engineering design process and the decision making skills. In the context of these findings, the elements in the activity worksheets prepared in the context of engineering design process were examined in the context of the analysis of the decision making process prepared by the researcher. It would be useful to talk about the results of the development of each step of the decision making process in the direction of the findings of the research separately. The steps of the decision making process are as follows: Definition of problem, creation of alternatives, evaluation of alternatives, decision aid applications, decision and application.
It has been found that in the "identifying the problem" step of the decision making process, pre-service science teachers are more successful towards the end of the DBL program of determining success criteria and constraints of grand design challenges. This step in the decision making process can be associated with "identifying need or problem" and "research need or problem" in the engineering design process. These steps are defined as the identification of the design problem in the engineering design process and the determination of the criteria and constraints for the product or system that will be the solution for the given problem situation give to the students [28,29,36]. The development of pre-service science teachers in the determination of criteria and constraints for their design challenges, are a supporting factor in showing that they have not developed in the step of identifying problem in the decision making process. Indeed, the identification of criteria and constraints is important for a better understanding of the problem for the engineering design process [36,56,70].
It has been found that the number of alternative solutions suggested when the activity documents of the groups are examined in the order in which the activities are carried out during the "generating alternatives" step of the decision making process does not show a linear increase during the process. In other words, the practices that have been carried out in the process have not made any direct improvement in the way that the pre-service science teachers can generating alternatives in the steps of the decision making process. As a matter of fact, the reason for this result is that the pre-service science teachers in Sea and Nature's group were able to make three suggestions in the first three of the grand design challenges, and only two suggestions in the last grand design challenge and also that in Sun's group, the pre-service teachers came up with two solution suggestions for the first grand design challenge, three for the second grand design challenge, two for the third grand design challenge, and only one for the final grand design challenge. Likewise, in Soil's group, two, three, one and three suggestions were made in the grand design challenges. While Rock's group had three suggestions for the first three grand design challenges, only one suggestion for the grand design challenge was given.
It can be said that the engineering design process as the process of producing creative solutions for human needs is determining the possible solutions of the step that requires the most creativity [19,36,56]. Indeed, there is no single correct solution for engineering design problems in real life, and there are usually many solutions [56,71]. In this context, the design problems presented to pre-service science teachers during grand design challenges during the process are aimed at producing a large number of solutions. However, it has been found that as the process progresses, pre-service teachers cannot create more alternatives related to design challenges. In the field of learning through engineering design, it is emphasized that in order to enable students to produce a large number of alternative solutions in the design process, it must entail group work, usually working with small groups, brainstorming, and if the problem consists of several parts, making proposals for each piece by dividing the pieces [28,56]. While preparing the structure of activity worksheets, literature was taken into consideration. In this case, it is considered that the pre-service teachers that there may be more effective factors in the process of generating more solution suggestions regarding the design challenges during the process than just experience with the process. However, the findings of this study suggest that development in the step of identifying the problem in pre-service science teachers may prevent them from needing more solutions. Because pre-service science teachers have succeeded in the implementing the decision step even though they have not produced many solution suggestions in the last two activities.
It has been found that when the activity documents of the groups are examined in the order of performing the activities, in the "evaluating alternatives" and "choosing an alternative" of the decision making process, pre-service science teachers have shown improvement in evaluating the alternatives in terms of criteria and constraints and in the formation of decision matrices. As a matter of fact, in the first grand design challenge, decision matrices are presented ready prepared, and after evaluating the criteria and limitations from the pre-service science teachers, they are expected to evaluate them by placing them in the matrices, and in this case, the pre-service teachers did not have any problems in forming decision matrices. However, some pre-service science teachers had problems developing matrices themselves and then developed towards the end of the process. After identifying many possible solutions in the design process, it is necessary to choosing an alternative of the solutions by analyzing the criteria and the constraints [4,28,36,56,71]. At this step, evaluations are made in terms of success criteria and constraints in order to determine the most appropriate solution for the design challenge [4,28]. Emphasis is placed on the formulation of decision making matrices for these evaluations in the engineering design process [56]. The fact that the steps of the engineering design process are parallel to the decision making process, and the pre-service science teachers improving at these steps seems to be supported by literature. Similar to the decision making process, Jonassen & Kim [72] emphasize that in the design process, debate is carried out in line with the available data to make a decision after alternative solutions have been established for each decision. Jonassen [35] also notes that decision matrices can be used in similar ways in both processes.
It has been determined that the pre-service science teacher's evaluations of the criteria and constraints in the decision matrices have been improved in a way that reflects the scientific reasoning and the multi-criteria decision making process. This is reflected in the "implementing the decision" step of the decision making process. Pre-service science teachers also showed improvement during the step. Indeed, determining the best solution in the engineering design process involves a decision making process [6]. It can be said that the decision making processes of pre-service teachers has achieved successful results when they test the grand design challenge, which they decided after the first design challenge in the "evaluating decision effectiveness" step and they showed improvement during the program. The process of constructing the prototype in the engineering design process can be thought of as the practical reflection of the solutions put forward in theory by engineers [4,36]. In order to address this phase, which is similar to the application phase of the decision making process, it was expected that in the process, the pre-service science teachers would build their decisions on grand design challenges. Gaining experience in this process has positively affected the development of pre-service science teachers towards these steps.
Denson [33], in a study where a synthesis of experience in engineering design approach in STEM education was carried out, emphasizes that future researchers who want to conduct research on engineering design approach should focus on decision making skills. In a study conducted by Ercan & Bozkurt [6] with middle school students, it was determined that the out-of-school applications in which DBL is implemented provide the development of decision making skills of the students. Jonassen [35] draws attention to the relationship between 2902 The Effect of Design Based Learning on Pre-service Science Teachers' Decision Making Skills decision making process and engineering design process, explaining the relation between these two processes by drawing a three-dimensional model. It is stated in the model that the spiral is related to the engineering design process, and that the decision process in each step supports the design phases. Although Denson [33] and Jonassen [35] have not conducted an empirical study of the decision making skills of the engineering design process, they theoretically pointed to the similarity between these two processes. The result of this research, which starts off from the theoretical connection between the engineering design process and the decision making process, shows that the decision making skills of the pre-service science teachers are skills that can be improved by the engineering design process.
"Decision making skills", which are among the skills of life in the Science curriculum, are an important skill that all human beings must possess, as in daily life, people make many decisions. In a typical day, a multitude of decisions need to be made that differ in content and outcome, and in each decision making, the individual tries to choose the best way to reach the desired outcome from their choices [73]. Indeed, it is very important to choose the best to be successful in life. In this context, it is important for pre-service science teachers to have the ability to look at the multiple viewpoints of the decision making process [74] for the development of a generation with decision making skills. In this study, it was determined that the decision making process of the pre-service teachers about the daily life decisions was developed by the quantitative data collection tool and the decision process related to the design challenges revealed from the real life situations were identifying the problem, evaluating alternatives, choosing an alternatives, implementing the decision, evaluating decision effectiveness steps. When the role of engineering designs in our daily lives is considered, it is clear that members of the future society will be directly or indirectly influenced by engineering decisions [36] As a matter of fact, beginning to look from the critical decision making viewpoint using the decision making process of engineers is very important in understanding the world that is molded by engineers [36].
Pre-service science teachers reported positive opinions in the middle and final of the DBL program that the program has provided the development of decision making skills. However, at the end of the DBL program, pre-service teachers were able to better correlate the steps of the decision making process in the final interviews with the engineering design process, while expressing their views on the development of decision making skills. In other words, it was determined that in the first round of interviews, these skills were developed by emphasizing only a few steps of the decision making process, while in the final interviews, they were explaining the development of decision making skills, with experiences gained throughout the process. This result of the study shows that the awareness of the pre-service teachers' decision making skills is increased and they can establish a relationship between engineering design process and decision making process at the end of the DBL program. It has been determined that pre-service science teachers report opinions that they feel that decision making skills have improved both and that this is noticeable in daily life in the middle and end of the DBL program, but at the end of the program it has been determined that decision making skills in daily life are expressed with the use clearer expressions, they can give examples and can be evaluated in the context of decision making process. Pre-service science teachers being able to explain their opinions on the development of decision making skills by presenting examples from everyday life and their use of clearer expressions can be considered as an increased awareness of participants' decision making skills.
In literature, no practical research was found to directly relate DBL to the influence of pre-service science teachers 'decision making skills, or to the relationship between pre-service science teachers' engineering design process and decision making skills. Indeed, the study carried out by Sungur Gül and Marulcu [75] is indirectly parallel to the results of the study, as they have the opinion that the thinking skills of teachers can be developed with DBL. Because decision making skills also involve thinking processes. In addition, the fact that pre-service science teachers' decision making skills have evolved throughout the process, and that they have successfully accomplished what has to be done in steps of the decision making process by reflecting this development in their activity has also been a supporting result. In addition to pre-service teachers' decision making abilities having been developed with DBL, there was also awareness of this development by the participants themselves and they were able to evaluate their decision making in daily life in terms of their progress.

Conclusions
Science Teaching Laboratory Applications I conducted with DBL has the effect on the pre-service science teacher's development of decision making skills. The quantitative data relates to the selection of the best solution by considering the multiple choice and alternative solutions of real life problems and the relative importance of the criteria in the decision making process or the determination of the criterion for the preferred solution by going through the selected solution for a real life problem. In this direction, the quantitative data obtained by DMST have given results regarding the decision making skills of pre-service science teachers in the "identifying the problem", "evaluating alternatives" and the "choosing an alternative" steps. In the qualitative data, it was concluded that DBL has the effect on development of from six steps on five. The five steps contains which are "identifying the problem", "evaluating alternatives", "choosing an alternatives", "implementing the decision", "evaluating decision effectiveness". In the qualitative data, "generating alternatives" step of the decision making does not show a linear increase during the process.
It was concluded that pre-service science teachers have positive opinions in the reported positive opinions in the middle and final of the DBL program that the program has provided the development of decision making skills. At the end of the process, it was determined that awareness of the decision making and the DBL process evolved, making this decision making skill and DBL process more relevant.

Suggestions
Today's research has revealed the necessity of enriching the science education, including science, technology, engineering and mathematics, based on inquiry / research, which has been supported for many years in science education, with an engineering design approach and a report by the National Research Community (NRC) has renewed the standards in science education in this accord [4]. When the new generation science standards are examined, it is seen that the engineering design process is included in science education based on inquiry [13]. In this respect, it is suggested that DBL should be included in teacher training programs when considering the importance of educating pre-service science teachers who will be practitioners of the developments, so that they may be aware of new methods and techniques.
The effectiveness of the DBL on decision making ability can be explored in depth by developing a measurement tool that can measure decision making skills, including all phases of the decision making process.
While the emphasis on the use of engineering design approach in science education in the world and the importance of interdisciplinary science education has been increasing day by day and while inquiry based approach in science education programs has been enriched by the engineering design approach, it has been found that pre-service science teachers' have the knowledge, approach and quality that can develop science-technology-society gains with DBL.