A Neural Network-b ased Hybrid Method to Generate Feasible Neighbors for Flexible Job Shop Scheduling Problem

In this paper, a hybrid method is proposed to generate feasible neighbors for the ﬂexible job shop scheduling problem. Many of the optimization and artiﬁcial intelligence methods have been used to solve this important and NP-hard combinatorial problem which provide the basis for solving real-life problems. It is well-known that for such problems the hybrid methods obtain better results than the other approaches. For instance, the applied non-hybrid neural networks for the combinatorial problems, as the Hopﬁeld neural network, usually converge early. Also, their results almost always contain large gaps. These shortcomings prevent them to ﬁnd good results. Another necessity for a quality search is to ﬁnd suitable neighbors of the obtained solutions, however, it is possible to create infeasible neighbors during the optimization process. The aim of this study is to overcome these deﬁciencies. In the suggested approach, at ﬁrst, an initial solution is generated and then using the left shift heuristics, its gaps are removed. Based on the critical path and critical block concepts, 6 neighbors are constructed for the obtained solution. After the generation of each neighbor, a neural network runs and controls the constraints of the problem. If the achieved neighbor is feasible it is saved. Else if it is infeasible, the neural network tries to transform it into a feasible solution. This is done by applying penalties to the start time of the operations on the violated constraints, which shifts them to the right or the left. During this process, if there are not any violated constraints, the neural network reaches the stable condition so it stops and the obtained solution is saved as a feasible neighbor. Otherwise, after a certain number of the iterations, it stops without any feasible neighbors. Then these steps are repeated for the other created neighbors. This constraint-based process provides an effective and diverse search. Finally, the obtained neighbors, are improved using the left shift heuristics. Also to demonstrate the importance of the initial solutions, they are generated randomly and also using the Gifﬂer and Thompson’s heuristic. The comparison between the proposed approach and the methods from the literature shows that it constructs better neighbors. However, using the Gifﬂer and Thompson heuristic to create the initial solution improves the results signiﬁcantly.


Introduction
The flexible job shop scheduling problem (FJSSP) is one of the well-known NP-hard scheduling problems because of its application and theoretic aspects. Many of the real-life scheduling problems are its extensions. Previously, several methods based on the mathematical optimization and the computational intelligence-based approaches have been proposed in the literature to solve the FJSSP. Actually, there aren't any strict differences between these approaches and most of the applied neural networks (NNs) for the FJSSP are formulated as the mixed integer programming models. The Hopfield neural network (HNN) that is applied for solving many of the combinatorial problems is one of them. In some 2 A Neural Network-Based Hybrid Method to Generate Feasible Neighbors for Flexible Job Shop Scheduling Problem studies of the literature, NN constructs a solution, whereas, in some others, it prevents infeasible solutions by checking the constraints. When a NN constructs solutions, generally some deficiencies like premature convergence and a large number of gaps in the achieved schedules occur. So it is necessary to use heuristics like the local and global left shifts to solve these troubles. Early convergence, prevents to achieve the global optimal results. When a NN is used as a preventive tool by checking prohibited moves of the operations by checking the constraints of the problem, the limited move set reduces the diversity of the search. All these reasons point out that in a non-hybrid approach, using NN cannot achieve good results. While combining them with the heuristics methods improves their performance with escaping from the local optimal points for the problems with high complexity.
Also, to provide a quality search, it is necessary to define effective neighborhood structures. Otherwise, a blind search will be done. Sometimes, during the process of the neighbor generation, infeasible results are obtained. In this study, a hybrid method that is easy to implement is proposed that constructs feasible neighborhoods for the FJSSP. Different from the previous works, the proposed NN is used to transform infeasible neighbors to the feasible ones considering the constraints of the problem. The general steps of this approach can be summarized as that at first, an initial solution is generated, then the local left shift heuristic is used to convert the solution to a semi-active solution. Two approaches are used to construct the initial solutions. In the first approach, they are created randomly, while the Giffle and Thompson's (GT) algorithm is used to generate them in the second approach. The critical paths and critical blocks of the initial solution are found and the neighbors are created by applying 6 different types of moves to the operations on them. These moves are applied even if some of them are forbidden according to the constraints of the problem. Due to the fact that the operations on these structures determine the makespan, the changes made on them significantly affect the objective function. Also, using more than one neighborhood structure leads to a better quality result. When a neighbor is constructed, the NN runs and controls the constraints of the problem, which are related to the start time of the operations and the order of them on the jobs and the machines. If the solution is feasible, it is saved, otherwise, penalties are applied to the violated constraints for adjusting them. During this process, the operations of the violated constraints are shifted to the left or the right. This means that their start time decreases or increases. These changes can lead to new violations, so after each iteration, the neural network attempts to correct the constraints that are violated again. Meanwhile, if a feasible solution is obtained, it is saved. Applying all moves, including the infeasible ones and then adjusting the violations, increases the diversity of search. After a pre-defined number of the iterations, if any feasible solution is not obtained, the NN stops. If a feasible neighbor is achieved, it transforms into a more effective solution by applying the global and local left shifts. Thereafter, all of these steps are repeated for the neighbors constructed by the neighborhood generating unit.
In the second section of the article, the FJSSP definition and its benchmarks are described. Literature review, the NNs presented for the scheduling problems in previous studies, description of the proposed algorithm and its design steps, implementation and the obtained results, comparison and discussion are the next sections of the article. Conclusion and future works are explained in the last section.

Problem Definition and Test Instances
In the FJSSP there is a set of n jobs (J = {J 1 , J 2 , ..., J n }), each consists of n i operations (O j1 , O j2 , ..., O jni ) that plan to be processed on a subset of m machines (M = {M 1 , M 2 , ..., M m }) as M j1 , M j2 , ..., M jni [43]. Other assumptions about the problem are: (i) the number of jobs and operations are known in advance, (ii) assignable machines and processing time of the operations on them are predefined, (iii) the start times and the processing times of all operations are integer, (iv) jobs are independent of each other, (v) interruption during the process of operations isn't allowed, (vi) each job has a consecutive set of operations according to the sequence constraints, (vii) each machine can process at most one operation at any time according to the resource constraint, (viii) setup time of the machines are negligible, (ix) each machine can work independently from the other machines with the maximum performance and there aren't any maintenance and breakdown times [39] [46].
In this paper, O jik stands for operation i of job j to be processed on machine k with R jik release time, ST jik start time, P jik processing time and C jik completion time. The objective function is defined as to minimize the makespan, which is formulated in Equation 1.
The constraints of the problem are as Equations 2, 3, 4 and 5. According to the start time constraint, each operation Universal Journal of Applied Mathematics 6(1): 1-16, 2018 3 can start at a positive time that is greater than its release time.
As stated in the assumption of the problem, according to the sequence constraint, the operations of each job must be processed in a certain order. So as seen in Equation 3, if operation O ij and O i(j−1) are assigned to machines l and k respectively, then the difference between their start times must be at least as much as P i(j−1)k that is the processing time of O i(j−1) on machine k.
Due to that each machine can process at most one operation at any time and interruption of the processes is forbidden, if an operation is not in the first order on the assigned machine, its start time must be after the finish time of the previous operation on the machine. So if operation O ij starts before operation O pq on machine k then the related resource constraint is Equation 4.
But if operation O ij starts on machine k after the completion time of operation O pq then the resource constraint is like Equation 5.
Only one of Equations 4 and 5 are valid at a time based on assignments of the operations [14][34] [42].
There are several test instances for the FJSSP like Behnke and Geiger [5], Brandimarte [7], Hurink et al. [19], Dauzère-Pérès and Paulli [10], Chambers and Barnes [9], Kacem et al. [20], Fattahi et al. [13] instances. If all machines of a problem can process all of the operations, that is a FJSSP with total flexibility. In the partial FJSSP, each operation can be processed on a subset of the machines set [20] [44]. Partial FJSSPs are more difficult than the Total FJSSP [4].

Neural Networks Application for Solving Scheduling Problems in the Literature
One the most applied neural networks is the Hopfield neural network (HNN). After the first use of this method for an optimization problem by Hopfield, its hybrid types have been successfully applied for many of the problems in the communication, the pattern recognition and the combinatorial optimization. The analog to the digital conversation problem, the traveling salesman problem (TSP), the optimal process plan selection, the terminal assignment problem (TAP), the frequency assignment problem (FAP), the assignment of cells to switches in a cellular mobile network, the scheduling problems and the linear and non-linear programming problems are some examples of these problems [18] [47]. The HNN is a dynamic and recurrent network in which each state is dependent on the previous state. The Boltzman machine, which can be considered as the stochastic type of the HNN, is very suitable for solving the scheduling problems. For instance, Fnaiech et al. have proposed a HNN for the production scheduling problem with taking account the different random maintenance periods [14]. Wang and Yang have summarized these NNs deficiencies as they are generally non-adaptive networks, with the predefined biases and the weights of the connection. Generally, the NNs cannot convergence to the valid solutions and don't locate in the global minimum and have poor scaling properties due to the use of the quadratic energy functions [41] [47]. There are some studies about defining more proper energy functions. Foo et al. have presented a modified Tank-Hopfield linear programming network for solving mixed integer linear programming. They have defined a linear energy function that is more practical than the quadratic energy function. However, their algorithm needs more processing elements [15]. Simon and Takefuji have proposed an integer linear programming neural network (JLPNN) based on the Tank and HNN modifications. They have reported that there are several integer linear programming models for the resource scheduling problems but they are impractical in general. So they have presented a special structure of the integer linear programming as NN that seeks to minimize start Real-life scheduling problems of the production systems about the staff, resource and the maintenance process are generally stochastic and dynamic problems with many constraints [1][17] [29]. So artificial learning methods application is a useful approach to solve such problems. Even notwithstanding the shortcomings of the NNs in the combinatorial problems, using them combined with the other optimization methods causes to good results. Ming and Mak have presented a hybrid HNN combined with a genetic algorithm (GA) that achieves quality solutions. The energy function (or the Lyapunov function) of this HNN is specially designed for solving their problem which is the process plan selection problem. The HNN generates initials solutions and the GA search for the better solutions, hereby the algorithm achieves near global optimal results [24]. Salcedo-Sanz and Yao have presented a hybrid HNN to solve the terminal assignment (TA) and cells to switches in a cellular mobile network problem. The HNN controls the constraints of the problem and the GA searches for better solutions [31] [32]. In a similar study, Sancho Salcedo-Sanz et al. have suggested a hybrid HNN combined with a simulated annealing algorithm (HopSA), for the frequency assignment problem (FAP) for the satellite communication systems [30]. Tang and Zhang have proposed a NN combined with heuristics for the flow shop scheduling problem with sequence dependent setup times [37]. Yu and Liang have suggested a NN and GA based hybrid algorithm for the expanded job shop scheduling problem (EJSSP), in which the GA optimizes the sequences and the NN adjusts the start times [49].  [50]. The optimal schedule is an active schedule but not necessary a non-delay schedule [45].  Figure 1. Classification of the schedules [27] Many algorithms use the GT algorithm for solving the scheduling problems, in one or more stages as the initial solution creation or as a part of a meta-heuristic or heuristic methods. It is a systematic recursive procedure to generate all or some of the active schedules. With some modifications, this algorithm can construct the non-delay schedules. It isn't a complex procedure, while it is a useful method, because it restricts the search space effectively to the active or the non-delay schedules spaces [27].
NNs are also used for improving the neighborhood search methods of the scheduling problems [51]. In the graph theory, the neighborhoods and the local search are related to the critical operations, the critical path and the critical blocks. The longest path from the first operation to the last one is called the critical path and the operations on the critical path are the critical operation. Makespan is equal to the length of the critical path. It is possible to decompose the critical path Universal Journal of Applied Mathematics 6(1): 1-16, 2018 5 into blocks, that is the maximal sequence of adjacent critical operations on the same machine [16][25] [40]. Several neighborhood structures are defined for the scheduling problems [48]. For example, in N 1 structure, one machine is chosen and all possible swaps for the jobs that are assigned to this machine are applied. In N 2 , two machines are chosen and all possible swaps of the jobs from these different machines are considered. In N 3 , randomly a machine and a job that is not assigned to it are chosen. Then the selected job is transferred to a valid position on the chosen machine [3][11] [28]. In the N 4 neighborhood structure, all operations of the critical block are moved to the very beginning or very end of the block. Nowicki and Smutnicki have proposed the N 5 neighborhood structure that is smaller than the N 1 and N 4 , which searches for new neighbors on the borders of the blocks. The N 6 neighborhood structure switches a schedule to another schedule with interchanging two operations, which are on the same block of the critical path [6][11] [25].

The Proposed Method
The general steps of the proposed method are summarized in Algorithm 1, where at first, an initial solution is generated, then it is transformed into a semi-active schedule by applying local left shifts. Then 6 neighbors of this solution are generated according to Algorithm 2. The changes are made based on the start time of the operations instead of the order of them, then the NN runs to check and adjust infeasible neighbor. The structure of the applied NN is similar to the NNs in the literature for the scheduling problems [45]. It has the units which stand for the start time, the sequence and the resource constraints of the operations. Each unit has an input (I), an activation function (AC) and an output (O). The start time neurons are as much as the number of operations, which is equal to n × m. The bias value of these neurons is zero. The start times of the operations in the random solution are the input values of these neurons. The activation function is like Figure 2 and Equation 6 that is to prevent the start times less than the release times of the operations. The output of this activation function is a feasible value for the operations start time.
There is a priority relationship between the operations of each job. For example, as seen in Figure 3, if O 8 and O 7 are from the same job, Equation 7 must be valid, which means that the start time of O 8 must be greater than the start time of O 7 at least as much as the processing time of O 7 on the assigned machine. Because according to the sequence constraint, the start time of O 8 must be after the finish time of O 7 . The connections that are black in Figure 3, and their weights (+1 or -1 ) represent the coefficient of the start times in the sequence constraint that is as Equation 7.
The activation function of the sequence constraint unit is as Figure 6 and Equation 10. If there is a violated sequence constraint, the value of the corresponding neuron is negative and the constraint is added to the list of the violated constraints. The total number of the sequence constraints equals to nm − n.  According to the resource constraint that specifies the order of operations on the machines, a machine cannot process more than one job at any time. Therefore, only after that an operation is finished, the other operation can start. The number of these constraints is nm − m. Suppose that O 2 and O 6 are sideways on machine m. If O 6 precedences O 2 , then the relation between the neuron of the corresponding constraint is as Figure 4 and Equation 8. . By applying the activation function that is as in Figure 5 and The neurons corresponding to the violating sequence constraints must be adjusted using the feedback weights seen in Figure 7 as the black arrows. This means that the start time of the corresponding operations decreases or increases, and the operations are shifted to the left or the right on the time axis. If P and V stand for the value of the penalty and violation, the adjustment value is P × V , as it is in Equation 11, which is done with taking the start time constraints into account, so these constraints will not be violated during this process. In this way, the start times of the violated constraints at the next iteration (t+1) are formed based on their values in the current iteration (t).
After a similar adjustment process for the neurons related to the resource constraints, the NN passes to the next iteration. At the end of each iteration, the resource constraints are updated because it is possible to change the order of the operations on the machines. Also, new violations may occur during the adjustment process. So it is necessary to control all of the constraints unless the start time constraints, which are not violated. The iteration number of the NN is defined as 2500, but when a feasible solution is found, the NN stops and similar steps are repeated for the next neighbor (N i+1 ). Also, if there isn't any feasible result for N i after the last iteration, the algorithm passes to N i+1 . However, the NN generally converges before than 100 iterations. When these processes are finished for the last neighbor (N 6 ), the algorithm totally stops and the best neighbor is selected between the obtained ones. The general steps of the NN are described in Algorithms 3.

Illustrative example
Suppose that a random solution is created as it is in Figure 9 for the problem that is summarized in Figure 8. The first row shows the order of the operations while the second row shows the assigned machines.  The corresponding schedule for this solution is shown in Figure 10, which is not a semi-active schedule and contains some gaps. With applying some local left shifts, the random solution transforms into the schedule seen in Figure 11. The black line shows the critical path. The corresponding NN to the random solution is as in Figure 12.
Universal Journal of Applied Mathematics 6(1): 1-16, 2018 9 Figure 12. Corresponding NN to the random solution As a simple move, as in Figure 13, the start time of O 5 changes to 1, then due to that the order on machine 2 is changed the structure of the resource constraint neurons (RC) are changed as in Figure 14. In the new state, all of the constraints are checked and it is noticed that the resource constraint between operations O 5 and O 2 is violated. So in the corresponding NN, that is depicted in Figure 14, the output of RC 5,2 is negative.  As seen in Figure 15, in the next iteration, the NN shifts O 2 to the right but in this case the sequence constraint between this operation and O 3 is violated. Figure 15. The NN applies a right shift to O 2 O 3 is shifted to the right and as seen in Figure 16, a feasible neighbor is obtained, so the NN stopes and the algorithm passes to the next constructed neighbor and repeats the previously stated steps. Finally, since the obtained schedule is not semi-active, a left shift is applied that improves the results as shown in Figure 17.

Experimental Study and the Performance of the Suggested Approach
The described method is implemented in MATLAB 2013a on a system with a Pentium dual-core 2.20 GHz processor and 3.00 GB RAM. Two approaches are used to generate the initial solutions. In the first approach, they are created randomly, while the GT algorithm is used to construct them in the second approach. To show the effectiveness of the proposed method, it is compared with the local left shift and the global left shift heuristics based on the MK benchmarks from the literature. In the first approach, 10 random solutions are generated for each of the benchmarks and the results in Table 1 are the average of the results obtained by the algorithms. In the second approach, 10 initial solutions are generated for each of the benchmarks according to the GT algorithm. The same procedures are applied to these solutions and the results in Table 2 are the average achieved results. Brandimarte [7] using the constraint programming techniques has obtained the results which are shown in Table 3. Table 3. Makespans obtained by the constraint programming-based algorithms [5] Instances The constraint programming  mk1  40  mk2  27  mk3  204  mk4  60  mk5  174  mk6  59  mk7  143  mk8  523  mk9  307  mk10  214 As seen in Tables 1 and 2, the proposed approach can generate good neighbors especially if the initial solutions are generated using the GT algorithm. Actually, these results are obtained by applying some changes on the initial solutions and not by a global optimization process. However, a comparison between these results and those are in Table 3 demonstrates that the achieved improvements are significant. Using this method as a part of the global optimization methods can improve the quality of search. Also, if it is applied for solving real-time scheduling problems, it can generate fair results in a little time, where the exact optimization methods application is hard and time-consuming.

Conclusion and Future Works
In this study, a hybrid method is proposed, which generates feasible neighbors for the FJSSP. It applies effective moves to the operations on the critical paths and the critical blocks, then by taking the constraints into account, constructs feasible solutions. It is well-known that effective results cannot be achieved when the neighbors are constructed based on the operations that are not on the critical paths and the critical blocks. Because in a schedule, the operations on the critical path specify the makespan. Also in the multi-objective models, with moving the operations on the critical path, the total workload and maximum workload of the machines also change. Before creating a neighbor for a random solution, Flexible Job Shop Scheduling Problem it should be converted to a semi-active or active schedule with the left shifts. Because it is not possible to find the critical paths and the critical blocks in a schedule that is not semi-active. It is possible to obtain infeasible solutions during the neighbor generation process. Actually, if the construction of the infeasible solutions is prevented, the quality and the diversity of the search may reduce. The NN in this work converts infeasible results into the feasible ones by applying penalties to the violated constraints. During this process, these operations are shifted to the right or to the left. In general, NN will converge sooner if only one of the operations are shifted to the right or to the left, as much as the violation value. The adjustment process may also result in the new violations, which must be adjusted in the next iterations. When the NN becomes stable, it means there are not any violated constraints and the achieved solution is feasible. But if after a certain number of iterations, the NN is not stable, a new neighbor is generated and the search continues. The proposed NN generally converges before 100 iterations. In the scheduling problems, usually, the solutions that are obtained by a NN contain gaps. If there are any gaps in the result, it is necessary to apply the left shifts to reduce the value of makespan.
Another important matter is that as in the global search, without decreasing the diversity, the space of the search should be appropriately restricted in the local search. So it can be a good approach to generate initial solutions and make changes to them using the heuristics such the GT algorithm that their results are near the optimal solutions. Similar to the neighborhood structures, the proposed method transforms a solution into other solutions which are its neighbors. So it cannot be considered as a global search method. That is why it is compared with the left shift heuristics which are based on a similar approach. Actually, this method must be used combined with the global search methods, which improves their performance.
In future works, we plan to design a NN based on the mixed integer programming to solve the dynamic staff scheduling problem.
(The MATLAB codes would be accessed through the corresponding author's email.)