Study of the New Finite Mixture of Weibull Extension Model: Identifiability, Properties and Estimation

Finite mixture models have been used in many fields of statistical analysis such as pattern recognition, clustering and survival analysis, and have been extensively applied in different scientific areas such as marketing, economics, medicine, genetics and social sciences. Introducing mixtures of new generalized lifetime distributions that exhibit important hazard shapes is a major field of research aiming at fitting and analyzing a wider variety of data sets. The main objective of this article is to present a full mathematical study of the properties of the new finite mixture of the three-parameter Weibull extension model, considered as a generalization of the standard Weibull distribution. The new proposed mixture model exhibits a bathtub-shaped hazard rate among other important shapes in reliability applications. We analytically prove the identifiability of the new mixture and investigate its mathematical properties and hazard rate function. Maximum likelihood estimation of the model parameters is considered. The Kolmogrov-Smirnov test statistic is used to fit two famous data sets from mechanical engineering to the proposed model, the Aarset data and the Meeker and Escobar datasets. Results show that the two-component version of the proposed mixture is a superior fit compared to various lifetime distributions, either one-component or two-component lifetime distributions. The new proposed mixture is a significant statistical tool to study lifetime data sets in numerous fields of study.


Introduction
Historically, survival analysis has been studied with the classical statistical distributions such as Gamma, exponential and Weibull distributions among others. Generalizations of such lifetime distributions have been introduced in the literature to fit a wider variety of data sets and to present multiple shapes of useful hazard functions. The Weibull distribution is perhaps the most widely used lifetime distribution model because of its flexibility and simple expressions for the density, survival, and hazard functions. However, it cannot capture the behavior of lifetime data that exhibit non-monotone hazard shapes (such as bathtub, inverse bathtub (unimodal)…etc.), often encountered in reliability engineering studies. A number of distributions were developed as generalizations or modifications of the Weibull distribution. Reference [1] introduced the Inverse generalized Weibull and generalized Inverse generalized Weibull (GIGW) distributions. They investigated the mixture model of two-component GIGW distributions.
Reference [2] introduced the four-parameter new generalized Inverse Weibull distribution. They derived various structural properties and MLEs for the new modified distribution. Reference [3] studied a new generalization of the flexible Weibull distribution with three parameters referred to as the exponentiated flexible Weibull extension. Their new distribution exhibits bathtub shape hazard rate function. Reference [4] proposed a generalized form of the Weibull distribution known as the Fré chet-Weibull distribution by using the T-X family. Reference [5] presented a detailed statistical analysis for their new proposed three-parameter Weibull Inverse Rayleigh distribution.
Our distribution of interest, the Weibull extension model (WEM), presented by [6], is an example of such generalizations that exhibits bathtub shape hazard rate function among other important shapes. The Weibull extension Model (WEM) is considered a generalization of the model studied by [7] by adding a scale parameter in order to enhance its capability to fit diverse lifetime data. The model has been fitted to the failure times of Aarset data and the failure times of a sample of devices in a large system (see, [8]). Reference [9] investigated the estimation of the model parameters based on generalized order statistics.
The probability density function , the cumulative distribution function , survival and hazard functions of the three-parameter Weibull extension model are given by, (see, [5]). where , are shape parameters and is a scale parameter. For analysis purpose, the product is referred to as the parameter .
The shape of the hazard rate depends only on the shape parameter . For ≥ 1, ℎ( ) → ∞ as → ∞.
The change point * is directly proportional to and inversely proportional to .
We reproduced the proof of the hazard shapes of the WEM by applying Glaser's theory (see, [10]) in Appendix A. Survival mixture models have been widely applied as an efficient statistical tool for modeling and describing population heterogeneity in reliability, life testing applications, and in many fields of statistical analysis such as pattern recognition, clustering and survival analysis. Notably, no previous work in the literature presented the finite mixture of the Weibull extension model. This article introduces the new −component mixture of Weibull extension distributions. The identifiability of the proposed mixture is proved. No statistical analysis is validated for a new proposed model without the latter being mathematically identifiable. Moreover, the usefulness of the model arises from the flexibility of its failure rate which accommodates increasing, bathtub and modified bathtub among other hazard patterns. These three patterns have been widely accepted in several fields, especially reliability and engineering fields. When fitting our model to two real datasets, it tends to be significantly superior to all competing distributions either one-component or two-components. It outstands the model by [7], this highlights the importance of adding scale parameters in the mixture case and that the two-component WEM mixture is a better representation for both datasets. In addition to the detailed mathematical presentation of the new mixture, identifiability and properties, we worked out a new general analytical form for the first derivative of the function ( ) in the mixture case. This form compares to the first derivative of the hazard rate function but is more tractable and useful to the analysis of the hazard shapes for more complex models. This proof in presented in Appendix B.
Following this introductory section, Section 2 presents the definition of the −component mixture of WEM, its density, survival and hazard functions whereas the two-component mixture is presented in details with the shapes of the density function at various combinations of parameters. Section 3 presents the full analytical proof of the identifiability of the proposed mixture. Section 4 gives a detailed presentation of the mixture mathematical properties and a full study of its hazard function. Section 5 considers the maximum likelihood estimation of the model parameters. Finally, Section 6 gives an analysis and fit of two important datasets in reliability analysis, the Aarset data and the Meeker and Escobar data. A detailed comparison of fitting the data using the proposed model to several lifetime models is given as well.

Finite Mixture of the Weibull Extension Model (FMWEM)
A random variable follows a finite mixture of WEM (FMWEM) distribution if it has the following , where, for any ∈ {1, … … }, ( | ) is the pdf of the -th component defined by (1.1) with , and replaced by , and respectively, is the mixing proportion of the -th component with , is the vector of parameters of ( | ) and Ψ = ( 1 , … … . , ; 1 , … … . . , ) is the whole parameters' vector. The , reliability function, hazard function of the FMWEM are given by, where ( | ) and ( | ) are the cdf and the reliability function of the -th component defined by (1.2) and (1.3) respectively.
The hazard rate function is the ratio of the density function to its survival function (see, [11]). The failure rate function of the -component mixture of a WEM is given by,  [12]). Note that: Figures legend is described as follows: "pdf" refers to the mixture density, "pdf1" refers to the first component density and "pdf2" refers to the second component density. Let = , Figure 1 represents the density functions for ≥ 1, 0 < < 1. Figure 2 represents the density functions for ≥ 1, > 1 . Figure 3 represents the density functions for < 1, > 1. Figure 4 represents the density functions for = 1, < 1 and ≥ 1 . Figure 5 represents the density functions for < 1, 0 < < 1. Finally, Figure 6 displays the density functions for the case where for one component is >1 whereas < 1 for the other component. From figures 1-6, the following is observed.  As expected, the shape of the mixture is closer to the shape of the of the component associated with the larger mixing proportion.  The of the two-component mixture model is very flexible. By considering various choices of (0 < < 1, = 1, or > 1) and (< 1 or ≥ 1), the mixture model can take strictly decreasing, unimodal, bimodal, and decreasing followed by unimodal shapes among other shapes.
In summary, the shapes of the new mixture density at a wide range of parameters' values indicate its capability to represent multiple datasets in diverse scientific areas.

Identifiability of the Model
For a new proposed mixture model, a researcher can meaningfully discuss the model mathematical properties, estimation of parameters, hypotheses testing about parameters only if the family of mixing distributions is said to be identifiable. Reference [13] stated and proved the following theorem: Theorem: Let be the set of all finite mixing distributions related to a class Φ of probability distributions for = 1,2, … , , Suppose for each ∈ Φ there will be associated a transform having domain of definition , and suppose that the mapping : → is linear. Suppose there exists a total ordering (≤) of Φ such that (i).
For each 1 ∈ Φ, there exists some 1 in the closure Then the class of all finite mixing distributions is identifiable relative to Φ.
Applying Chandra's approach, we proved the following proposition. Proposition: The class of all finite mixing distributions relative to the WE distribution is identifiable.
Proof: Let be a random variable having pdf and cdf of the WEM given in (1.1) and (1.2), respectively. Then the ℎ moment of the WEM is given by The lower limit 1 yields a divergent integral, we must have { > −1} for the integral to converge; i.e.
Thus lim

Properties of the FMWEM
A detailed statistical analysis of the WEM was given in [12]. Reference [14] referred to the WEM as the modified Weibull distribution. He derived explicit algebraic formulas for the ℎ moment (about 0) of the distribution with 1 ⁄ being a non-negative integer. The following presents the mathematical properties of our proposed FMWEM.
Moment Generating Function, Mean, Mode and Median, Variance, Skewness, Kurtosis Let be a random variable having and of the WEM given in (1.1) and (1.2), respectively, The moment generating function of the FMWEM is given by where ( ), the ℎ moment of the FMWEM is given by where ( ), the ℎ moment of the WEM is given by (see, [12]) Using the transformation = exp ( ) , where = .
The mean of the FMWEM is given by, = 2 in the two-component case and (4.6) The mode (modes) of the FMWEM is (are) obtained by differentiating the pdf of the WEM and by solving the following non-linear equation, where = 2 in the two-component case. The quantile function of the FMWEM is derived by finding the value of for which ( ) = where is the probability of the quantile . The value of the median is the value of for which = 0.5. The variance of a two-component mixture is defined as follows, (see, [15]) ( ) = 1 1 2 + 2 2 2 + 1 2 ( ′ 11 − ′ 12 ) 2 . (4.9) where 1 2 , 2 2 are the variances of the two components respectively, ′ 11 , ′ 12 are the means of the two components respectively.
The Fisher skewness of the two-component mixture of the WEM is given by, where 3 is the third central moment of the two-component mixture and is given by where ′ 3 is the 3 non-central moment of the two-component mixture.
The Fisher kurtosis of the two-component mixture of the WEM is given by, where 4 is the fourth central moment of the two-component mixture and is given by is the 4 ℎ non-central moment of the two-component mixture.

Limits of the Hazard Rate Function
The hazard rate function of the two-component WEM satisfies the following limits.

Shapes of the Hazard Rate Function
It is well known that mixtures of decreasing failure rate (DFR) distributions are always DFR (see, [16]). On the other hand, mixtures of increasing failure rate (IFR) distributions can decrease, at least in some intervals of time. Note that IFR distributions are often used to model lifetimes governed by ageing processes, which means that the operation of mixing can dramatically change the pattern of population ageing, e.g., from positive ageing (IFR) to negative ageing (DFR), (see, [17])). Therefore, the study of the shape of the observed (mixture) failure rate in a heterogeneous setting is important in many applications (reliability, demography, risk analysis, etc.). It is worth noting that the shape of the failure rate function (and the mixture failure rate as well) is often based on the Glaser's approach ( [10]), which states that the shape of the failure rate is mainly affected by the shape of the following function: which is easier to handle in many cases than the shape of the corresponding failure rate. For instance, when ( ) is increasing (decreasing) the failure rate is also increasing (decreasing). We reproduced the derivation of the shapes of the hazard function of the WEM in terms of the functions ( , ) and ′ ( , ) in Appendix (A). In the mixture case, reference [16] showed that  Note that ( , ), = 1,2, is defined as ( , ) with , , being replaced by , , respectively. Using The proof of (4.18) is given in Appendix (B). An analogous expression for the derivative of the hazard rate in terms of the derivatives of the components 'hazards was given, for example, by [18] and was used by several authors for studying and interpreting the shapes of the hazard rate. Equivalently, this can be done applying(4.18).
Figures 7-10 display multiple shapes of the two-component mixture hazard rate function along with the hazard function of the individual components at various combinations of the parameters. Figure 7 represents the hazard rate functions for 1 = 2 = 1. Figure 8 represents the hazard rate functions for > 1. Figure 9 represents the hazard rate functions for 0 < < 1. Finally, Figure 10 displays the hazard rate functions for the case where for one component is >1 whereas < 1 for the other component. In all figures of the hazard functions, the parameter is taken to be greater than 1 or less than 1.
Based on the limits proved of the two-component mixture hazard rate function and figures 7-10, the following comments are valid.  Based on the proved limits of the hazard function, the latter cannot decrease or take an inverse bathtub (unimodal) shape. However, it can still decrease or have a unimodal shape for a certain interval of time.
(see, figure (10))  The mixture hazard rate tends with time to the hazard of the component associated with the smaller value for the shape parameter . This can be explained by noting that the hazard rate of the mixture distribution always tends to the hazard rate of the stronger population (the component having the smaller hazard rate). It can be shown that the stronger population is the one having the smaller value of (when → ∞) holding the other parameter fixed.  A mixture hazard function having one component with an increasing hazard rate ( ≥ 1 ) and the second component having a bathtub shaped hazard rate ( < 1) may have a modified bathtub shape (increasing followed by bathtub shaped hazard) as in figure (10) for all intervals of .
In short, the graphs of the hazard rate illustrate the flexibility of the proposed mixture to describe different failure patterns, namely increasing, bathtub, modified bathtub and decreasing-increasing-decreasing-increasing shapes. Figures (7-10). Hazard Rate Function of the two-component mixture of WEM for different values of parameters ( , 1 , 2 , 1 , 2 , 1 , 2 ).
Note that: Figures legend id described as follows "h.r.m" refers to the mixture hazard function, "h.r.1" refers to the first component hazard function and "h.r.2" refers to the second component hazard function. In industrial reliability studies of repair, replacement and other maintenance strategies, the MRL function may be more relevant than the hazard rate function. The hazard rate function relates only to the risk of immediate failure whereas, the MRL function summarizes the entire residual life distribution (see, [19]).
The MRL function of the -component mixture of a WEM is given by,

Parameter Estimation
Given a sample of independent observations from a mixture model defined by (2.1), the complete-data likelihood function is We can obtain the partial derivatives involved in the normal equations as follows, The normal equations do not have explicit solutions and are solved simultaneously using numerical methods.

Data Analysis
This section illustrates the flexibility of the proposed two-component mixture of WEM for fitting and representing various patterns of data. It includes the analysis and fit of two famous datasets in reliability analysis, the Aarset ( [21]) and the Meeker and Escobar ( [22]) datasets (we will refer to the latter as MeekE data). A preliminary analysis of both datasets was carried out by drawing the total time on test (TTT) plot to measure the variability and the hazard pattern of the data, see figures (11(a)) and (11(b)). Based on the TTT plot theory, the shape of the failure rate curve of the data uniquely determines the shape of the empirical TTT plot and vice versa. Thus, the TTT plot indicates whether a model with a monotonic, unimodal, or bathtub-shaped hazard rate is suitable for the data, (see, [23]). Reference [21] showed that if the curve approaches a straight diagonal function, a constant hazard rate is adequate. Whereas when the curve is convex or concave, the hazard rate is monotonically decreasing or increasing respectively and when the curve is convex then concave a bathtub shaped failure rate is accurate. Otherwise, a unimodal hazard rate function is more adequate. The scaled TTT transform ( ) of a distribution with is given by, The empirical TTT plot of both the Aarset and the MeekE datasets indicates that a bathtub shaped hazard rate function is more adequate to fit the data. Hence the two-component MWEM is an appropriate model in both cases. Table (3) summarizes the fit results of both data sets to our two-component WEM mixture, including parameters 'estimates, K-S values, p-values and estimated log-likelihood functions. Table (4) displays the results of fitting multiple lifetime distributions to the Aarset data. The competitive distributions are the additive Weibull (AddW) in [24], the WEM in [6], the modified Weibull in [25], the exponential flexible Weibull Extension (EFWE) distribution in [3], the two-component mixtures of three-parameter Weibull (M3pW) (see, [11]), exponentiated Weibull (MEW) (common scale parameters in both components) (see, [26]), Burr type III (MBurrIII) (see, [27]) and WEM are fitted as well. A reduced version of MWEM is the two-component mixture of the Chen model when the scale parameter = 1 for both components (MWEMR1).
Results in Table (3) assure the competence of the proposed two-component mixture of WEM to best fit both datasets in mechanical engineering. The descriptive statistics of the datasets (see, Table (2)) reveal heterogeneity and that it can be better represented by a mixture model with a hazard rate that exhibits bathtub shape. This seems evident when comparing the results of the one component distributions fitted with the results of our proposed mixture model. It is also noticed that the two-component mixture of Chen model, which is a reduced model of MWEM does not fit the data which shows the importance of including the scale parameters 1 and 2 .
Table (5) shows the results of fitting multiple lifetime distributions to MeekE data. The one-component distributions fitted to the data are the additive Weibull (AddW), the WEM, the modified Weibull and the new modified Weibull (NMW) in [28]. We added the four-parameter Fré chet-Weibull (F-W) by [4] in our fit of the MeekE data. We tried to fit the latter to the Aarset data as well but estimation results were not good. Trying to fit the mixture model of F-W to the MeekE data was not a good choice based on the results of its one-component version.
The two-component mixtures of the Chen model and WEM are fitted as well. Although the six models fitted the data but the WEM mixture has the lowest K-S value and highest p-value.

Conclusion
This paper introduces a new finite mixture of a generalized form of the standard Weibull distribution, the Weibull Extension model introduced by [6]. The identifiability of the new mixture is analytically proved. For mathematical robustness of the study, the identifiability condition on the shape parameters is considered in all parameter choices and analysis. All the mathematical properties of the model are investigated. Maximum likelihood estimation of the new mixture is studied. Various combinations of the model parameters are applied to demonstrate diversity of the shapes of the density function and the hazard function. The new mixture exhibits increasing, bathtub and modified bathtub failure rates which makes the new lifetime model useful and flexible in representing many life data in practice. The two-component version of the new mixture is fitted to the Aarset and the Meeker and Escobar datasets. The model is the best fit, for both datasets, compared to many lifetime distributions, either one component or mixture distributions. In conclusion, the new FMWEM is identifiable and has tractable mathematical properties. The diversity of the density, hazard shapes and the results of fit assures the competence of our proposed mixture and usefulness in many scientific areas and applications.

APPENDIX A:
Derivation of the shapes of the hazard rate of the WEM in terms of ( , ) and ′ ( , ) We applied Glaser's theorem ( [10]) to the one-component Weibull extension model using the exponential family density to prove the shapes of the hazard function. Evidently, if ≥ , the derivative ′ ( , ) will be greater than or equal to 0; as the first two positive terms in the previous equation dominate the 3 negative term. This implies an increasing failure rate (IFR) for the one component WEM. Hence, at a certain value 0 , such that 1 − 1 < 0 < 1 , ′ ( , ) = 0.
Noting that lim →0 ( ) = ∞, the hazard rate function for the one component WEM has a bathtub shape (BT) for 0 < < 1.

APPENDIX B:
Derivation of ( . ) In the mixture case, recall that  The first derivative of the function ( , ) in the mixture case, given in (4.18), represents an alternative to the first derivative of the hazard function ℎ( ). This assures the ability of the function ( , ) to interpret shapes of the failure rate function.