Markov Chain: First Step towards Heat Wave Analysis in Malaysia

Extreme temperature has been carried out around the world to provide awareness and proper opportunity for the societies to prepare necessary arrangements. In this present paper, the first order Markov chain model was applied to estimate the probability of extreme temperature based on the heat wave scales provided by the Malaysian Meteorological Department. In this study, the 24-year period (1994-2017) daily maximum temperature data for 17 meteorological stations in Malaysia was assigned to the four heat wave scales which are monitoring, alert level, heat wave and emergency. The analysis result indicated that most of the stations had three categories of heat wave scales. Only Chuping station had four categories while Bayan Lepas, Kuala Terengganu, Kota Bharu and Kota Kinabalu stations had two categories. The limiting probabilities obtained at each station showed a similar trend which the highest proportion of daily maximum temperature occurred in the scale of monitoring and followed by the alert level. This trend is apparent when the daily maximum temperature data revealed that Malaysia is experiencing two consecutive days of temperature below 35 ̊C.


Introduction
One of the most important climate parameters which affect natural and social phenomena is the temperature. The temperature that has exceeded the average temperature for a given area is considered as a heat wave. Recently, on February 2019, the Malaysian Meteorological Department (MMD) issued a level 1 alert for ten areas in the country amidst the ongoing nationwide heat wave. The ten areas are located in the western region of Peninsular Malaysia including Perlis, Kedah, Perak, Kuala Lumpur and Johor [1]. The level 1 alert was issued as the daily maximum temperature exceeding 35˚C for three consecutive days.

Background of Study
Stochastic processes are progressively being used for modeling and predicting climate data. The arising need of using this approach in climate and weather models is caused by the inability to resolve all necessary processes and scales in comprehensive numerical weather and climate prediction models [2]. One of the famous stochastic models applied to evaluate climate data is Markov chain. The Markov chain model is a commonly used tool for simulating time series of discrete random variables [3]. This model is considered as a memoryless process due to the probability of transitioning from one state to another depends only on the current state and not on the past. The detailed theory and structure of Markov chain can be referred to in some textbooks stochastic processes [4,5]. Many researchers around the world have applied Markov chain model to analyze the climate data. In Malaysia, several similar analyses also have been attempted by the researchers to model the daily observation of rainfall [6], wind speed [7] and maximum temperature [8]. In the year 2015, Hasan [8] proposes the application of Markov chain onto daily maximum temperature data. The study covers only the northern region of Peninsula Malaysia. A scale for the state of transition was determined by using the physiological equivalent temperature (PET). The same scale was applied by Hassan and Hasan [9] in the year 2017 to determine the steady-state probability for the daily maximum temperature across Peninsular Malaysia. This study is the extension of these two works. The main purpose of the current work is to describe the heat wave condition in Malaysia by applying Markov chain model onto the daily maximum temperature data.

Data Characteristics
The data used in this study are daily maximum temperature measured in degree Celsius (˚C) from 17 meteorological stations across Malaysia. These data are provided by Malaysian Meteorological Department (MMD) with less than 2% missing data. Each station has at least 24 years of daily data except KLIA station (19 years). The location of each station is illustrated in Figure 1

Heat Wave Scales
The daily maximum temperature model based on a Markov chain describes the temperature changes by following the heat wave scales. Four levels of heat wave scale provided by the Malaysian Meteorological Department are used in this study. The daily temperature data in this study is assigned to its heat wave scales. The descriptive terms for each scale together with the range of daily maximum temperature are listed in Table 1.

A Markov Chain Model for Daily Maximum Temperature
There are two properties need to be distinguished when applying Markov chain model to the climate data. The first property is the "state", defined as the number of different values that the variable can have and the second property is the "order", described as the number of previous values used to determine the state-to-state transition probabilities [3]. In this study, the Markov chain model applied on the daily maximum temperature is the first order model with four states which can be described as follows. Let t X be defined as the state of temperature at th t day where t Xi = , 1,2,3,4 i = 1. if day t is at state of monitoring, 2. if day t is at state of alert level, 3. if day t is at state of heat wave, 4. if day t is at state of emergency.
Assuming that the state of temperature is dependent on the condition of the previous day, then t X follows the first-order Markov chain. As an example, the below equations express the conditional probabilities of the temperature at state j on day t depending on the temperature at state i on day 1 t − . The transition probabilities estimated from the historic measurements, signify the probabilities of temperature from state i to state j . In general, the transition probability from state i into state j can be written as

Development of Transition Probability Matrix
To obtain the transition probability matrix P , the data with the previous state i and current state j are first counted. Then, the counted values, ij m will be sorted and placed into a transition count matrix M . Since there are four states of transition used in this study, the transition of daily maximum temperature can be divided into the 16 cases which can constitute the following count matrix M . Define

Development of Limiting State Probabilities
The n-step transition probabilities can be defined as follows:     represent the mean occurrence probabilities of daily maximum temperature will be in each state, namely monitoring, alert level, heat wave and emergency state respectively after a sufficiently long time. These probabilities are also called the limiting or stationary state probabilities. The limiting state probability can be obtained by first, multiplying the transition probability matrix by itself. Then the resulting matrix is iterated until all the transition probability matrices reach the equilibrium [8].

Analyzing Markov Chain Using R Software
R is a free software environment used for statistical computing and graphics. R comes with a standard set of packages and other packages that can be downloaded and installed. These packages contain R functions, data and compiled code in a well-defined format. In this study, the package named 'markovchain' has been used to manage discrete time Markov chain more easily. It was developed and maintained by Giorgio Alfredo Spedicato and published on 21 st January 2019.
An individual transition count matrix for each station can be created using a createsequenceMatrix function while the transition probability matrices can be computed using markovchainFit function. Using a plot function, a better understanding of the Markov Chain model can be gained through a transition diagram. This transition diagram is a graphical representation of a Markov chain which is equivalent to its transition probability matrix [10]. Furthermore, a steady-state distribution or a limiting state probability can also be computed using steadyStates function.

Descriptive Statistics
The descriptive statistics consisting of mean, maximum and minimum values (˚C) of the daily maximum temperature from the selected metrological stations are presented in Table 2. The mean values of the observed data range from 31.30˚C to 32.94˚C. The highest value of maximum temperature is recorded at Chuping (40.10˚C) followed by Alor Setar (39.10˚C) station. It is interesting to note that both stations are located in the northern region of Peninsular Malaysia.

Count and Transition Probability Matrices
The number of categories indicates the dimension of their respective count and probability matrices. To obtain the count matrix, the transition between two consecutive days is tallied and totaled. We notice that a large number of days are accumulated in cell 11 m compared to the other cells. This common trend explains that all of the stations tend to experience two consecutive days at the state of monitoring. The second highest counted values are observed at cell 12 m (the transition of temperature from state monitoring to alert level) for most of the station and at cell 22 m (the temperature stays at alert level state for two consecutive days) for Chuping and Alor Setar stations. As explained in the methodology section, the transition probability matrix, presented in Figure 2, can be computed from the count matrix. The result evaluated from the transition probability matrix shows that the daily maximum temperature has the highest probability (p > 0.900) to remain at the state of monitoring for two consecutive days.
Furthermore, to obtain a better understanding of transition probability for each of the station, the transition diagram, as shown in Figure 3, is plotted. Having obtained both transition probability and transition diagram, the behavior of the daily maximum temperature can be assessed and the existence of heat wave at the studied location can be determined. Referring to the transition diagram, all states communicate with each other. Thus, this Markov chain model is irreducible. The transition of temperature from the monitoring or alert level to heat wave state is probably happening at Chuping, Alor Setar, Subang, KLIA, Seremban, Melaka, Mersing, Senai, Muadzam Shah, Kuantan and Kuching stations as their transition probability value, 13 0.000 p  or 23 0.000 To be considered as heat wave, the temperatures have to be outside the historical averages for a given area for at least two or more days [11]. As mentioned above, the daily maximum temperature in Malaysia falls into heat wave category (Scale = 3) when the temperature is above 37˚C. Based on the transition probability and transition diagram, several stations located in the western region of Peninsular

Limiting State Probabilities
The limiting state probabilities can be interpreted as the probabilities that remain unchanged in the Markov chain as time progresses. These probabilities are computed by multiplication of the transition probability with itself until the stationary distribution is attained. After a sufficiently long time, the limiting state probabilities are achieved when each row of the transition probability has the same probability value.

Conclusions
In this paper, the behavior of daily maximum temperature data from 17 meteorological stations across Malaysia is assessed through Markov chain modeling. Previous studies have used physiological equivalent temperature (PET) scale to determine the state of transition. For the data analyzed here, the heat wave indications; monitoring, alert level, heat wave and emergency are used to categorize the data. The data classification result shows that the data can be categorized into four states of transition; monitoring, alert level, heat wave and emergency. The majority of the stations have three states of transition; monitoring, alert level and heat wave except for Bayan Lepas, Sitiawan, Kota Bharu, Kuala Terengganu, Labuan and Kota Kinabalu stations which have two states and Chuping possesses all four states.
It is not surprising to note that the highest transition probability value for each station is to remain at the state of monitoring for two consecutive days. Only five stations located in the western region of Peninsular Malaysia experience state of heat wave for two consecutive days. Investigation of the limiting state probabilities shows the decreasing trend from the state of monitoring towards the emergency state. After a sufficiently long time, all the stations have the highest proportion of time to experience monitoring state (temperature below 35˚C). This paper is limited to only investigate daily maximum temperature data. A more complete study of heat wave might consider other variables including daily minimum temperature or measure of heat stress.