Organizational Levels Model for Measuring the Effectiveness of Enterprise Resource Planning System (Case Study TUGA Company, Iran)

Enterprise Resource Planning systems (ERPs)are described as computer-based information systems designed to process an organization’s transactions and facilitate integrated and real-time planning, production, and customer response [23]. These systems are designed to address the problem of fragmentation Implementation of ERPs. ERP systems are complex and costly processes. In spite of ERP systems have provided many tangible and intangible advantages and capabilities, some of them result in serious failures. Many studies and research projects have been conducted in identifying ERP Critical Success Factor and challenges in ERP implementation projects. Still, there are a little studies which investigated on the effective implementation ERP systems. In this research, a model is developed from the strategy and objectives hierarchy in corresponding with organizational levels. While doing this research, for participation, cooperation and support in developing this model by TUGA Company, we’ve named this model TUGA. The results of this study have provided a very useful reference and framwork for scholars and managers to measure the effectiveness of ERP system.


Introduction
Many companies were increasingly interested in the ERP systems because of some evolution in such systems. Furthermore, these systems cost a lot around the world [16].Many businesses have adopted ERP as a tool to achieve strategic competitive advantages [22]. ERP System is a significant device to help change the enterprise business and obtain a sustainable competitive advantage over the competitors. Implementing an ERP can facilitate the decision-making for organizations; moreover it makes the services given to the customers and the formats of data better.
ERP is the state-of-the-art information system which can enhance the function of an organization and the competition spirit by omission of duplication in data and work and improving the business process. There are many reasons for managers to utilize ERP, for example the improvement in productivity, reduction in costs, more effectiveness in operation, the better control on customer relationship and greater chain management [2].
Using ERP has three main benefits which are the integrity it can create in the business process, access to the common data and the process of business in the organization, and data production in real time [11].
It's so expensive to adopt and implement ERP system because it needs so much energy and exact analysis of influential factors in utilizing such systems.
The advantage of ERP system is that it can make it possible for an organization to unite its entire business works to improve the effectiveness and competitiveness spirit [9].We can't expect any impending improvement in productivity and competitiveness spirit unless we implement the ERP system in an organized way [20].
Therefore, enterprises must continuously measure the efficiency of the ERP system from different aspects in order to ensure that it can obtain expected advantages of ERP system implemented in organization [3] The ERP system is aimed to assess, design and perform the projects better and facilitate the decision-making by managers in organization. As a result, ERP and other information systems are different in some aspects like scale, domain, complexity, organizational changes, the cost of projects and requirement for reengineering of business process [17]. Dezdar [7] states that there are so many aspects about the implementing the ERP system successfully. However, the popular aspects are on the basis of two variables which are the satisfaction of user and the impact of organization, the former of which is influential in the information system success. Moreover, it's an alternative for measuring the IS success in environment where computing matters. He also says thata system without user satisfaction is less likely to be used and to produce beneficial results to a user community and the organization. Dezdar [7].believes that the usefulness and utility of an ERP system can be measured based on the user's attitude and feedback. Three reasons represented by DeLone about the satisfaction of users have been criteria for measuring the IS systems. These reasons are high degree of face validity, development of reliable tools for measure, and conceptual weakness and non-access to other measures [4]. Ifinedo said that his studies showed that organizational structure is positively related to the Effectiveness of ERP system (Ifinedo,2006), thus regarding this fact that the structure of every organization has three levels, namely strategy, middle and operational, we have studied the Effectiveness of ERP system considering each of these levels and specific user in a level. Therefore, we are going to divide the organization into three levels of strategy, middle, and operation. Competition criteria strategy level, organization efficiency in management level and effectiveness in operational level are of great significance.

Literature Review
The Effectiveness of information systems has been measured in two main domains of research, namely acceptance and user satisfaction, additionally various models have been developed during the time. These two domains have created two different branches in studying the Effectiveness of information technology. User acceptance branch is introduced by the technology-acceptance model. DeLone and McLean model [4] is the most famous representative in user satisfaction branch.
A few researches have been done relating measurement of information systems although so many studies are done in various domains of ERP systems, so there is not generous review of literature in this domain. Lacking sufficient knowledge, organizations accepting ERP system are not usually aware of the way they can measure the Effectiveness of this system [12,13,15]Majority of researches about the measurement of ERP system have considered a few aspects only and ignored specific characteristics of ERP system. Furthermore, most of systems are developed in a traditional structure of information system, so they are not appropriate in measuring the Effectiveness of ERP system [8]. The main reason for this fact is the specific features of ERP system leading to requiring special and unique factors to measure this system. In general, measuring the Effectiveness of ERP system is challenging due to its complicated nature. Gable's researches [8] are of the most important works having been done in the area of measurement of ERP system. He began studying 27 Australian official organizations utilizing SAP R/3 system in 2001 and examined aspects and criteria which were effective in the Effectiveness of ERP system. His statistical sample was users from all levels. Generally, he recognized 458 criteria and the effect of ERP system, and then he classified them. His goal was obtaining a simple framework having utility beyond its examination. Gable continued his studies based on DeLone and McLean [4] and used the framework of ERP system benefits represented by Shang and Seddon [25].

Figure1. Gable's model of measuring the Effectiveness of ERP system
Ifinedo extended Gable's work in 2006 [12,15]. He said that since the philosophy of ERP systems is fundamentally effective in efficiency and improvement of intersectional function of accepting organizations, furthermore, intersectional functions (cooperation) and cooperation are ranked 3 and 6 respectively in Effectiveness key factors of Akkermans &Helden [1].we should consider these factors too.Ifinedo also added vendor/consultant aspect to Gable's model. He, using data gathered from 16 organizations in Finland and Estonia, stated that organizations accepting ERP systems believe that the role of software providers (consultants and vendors) is related to the thorough Effectiveness of their organizations [12,13,15]. Moreover, literature of information systems supports vendor/consultant quality in all cycles of information systems life very well [5,18,27,26,19]. Ifinedo model [12,15] which is in fact the developed model of Gable is represented.

Research Methodology
Literature review represented the research gap existing to be bridged in area of measuring the Effectiveness of ERP systems focusing on organizational structure and Ifinedo's findings showing that organizational structure is positively related to the Effectiveness of ERP system, in SAP strategic committee of TUGA Company we decided to concentrate on various organizational levels (strategy, middle and operation Universal Journal of Industrial and Business Management 2(2): 25-30, 2014 27 level) in order to measure the Effectiveness of ERP system. The conceptual model of this research is represented in figure3. All criteria for measuring the quality of various information systems and ERP systems which studied before and then they were represented to the SAP strategic committee of TUGA Company including 12 managers, moreover; we interviewed different users in all other organizational levels of TUGA. Doing in such a way, we managed to obtain the criteria which were not given in the gathered ones but were of great importance in measuring the Effectiveness of ERP system in their views. Finally, 53 various criteria for measuring the Effectiveness of ERP system were obtained. Furthermore, to determine the organizational level (strategy, middle, and operation), a questionnaire was prepared and then the effective criteria in the effectiveness of ERP system relating to each organizational were classified based on focus group.

Sampling
Having corresponded with TUGA and other colleague companies, we selected two other companies in which we chose some people based on our criteria used in TUGA. We, then, distributed 279 questionnaires in these three companies (84 questionnaires in strategy level, 96 questionnaires in middle level, and 99 questionnaires in operation level). The goal and methodology of research were explained to people in a session held with the cooperation of their managers in addition to answering their questions. Finally, we gathered questionnaires, so we got 100%of questionnaires distributed.

Measurement and Exploratory Factor Analysis
We utilized Likert scale consisting of seven levels to measure the importance of these criteria in measuring the Effectiveness of ERP system. Number 1 shows low importance, 4 represents moderate importance and 7 shows high importance. Filling in the questionnaires, we pay careful attention to the quality. Because the person in question might make a mistake in understanding that the questionnaire was about measuring the Effectiveness of ERP system for so many times, while he/she had to fill in the advantages based on the importance of studied criterion in measuring the Effectiveness of the ERP system.
To validate the different basic dimensions of data set, we used exploratory factor analysis (EFA). Then we studied the responses by the use of principle-components factor analysis as a tool and varimax as the orthogonal rotation method. Extracting a stable factor structure, we used three regular decision rules to eliminate items, the first of which is eigenvalue less than 1, the second one is loadings of less than 0.35 on all factors and the third rule is loadings greater than 0.35 on two or more factors [24].
While we analyzed factors for each questionnaire respectively, no factor was omitted, moreover; we classified different factors of every questionnaire in a distinct aspect regarding the factor classification resulted from varimax rotation. We then proposed an appropriate title for every aspect considering criterion essence and review of literature. Ultimately, we selected the titles after the SAP committee in TUGA Company confirmed them. The final results of strategic, management and operation indexes are summarized in table 1-3.

Validity and Reliability
To measure the validity of questionnaires, we used some experts in SAP committee of TUGA Company. Furthermore, Cronbach alpha technique was utilized to measure the reliability. The case is acceptable when Cronbach is greater than 0.7. Some people advocate this idea and suggested that the variables of research have to be refuted in the case of Cronbach's value being less than 0.35. Cronbach alpha of every structure aspects is represented in table 4.

KMO Index and Bartlett Test
Doing factor analysis, we must be sure that we can use existing data in order to analyze them. In other words, we should be sure if the data is appropriate for factor analysis. Thus, we utilize KMO index and Bartlett Test. KMO is a sampling index examining the smallness of partial correlation between variables and determine if variables' variance in research is affected by common variance of some latent and basic factors or not. This index ranges from 0-1. If the index amount approaches 1, data are appropriate for the factor analysis, otherwise (amounts less than 0.6) results from factor analysis can't be so suitable for the research. This test examines the time when the mathematically identified correlation matrix is a unitary and identity matrix, so it is not useful for recognizing the structure of the factor model. Moreover, if the correlation matrix between variables is not unitary and identity, there is a meaningful relationship between variables. Therefore, if sig in Bartlett Test is less than 5%, factor analysis is suitable for recognizing the structure of the factor model because the hypothesis of recognition of correlation matrix is rejected.

Representing the Model and Assessing the Effectiveness of ERP in TUGA Company
Having done exploratory factor analysis (EFA) in this research, main aspects (strategy, management, and operation) were divided into four different elements. Figure 4 repents the TUGA model.

Determining The Weight To The Model Aspects
Having been confirmed for the effectiveness assessment of ERP in Tuga Company, we had to determine the weightof the model regarding the business condition in the company and then assessing it. To do so, we held a session in which the company president, vice-president and the system president attended and the weight of the model was done using AHP technique and Expert Choice. In short, firstly we considered identical weight for each of the main aspect (strategic, middle and operational) on the basis of every aspect and at the next step the weight of each separate factor was obtained by sub-factors of aspects on the basis of decisions made by the members and AHP technique. The results are shown in the Table 5.

Effectiveness Assessment
To assess the effectiveness of the model, we are to distribute some questionnaire among the Tuga users to assess the success of the ERP. We used the formula 1 to determine the sample size in which 134 was determined. Regarding the fact that the return rate for the questionnaire in this company is 80 percent, we distributed 150 questionnaires. Some questionnaires were sent by email and in some occasion they were posted. Generally speaking 84 percent of the questionnaires were returned by the answerers. Descriptive statistics was used to analyze the data collected.

Conclusion
To assess the ERP effectiveness, we have to determine short term, middle term and long term goals before assessing the effectiveness in detail. Then it's possible to assess the effectiveness of the ERP by comparing the current and expected situation. Unfortunately, this stage has not been done in this company, so we regarded two separate parts in the final questionnaire for scoring these indexes. Each user firstly should score his/her expectation level from the ERP regarding the system maturation and secondly decides another score for the same index to assess the success rate he/she observed. Finally the success rate of ERP was achieved by comparing the expected situation and the observed situation. In this article, the assessment findings are not represented. TUGA Company policy does not permit for revealing findings result. Based on finding a few initiatives are defined to fill the observed gaps. This assessment represents that education and training are key factors in the success and effectiveness of such systems.
In the research, paid close attention to the user's satisfaction in each organizational level. We suggest considering organizational objectives in the future researches. ERP has encompassed all the activities done in an organization and should help the organization to achieve its long term goals.