Location and Orientation Based Augmented Reality Mobile Application for Enhancing Heritage Landmark History and Information

Augmented Reality technology in Mobile Device can provide better experience for people to to access and gather information at the location of cultural heritage or tourism landmark point in cultural heritage and historical tourism. Currently, the AR implementation research in cultural aspect hardly explored about the sensing algorithm of the cultural object that ignite the AR and the algorithm’s complexity in detail. In this paper, we we propose geodetic location based tracking algorithm that has low computational complexity compare to the object based tracking which required heavy computational complexity for image processing algorithm. Based on the testing and implementation evaluation, our application can mark the cultural heritage or historical landmark and overlay it with the story behind the place to enhance the visitor experience.


Introduction
Nowadays mobile computing technology improvement and expansion increase significantly. It is boosted up by the advance of the special and dedicated sensors in mobile phone that is able to attain any environmental information all over us, such as camera, accelerometer, orientation, location, gyroscope, e.t.c. This development, drive the interactive technologies such as Augmented Reality (AR) to become more possible and easy to implement widely. Augmented Reality is a visualization technology that overlay text, graphics, video, sensors data and other formats on the top of camera view in mobile phone /computer or other devices.
Presently AR technology is implemented in various field, such as Military and law enforcement, vehicles, medical, tourism, architecture, industry, entertainment, education, art, weather forecasting and heritage culture to casual users. In cultural heritage and historical tourism, AR based mobile application provide a more authentic and interactive way to access and gather information at the location of cultural heritage or tourism landmark point.
Some AR research in cultural field (Museums, City landmark, e.t.c) are carried out in term of finding the increasing of visitor enhancement after applying AR [1], adding sensing system to the AR apps to adapt apps visualization with psychological state of visitor [2], proposing archetypal development framework to accommodate specific requirement of users in AR Apps for tourists sectors [3][16]. However, these research hardly explored about the sensing algorithm of the cultural object that ignite the AR and the algorithm's complexity in detail.
Another research proposed environment sensing and retrieval with comparing the real object to some dataset for tracking the object feature with relatively heavy computational complexity for image processing algorithm and required extensive storage for the datasets [4,5,6].
The choice of the AR sensing algorithm is important especially for the cultural place in rural area with limited communication infrastructure since it required more light computational complexity during sensing process. In this work, we propose geodetic location based tracking algorithm that has less computational complexity than real object tracking based algorithm in [4,5,6] and implement it in android mobile device.
This research objective is to develop an augmented reality mobile application that mark the cultural heritage or historical landmark and overlay it with the story behind the place to enhance the visitor experience.

Augmented Reality in Mobile Computing
Augmented Reality is defined as "an enhanced version 40 Location and Orientation Based Augmented Reality Mobile Application for Enhancing Heritage Landmark History and Information of reality created by the use of technology to overlay digital information on an image of something being viewed through a device (such as a smartphone camera)" [13]. AR can be implemented in various devices, such as head-mounted displays (HMD), projector devices, and also mobile devices, since nowadays mobile device has computational ability that enables AR technology. The AR implementation in mobile devices usually comprise two main elements: the real data that sensed by the device (Camera capture, current location, current direction of device e.t.c) and the composed data used for augmentation (Additional information, image, animation e.t.c). The two elements is then complemented each other to build an AR application in a way that the composed data will overlay on the real live data. To develop the AR Application in mobile device, we need to utilize the device camera, graphics and sensors using related APIs.
In AR application development, tracking technique is a key challenge to be counted carefully. The tracking technique is selected, generally, base on the resources provided in mobile device especially the type of sensors available in the mobile devices i.e; camera, GPS, orientation sensor, accelerometer, gyroscopes e.t.c. . The several tracking techniques is discussed in the following: a. Marker-based tracking is commonly used in limited sensor function mobile devices, since it can be simply processed and identified by camera with low computational complexity. However, this tracking technique is more suitable for indoors tracking. b. Marker-less Tracking usually employs more complex image recognition and image processing algorithms to identify and recognize the AR targets, since it must recognize the real objects that have various shape and perspective. This condition, results in the significant increase of the system computational complexity which in turn requires higher resources (higher power, storage, computation speed etc). Thus, the technique is difficult to implement in mobile device since it has limited storage, power, etc. c. Location Based Tracking can be used for outdoor tracking with relatively low computational complexity. However, in term of accuracy this tracking can not be guaranteed since the use of GPS base on Base Station Triangulation has wide area of tracking.
To achive more accuracy in tracking the AR target object, we use Low complex Location Based Tracking that combined with the azimuth calculation based on Orientation Sensor listener in android mobile device.

Overview Scenario and System Architecture
Define During the cultural heritage or historical landmark observation process, visitor will hold the mobile device point to the certain location facing to the target object. When the point of interest is displayed at the center of the camera, the augmented reality content will also displayed and superimposed upon the point of interest. In this work, the location based augmented reality Mobile Application is developed using Android Studio Tool. The main objective of this system is to recognize the point of interest (i.e the heritage landmark) and then overlay the camera view of the real-life point of interest with the augmented reality object. The overlayed AR object, therefore, is expected to enhance the visitors' interest and sensation during observing the real object. The augmented reality content is being adjusted with the history and cultural background of the place/landmark. The challenge that are faced for developing this application is how to simply detect the point of interest and display the relevant content that can promote the information about the point of interest.

Tracking Algorithm
The Augmented Reality system generally exploit image processing algorithms to identify and recognize some special feature of object (point of interest) in the camera and then display the right AR feature at the real object camera image. Since the image processing algorithm require relatively heavy computation the system developed in this project used location based real object identification algorithm. The Algorithm implemented in the system is based on the geodetic Azimuth combine with the location data of the point of interest. Azimuth is defined as an angle between a point and x-axis in the x y plane in spherical coordinates (q) as seen in figure 2 [12] [13].
In case of earth, we can define the geodetic azimuth as angle measured from the reference line, usually from the north. We can utilize the azimuth to identify our point of interest combining it with the location data since the azimuth only represent object direction. The right direction and the right location will designate the exact position of our point of interest and the right AR object can be generated and displayed in the real position through camera view.
Where r = P2 range from zero coordinates; v and z are vertical and zenith angle respectively [15] Geodetic azimuth in equation (2) doesn't consider the earth curvature or usually called planar approximation, hence, in the real earth condition for the point of interest location that relatively far from north pole, such as Indonesia and other countries located at equator, this azimuth calculation have high distortion. Therefore, in this work we plan to implement spherical geodetic azimuth algorithm that can be more precisely recognize the point of interest to be overlayed by AR object. The spherical approximation for calculating azimuth is chosen since it indicate error less than 1% for flattening in the order of 10 -3 [16]. The spherical geometry is showed in figure 4 To determine the Azimuth A we need to implement cosine and sine laws:

System Implementation
The system exploits Location Service and Motion Sensor in Android Mobile Device. The location service will generate the latitude and longitude data of the current location. This data will be used to calculate the theoretical azimuth by using algorithm in section 3.b. , while the motion sensor is used to generate Real Azimuth Data. To recognize and mark the point of interest, we need to compare the theoretical and real azimuth. If both azimuth match each other, the system will invoke the augmented reality event.The augmented reality event then will overlay the AR object/model on the PoI camera view. This AR model can be a landmark Icon, historical image or a historical story to enhance visitor experience during their visit at the landmark site. Figure 6 depict the implementation of the algorithm for the AR system.  The application development method is using prototyping method. The first prototype's output is simply implement geodetical data only without addition process with location data before the overlaying process. This prototye therefore display the augmented reality object when the azimuth of the place is match with the calculated azimuth even if the exact location is different. Afterward, the last prototype implement additional location data of the point of interest before overlaying the augmented reality object to improve the accuracy and the pressision of the application.

Result
By utilizing camera phone, the developed application has been tested to ensure whether the tracking algorithm employed android device's location service and motion sensors can display the Augmented Reality Model. In the case of prototype 1 the AR object is displayed not only in the exact location of the point of interest but also at the place with same azimuth data, even it has some long distance from the real Point of Interest Location (Figure 7) The improved prototype testing result showed that the Augmented reality object show up at exact position of the point of interest, i.e. the "Gedung Sate" the famous landmark of Bandung City West Java Indonesia (figure 8). The Augmented Reality object is the historical information about Gedung Sate, that is, a battle of 7 Indonesian warrior and the allied forces during world war II.

Conclusions
The low complexity Augmented Reality mobile application that exploit geodetic information (location and orientation) of the mobile device to enhance the city landmark visitor's experienced is developed. Based on the testing process it is shown that this application can display the historical background information of Bandung Indonesia's landmark Gedung Sate at the right location.
The implication of the application is by enhancing the more location of the point of interests, the tourist visit to Indonesia specifically Bandung can be raised. Moreover it can increase the government income to support the development of Indonesia as a developing country. In the 44 Location and Orientation Based Augmented Reality Mobile Application for Enhancing Heritage Landmark History and Information education field it can expand the student experience in history and geography class to increase their love to their homeland. The future works is to increase the application's point of interest and integrate it with the history and geography curricullum in elementary and secondary school in Indonesia. Then, the evaluation in the implementation impact need to be carried out.