Sarajevo Graphics Group‘s experience in performing qualitative studies is described in the paper: VR Video Storytelling for Intangible Cultural Heritage Preservation to be presented at the 16th EUROGRAPHICS Workshop on Graphics and Cultural Heritage [1]. In this paper, we investigate if 360 degree VR videos further contribute to user immersion in the preservation of intangible cultural heritage. It describes a case study of the Mostar bridge diving project, aimed to present and preserve the bridge diving tradition from the Old Bridge in Mostar, Bosnia and Herzegovina. It is a virtual reality application which enables the user to virtually jump off the bridge after watching 360-degree video stories about its history and the bridge diving tradition and upon successfully completing the quiz evaluation of the knowledge gained from the stories. In our study, we combined semi-structured interviews and contextual observations of users, as it is suggested for the research question related to pilot phase of development and understanding the phenomenon in focus of the application [2]. The main purpose of the study was to explore user needs and behaviors, compare their experience and performance (with respect to different levels of preconditioning), and identify strengths and limitations of the approach proposed in the paper. The user experience evaluation study shows that our method was successful in preserving a form of intangible heritage and provide suggestions that can be used in developing an intangible heritage preservation framework.
Our experience in quantitative studies includes: standard user experience evaluation as a part of the storytelling life-cycle and developing an instrument for benchmarking multimedia applications.

We have designed a quantitative user experience surveys evaluating the storytelling work within the H2020 iMARECULTURE project in order to evaluate Kyrenia – the pilot hyper storytelling application [3]. Hyper storytelling offers a solution for narrative paradox in introducing motivation factor into the interactive digital story. The purpose of the application is to introduce the Internet users to the oldest sunken ship from 280 B.C., found in 1975 near Kyrenia, Cyprus. Users were invited to view digital stories and embark the virtual model of Kyrenia, and after that to fill in the Internet-based survey. We have recognized different user types regarding their professional background (art, engineering, history) and computer proficiency (basic, intermediate, and professional). At this pilot phase, our primary goal was usability testing aimed to improve the pilot application. In the same time, we wanted to assess our evaluation approach, as this was the first step in developing the more generalized instrument for measuring features as edutainment and immersion.
Other User Experience evaluation study within the H2020 iMARECULTURE project was described in “A serious game for understanding ancient seafaring in the Mediterranean sea.” [4]

We pursued the objective to develop a simple and efficient instrument capable to highlight both main qualities and major difficulties; and to provide, in the same time, a tool for a comparison of different applications, or a different version of the same application. In our research, we selected the White Bastion 3D VR presentation as a case study, as described in “Measuring immersion and edutainment in multimedia cultural heritage applications” [5]. The motivation for our research was to obtain a reliable measure of the success of multimedia digital heritage applications, especially measuring success in achieving features as edutainment and immersion. We did not expect that benchmarking can be a solution in removing interaction flaws, not even to provide precise identification of the flaws, but this simple and efficient instrument can guide developers towards the most needed improvements in designing interaction of multimedia heritage applications. This is the reason that we did not aim to use the benchmarking process as ranking, but as the process of identifying and learning from good practices.

[1] Selmanovic, E., Rizvic, S., Harvey, C., Boskovic, D., Hulusic, V., Chahin, M. and Sljivo, S., 2018. VR Video Storytelling for Intangible Cultural Heritage Preservation. In: The 16th EUROGRAPHICS Workshop on Graphics and Cultural Heritage (EG GCH), 12-15 November 2018, Vienna, Austria. (In Press)
[2] Saldaña J.: The coding manual for qualitative researchers. SAGE London, 2009.
[3] Rizvic, S., Boskovic, D., Okanovic, V. and Sljivo, S., 2017. Kyrenia-Hyper Storytelling Pilot Application. 15th EUROGRAPHICS Workshop on Graphics and Cultural Heritage (EG GCH), November 2017, Vienna, Austria.
[4] Philbin-Briscoe O, Simon B, Mudur S, Poullis C, Rizvic S, Boskovic D, Liarokapis F, Katsouri I, Demesticha S, Skarlatos D. A serious game for understanding ancient seafaring in the Mediterranean sea. In2017 9th International Conference on Virtual Worlds and Games for Serious Applications (VS-Games) 2017 Sep 1 (pp. 1-5). IEEE.
[5] Boskovic, D., Rizvic, S., Okanovic, V., Sljivo, S. and Sinanovic, N., 2017, October. Measuring immersion and edutainment in multimedia cultural heritage applications. In Information, Communication and Automation Technologies (ICAT), 2017 XXVI International Conference on (pp. 1-6). IEEE.