M5SAR
MOBILE FIVE SENSES AUGMENTED REALITY SYSTEM FOR MUSEUMS
Financed by Portugal2020, CRESC Algarve 2020 I&DT, n.º 3322,
promotor SPIC - Sonha Pensa Imagina Comunica, Lda, co-promotor University of the Algarve
1 Jan. 2016 - 31 Out. 2018
Collaboration: Profª. Drª Alexandra Gonçalves (Direção Regional de Cultura do Algarve)
Acknowledgements: Museu Municipal de Faro
Synopsis
Imagine visiting a museum where you can see, hear, feel, smell and maybe even taste what existed at the time when the piece was developed or relative to its contents. This project aims to develop an AR system, consisting of an application (APP) platform and a device (usually referred to as "gadget") to integrate in the mobile devices (phablet and tablet) that explores the 5 senses (5S) human (vision, hearing, touch, smell and taste). The system focuses on being used as a guide in cultural, historical and museum events. The system will consist of three products: (a) Application (APP) for mobile devices (software) where the download will be made in the "store" or on the website, which will work on the latest devices phablet and tablet, focusing on four of the five senses: sight, hear, touch and taste. The second product consists of (b) the device (hardware) that can be sold separately, or may be rented at the museum. This device focuses on 3 of the 5 senses: touch, smell and taste. (c) Both the APP (a) and the device (b) may be associated, deepening the user experience, meaning that one of the device characteristics (b) will be the easy coupling to the mobile device (phablet and tablet), as well as the complete integration with the APP.
Team UAlg
Professors
Research Grants
João Pereira
06/2016 - 10/2018 (WP#1)
João Duarte
06/2016 - 10/2018 (WP#3)
Marco Freitas
09/2016 - 03/2017 (WP#2)
Roman Bajireanu
05/2017 - 10/2018 (WP#2)
Ricardo Veiga
12/2017 -10/2018 (WP#2)Localization
Lab. Projeto em Computação Visual (Visual Computing Lab.) Room 163 ISE/UAlg (+351 289800100 ext. 6351)
Publications
Book Chapter
1. Rodrigues, J.M.F, Cardoso, P.J.S., Lessa, J., Pereira, J.A.R., Sardo, J.D.P., Freitas, M., Semião, J., Monteiro, J., Ramos, C.M.Q., Lam, R., Esteves, E., Figueiredo. M., Gonçalves, A., Gomes, M., Bica, P. (2018) An Initial Framework to Develop a Mobile 5 Sense Museum System, Chapter 5 of Technological Developments for Cultural Heritage and eTourism Applications (pp. 97- 119), IGI Global. DOI: 10.4018/978-1-5225-2927-9.ch005
Journal
1. Rodrigues, J. M. F., Ramos, C., Pereira, J., Cardo, J., Cardoso, P. J. S. (2019) Five Senses Augmented Reality System: technology acceptance study, IEEE ACCESS. DOI: 10.1109/ACCESS.2019.2953003
PDF: https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8896028
2. Cardoso, P. J. S., Rodrigues, J. M. F. , Pereira, J., Nogin, S., Lessa, J., Ramos, C., Bajireanu, R., Gomes, M., Bica, P. (2019) Cultural Heritage Visits Supported on Visitors’ Preferences and Mobile Devices, Universal Access in the Information Society
PDF: https://link.springer.com/article/10.1007/s10209-019-00657-y
3. Bajireanu, R., Pereira, Joao A.R. , Veiga, Ricardo J.M. , Sardo, João D.P., Cardoso, Pedro J.S. , Lam, Roberto, Rodrigues, João M.F. (2019) Mobile Human Shape Superimposition: An Initial Approach using OpenPose. International Journal of Computers, 3, 1-8 (invitation from 18th International Conference on Applied Computer Science, Dubrovnik, Croatia, September 26-28, 2018)
4. Sardo, João D. P., Pereira, João A. R., Veiga, Ricardo J. M., Semião, Jorge, Cardoso, Pedro J. S., Rodrigues, João M. F. (2018) Multisensorial Portable Device for Augmented Reality Experiences in Museums. International Journal of Education and Learning Systems, 3, 60-69 (invitation from 18th International Conference on Instrumentation, Measurement, Circuits and Systems, Paris, France, 13-15 April)
Conferences
1. HCI International 2019, 26 - 31 July 2019, Orlando, Florida, USA
Rodrigues J.M.F., Veiga R.J.M., Bajireanu R., Lam R., Cardoso P.J.S., & Bica P. (2019). AR Contents Superimposition on Walls and Persons. In Antona M., Stephanidis C. (eds) Universal Access in Human-Computer Interaction. Theory, Methods and Tools. HCII 2019. LNCS 11572, pp. 638-645, Springer International Publishing. DOI: 10.1007/978-3-030-23560-4_46
PDF: https://link.springer.com/chapter/10.1007/978-3-030-23560-4_46
2. International Conference on Applied Computer Science, September 26-28, 2018, Dubrovnik, Croatia
Bajireanu, R., Pereira, Joao A.R. , Veiga, Ricardo J.M. , Sardo, João D.P., Cardoso, Pedro J.S. , Lam, Roberto, Rodrigues, João M.F. (2019) Mobile Human Shape Superimposition: An Initial Approach using OpenPose. International Journal of Computers, 3, 1-8 (invitation from 18th International Conference on Applied Computer Science, Dubrovnik, Croatia, September 26-28, 2018)
3. International Conference on Instrumentation, Measurement, Circuits and Systems, 13-15 April, 2018, Paris, France
Sardo, João D. P., Pereira, João A. R., Veiga, Ricardo J. M., Semião, Jorge, Cardoso, Pedro J. S., Rodrigues, João M. F. (2018) Multisensorial Portable Device for Augmented Reality Experiences in Museums. International Journal of Education and Learning Systems, 3, 60-69 (invitation from 18th International Conference on Instrumentation, Measurement, Circuits and Systems, Paris, France, 13-15 April, 2018 )
4. HCI International 2018, 15-20 July 2018, Las Vegas, Nevada, USA
Rodrigues, J.M.F., Veiga, R.J.M., Bajireanu, R., Lam, R., Pereira, J.A.R., Sardo, J.D.P., Cardoso, P. J.S., Bica, P. (2018) Mobile Augmented Reality Framework – MIRAR, In M. Antona and C. Stephanidis (Eds.): Universal Access in Human-Computer Interaction 2018, LNCS 10908, pp. 102–121, 2018. DOI: 10.1007/978-3-319-92052-8_9 (HCII 2018 Conf.)
PDF: https://link.springer.com/chapter/10.1007/978-3-319-92052-8_9
5. HCI International 2018, 15-20 July 2018, Las Vegas, Nevada, USA
Cardoso, P. J.S., Guerreiro, P., Monteiro, J., Rodrigues, J.M.F. (2018) Applying an Implicit Recommender System in the Preparation of Visits to Cultural Heritage Places, In M. Antona and C. Stephanidis (Eds.): Universal Access in Human-Computer Interaction 2018, LNCS 10908, pp. 421–436, 2018. DOI: 10.1007/978-3-319-92052-8_33 (HCII 2018 Conf.)
PDF: https://link.springer.com/chapter/10.1007/978-3-319-92052-8_33
6. International Conference on Applied Mathematics and Computer Science (AMACS 2018), October 26-28, London, UK
Veiga, R.J.M., Pereira, J.A.R., Sardo, J.D.P., Bajireanu, R., Cardoso, P. J.S., Rodrigues, João M.F. (2018) Augmented Reality Indoor Environment Detection: Proof-of-Concept, In Procs 3rd International Conference on Applied Mathematics and Computer Science (AMACS 2018), London, UK , October 26-28
7. Portuguese Conference on Pattern Recognition, 26 Oct. Coimbra, Portugal,
Bajireanu, R., Veiga, R.J.M., Pereira, J.A.R., Sardo, J.D.P., Roberto, L., Cardoso, P. J.S., Rodrigues, João M.F. (2018) Mobile Human Shape Superimposition using OpenPose: An Initial Approach, In Procs Portuguese Conference on Pattern Recognition, Coimbra, Portugal, 26 Oct.
PDF: https://aprp.pt/wp-content/uploads/proceedings_recpad2018.pdf
8. International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting Info-exclusion, 20-22 June, Thessaloniki, Greece
Ramos, C., Mendonça, M., Rodrigues, J.M.F. (2018) Senior users of social networks: technological experience and use habits analysis, In Procs 8th International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting Info-exclusion, Thessaloniki, Greece, Greece, 20-22 June
9. International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting Info-exclusion, 20-22 June, Thessaloniki, Greece
Pedro J.S. Cardoso, Pedro Guerreiro, João A. R. Pereira, Ricardo J.M. Veiga (2018) A Route Planner Supported on Recommender Systems Suggestions: Enhancing Visits to Cultural Heritage Places. In Procs 8th International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting Info-exclusion, Thessaloniki, Greece, 20-22 June
10. INternational CongRess on Engineering and Sustainability in the XXI cEntury, INCREaSE 2017, 11-13 October, Faro, Portugal
Pereira, J., Freitas, M., Sardo, J., Veija, R. Cardoso, P., Rodrigues, J. (2017) MIRAR: Mobile Image Recognition based Augmented Reality Framework, In: Mortal A. et al. (eds) INCREaSE. Springer Cham, pp. 321--337. https://doi.org/10.1007/978-3-319-70272-8_27 (INCREaSE 2017 Conf.)
PDF: https://link.springer.com/chapter/10.1007/978-3-319-70272-8_27
11. INternational CongRess on Engineering and Sustainability in the XXI cEntury, INCREaSE 2017, 11-13 October, Faro, Portugal
Sardo, J. D. P. , Semião, J., Monteiro, J. M., Esteves, E., Pereira, J., Freitas, M., Rodrigues, J. M. F. (2017) Portable Device for Touch, Taste and Smell Sensations in Augmented Reality Experiences, In A. Mortal et al. (eds.), INCREaSE, Springer Cham, pp. 305–320. https://doi.org/10.1007/978-3-319-70272-8_26 (best paper) (INCREaSE 2017 Conf.)
PDF: https://link.springer.com/chapter/10.1007/978-3-319-70272-8_26
12. HCI International 2017, 9 - 14 July 2017, Vancouver, Canada
Rodrigues, J.M.F., Pereira, J.A.R., Sardo, J.D.P., Freitas, M.A.G., Cardoso, P.J.S., Gomes, M., Bica, P. (2017) Adaptive Card Design UI Implementation for an Augmented Reality Museum Application, In M. Antona and C. Stephanidis (Eds.): Universal Access in Human-Computer Interaction 2017, Part I, LNCS 10277, pp. 433–443, DOI: 10.1007/978-3-319-58706-6_35 (HCII 2017 Conf.)
PDF: https://link.springer.com/chapter/10.1007/978-3-319-58706-6_35
13. HCI International 2017, 9 - 14 July 2017, Vancouver, Canada
Cardoso, P.J.S., Rodrigues, J.M.F., Pereira, J.A.R., Sardo, J.D.P. (2017) An Object Visit Recommender Supported in Multiple Visitors and Museums, In M. Antona and C. Stephanidis (Eds.): Universal Access in Human-Computer Interaction 2017, Part I, LNCS 10277, pp. 301–312, 2017. DOI: 10.1007/978-3-319-58706-6_24 (HCII 2017 Conf.)
PDF: https://link.springer.com/chapter/10.1007/978-3-319-58706-6_24
14. Portuguese Conference on Pattern Recognition, 28 Oct., Aveiro, Portugal
Veiga, R.J.M., Bajireanu, R., Pereira, J.A.R., Sardo, J.D.P., Cardoso, P. J.S., Rodrigues, J.M.F. (2017) Indoor Environment and Human Shape Detection for Augmented Reality: an initial study, In Procs 23rd edition of the Portuguese Conference on Pattern Recognition, Aveiro, Portugal, 28 Oct., pp. 67-68.
PDF: https://aprp.pt/wp-content/uploads/2017/10/proceedings_recpad2017.pdf
15. HCI International 2015, 2 - 7 August 2015, Los Angeles, CA, USA
Pereira, J., Nogin, S., Cardoso, P.J.S., Rodrigues, J.M.F. (2016) A Cultural Heritage and Points of Interest Multi-Criteria Router Supported on Visitors Preferences. In Proc of the 7th International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting Info-exclusion (DSAI 2016). ACM, New York, NY, USA, 392-399. DOI: 10.1145/3019943.3019999.
16. International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting Info-exclusion, December 1 - 3, 2016, Vila Real Portugal
Rodrigues, J.M.F., Lessa, J., Gregório, M., Ramos, C., Cardoso, P.J.S. (2016) An Initial Framework for a Museum Application for Senior Citizens. In Proc. of the 7th International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting Info-exclusion (DSAI 2016). ACM, New York, NY, USA, 414-421. DOI: 10.1145/3019943.3020002
Thesis
1. João Sardo (2017) Portable Device for Augmented Reality Five-Sense Experiences, Mestrado em Engenharia Elétrica e Eletrónica, Instituto Superior de Engenharia, Universidade do Algarve
2. João Pereira (2017) Smart Augmented Reality Application for Enhanced Museum Experience, Mestrado em Engenharia Elétrica e Eletrónica, Instituto Superior de Engenharia, Universidade do Algarve
3. Roman Bajireanu (2019) Real-Time Human Body Detection and Tracking For Augmented Reality Mobile Applications, Mestrado em Engenharia Elétrica e Eletrónica, Instituto Superior de Engenharia, Universidade do Algarve
Press
RTP/RDP
RTP1
