An augmented reality application for improving shopping experience in large retail stores

 

Authors

Cruz, Edmanuel; Orts-Escolano, Sergio; Gomez-Donoso, Francisco; Rizo, Carlos; Rangel, Jose Carlos; Mora, Higinio; Cazorla, Miguel

Format

Article

Status

publishedVersion

Description

In several large retail stores, such as malls, sport or food stores, the customer often feels lost due to the difficulty in finding a product. Although these large stores usually have visual signs to guide customers toward specific products, sometimes these signs are also hard to find and are not updated. In this paper, we propose a system that jointly combines deep learning and augmented reality techniques to provide the customer with useful information. First, the proposed system learns the visual appearance of different areas in the store using a deep learning architecture. Then, customers can use their mobile devices to take a picture of the area where they are located within the store. Uploading this image to the system trained for image classification, we are able to identify the area where the customer is located. Then, using this information and novel augmented reality techniques, we provide information about the area where the customer is located: route to another area where a product is available, 3D product visualization, user location, analytics, etc. The system developed is able to successfully locate a user in an example store with 98% accuracy. The combination of deep learning systems together with augmented reality techniques shows promising results toward improving user experience in retail/commerce applications: branding, advance visualization, personalization, enhanced customer experience, etc.
In several large retail stores, such as malls, sport or food stores, the customer often feels lost due to the difficulty in finding a product. Although these large stores usually have visual signs to guide customers toward specific products, sometimes these signs are also hard to find and are not updated. In this paper, we propose a system that jointly combines deep learning and augmented reality techniques to provide the customer with useful information. First, the proposed system learns the visual appearance of different areas in the store using a deep learning architecture. Then, customers can use their mobile devices to take a picture of the area where they are located within the store. Uploading this image to the system trained for image classification, we are able to identify the area where the customer is located. Then, using this information and novel augmented reality techniques, we provide information about the area where the customer is located: route to another area where a product is available, 3D product visualization, user location, analytics, etc. The system developed is able to successfully locate a user in an example store with 98% accuracy. The combination of deep learning systems together with augmented reality techniques shows promising results toward improving user experience in retail/commerce applications: branding, advance visualization, personalization, enhanced customer experience, etc.

Publication Year

2019

Language

eng

Topic

Smart shopping
Deep learning
Augmented reality
Retail stores
User experience
Human–computer interaction
3D visualization
Smart shopping
Deep learning
Augmented reality
Retail stores
User experience
Human–computer interaction
3D visualization

Repository

RI de Documento Digitales de Acceso Abierto de la UTP

Get full text

https://link.springer.com/article/10.1007/s10055-018-0338-3
https://ridda2.utp.ac.pa/handle/123456789/9436

Rights

embargoedAccess

License