Image
Mobile application with Machine Learning based on packaging recognition technology
The general practice is that merchandisers check products’ availability on the store shelves manually. They record what remains of them, compare their display with the requirements, and evaluate the goods’ presentation. We, on our account, have come up with a solution for the issue of automation and simplified this process.
Image
Our solution can automatically match boxes from showcase photos to the SKU (the unique identifier of each product) with accuracy and determine the presence of each showcase’s brand. This can become a daily operation tool for the customer's merchandisers and fully automate merchandise package shelf auditing.
The solution consists of three parts:
- An artificial neural SKU recognition network trained on real field staff photos
- Continuous artificial intelligence learning system to add new product images and improve recognition accuracy
- Mobile app on the Android platform for stand recognition and result rendering
Consequently, the solution provides a recognition accuracy of over 95%, saves merchandisers the hassle of manually auditing their stores, and automatically transmits all the information to the company's analytical systems.
How an artificial neural network works
An artificial neural network plays the key role in recognizing packages in an image. It is trained on the company's field photos. The neural network accurately recognizes packages photographed from different angles and under different lighting. It even reads excessively illuminated photos, achieving human perception quality.
The neural network runs the input data through a "neurons" system, a system of simpler programs that interact with each other, and then produces the calculations results based on this interaction, taking into account the experience and errors of past runs.
To get started, the neural network first needs to be trained. In abridged version, this is the way the process goes:
- The neural network is provided a set of training examples (pictures of merchandise bundles).
- Each sample in the set is processed within the neural network structure. The output signal of the network is calculated in the process. It is then compared with the corresponding target vector value representing the network’s required output.
- Then, the error coefficient is calculated based on a certain rule.
- Weighting is adjusted to reduce the error and new weighting is set for the connections in the neural network.
- The learning process continues until the amount of error across the entire verification array of photos reaches an acceptably low level. At that point, the neural network is considered trained.
Image
Why we need a continuous artificial intelligence learning system
Packaging design is constantly changing and new brands keep entering the market. Meanwhile, our system always continues to learn, improving its recognition accuracy.
Machine learning, in comparison with other recognition methods, provides more accurate results and always continues to improve.
Image
We used TensorFlow, an open source software library developed by Google, to solve the issues of building and training a neural network to find and classify images automatically.
How the solution works in real life
To start, the merchandiser simply opens the app and points the camera at the product stand. The system would tell you when you have angled the most suitable view of the showcase to continue with the recognition process.
Image
Once the merchandiser has taken a photo, the image undergoes the processing cycle and it is transmitted to an artificial neural network for recognition. The processing result is shown as a characteristic for the merchandise’s representation on the shelf and is displayed on the work smartphone a few seconds later.
Image
As a result of the photo analysis, the neural network gives the merchandiser a list of SKUs, a rendering of the brands on the shelf and the option of optimal placement. Simultaneously, all the information is transferred to the company's analytical systems for general information collection.