Manufacturers often increase the number of product variants or offer uniquely configured products to serve individual customer needs and create a competitive advantage. This advantage, however, comes with a trade-off: increasing complexity and costs on all steps of the manufacturing process, from engineering to planning and production to service. To eliminate this increased complexity and rise in costs, manufacturers must transform their operations to agile automation that can quickly react to changes within the planning and production environment.
An example of how this trade-off can be eliminated is a solution for machine-learning based video quality assurance, deployed at Foxconn’s production lines for HPE systems in the Foxconn site in Kutna Hora, Czech Republic.
The complexity trade-off in IT-system manufacturing
The production of IT systems is particularly prone to the complexity trade-off. One reason for this is the number of potential product variants. For example, a specific server model can theoretically be equipped with between two and sixteen memory modules, which in turn can have 16, 32, 64 or 128 gigabytes. Thus, the memory options allow for several hundreds of product variants – and these, in turn, must be multiplied with the number of options available for the processors, fans, disk drives etc. Moreover, the IT industry is known for rapid innovation, with short product refresh cycles and frequent product updates as a result – adding to the variability of the production environment.
Quality assurance of these products not only requires assessment of the number and position of such system components, but also of their correct implementation. For example, a cable must not only be positioned in front of the correct port, but also checked to make sure it was actually plugged in successfully during manufacturing and assembly. Further, other product defects such as scratches on the surface of the server chassis must be accounted for.
Considering the above circumstances, these types of complex IT system quality assurance inspections can take a human several minutes each. For a manufacturer like Foxconn, whose factories produce tens or hundreds of thousands of IT devices every day, this is a considerable amount of cost and time added to the manufacturing process.
Automating quality assurance with machine-learning based video analytics
A solution for this problem is to deploy video analytics for quality assurance to automate the process: cameras capture high-resolution images of products on a conveyor belt and stream them to an embedded or attached IT system where the images are analysed by a video analytics application using machine-learning (ML) algorithms. Just like humans would do, ML compares the image of the actual product with reference images that display accurate and defective implementations. Thus, the machine learns if a cable is properly plugged into a port, if a memory module is properly inserted into its socket, or if there is a scratch on the chassis.
But also for ML-based video analytics, product variety is a challenge. To teach the analytics application to detect defects in configuration, implementation or any other damage of the product, it could require thousands of reference images that the analytics application can compare with the product image from the conveyor belt. This comes with two key disadvantages. First, it can take weeks to train the ML algorithms to enable a precise and reliable detection of product defects. Second, this approach is inflexible, requiring a new training cycle for every new configuration, product refresh or update.
Together with Relimetrics, a firm specialised on smart quality audits for Industry 4.0, HPE Pointnext implemented a solution to solve these problems. One initial action included the disaggregation of product images. The solution does not store reference images of complete servers, for example, but of components like memory modules in the proper slots, the processor socket with its fan, the hard disk, etc. For every product on the conveyor belt, the Manufacturing Execution System (MES) provides a list of materials to the analytics application, enabling it to assemble the relevant reference image components into a complete reference picture.
This approach has two key effects. First, the ML algorithms aggregate learning much faster and more efficiently because reference image components are frequently reused. At Foxconn’s factory in Kutna Hora, HPE Pointnext was able to train the ML model for new server quality assurance with roughly 1,000 configuration variants in just two days, completely automating the defect detection process. Second, this approach enables increased flexibility by combining image components according to the actual product configuration as provided by the bill of materials.
Edge analytics for real-time quality management
When using ML for managing a production process like quality assurance, another challenge is the amount of data produced by the video cameras. The implementation at Foxconn’s server production in its Kutna Hora factory, for example, employs 10 video cameras for every conveyor belt in order to capture extremely detailed images of every server and part. These cameras generate 3 gigabytes of image data per hour. Transmitting large amounts of data to ML-based video analytics systems quickly is critical for high-producing factories. It would be impracticable to transfer that data via internal or external networks to be processed on remote servers – because the latency would be too high, networks would be overloaded with these data volumes, and production systems would grind to a halt during network outages.
Therefore, HPE Pointnext deployed the ML-based video analytics solution on HPE Edgeline Converged Edge Systems – rugged, compact systems delivering enterprise-grade IT capabilities at the edge. These systems are designed with manufacturing environments in mind and also integrate operational technology (OT) like data acquisition systems, control systems and industrial networks to enable seamless bi-directional and deterministic communication and control of OT systems like video cameras, production machines or conveyor belts.
With this solution, the constant video camera stream is first pre-processed on HPE Edgeline Converged Edge Systems running in close proximity to the conveyor belt. The solution extracts images of the actual product and next analyses the data in real-time using ML algorithms to detect defects. Only a subset of the analysed images are transferred via the network to be archived for traceability and compliance.
To eliminate the complexity and cost trade-offs of production variability, manufacturers must transform their operations from static to agile automation. Key enablers to achieve that include artificial-intelligence methods like machine learning. However, as the example of machine-learning based video quality assurance confirmed, realising this promise requires smart solutions that take into account the specifics of the individual production process. Moreover, it requires that manufacturers establish a new edge infrastructure to turn video streams into intelligence and action.
According to Gartner, by 2025, as a result of digital business projects, 75 per cent of enterprise-generated data will be created and processed outside the traditional, centralised data centre or cloud, up from less than 10 per cent in 2018. Video analytics in quality assurance is just one of many examples that show why this prediction is valid.
Norbert Reil, Director Global IoT Services, HPE Pointnext, Hewlett Packard Enterprise