'The Brain is in the Details': Anomaly Understanding using Brain Twins
HEADS UP!
No, this is not a two-bodied conjoined Zebra, and we know you know that. In a split second you spotted that something is wrong with this image. A split second afterwards, your brain solved this anomaly by realising one zebra is facing you while the other is facing away. Can you tell which is which?
As you saw with the example above, humans are experts at certain tasks, one of which is the inherent ability to notice patterns and spot anomalies. This ability is implemented through our brain’s evolution [2] and our life experience. Our brains are evolved to constantly classify because the ability to recognize and quickly decide whether something is safe or dangerous is a critical survival mechanism (AKA: Fight-Flight-Freeze Mechanism). Our life experience is the accumulation of everything we learn about the world from the environment and from those around us throughout our lives. We apply this life experience on top of our brain evolution to every perception and decision we make, including identifying anomalies.
Brainvivo is on a journey of digitizing the human brain, making decision-making accessible and useful. With its cutting-edge neuro-technologies, Brainvivo is creating Brain Twins that mimic the way experts make decisions.
For the purpose of this blog, we have chosen to share one project we have been working on with a top tier semiconductor company. The goal of this project was to build a Brain Twin of human experts that make thousands of decisions on semiconductors every day. These experts are crucial to the manufacturing processes as they are the final line of review, helping to maximize factory yield.
Why are Human Experts needed on the Semiconductor Fabrication Line?
Semiconductors (also known as microchips) are an essential component in practically every existing electronic system or product. You are using at least one to read this whether you are on your phone, tablet, or computer, or anytime you are traveling by bus, train, or your car. The creation of semiconductors is highly complex and involves rigorous quality assurance. Several checks and tests are used to verify that each chip has been assembled correctly, looking for defects and determining whether these defects are ‘killer’ (meaning they render the chip unusuable) or ‘non-killer’.
Why not use AI to solve this problem?
All semiconductor manufacturers are, in fact, using cutting-edge technologies including advanced AI algorithms to tackle this problem.
While most anomaly detection tools are automated using state-of-the-art image processing, machine learning (ML), and deep learning (DL), these methods cannot provide a yield of 100%. Though exact numbers are not disclosed, estimates suggest manufacturers achieve 90-95% yield via autonomous processes, leaving behind 5-10% defective units. With over a trillion semiconductors manufactured annually [3] in an industry worth over $500bn [4], the potential loss to the semiconductor industry is $50-100bn. The effect is even more significant as this loss directly impacts sales for the consumer electronics, tech, and e-commerce industries, each worth several trillions of dollars.
ML/DL systems alone cannot solve everything, especially when every new chip design means retraining the algorithms with newly generated and annotated data. If you take into account that some of the anomaly detection processes involve high-resolution images from electron microscopy (rather than low-resolution optical imaging), you add time constraints to the semiconductor manufacture as higher resolution means more pixels to test.
This is why we still need expert human reviewers to identify whether there are any defects, and distinguish between killer and non-killer defects.
Why are humans not always good enough?
Unfortunately, even the world’s top experts are affected by the same difficulties as all humans: stress, tiredness, and distraction, all of which lead to lower accuracy and inefficiency. Additionally, the last couple of years taught us that workers cannot be physically present all the time, so you must find a way to automate their decision making as well.
Brainvivo set out to address this human factor challenge by creating a Brain Twin of a semiconductor reviewer expert. Like his/her owner, this expert’s Brain Twin makes judgment calls on defects just like his owner by replicating the expert’s brain patterns of deciding between Go and No-Go. Click here to learn more about how our technology works.
How do Brain Twins help experts do their job better and faster?
Human experts reviewing semiconductors in the fab review station will typically look at several thousand images per shift. We built an expert Brain Twin that was trained on just one high-resolution semiconductor image containing roughly 100 defects, then tested it on 20 unseen images containing ~2000 defects. Despite being trained on such a low amount of data, Brainvivo's Brain Twin successfully identified semiconductor defects with 99.7% accuracy!
The Brain Twin has the potential to benefit the semiconductor industry by helping human experts minimise mistakes and complementing conventional AI methods to let manufacturers maximise their yield.
This is the first time a Brain Twin is being built and used to replicate expert human judgement. Since a trained human brain is already wired to identify patterns and recognize anomalies, Brainvivo's Brain Twins are able to easily capture and mimic this incredible ability. In this successful project, Brainvivo demonstrates its ability to export an expert brain capability in order to solve a real pain-point with minimal training data.
The potential of Brainvivo's Brain Twins is limitless. Stay tuned to find out more, and contact us if you are an expert who wants their expertise digitized.
Lodhi, S. (2019) Zebra crossing. [Photograph]. Sarosh Lodhi Photography, [online] available at: <https://i.insider.com/5f05e436aee6a81b56590a14?width=1136&format=jpeg>
Assaf, Y., Bouznach, A., Zomet, O., Marom, A. and Yovel, Y., 2020. Conservation of brain connectivity and wiring across the mammalian class. Nature Neuroscience, 23(7), pp.805-808. This article published in Nature Neuroscience was written by our co-founder and CTO, Professor Yaniv Assaf, on the subject of the evolution and connectivity of the mammalian brain.
Semiconductor Industry Association. (2022) ‘Semiconductor Sales Increase 23.5% Year-to-Year in November; Industry Establishes Annual Record for Number of Semiconductors Sold’. [online] Available at: <https://www.semiconductors.org/global-semiconductor-sales-increase-23-5-year-to-year-in-november-industry-establishes-annual-record-for-number-of-semiconductors-sold/>
bbc.com. (2019) ‘Semiconductors: the humble mineral that transformed the world’. BBC, [online] Available at: <https://www.bbc.com/future/bespoke/made-on-earth/how-the-chip-changed-everything/>