Discover the computers that use artificial intelligence for mysterious black-box learning

The term "black box" has different meanings for different people. Some may think of the recording devices that are used to investigate airplane crashes. Others may think of the small theaters that have minimal decorations. But there is another meaning of "black box" that is relevant to artificial intelligence. "AI black boxes" are AI systems that hide their inner workings from the user. They only show the input and the output, but not the code or the logic that generated the output.

Machine learning is a common subfield of artificial intelligence. It is the basis of generative AI systems like DALL-E 2 and ChatGPT. AI has three components: a model, training data, and an algorithm or a set of algorithms. An algorithm is a sequence of steps. In machine learning, an algorithm learns to recognize patterns from a large number of examples, or the "training data". A machine-learning model is the result of training a machine-learning algorithm. The model is used by people.

For example, a machine-learning algorithm could be designed to look for patterns in images, and the training data could be images of dogs. The resulting AI model would be able to spot canines. It would take an image as input and decide whether and where a group of pixels in the image corresponds to a dog.

The three components of a machine-learning system can be hidden or enclosed in a black box. The algorithm is often well-known, as is usually the case, making it less useful to keep it secret. Therefore, AI developers often put the model in a black box to protect their intellectual property. Another method software developers use is to put the training data in a black box, or hide the data used to train the model.

A glass box is sometimes used as the opposite term of a black box. In an AI glass box, the system's algorithms, training data, and model are all transparent to the public. However, even in these cases, researchers may still label some aspects of the system as black boxes.

That is because scientists do not fully understand how AI algorithms, especially deep learning algorithms, work.

Why Black Box is Important:

 Often, there is a good reason to be wary of black-box AI algorithms and models. Suppose that a machine-learning model has diagnosed your health condition. Would you rather have a glass or black box for the model? What about the treatment plan that your doctor has prescribed you? She might want to know how the model reached its decision.

Suppose that an AI model that determines if you qualify for a business loan from a bank rejects you. Would you want to know why? If you did, you could either change your situation to increase your chances of getting a loan the next time or challenge the decision more convincingly.

Moreover, black boxes have significant impacts on software system security. For years, many in the computer industry thought that putting software in a black box would prevent hackers from accessing it and make it secure. However, this assumption was largely wrong, as hackers can reverse-engineer the software. They can create a copy by closely observing how a piece of software works and finding vulnerabilities to exploit.

Software testers and hackers with good intentions can inspect software in a glass box to discover flaws and report them to the developers, lowering the risk of cyberattacks.