IBM uses AI to design better AI models

Deep learning uses algorithms to extract insights from data that we have never been able to obtain before. However, no algorithm is suitable for every task and it is a long process to find the most optimal architecture. A new system developed by IBM uses AI to accelerate that process.

Martin Wistuba, data scientist at IBM Research in Ireland, describes in a blog post how this works. He claims that the system is 50,000 times faster than other methods, with only a slightly higher error rate of up to 0.6 percent.

At IBM, engineers and scientists select the best architecture for a deep learning model from a large number of possible candidates. Today, this is a time-consuming manual process. Using a more powerful automated AI solution to select the neural network can save time and allow non-experts to apply deep learning faster, the researcher explains. The selection of a suitable algorithm would thus still be a matter of hours.

Evolutionary algorithm

Wistuba talks about an evolutionary algorithm that considers a convolutionary neural network architecture as a sequence of neurocells. By applying a series of mutations, a structure is sought that improves the performance of the neural network for a particular dataset and machine learning task.

This approach significantly reduces network training time. The mutations change the structure of the network, but do not change the network’s predictions and can, among other things, add layers, add new connections, or broaden kernels or layers, explains Wistuba.

Read this: Jonathan Berte: Artificial intelligence is a storm coming at us.

Wistuba used its method to find a suitable classification model for the CIFAR-10 and CIFAR-100 datasets. These are collections of labelled images that were made publicly available by the University of Toronto to train image recognition algorithms.

My algorithm had a slightly higher classification error, but required significantly less time compared to state-of-the-art man-designed architectures, results from reinforcement learning search methods and results for other automated methods based on evolutionary algorithms.

The IBM researcher hopes that his method can eventually be integrated into IBM’s cloud services and extended to larger datasets in other domains. Although evolutionary algorithms are not new in themselves, Wistubas’ system does represent an important step forward if it turns out to work as well as he claims it does.