IBM’s AI creates new sets of labeled images with semantic content

Scientists from IBM, Tel Aviv University and Technion describe in a paper a new model design for Artificial Intelligence (AI) that is designed to create new sets of labeled images, based on previously labeled images. The model is called Label-Set Operations (LaSO) networks.

The model can combine pairs of labelled images – for example a picture of a dog with the annotation ‘dog’ and a picture of a sheep with the annotation ‘sheep’ – to create new examples that use the labels of the first images. According to the researchers, the LaSO networks can be used in the future to supplement sets that contain too little data from the real world, writes Venturebeat.

LaSO networks learn to manipulate label sets of samples and develop new ones that match combined label sets by using photos of different types as input and identifying shared semantic content. Only then are real concepts in a sample that also occur in another sample removed.

Categories

The AI model works directly on image representations and requires no additional input for the manipulations. Therefore, it is able to generalize to images that contain categories that were not seen during the training.

This is important because few-shot learning – where an AI model receives very little training data – often involves little or only one sample per category. Most image classification approaches include only a few labels, with each training image containing a single object with a matching label. In the paper, however, the scientists investigate multi-label few-shot learning, in which training images contain multiple objects from different category labels.

The researchers trained multiple LaSO networks as a single multi-task network, based on a set of multiple labels per image belonging to the objects in that image. They then evaluated how well the network could classify the output examples, using a classifier that had been trained on multi-label data. In a separate experiment, the team used the networks to create additional examples from random pairs of the few training examples, and designed a new benchmark for multi-label few-shot classification.