Facebook wants to become more interesting for people. To this end, it is now carrying out extensive research into a neural network that should generate captions for photos that are more interesting for people. These are different tones that are touched and expressions that should make people think and interact.

Researchers at the Facebook AI unit want, for example, an expression like it was a beautiful day at the match to be added to a picture of a football stadium. Or a picture of a house that looks somewhat dark, a caption like Through this house and this street makes me feel so uncomfortable. In this way Facebook hopes to be more attractive for people.

Add personality

Researchers at Facebook have therefore developed a way to train a machine learning model to not only make dry factual representations of visual material, but also to respond in various ways to what is in the image. This should be more interesting for a person and, above all, should contain a certain personality.

Traditional machine learning tasks can only successfully describe a photo. Those descriptions as captions are not attractive to people. By adding some personality to the descriptions, this will change. The descriptions may vary, from a sweet message to something arrogant or anxious. A picture of a sandwich can be described as a delicious sandwich or I can make something better than this.

Several new techniques

The model performs a number of tasks. First, it analyses the image to determine its content. Secondly, a sentence is generated from that content. Both techniques are relatively new and the combination is revolutionary according to Facebook. However, the TransResNet model, as it is called, does not always work flawlessly. The image below shows the results that are produced with different images.

However, according to the developers, TransResNet is better than the standard benchmarks that are applied to images in order to generate a description of them. The vast majority of people in trials indicated that they found the caption with personality much more attractive than the one without.

This news article was automatically translated from Dutch to give Techzine.eu a head start. All news articles after September 1, 2019 are written in native English and NOT translated. All our background stories are written in native English as well. For more information read our launch article.