Fujitsu Laboratories in Japan wants to catch up with the trend of systems having to process more and more data. Today, the company announces that it has developed a new technology that can help with the rapid processing of big data in distributed storage systems in which information is stored on multiple disks.
The new technology has already been tried out in the open source Ceph framework and, according to Fujitsu, it works well enough to enable the smooth reading of data on such storage systems. In an extensive blog, the company explains exactly how the system works.
Delay due to file transfer
According to Fujitsu engineers, the large amounts of data going up and down between storage and servers cause the main delays in analytical systems. By already processing the data on the storage servers, the technicians think they can speed up the processes considerably, because it is not necessary to have the data switched at all from one server to another.
Fujitsu’s new technology, called Dataffinic Computing, connects multiple servers within a network, but also ensures that the servers retain their original storage functionality. The data that is spread over several servers can therefore be processed individually. This preserves the scalability of the system as a whole, while improving performance.
The improvements that have been achieved are considerable. According to Fujitsu, a Dataffinic test setup consisted of five storage locations and five servers connected to each other with a network that supported a speed of 1 Gbps. Within the test set-up, 50 gigabytes of video material had to be processed, which happened within fifty seconds. This is ten times faster than normal, because conventional methods would take about 500 seconds.
The next step that the developers want to take is also to confirm whether Dataffinic works with commercial applications. If this is the case, a new product with this architecture will have to be launched on the market in 2019.
This news article was automatically translated from Dutch to give Techzine.eu a head start. All news articles after September 1, 2019 are written in native English and NOT translated. All our background stories are written in native English as well. For more information read our launch article.
10 hours ago
