Facebook’s research division has open-sourced the code for PyTorch3D. PyTorch 3d is a toolkit designed to improve the development of artificial intelligence (AI) models that can work in three-dimensional environments.
The ability of AI models to understand 3D spaces is essential for certain applications. The most obvious fields are robotics, but also virtual and augmented reality. In addition, a traditional image recognition model, which normally only analyses two-dimensional photographs, can also benefit from knowledge of the actual, three-dimensional shape of an object. The extra context can help improve the accuracy of such a model.
Facebook argues that the development of this type of 3D recognition model has been relatively slow due to a lack of appropriate and targeted development tools. With the release of PyTorch3D, the social network is trying to overcome this, and thus make the learning curve less steep for the implementation of this type of models in applications.
Functionality
The first component of PyTorch3D is a data structure called Meshes. Digital 3D models are made up of so-called ‘meshes’, polygons that occur in many different variants, and can therefore sometimes be difficult to edit. The Meshes structure provides a uniform format for organising such shapes, which, according to Facebook, makes it easier for AI developers to work with their data.
Furthermore, developers can also build AI models to process their 3D models, using a number of loss functions and operators that are standard part of PyTorch 3d. With loss functions, errors in the training of an AI model can be recognised, making training faster. Operators are shortcuts in code that perform complex tasks (e.g. calculations for 3D models).
There is also a function in PyTorch3D with a rendering engine that converts 3D data into 2D images. These can then be viewed by developers or processed by applications. The engine can also add light and shadow effects. In addition, the engine has an API that makes it possible to export projects to widely used deep learning frameworks such as PyTorch, after which the 3D version is named.
17 hours ago
