Microsoft CEO Satya Nadella describes Fabric as perhaps the biggest launch of a data product since SQL Server.
Indeed, Fabric is intended to provide a new foundation for enterprise data and AI applications. If you have a mountain of quality data as an organization and want to build an AI application with it, you already have the first part in place. However, the data must be able to rely on a good architecture, and that is where many large companies today tend to go wrong. They have greatly expanded their data infrastructure by embracing many specialized tools. Those tools are very good at what they are supposed to do, but the tool explosion also makes it difficult to deploy data properly throughout the organization.
OneDrive for data
Microsoft wants to change that first by looking at what could be better within its data product offerings. The tech company is implementing some rebranding, optimizations and combinations. The result is Fabric. This platform combines Power BI for data visualization, Azure Synapse Analytics for warehousing and Azure Data Factory for integration. In this way, Microsoft is working toward a single platform that supports multiple data workloads. Data scientists, engineers and other professionals can have as much access as possible to the tools they need for their work within this environment.
Microsoft has also established a strong link with its new solution, OneLake, to make Fabric possible. This tool is also described as the “OneDrive for data.” OneLake is a data lake that should simplify data management and promises to reduce duplication of data. Optimizing data for different analytics engines, such as Spark and T-SQL, should be a thing of the past because, within OneLake, all data is stored in Delta Parquet format. OneLake is automatically linked to every Fabric workload.
Data for every employee
This also takes us a little closer to another new endeavour of Fabric: it should enable a broad data culture in a company. In such a culture, employees in different departments and functions use more insights from reports and analytics than before. For example, HR staff and sales representatives could use them more, but if they want to do so now they often have to turn to data scientists and data engineers. They may or may not engage with the request and develop a desired insight after hours, days or weeks. A tool that promotes a broader data culture could, in theory, deliver specific insights and reports quickly without putting all requests directly to data experts.
An example of how Microsoft now wants to make this happen is the Direct Lake mode in visualization tool Power BI. This allows a real-time connection from a Power BI report to data in OneLake. This integration creates only one copy of data, so business users can always use the data without data fragmentation. The insights from this data can be used in the Microsoft 365 tools that are popular among business users. For example, Microsoft Teams, Outlook and Excel are supported.
Integrating data sources into Fabric
We already cited that Fabric aims to simplify the enterprise data landscape. Microsoft also refers to this growth of tools in the data infrastructure as the “modern data estate”: an environment with multiple clouds, accounts, databases, domains and engines. These things will continue to run within companies for quite some time if all goes well, simply because they are needed for data-related work. However, what OneLake wants to change is making it easier to bring data together in one place.
First, this simplification should take place with what Microsoft calls shortcuts. This feature allows the data team to virtualize data in OneLake without moving and duplicating the data. The shortcuts can combine data from different clouds, accounts and business lines for a data product. There are shortcuts for Azure Data Lake Storage Gen2, Amazon S3 and Microsoft Dataverse. In addition, Microsoft is making it possible to interface with Fabric in the Power Apps environment. In addition to data from the Power Platform, this directly links data from the ERP system Dynamics 365 to Fabric. This logically brings much knowledge about business processes to Microsoft’s new data environment.
A second feature with which Microsoft wants to achieve more simplicity is what the company calls Mirroring. This feature lets users add existing data warehouses and databases to Fabric. This will be useful for large companies that have been around for decades, as they often have many databases and warehouses with crucial data that cannot simply be replaced. Mirroring, therefore, replicates a snapshot of the database to OneLake in Delta Parquet format and keeps the replica up to date as much as possible. Initially, there will be support for Azure Cosmos DB, Azure SQL DB, Snowflake and Mongo DB, although Microsoft will work to add more data sources.
Role for Copilot as well
Finally, Microsoft is adding generative AI to Fabric in the form of Copilot. Over the past few months, the tech giant has been busy adding the generative AI assistant to various applications, especially focusing more and more on AI in business environments. In Fabric, this is reflected in how Microsoft supports data scientists and engineers. They can use Copilot for day-to-day work. An example of this is assistance in writing code: Copilot can, in fact, automatically finish the code. But it is also possible to use Copilot’s templates for building data pipelines and models. There is then always a basis, which, if all goes well, quickly fills in some of the work. Data scientists and engineers can then do the further perfecting and pro-code work.
All in all, Microsoft is taking quite a few steps with Fabric to bring about something in the data world. Whether it will match or surpass SQL Server remains to be seen. After all, Microsoft is not the first tech vendor to want to cause a stir in such a way, but a solid ambition certainly can’t hurt.
After a period of previews, Fabric has recently become generally available to any interested company.
2 days ago
