Microsoft Fabric metadata collector

concept australia Updated Jul 29, 2026 9 min read microsoft-fabric-metadata-collector.md
Raw file servicenow.com

Provides read-only access to metadata from a Microsoft Fabric account.

The collector harvests metadata from Microsoft Fabric workspaces and their child resources, including items commonly found in Power BI.

Metadata cataloged

The collector catalogs the following information.

Object

Information cataloged

Workspaces

ID, Name, Description

Warehouses

ID, Name, Description, Created Date, Last Modified Date, Created By, Modified By, Collation Type, Connection String, JDBC URL, Endorsement (Certified By, Endorsement Badge)

Lakehouses

ID, Name, Description, Created Date, Last Modified Date, Created By, Modified By, OneLake Tables Path, OneLake Files Path, Connection String, JDBC URL, Endorsement (Certified By, Endorsement Badge)

Fabric Data Pipelines

ID, Name, Description, Created Date, Last Modified Date, Created By, Modified By, Parameters, Variables, Endorsement (Certified By, Endorsement Badge), Refresh Schedule

Eventhouses

ID, Name, Description, Created Date, Last Modified Date, Created By, Modified By, Endorsement (Certified By, Endorsement Badge), Ingestion Service URI, Query Service URI, State

Dataflows

ID, Name, Description, Last Modified Date, Refresh Schedule, Created By, Last Modified By, Endorsement (Certified By, Endorsement Badge)

Dataflow Gen2

ID, Name, Description, Created By, Last Modified Date, Last Modified By, Endorsement (Certified By, Endorsement Badge), Refresh Schedule

Dataflow Gen2 CI/CD

ID, Name, Description, Created Date, Created By, Last Modified Date, Last Modified By, Endorsement (Certified By, Endorsement Badge), Refresh Schedule, State

Mirrored Databases

OneLake Tables Path, Mirroring Status

Fabric Mirrored Database Table

Name, Description, Table Mirroring Status Extended Metadata: Created Date, Modified Date

Fabric Delta Lakehouse Table

Name, Description, Created Date, Last Modified Date, ABFS File Path

Fabric Lakehouse Column

Name, Description, Column Type, Is Nullable, Column Size, Column Index, Decimal Digits, Partition Index

Notebooks

ID, Name, Description, Created Date, Last Modified Date, Created By, Modified By, Definition, Endorsement (Certified By, Endorsement Badge), Refresh Schedule

Spark Job Definitions

ID, Name, Description, Created Date, Last Modified Date, Created By, Modified By, Endorsement (Certified By, Endorsement Badge), Refresh Schedule

SQL Analytics Endpoints

ID, Name, Description, Last Modified Date, Created By, Modified By, Provisioning Status, Connection String, Endorsement (Certified By, Endorsement Badge)

Lakehouse Folders

ID, Name, Description, Created Date, Last Modified Date, ABFS File Path

Lakehouse Files

ID, Name, Description, Created Date, Last Modified Date, ABFS File Path

Schemas

Name Extended Metadata: Created Date, Modified Date

Fabric Database Tables

Name, Description, Primary Key, ABFS File Path Extended metadata: Created date, Modified date

Database Columns

Name, Description, JDBC Type, Column Type, Is Nullable, Default Value, Key Type (Primary, Foreign), Column Size, Column Index, Decimal Digits

Views

Name, Description, SQL Definition

Reports

ID, Name, Description, Type, Preview Image (supported for paginated reports), Created Date, Last Modified Date, Created by, Modified by, External URL, Embed URL, Endorsement (Certified By, Endorsement Badge)

Report Pages

Name, Is Hidden

Dashboards

ID, Name, External URL, Embed URL

Dashboard Tiles

Name, Embed URL

Semantic Models

ID, Title, Description, Created Date, Created By, External URL, Refresh Schedule

Data Sources

Name, Type, Connection Details

Fabric Logical Tables

Name, Description, Is Hidden, Is Entered Data, Expression

Fabric Calculated Tables

Name, Description, Is Hidden, Is Entered Data, Expression

Fabric Logical Columns

Name, Description, Data Type, Column Type, Is Hidden, Expression

Measures

Name, Description, Is Hidden, Expression

Data Pipeline Activity

Name, Description, Type, Inactivity Status, Activity Policy (Retry, Timeout, Retry Interval In Secs, Secure Input, Secure Output)

Data Pipeline Run

Start Time, End Time, Invoke Type, Run Status

Activity Run

Start Time, End Time, Run Status, Output Message, Error Message

Job Run

ID, Job Type, Start Time, End Time, Run Status, Invocation Type, Failure Reason

KQL Database

ID, Name, Description, Created Date, Last Modified Date, Created By, Modified By, State, Endorsement (Certified By, Endorsement Badge)

App

ID, Name, Description

Org App

ID, Name, Description, Created Date, Last Modified Date, Created By, Modified By, Endorsement (Certified By, Endorsement Badge)

GraphQL API

ID, Name, Description, GraphQL Api Definition, Created Date, Last Modified Date, Created By, Modified By, Endorsement (Certified By, Endorsement Badge)

Refresh Schedule

ID, Frequency, Recurrence Interval, Time Of Day, Day Of Week, Day Of Month, Start Time, End Time, Enabled

Eventstreams

ID, Name, Description, Created Date, Last Modified Date, Created By, Modified By, event throughput level, retention time in days, state, Endorsement (Certified By, Endorsement Badge)

Eventstream Source

ID, Name, Status, Type

Eventstream Destination

ID, Name, Status, Type

Environments

ID, Name, Description, Created Date, Last Modified Date, Created By, Modified By, State, Library Dependency (name, library version, library type), Publish State, Spark Compute (driver cores, driver memory, executor cores, executor memory), Dynamic Executor Allocation (dynamic executor allocation state, max executors, min executors) Instance Pool (Type, Id, Name), Spark Property (key, value), Run Time Version, Publish Start Time, Publish End Time, Spark Libraries Publish State, Spark Settings Publish State, Endorsement (Certified By, Endorsement Badge)

Database resource types from SQL Server

For a complete list of metadata harvested from SQL server, see the Microsoft SQL Server collector documentation.

Variable library

ID, Name, Description, Created Date, Last Modified Date, Created By, Last Modified By, Active Value Set Name, Variable (variable type, value, note)

The Microsoft Fabric collector supports the profiling and sampling specific parameters which the SQL Server collector supports and these apply to Warehouses and Lakehouses. When profiling and sampling parameters are enabled, the following additional column information is cataloged:

Object

Information cataloged

Column

  • Average Length (sample)
  • Average Value (sample)
  • Data Distribution
  • Distinct Values
  • Estimated Distinct Values
  • Estimated Non-null Values
  • Maximum Length (sample)
  • Maximum Value (sample) sorted numerically or alphabetically (z-a)
  • Minimum Length (sample)
  • Minimum Value (sample) sorted numerically or alphabetically (a-z)
  • Non-null Values (sample)
  • Sample String Values (first 5 items in a column)

Table

  • Row Count
  • Sample Count (Target sample size)
## Relationship between objects

Catalog pages show relationships between the following data asset types:

Object Relationship
Workspace Warehouses, Lakehouses, Fabric Data Pipelines, Notebooks, Dataflows, Semantic Models, SQL Analytics Endpoints, Eventhouses, Spark Job Definitions, Reports, Dashboards, App, Org App, GraphQL API, Environments, Variable Library
App Workspace
Org App Workspace
GraphQL API Workspace
Warehouses Workspace, Schemas
Lakehouses Workspace, Schemas, Lakehouse Folders, Fabric Delta Lakehouse Table
Fabric Data Pipelines Workspace, Activities which belong to the Pipeline, Refresh Schedule, Variable Library, Data Pipeline Activities, Warehouses, Lakehouses, or SFTP files, Fabric Data Pipelines, Fabric Notebooks, Fabric Dataflows, or Fabric Spark Job Definitions executed by the Activity, Semantic Models refreshed by the Activity, Stored Procedures executed by the Activity
Eventhouses Workspace, KQL Database
KQL Databases Eventhouse
Dataflows Workspace, Logical Tables, Data Sources
Notebooks Workspace, Job Runs, Refresh Schedule, Environments
Spark Job Definitions Workspace, Job Runs, Refresh Schedule, Environments
SQL Analytics Endpoints Workspace, Lakehouse Folders, Lakehouse
Lakehouse Folders Lakehouse, Child Folders, Lakehouse Files
Lakehouse Files Lakehouse Folder
Schemas Lakehouse/Warehouse/Database, Tables, Views
Fabric Database Tables Schema, Columns
Fabric Mirrored Database Table Schema, Columns, Source Database Table
Database Columns Table or View
Views Schema, Columns
Reports Workspace, Dashboard Tile, Report Pages
Report Pages Report
Dashboards Workspace, Dashboard Tiles
Dashboard Tiles Dashboard
Semantic Models Workspace, Logical Tables, Data Sources
Data Sources Semantic Models, Dataflows
Dataflow Gen2 Workspace, Logical Tables, Data Sources, Data Destinations, Refresh Schedule
Dataflow Gen2 CI/CD Workspace, Logical Tables, Data Sources, Data Destinations, Refresh Schedule, Job Runs
Fabric Lakehouse Delta Tables Column, Lakehouse, Lakehouse Folder
Eventstream Workspace, Eventstream source, Eventstream destination
Eventstream Destination Published data to lakehouse table, KQL database
Environments Notebooks, Spark Job Definitions, Variable Library, Workspace, Data Pipeline
Notebooks Environments
Spark Job DefinitionsEnvironments
Variable Library Workspace Data Pipeline

Lineage and dependencies for Microsoft Fabric

The following lineage information is collected by the Microsoft Fabric collector.

Object

Lineage available

Database View

The collector identifies the associated column in an upstream view or table:

  • Where the data is sourced from
  • That sort the rows via ORDER BY
  • That filter the rows via WHERE/HAVING
  • That aggregate the rows via GROUP BY

Note: For views, the collector first tries to parse the view SQL to harvest lineage metadata. If the SQL parser cannot parse the view SQL, the collector catalogs some lineage relationships using the [dm\_sql\_referencing\_entities](https://learn.microsoft.com/en-us/sql/relational-databases/system-dynamic-management-views/sys-dm-sql-referenced-entities-transact-sql?view=sql-server-ver16#table-returned) system function, when available. For each row in the referenced entities, if is_selected or is_select_all is true, the collector catalogs a relationship between the referencing entity and the database column.

Semantic Model

Dataflows and Semantic Models this Semantic Model uses data from.

Dataflow

Other Dataflows this Dataflow uses data from.

Logical Table

Associated tables that the table sources its data from.

Note: The collector uses expressions returned by the Metadata Scan APIs to parse the lineage to the source columns and tables.

Calculated Table

Logical tables and columns from which the calculated table calculates its values.

Logical Column

Associated logical and database columns that the column sources its data from or calculates its values from.

Measure

Associated logical columns that the measure sources its data from.

Dashboard Tile

Associated Semantic Model.

Report

Associated Semantic Model and the Report this Report was published from.

Activity

Establishes lineage between tables or files an Activity copies data to and copies data from. For Copy Activities, see cross-system lineage for supported lineage.

Report Pages

Logical columns and measures from Semantic Model tables that the Report Page uses data from.

Fabric Delta Lakehouse Table

Sourced data from eventstream.

KQL Databases

Sourced data from eventstream.

**Supported cross-system lineage**

The following data sources support cross-system lineage with Microsoft Fabric.

Note: While other data sources are not formally supported, running the collector for those sources may still enable you to view cross-system lineage between Microsoft Fabric and those sources.

Object

Supported data sources

Semantic Models and Dataflows

  • Fabric Lakehouse
  • Fabric Warehouse
  • Oracle
  • Denodo
  • Snowflake
  • SQL Server
  • PostgreSQL
  • Redshift
  • Databricks
  • CSV documents

Fabric Data Pipelines

  • Fabric Delta Lakehouse Tables
  • Fabric Lakehouse Files and Folders
  • Fabric Warehouse Tables
  • SFTP Server Files
  • SQL Server Tables
  • SQL Server Stored Procedures
  • Oracle Tables

Dataflow Gen2 and Dataflow Gen2 CI/CD

  • Fabric Warehouses
  • Fabric Lakehouses
**Supported Power Query \(M\) functions and expressions for lineage metadata**

This section captures supported transformations, source expressions, calculated columns, and measure expressions when harvesting lineage metadata.

Note: Any table operations or transformations not listed in the following table as supported or unsupported are ignored.

Category

Supported and unsupported objects

Supported parameterized expressions

The collector parses source expressions that use parameters in place of the following elements: full source value, server or host value, warehouse value, database name, schema name, table name, and SQL expressions that incorporate parameters.

Supported data functions

Csv.Document, Excel.Workbook, File.Contents, Folder.Contents, Folder.Files, Json.Document, Odbc.DataSource, Odbc.InferOptions, Odbc.Query, Xml.Document, Web.Contents, Web.Headers, Web.BrowserContents, AmazonRedshift.Database, Sql.Database, Sql.Databases, Snowflake.Databases, PostgreSQL.Database, Databricks.Catalogs, Oracle.Database, Denodo.Contents, Databricks.Query, Lakehouse.Contents, Fabric.Warehouse

Supported table functions

Table.AddColumn, Table.AddIndexColumn, Table.RenameColumns, Table.NestedJoin, Table.ExpandTableColumn, Table.SplitColumn, Table.DuplicateColumn, Table.CombineColumns

Unsupported table operations

Table.Pivot, Table.PromoteHeaders, Table.DemoteHeaders, Table.PrefixColumns, Table.TransformColumnNames, Table.Unpivot, Table.UnpivotOtherColumns, Table.AddFuzzyClusterColumn, Table.AddJoinColumn, Table.AggregateTableColumn, Table.Combine, Table.CombineColumnsToRecord, Table.ExpandRecordColumn, Table.Join, Table.Transpose

Note: Contact support if you have expressions that use these unsupported table operations.

Supported dataflow functions

PowerPlatform.Dataflows, PowerBI.Dataflows

Supported value functions

Value.NativeQuery

Supported calculated columns

Lineage from calculated column expressions containing columns with and without table references. Columns or tables with alphanumeric characters, spaces, hyphens, and underscores are supported.

Supported measures

Lineage from measure expressions containing columns or tables with alphanumeric characters, spaces, hyphens, underscores, and surrounding quotes are supported.

**Dependencies**

Dependencies between Microsoft Fabric resources are cataloged from the Fabric metadata scan APIs. These relationships are visible in the Lineage Explorer.

Authentication supported

The collector supports Service principal authentication method.

Parent Topic:Configuring metadata collectors

Search documentation
Search document titles and file paths on the australia branch