Variable library integration with pipelines

The Variable library is a new item type in Microsoft Fabric that allows users to define and manage variables at the workspace level, so they could soon be used across various workspace items, such as pipelines, notebooks, Shortcut for lakehouse and more. It provides a unified and centralized way to manage configurations, reducing the need for hardcoded values and simplifying your CI/CD processes, making it easier to manage configurations across different environments.

How to use Variable library with pipelines

1. Navigate to your workspace and create a new item.

   

2. Use the filter to find Variable library or scroll down to the Develop data section.

   

3. Select Variable library to create a new Variable library. Choose a name and hit Create.

   

4. Once the Variable library is created, you are taken to the homepage. Click + New to add a new variable.

   

5. Once you add a new variable, you can configure your variable and set the Name, Type, and Default value set. You can also add Notes.

   

6. Add Alternative value sets as you need (for example, different values for different deployment pipeline environments). After you set a name and click Create, you can edit the variable values and choose value set to make active.

   

   

7. Save your changes once you're done.

Use Variable library variables in your pipeline

1. To use a Variable library variable in your pipeline, create a new pipeline or navigate to an existing pipeline.

   

2. In your pipeline, you need to create a reference to your variable library variable in the bottom panel.

   

3. After you click + New, a pop-up will open that shows you all your Variable libraries. You can use this picker to select your Variable library variable and click Ok.

   

   

   

4. Once you have added your library variable references, add your pipeline activities. In the activity settings, click Add dynamic content for the setting you want to parameterize.

   

5. The expression builder opens. If you don't see Library variables, click the 3 dots next to Functions and select Library variables.

   

6. Click on your Library variable reference to add a new expression to the expression builder. Then, click Ok to add your expression.

   

   

7. You'll now see that the dynamic content is added to your activity setting.

   

8. You can preview your data by clicking Preview data. This will open a new pane that shows the value of your library variable. Click Ok to get a preview of your data.

   

   

9. Save and run your pipeline as you normally would. You'll see that the value passed is whatever value is set as Active in your Variable library.

   

Known limitations

The following known limitations apply to the integration of Variable library in pipelines in Data Factory in Microsoft Fabric:

• The Variable library supports Boolean, Datetime, Guid, Integer, Number, and String as data types. In your pipeline, you will see Boolean as Bool type, Datetime as String type, Guid as String type, Integer as Int type, and String as String type. Number types are not supported in pipelines.
• External connection parameterization is supported with variable library integrated with pipelines. However, you must look up the GUID for your connection from Settings | Manage connections and gateways. There you will find the GUID for your connection by selecting Settings next to your connection name.

Related content

• CI/CD for pipelines in Data Factory
• Parameters in pipelines
• Introduction to the CI/CD process as part of the ALM cycle in Microsoft Fabric