how to get latest updates here to my deployed fabric framework

[khalednsair]

how to get latest updates here to my deployed fabric framework, process and technical steps ?
does those updates overwrites any changes i have done to pipelines / config database?

[pballada]

Updating is the same as deploying: run the setup notebook again. Here's how it works and what it overwrites, based on setup/NB_SETUP_FMD.ipynb.

Steps

1. Download the latest setup/NB_SETUP_FMD.ipynb from this repo and import it into your workspace, replacing your old copy.
2. Fill in the configuration cells with the same values you used the first time: domain names, capacity, workspace roles and so on. The notebook then targets your existing workspaces and database instead of creating new ones.
3. Decide on the two switches below, then run it.

When it runs, the notebook downloads src and config fresh from GitHub (repo_owner = "edkreuk", repo_name = "FMD_FRAMEWORK", branch = "main"). So you always get the current version of the framework items, not the ones from your first install.

What gets overwritten

• Notebooks, environments, the variable library and data pipelines are imported with fab import <workspace>.Workspace/<item> -i <path> -f. The -f replaces an existing item of the same name. Changes you made inside the framework's own pipelines or notebooks are lost. Keep your customisations in items with your own names, or export them first and reapply them.
• Variable library: overwrite_variable_library = True is the default ("By default the Library is overwritten, change this to False if you have custom changes"). Set it to False if you edited values there.

What is kept

• Configuration database metadata. Connections, data sources, workspaces, lakehouses, pipelines and entities are registered through upsert procedures ([integration].[sp_UpsertConnection], sp_UpsertDataSource, sp_UpsertWorkspace, sp_UpsertPipeline, sp_UpsertLakehouse, sp_UpsertLandingzoneEntity, sp_UpsertBronzeLayerEntity, sp_UpsertSilverLayerEntity). The framework's own rows get updated, and the rows you added stay.
• Demo data: set load_demo_data = False on the re-run if you don't want the demo entities registered again.

Before you run it

• Put the workspaces under Git integration, or export the items you changed, so you can compare and roll back.
• If you can, run it against a test domain first.
• After it finishes, check that your pipelines still have their connections. The deployment doc explains that pipelines deployed before CON_FMD_FABRIC_SQL exists end up with inactive audit activities, and a second run fixes that.