Skip to content

Propagate and reconcile SharePoint metadata

Draft documentation

This guide is a draft. Validate it against the approved release package and customer operating procedure before using it for a production operation.

Use the metadata propagation and reconciliation tool to make selected Brief Connect record metadata available on documents in SharePoint Document Sets. You can also compare SharePoint values with the Brief Connect database and apply reviewed corrections.

The tool uses the Brief Connect database as the source of truth. It updates Document Set parent values and relies on SharePoint Shared Fields to propagate those values to child documents. It does not update child documents directly.

Use an authorised operator

Run this tool only from the approved Election Readiness tools virtual machine (VM). The operator needs access to the tool package and its output folder but does not need access to the secrets stored in Azure Key Vault.

Before you begin

Confirm that an administrator has:

  • Installed the complete win-x64 self-contained tool package on the VM.
  • Enabled the VM system-assigned managed identity and granted it access to the required Key Vault secrets.
  • Configured the Brief Connect Server App certificate and the required Microsoft Graph and SharePoint permissions.
  • Allowed network access from the VM to Key Vault, Microsoft Graph, SharePoint, and the active Brief Connect database services.
  • Granted you access to a protected output location with enough free space for all reports and logs.
  • Confirmed whether the environment uses the Modern or Legacy database model.

Use PowerShell for all commands in this guide. Run the commands from the folder containing E2.BriefConnect.MetadataPropagation.exe.

Create the run configuration

Create a separate configuration for the environment and scope you intend to process. Do not include passwords, certificates, access tokens, or connection strings. Store only resource identifiers and Key Vault secret names in the file.

The following example configures a Modern database run for one site, library, record type, and field:

{
  "Environment": {
    "KeyVaultUri": "https://<vault>.vault.azure.net/",
    "ApplicationDatabaseSecretName": "<database-connection-secret>",
    "MongoDatabasePasswordSecretName": "<database-password-secret>",
    "ClientId": "00000000-0000-0000-0000-000000000000",
    "TenantId": "00000000-0000-0000-0000-000000000000",
    "CertificateSecretName": "<server-app-certificate-secret>",
    "DbType": "Modern",
    "MongoDatabaseName": "BriefConnect"
  },
  "Run": {
    "WhatIf": true,
    "JobId": "metadata-review-20260918",
    "TriggerSharedFieldsPropagation": true,
    "OutputRoot": "runs",
    "CheckpointJournalPath": "runs/metadata-review-20260918/prepare/checkpoints.jsonl",
    "EventLogPath": "runs/metadata-review-20260918/prepare/events.csv",
    "CsvLogPath": "runs/metadata-review-20260918/prepare/report.csv",
    "DiagnosticLogPath": "runs/metadata-review-20260918/prepare/diagnostic.log",
    "JsonReportPath": "runs/metadata-review-20260918/prepare/report.json",
    "SharedFieldAudit": {
      "Scopes": [
        {
          "SiteUrl": "https://<tenant>.sharepoint.com/sites/<site>",
          "LibraryServerRelativeUrl": "/sites/<site>/<library>",
          "RecordTypeName": "<record-type>",
          "RecordPartitionKey": "<exact-partition-key>",
          "RecordIdentityFieldInternalName": "ABCRecordId",
          "CompareWithDatabase": true,
          "MaxDocumentSets": 1000,
          "MaxDatabaseRecords": 10000,
          "MaxAuditRows": 100000
        }
      ]
    }
  },
  "Sites": [
    {
      "SiteUrl": "https://<tenant>.sharepoint.com/sites/<site>"
    }
  ],
  "RecordTypes": [
    {
      "Name": "<record-type>",
      "AllFields": false,
      "Fields": [
        {
          "InternalName": "<field-internal-name>"
        }
      ]
    }
  ]
}

Set RecordTypes[].AllFields to true only when every active field on that record type is in scope. When AllFields is true, leave Fields empty.

For a Legacy database environment, obtain the approved Legacy configuration values from your Brief Connect administrator. A run must use only one database model and cannot fall back to the other model.

Keep output paths together

Set all five output paths explicitly and keep them under OutputRoot. The five paths are CheckpointJournalPath, EventLogPath, CsvLogPath, DiagnosticLogPath, and JsonReportPath. Use a different output folder for each phase.

Preview the Shared Fields preparation

Run preparation in its default what-if mode before making any SharePoint changes.

.\E2.BriefConnect.MetadataPropagation.exe `
  --config .\run.json `
  --prepare-shared-columns `
  --json-report .\runs\prepare\prepare.json `
  --csv-log .\runs\prepare\prepare.csv `
  --event-log .\runs\prepare\events.csv `
  --diagnostic-log .\runs\prepare\diagnostic.log `
  --checkpoint-journal .\runs\prepare\checkpoints.jsonl

Review the generated JSON and CSV reports. Confirm that:

  • The site, libraries, record types, content types, and fields match the approved scope.
  • The report contains no errors or unsupported fields.
  • The planned Shared Fields and propagation-trigger changes are expected.
  • ConfigurationFingerprint identifies the configuration you intend to use.
  • IsReadyForReconciliation is true, or the reported changes are safe to prepare.

If IsReadyForReconciliation is already true, retain the complete report generation and continue to Scan for metadata drift. Otherwise, perform live preparation.

Prepare Shared Fields

Run live preparation only after you have reviewed the what-if report. The confirmation value is case-sensitive.

.\E2.BriefConnect.MetadataPropagation.exe `
  --config .\run.json `
  --prepare-shared-columns `
  --what-if false `
  --confirm PREPARE `
  --json-report .\runs\prepare-live\prepare.json `
  --csv-log .\runs\prepare-live\prepare.csv `
  --event-log .\runs\prepare-live\events.csv `
  --diagnostic-log .\runs\prepare-live\diagnostic.log `
  --checkpoint-journal .\runs\prepare-live\checkpoints.jsonl

The tool adds existing configured fields to the Document Set Shared Fields configuration and pushes the configuration to relevant library content types. By default, it also sends an unchanged parent update to trigger SharePoint propagation. It does not create, remove, or change the type of SharePoint columns or content types.

Require exit code 0 and confirm that the live preparation report has IsReadyForReconciliation set to true. Retain the JSON, CSV, manifest, and associated artifact-generation folder together.

Scan for metadata drift

Use the retained preparation JSON to run a read-only comparison of database, Document Set parent, and child-document values.

.\E2.BriefConnect.MetadataPropagation.exe `
  --config .\run.json `
  --scan-drift `
  --preparation-report .\runs\prepare-live\prepare.json `
  --json-report .\runs\scan\scan.json `
  --csv-log .\runs\scan\scan.csv `
  --event-log .\runs\scan\events.csv `
  --diagnostic-log .\runs\scan\diagnostic.log `
  --checkpoint-journal .\runs\scan\checkpoints.jsonl

The scan is read-only even if WhatIf is false in the configuration.

Review the scan JSON, scan CSV, and the generated .shared-fields reports. Do not approve a scan that contains:

  • Errors, unsupported values, or stale outcomes.
  • Missing database records or missing Document Sets.
  • Parent, child, or database mismatches that you do not understand.
  • Truncated or incomplete coverage.
  • Sites, libraries, record types, fields, or record identifiers outside the approved scope.

Record the scan JSON ReportHash separately after approving the report. Do not edit the report. Editing a published report invalidates its integrity checks.

Apply the approved reconciliation

Use the exact retained preparation report, approved scan report, and approved ReportHash. The confirmation value is case-sensitive.

.\E2.BriefConnect.MetadataPropagation.exe `
  --config .\run.json `
  --apply-reconciliation `
  --approval-report .\runs\scan\scan.json `
  --preparation-report .\runs\prepare-live\prepare.json `
  --approval-hash <approved-64-character-report-hash> `
  --what-if false `
  --confirm APPLY `
  --json-report .\runs\apply\apply.json `
  --csv-log .\runs\apply\apply.csv `
  --event-log .\runs\apply\events.csv `
  --diagnostic-log .\runs\apply\diagnostic.log `
  --checkpoint-journal .\runs\apply\checkpoints.jsonl

Before each write, the tool re-reads the database and SharePoint values. It skips a unit when the approved value is stale or when the current value is already correct. It updates only the Document Set parent and then verifies the parent and child values.

Review RefreshRequired in the apply report. For each listed unit, run a new, narrowly scoped scan and obtain a new approval. Do not reuse the previous approval hash or use --force to bypass a stale result.

Verify Shared Fields

Run an independent read-only verification after apply or whenever you need to confirm parent-to-child propagation.

.\E2.BriefConnect.MetadataPropagation.exe `
  --config .\run.json `
  --verify-shared-fields `
  --preparation-report .\runs\prepare-live\prepare.json `
  --json-report .\runs\verify\verify.json `
  --csv-log .\runs\verify\verify.csv `
  --event-log .\runs\verify\events.csv `
  --diagnostic-log .\runs\verify\diagnostic.log `
  --checkpoint-journal .\runs\verify\checkpoints.jsonl

Require exit code 0. A mismatch, missing child value, Document Set without child documents, inventory failure, or configured-limit truncation causes a non-success result.

Resume an interrupted run

Preserve all reports, manifests, logs, and artifact-generation folders from the interrupted attempt. Resume from a supported CSV report or event log by adding --resume-log to the original phase command.

.\E2.BriefConnect.MetadataPropagation.exe `
  --config .\run.json `
  --scan-drift `
  --preparation-report .\runs\prepare-live\prepare.json `
  --resume-log .\runs\scan\scan.csv `
  --json-report .\runs\scan-resumed\scan.json `
  --csv-log .\runs\scan-resumed\scan.csv `
  --event-log .\runs\scan-resumed\events.csv `
  --diagnostic-log .\runs\scan-resumed\diagnostic.log `
  --checkpoint-journal .\runs\scan-resumed\checkpoints.jsonl

The tool validates resume evidence against the phase, configuration, unit, and relevant report hash. It confirms current state before skipping completed work. Use --force only when you intentionally need to reprocess completed units; it does not make stale or unsupported values safe.

Interpret exit codes

Exit code Meaning
0 The requested workflow completed successfully, or help was displayed.
1 The workflow completed with failures, mismatches, stale or unsupported units, missing records, or incomplete coverage.
2 Command, configuration, package, path, capacity, capability, or startup preflight validation failed.
130 The operator cancelled the run with Ctrl+C.

Handle a failed run

If preparation fails, correct the reported problem and repeat the what-if preparation before making another live attempt.

If scan or verification is interrupted, retain the partial generation and resume it. Never approve an incomplete report.

If apply is interrupted, assume that some Document Set parent values may already have changed. Retain all output, resume with the same approved report and hash, and run independent verification. The tool re-reads current values and treats already-correct units as no-write recovery outcomes.

If a report is truncated, divide the work into smaller, non-overlapping scopes and rerun it. Configuration properties that claim to allow truncation do not make truncated evidence complete.