[ilyakuz-db]

Changes

bundle config-remote-sync now picks up permissions added remotely to a deployed resource whose config declares none, and writes them back as a permissions: block.

CalculatePlan gets an opt-in readUnmanagedPermissions argument. Only config-remote-sync passes true. With it, every deployed resource that supports the permissions API and has no permissions in config or state gets a Skip .permissions entry, and the planner reads its remote ACL into RemoteState. ExtractChanges turns that ACL into a single permissions add on the resource. It leaves out IS_OWNER and the current user's CAN_MANAGE, because deploy adds those itself (FixPermissions).

Declared permissions, bundle-level permissions and grants are still skipped, as in #5979.

Why

Permissions changed in the UI on a job or pipeline that the bundle never managed were silently ignored, so the synced config drifted from what is deployed. In the direct engine the permissions node only exists when the config or state already has it, so config-remote-sync never read those ACLs.

Deploy, destroy, bind and migrate pass false, so their plans are unchanged. A failed ACL read (e.g. the user can't read the resource's permissions) only logs a warning instead of failing the sync. Resource keys that can't be referenced (spaces, quotes) are skipped, because the permissions node's object_id is a reference to the parent's ID.

Tests

New acceptance test config-remote-sync/unmanaged_permissions (job with two added users, pipeline with a group; the owner entry is dropped, and a second sync reports no changes). It replaces the commented-out permission stubs in job_fields/pipeline_fields, which also run on cloud where those example users don't exist. Added unit tests for the ACL filter and node selection. Ran config-remote-sync, resources/permissions, deployment/bind, migrate and plan acceptance tests locally; lint-q clean.

[github-actions]

Commit: 0725cb9e3 | Workflow run

Regression test report

Selected 3 changed tests (limit=20, 0 not selected)

6 tests: regression=2 unreleased=0 coverage=4 failing=0

test	branch	main (28bbf47)	latest (v1.26.0)
TestAccept/bundle/config-remote-sync/job_fields/DMS=	✅	✅	✅
TestAccept/bundle/config-remote-sync/job_fields/DMS=true	✅	✅	✅
TestAccept/bundle/config-remote-sync/pipeline_fields/DMS=	✅	✅	✅
TestAccept/bundle/config-remote-sync/pipeline_fields/DMS=true	✅	✅	✅
TestAccept/bundle/config-remote-sync/unmanaged_permissions/DMS=	✅	❌	❌
TestAccept/bundle/config-remote-sync/unmanaged_permissions/DMS=true	✅	❌	❌

TestAccept/bundle/config-remote-sync/unmanaged_permissions/DMS=: branch ✅ | main (28bbf47) ❌ | latest (v1.26.0) ❌

main (28bbf47):

acceptance_test.go:1373: Diff:
        --- bundle/config-remote-sync/unmanaged_permissions/output.txt
        +++ /tmp/TestAcceptbundleconfig-remote-syncunmanaged_permissionsDMS=610213820/002/output.txt
        @@ -7,40 +7,13 @@
         === Add permissions remotely to resources that declare none
         
         === Detect and save changes
        -Detected changes in 2 resource(s):
        +No changes detected.
         
        -Resource: resources.jobs.my_job
        -  permissions: add
         
        -Resource: resources.pipelines.my_pipeline
        -  permissions: add
        -
        -
        -
         === Configuration changes
         
         >>> diff.py databricks.yml.backup databricks.yml
        ---- databricks.yml.backup
        -+++ databricks.yml
        -@@ -14,4 +14,9 @@
        -             num_workers: 1
         
        -+      permissions:
        -+        - level: CAN_MANAGE
        -+          user_name: viewer@example.com
        -+        - level: CAN_MANAGE_RUN
        -+          user_name: admin@example.com
        -   pipelines:
        -     my_pipeline:
        -@@ -20,4 +25,7 @@
        -         - notebook:
        -             path: /Users/{{workspace_user_name}}/notebook
        -+      permissions:
        -+        - group_name: users
        -+          level: CAN_VIEW
        -
        - targets:
        -
         === Sync again: the permissions are now in the config
         No changes detected.

latest (v1.26.0): same output as main (28bbf47)

TestAccept/bundle/config-remote-sync/unmanaged_permissions/DMS=true: branch ✅ | main (28bbf47) ❌ | latest (v1.26.0) ❌

main (28bbf47):

acceptance_test.go:1373: Diff:
        --- bundle/config-remote-sync/unmanaged_permissions/output.txt
        +++ /tmp/TestAcceptbundleconfig-remote-syncunmanaged_permissionsDMS=t364340315/002/output.txt
        @@ -7,40 +7,13 @@
         === Add permissions remotely to resources that declare none
         
         === Detect and save changes
        -Detected changes in 2 resource(s):
        +No changes detected.
         
        -Resource: resources.jobs.my_job
        -  permissions: add
         
        -Resource: resources.pipelines.my_pipeline
        -  permissions: add
        -
        -
        -
         === Configuration changes
         
         >>> diff.py databricks.yml.backup databricks.yml
        ---- databricks.yml.backup
        -+++ databricks.yml
        -@@ -14,4 +14,9 @@
        -             num_workers: 1
         
        -+      permissions:
        -+        - level: CAN_MANAGE
        -+          user_name: viewer@example.com
        -+        - level: CAN_MANAGE_RUN
        -+          user_name: admin@example.com
        -   pipelines:
        -     my_pipeline:
        -@@ -20,4 +25,7 @@
        -         - notebook:
        -             path: /Users/{{workspace_user_name}}/notebook
        -+      permissions:
        -+        - group_name: users
        -+          level: CAN_VIEW
        -
        - targets:
        -
         === Sync again: the permissions are now in the config
         No changes detected.

latest (v1.26.0): same output as main (28bbf47)

[eng-dev-ecosystem-bot]

✅ Integration test report

Commit: 0725cb9

Run: 43200963893

	Env	🔄​flaky	💚​RECOVERED	✅​pass	🙈​skip	Time
💚​	aws linux-2core-8gb		1	276	16	13:29
🔄​	aws-windows-latest-4core-16gb	1	1	271	16	8:55
💚​	azure linux-2core-8gb		1	275	16	11:43
💚​	azure-windows-latest-4core-16gb		1	271	16	7:38
💚​	gcp linux-2core-8gb		1	276	16	16:41
💚​	gcp-windows-latest-4core-16gb		1	272	16	11:11

	Test Name	aws linux-2core-8gb	aws-windows-latest-4core-16gb	azure linux-2core-8gb	azure-windows-latest-4core-16gb	gcp linux-2core-8gb	gcp-windows-latest-4core-16gb
💚​	TestAccept	💚​R	💚​R	💚​R	💚​R	💚​R	💚​R
🔄​	TestSyncIncrementalFileSync	✅​p	🔄​f	✅​p	✅​p	✅​p	✅​p

Top 11 slowest tests (at least 2 minutes):

duration	env	testname
3:55	azure linux-2core-8gb	TestAccept
3:51	gcp linux-2core-8gb	TestAccept
3:50	gcp-windows-latest-4core-16gb	TestAccept
3:44	aws linux-2core-8gb	TestAccept
3:33	azure-windows-latest-4core-16gb	TestAccept
3:33	aws-windows-latest-4core-16gb	TestAccept
3:07	aws linux-2core-8gb	TestImportDirWithOverwriteFlag
2:15	gcp-windows-latest-4core-16gb	TestExportDir
2:13	gcp linux-2core-8gb	TestFsCpDirToDirFileNotOverwritten/dbfs_to_dbfs
2:03	gcp linux-2core-8gb	TestFsCpFileToFileFileNotOverwritten/dbfs_to_uc-volumes
2:03	gcp linux-2core-8gb	TestImportDirWithOverwriteFlag