Skip to content

aitools: keep installing plugins when the skills install fails - #7479

Open
anton-107 wants to merge 1 commit into
aitools/drop-proceed-confirmfrom
aitools/skills-failure-keeps-plugins
Open

aitools: keep installing plugins when the skills install fails#7479
anton-107 wants to merge 1 commit into
aitools/drop-proceed-confirmfrom
aitools/skills-failure-keeps-plugins

Conversation

@anton-107

@anton-107 anton-107 commented Nov 11, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Stacked on #7478.

Stacked on: aitools: drop the Proceed? confirm after the agent picker in install

Summary

databricks aitools install no longer stops when installing skill files fails. Agents that get a plugin (Claude Code) don't depend on the skill files, so their plugin is still installed. The skills error is still the command's error. The plugin is recorded in the install state without advancing state.Release, because the skill files on disk are still at the old release; otherwise aitools version and a pinned aitools update would report them as up to date. If skills from an install that predates state tracking are on disk with no state file, the plugin is not recorded at all, so aitools update and the install's legacy checks still see those skills.

 executePlan
   install skills for skills agents
-    on error: return error            # plugin agents never attempted
+    on error: remember skillsErr
   install plugins for plugin agents
-    on state write error: return err
+    on state write error: return errors.Join(skillsErr, err)
+    record plugins; if skillsErr, keep state.Release   # skills on disk are still at the old release
+    if skillsErr and untracked legacy skills on disk: don't create a plugin-only state file
+  if skillsErr: return skillsErr      # stays the top-level error
   if per-agent errors: return agentErrors

Changelog fragment: .nextchanges/cli/aitools-install-skills-failure-keeps-plugins.md.

Evidence

New test TestExecutePlanSkillsFailureStillInstallsPlugins (cmd/aitools/install_test.go): plan with Cursor (skills) and Claude Code (plugin), skills install returns an HTTP 503 manifest fetch error. Run with go test ./cmd/aitools -run TestExecutePlanSkillsFailureStillInstallsPlugins.

  • Before (cmd/aitools/install.go reverted to the parent): fails, no plugin is installed. 
    install_test.go:301: "[]" should have 1 item(s), but has 0
FAIL	github.com/databricks/cli/cmd/aitools
    After: passes. The error is still the fetch error (errors.Is), one plugin install is made, Claude Code's outcome is installed, and the saved state records the plugin while Release stays at the earlier v0.2.5.

New test TestExecutePlanSkillsAndStateWriteFailuresAreJoined: the skills install and the state write both fail. The returned error joins both (skills first), and its category is STATE_INVALID, because classifyInstallError ranks errors by type.

New test TestExecutePlanSkillsFailureKeepsUntrackedSkillsVisible: legacy skills on disk with no state file, and the skills install fails. The plugin is installed, but no state file is created, so HasUntrackedSkills (the signal aitools update and the install's legacy checks rely on) is still true. Without the guard, a plugin-only state file is written and the test fails.

Merge Danger

Door: two-way

Revert the commit to bring the old behavior back. Nothing is migrated. After a skills failure the state file now gets the plugin record (with Release unchanged), where before nothing was written. The exception is a global scope with skills on disk and no state file: there nothing is written, and the plugin is recorded by the next install whose skills step succeeds.

Blast Radius: small

  • The exit code doesn't change: a skills failure still fails the command with the same error. If the plugin step later also fails on writing the state, both errors are joined.
  • After a skills failure the command now keeps going and runs the plugin install, so the user waits for that work (and its network calls) before the command exits.
  • Only covered by unit tests with stubbed installers. Not run on Windows or macOS.

This pull request and its description were written by Isaac.

@anton-107
anton-107 requested review from a team and lennartkats-db as code owners November 11, 2026 14:27
@anton-107 anton-107 added the ai-auto PRs made by AI loops & factories, label Nov 11, 2026
@anton-107
anton-107 requested a review from a team as a code owner November 11, 2026 14:27
@anton-107
anton-107 requested a review from misha-db November 11, 2026 14:27
@anton-107 anton-107 mentioned this pull request Nov 11, 2026
Open
@anton-107
anton-107 force-pushed the aitools/drop-proceed-confirm branch from 7c162ba to 593c4fe Compare November 11, 2026 14:34
@anton-107
anton-107 requested a review from a team as a code owner November 11, 2026 14:34
@anton-107
anton-107 force-pushed the aitools/skills-failure-keeps-plugins branch from 891a0e3 to e15f2f9 Compare November 11, 2026 14:34
@github-actions

github-actions Bot commented Nov 11, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Commit: 2adef0363 | Workflow run

Regression test report

Selected 12 changed tests (limit=20, 0 not selected)

12 tests: regression=8 unreleased=0 coverage=4 failing=0
test branch main (8b4f052) latest (v1.26.0)
TestAccept/cmd/aitools/install-host-metadata ✅ ❌ ❌
TestAccept/cmd/aitools/list/explicit-ref ✅ ✅ ✅
TestAccept/cmd/aitools/list/latest ✅ ✅ ✅
TestAccept/cmd/aitools/list/latest-lookup-fails ✅ ✅ ✅
TestAccept/cmd/aitools/list/pinned ✅ ✅ ✅
TestAccept/cmd/aitools/telemetry/apps-init-tip ✅ ❌ ❌
TestAccept/cmd/aitools/telemetry/hint ✅ ❌ ❌
TestAccept/cmd/aitools/telemetry/install ✅ ❌ ❌
TestAccept/cmd/aitools/telemetry/install-failure ✅ ❌ ❌
TestAccept/cmd/aitools/telemetry/update-uninstall ✅ ❌ ❌
TestAccept/cmd/aitools/telemetry/workspace-unreachable ✅ ❌ ❌
TestAccept/cmd/version ✅ ❌ ❌
TestAccept/cmd/aitools/install-host-metadata: branch ✅ | main (8b4f052) ❌ | latest (v1.26.0) ❌

main (8b4f052):

acceptance_test.go:1213: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:46765" ("[DATABRICKS_HOST]")
    acceptance_test.go:1345: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1345
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1086
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:642
        	Error:      	Not equal: 
        	            	expected: "\n=== workspace unreachable: the background auth resolution prints nothing\n\n>>> [CLI] aitools install --skills-only --scope=global --agents=claude-code\nInstalling Databricks skills for Claude Code...\nUsing skills version test-ref\nFetching skills manifest...\nInstalled 1 skill.\n\n=== --debug still shows the hostmetadata line, at debug level\nDebug: [hostmetadata] failed to fetch host metadata for https://unreachable.test\n"
        	            	actual  : "\n=== workspace unreachable: the background auth resolution prints nothing\n\n>>> [CLI] aitools install --skills-only --scope=global --agents=claude-code\nWarn: [hostmetadata] failed to fetch host metadata for https://unreachable.test, will skip for 1m0s\nInstalling Databricks skills for Claude Code...\nUsing skills version test-ref\nFetching skills manifest...\nInstalled 1 skill.\n\n=== --debug still shows the hostmetadata line, at debug level\n\nExit code: 1\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -4,2 +4,3 @@
        	            	 >>> [CLI] aitools install --skills-only --scope=global --agents=claude-code
        	            	+Warn: [hostmetadata] failed to fetch host metadata for https://unreachable.test, will skip for 1m0s
        	            	 Installing Databricks skills for Claude Code...
        	            	@@ -10,3 +11,4 @@
        	            	 === --debug still shows the hostmetadata line, at debug level
        	            	-Debug: [hostmetadata] failed to fetch host metadata for https://unreachable.test
        	            	 
        	            	+Exit code: 1
        	            	+
        	Test:       	TestAccept/cmd/aitools/install-host-metadata
        	Messages:   	cmd/aitools/install-host-metadata/output.txt vs /tmp/TestAcceptcmdaitoolsinstall-host-metadata3917531813/002/output.txt

latest (v1.26.0):

acceptance_test.go:1213: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:39795" ("[DATABRICKS_HOST]")
    acceptance_test.go:1345: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1345
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1086
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:642
        	Error:      	Not equal: 
        	            	expected: "\n=== workspace unreachable: the background auth resolution prints nothing\n\n>>> [CLI] aitools install --skills-only --scope=global --agents=claude-code\nInstalling Databricks skills for Claude Code...\nUsing skills version test-ref\nFetching skills manifest...\nInstalled 1 skill.\n\n=== --debug still shows the hostmetadata line, at debug level\nDebug: [hostmetadata] failed to fetch host metadata for https://unreachable.test\n"
        	            	actual  : "\n=== workspace unreachable: the background auth resolution prints nothing\n\n>>> [CLI] aitools install --skills-only --scope=global --agents=claude-code\nWarn: [hostmetadata] failed to fetch host metadata for https://unreachable.test, will skip for 1m0s\nInstalling Databricks skills for Claude Code...\nUsing skills version test-ref\nFetching skills manifest...\nInstalled 1 skill.\n\n=== --debug still shows the hostmetadata line, at debug level\n\nExit code: 1\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -4,2 +4,3 @@
        	            	 >>> [CLI] aitools install --skills-only --scope=global --agents=claude-code
        	            	+Warn: [hostmetadata] failed to fetch host metadata for https://unreachable.test, will skip for 1m0s
        	            	 Installing Databricks skills for Claude Code...
        	            	@@ -10,3 +11,4 @@
        	            	 === --debug still shows the hostmetadata line, at debug level
        	            	-Debug: [hostmetadata] failed to fetch host metadata for https://unreachable.test
        	            	 
        	            	+Exit code: 1
        	            	+
        	Test:       	TestAccept/cmd/aitools/install-host-metadata
        	Messages:   	cmd/aitools/install-host-metadata/output.txt vs /tmp/TestAcceptcmdaitoolsinstall-host-metadata833715022/002/output.txt
TestAccept/cmd/aitools/telemetry/apps-init-tip: branch ✅ | main (8b4f052) ❌ | latest (v1.26.0) ❌

main (8b4f052):

acceptance_test.go:1213: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:41825" ("[DATABRICKS_HOST]")
    acceptance_test.go:1213: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:41825" ("[DATABRICKS_HOST]")
    acceptance_test.go:1345: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1345
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1086
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:642
        	Error:      	Not equal: 
        	            	expected: "\n=== apps init with flags prints the tip: one hint-shown event per hint printed\n\n>>> [CLI] apps init --template ./template --skip-install --name app-tip\nDatabricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install\n✔ Creating project\n\n✔ Project created successfully!\n\n  Location: [TEST_TMP_DIR]/app-tip\n  Files: 3\n\nTip: coding agents detected without Databricks skills. Run 'databricks aitools install' to install them.\n\n>>> print_telemetry\n{\"command\":\"apps_init\",\"exit_code\":0,\"aitools_hint_event\":{\"hint_kind\":\"INSTALL_TEXT\"}}\n{\"command\":\"apps_init\",\"exit_code\":0,\"aitools_hint_event\":{\"hint_kind\":\"APPS_INIT_TIP\"}}\n\n=== plugin installed: no hint, no tip, no event\n\n>>> [CLI] apps init --template ./template --skip-install --name app-plugin\n✔ Creating project\n\n✔ Project created successfully!\n\n  Location: [TEST_TMP_DIR]/app-plugin\n  Files: 3\n\n\n>>> print_telemetry\n"
        	            	actual  : "\n=== apps init with flags prints the tip: one hint-shown event per hint printed\n\n>>> [CLI] apps init --template ./template --skip-install --name app-tip\nDatabricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install\n✔ Creating project\n\n✔ Project created successfully!\n\n  Location: [TEST_TMP_DIR]/app-tip\n  Files: 3\n\nTip: coding agents detected without Databricks skills. Run 'databricks aitools install' to install them.\n\n>>> print_telemetry\n\n=== plugin installed: no hint, no tip, no event\n\n>>> [CLI] apps init --template ./template --skip-install --name app-plugin\n✔ Creating project\n\n✔ Project created successfully!\n\n  Location: [TEST_TMP_DIR]/app-plugin\n  Files: 3\n\nTip: coding agents detected without Databricks skills. Run 'databricks aitools install' to install them.\n\n>>> print_telemetry\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -15,4 +15,2 @@
        	            	 >>> print_telemetry
        	            	-{"command":"apps_init","exit_code":0,"aitools_hint_event":{"hint_kind":"INSTALL_TEXT"}}
        	            	-{"command":"apps_init","exit_code":0,"aitools_hint_event":{"hint_kind":"APPS_INIT_TIP"}}
        	            	 
        	            	@@ -28,2 +26,3 @@
        	            	 
        	            	+Tip: coding agents detected without Databricks skills. Run 'databricks aitools install' to install them.
        	            	 
        	Test:       	TestAccept/cmd/aitools/telemetry/apps-init-tip
        	Messages:   	cmd/aitools/telemetry/apps-init-tip/output.txt vs /tmp/TestAcceptcmdaitoolstelemetryapps-init-tip2523280591/002/output.txt
    acceptance_test.go:1123: LOG.apps-init: 
    acceptance_test.go:1123: LOG.apps-init-plugin:

latest (v1.26.0):

acceptance_test.go:1213: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:42743" ("[DATABRICKS_HOST]")
    acceptance_test.go:1213: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:42743" ("[DATABRICKS_HOST]")
    acceptance_test.go:1345: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1345
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1086
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:642
        	Error:      	Not equal: 
        	            	expected: "\n=== apps init with flags prints the tip: one hint-shown event per hint printed\n\n>>> [CLI] apps init --template ./template --skip-install --name app-tip\nDatabricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install\n✔ Creating project\n\n✔ Project created successfully!\n\n  Location: [TEST_TMP_DIR]/app-tip\n  Files: 3\n\nTip: coding agents detected without Databricks skills. Run 'databricks aitools install' to install them.\n\n>>> print_telemetry\n{\"command\":\"apps_init\",\"exit_code\":0,\"aitools_hint_event\":{\"hint_kind\":\"INSTALL_TEXT\"}}\n{\"command\":\"apps_init\",\"exit_code\":0,\"aitools_hint_event\":{\"hint_kind\":\"APPS_INIT_TIP\"}}\n\n=== plugin installed: no hint, no tip, no event\n\n>>> [CLI] apps init --template ./template --skip-install --name app-plugin\n✔ Creating project\n\n✔ Project created successfully!\n\n  Location: [TEST_TMP_DIR]/app-plugin\n  Files: 3\n\n\n>>> print_telemetry\n"
        	            	actual  : "\n=== apps init with flags prints the tip: one hint-shown event per hint printed\n\n>>> [CLI] apps init --template ./template --skip-install --name app-tip\nDatabricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install\n✔ Creating project\n\n✔ Project created successfully!\n\n  Location: [TEST_TMP_DIR]/app-tip\n  Files: 3\n\nTip: coding agents detected without Databricks skills. Run 'databricks aitools install' to install them.\n\n>>> print_telemetry\n\n=== plugin installed: no hint, no tip, no event\n\n>>> [CLI] apps init --template ./template --skip-install --name app-plugin\n✔ Creating project\n\n✔ Project created successfully!\n\n  Location: [TEST_TMP_DIR]/app-plugin\n  Files: 3\n\nTip: coding agents detected without Databricks skills. Run 'databricks aitools install' to install them.\n\n>>> print_telemetry\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -15,4 +15,2 @@
        	            	 >>> print_telemetry
        	            	-{"command":"apps_init","exit_code":0,"aitools_hint_event":{"hint_kind":"INSTALL_TEXT"}}
        	            	-{"command":"apps_init","exit_code":0,"aitools_hint_event":{"hint_kind":"APPS_INIT_TIP"}}
        	            	 
        	            	@@ -28,2 +26,3 @@
        	            	 
        	            	+Tip: coding agents detected without Databricks skills. Run 'databricks aitools install' to install them.
        	            	 
        	Test:       	TestAccept/cmd/aitools/telemetry/apps-init-tip
        	Messages:   	cmd/aitools/telemetry/apps-init-tip/output.txt vs /tmp/TestAcceptcmdaitoolstelemetryapps-init-tip753001004/002/output.txt
    acceptance_test.go:1123: LOG.apps-init: 
    acceptance_test.go:1123: LOG.apps-init-plugin:
TestAccept/cmd/aitools/telemetry/hint: branch ✅ | main (8b4f052) ❌ | latest (v1.26.0) ❌

main (8b4f052):

acceptance_test.go:1213: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:34469" ("[DATABRICKS_HOST]")
    acceptance_test.go:1213: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:34469" ("[DATABRICKS_HOST]")
    acceptance_test.go:1345: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1345
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1086
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:642
        	Error:      	Not equal: 
        	            	expected: "\n=== hint printed on an authenticated command: one hint-shown event is uploaded\n\n>>> [CLI] jobs list\nDatabricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install\n\n>>> print_telemetry\n{\"command\":\"jobs_list\",\"exit_code\":0,\"aitools_hint_event\":{\"hint_kind\":\"INSTALL_TEXT\"}}\n\n=== hint printed on a command that resolves no auth: nothing is uploaded\n\n>>> [CLI] version\nDatabricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install\nDatabricks CLI v[CLI_VERSION]\n\n>>> print_telemetry\n\n=== plugin installed: no hint, no event\n\n>>> [CLI] jobs list\n\n>>> print_telemetry\n"
        	            	actual  : "\n=== hint printed on an authenticated command: one hint-shown event is uploaded\n\n>>> [CLI] jobs list\nDatabricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install\n\n>>> print_telemetry\n\n=== hint printed on a command that resolves no auth: nothing is uploaded\n\n>>> [CLI] version\nDatabricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install\nDatabricks CLI v[CLI_VERSION]\n\n>>> print_telemetry\n\n=== plugin installed: no hint, no event\n\n>>> [CLI] jobs list\n\n>>> print_telemetry\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -7,3 +7,2 @@
        	            	 >>> print_telemetry
        	            	-{"command":"jobs_list","exit_code":0,"aitools_hint_event":{"hint_kind":"INSTALL_TEXT"}}
        	            	 
        	Test:       	TestAccept/cmd/aitools/telemetry/hint
        	Messages:   	cmd/aitools/telemetry/hint/output.txt vs /tmp/TestAcceptcmdaitoolstelemetryhint3769298487/002/output.txt

latest (v1.26.0):

acceptance_test.go:1213: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:46503" ("[DATABRICKS_HOST]")
    acceptance_test.go:1213: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:46503" ("[DATABRICKS_HOST]")
    acceptance_test.go:1345: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1345
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1086
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:642
        	Error:      	Not equal: 
        	            	expected: "\n=== hint printed on an authenticated command: one hint-shown event is uploaded\n\n>>> [CLI] jobs list\nDatabricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install\n\n>>> print_telemetry\n{\"command\":\"jobs_list\",\"exit_code\":0,\"aitools_hint_event\":{\"hint_kind\":\"INSTALL_TEXT\"}}\n\n=== hint printed on a command that resolves no auth: nothing is uploaded\n\n>>> [CLI] version\nDatabricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install\nDatabricks CLI v[CLI_VERSION]\n\n>>> print_telemetry\n\n=== plugin installed: no hint, no event\n\n>>> [CLI] jobs list\n\n>>> print_telemetry\n"
        	            	actual  : "\n=== hint printed on an authenticated command: one hint-shown event is uploaded\n\n>>> [CLI] jobs list\nDatabricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install\n\n>>> print_telemetry\n\n=== hint printed on a command that resolves no auth: nothing is uploaded\n\n>>> [CLI] version\nDatabricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install\nDatabricks CLI v[CLI_VERSION]\n\n>>> print_telemetry\n\n=== plugin installed: no hint, no event\n\n>>> [CLI] jobs list\n\n>>> print_telemetry\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -7,3 +7,2 @@
        	            	 >>> print_telemetry
        	            	-{"command":"jobs_list","exit_code":0,"aitools_hint_event":{"hint_kind":"INSTALL_TEXT"}}
        	            	 
        	Test:       	TestAccept/cmd/aitools/telemetry/hint
        	Messages:   	cmd/aitools/telemetry/hint/output.txt vs /tmp/TestAcceptcmdaitoolstelemetryhint3999197819/002/output.txt
TestAccept/cmd/aitools/telemetry/install: branch ✅ | main (8b4f052) ❌ | latest (v1.26.0) ❌

main (8b4f052):

acceptance_test.go:1213: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:41497" ("[DATABRICKS_HOST]")
    acceptance_test.go:1213: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:41497" ("[DATABRICKS_HOST]")
    acceptance_test.go:1345: Diff:
        --- cmd/aitools/telemetry/install/output.txt
        +++ /tmp/TestAcceptcmdaitoolstelemetryinstall4020815420/002/output.txt
        @@ -8,14 +8,7 @@
         Installed 1 skill.
         
         >>> print_telemetry
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STARTED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"FLAGS_VALIDATED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"SCOPE_RESOLVED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"AGENTS_SELECTED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"PLAN_CONFIRMED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"SKILLS_INSTALLED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
         {"command":"aitools_install","exit_code":0,"aitools_install_event":{"agents":["CLAUDE_CODE"],"scope":"GLOBAL","error_category":"TYPE_UNSPECIFIED"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"FINISHED","outcome":"SUCCESS","interactive":false,"install_mode":"SKILLS_ONLY","agents":["CLAUDE_CODE"],"scope":"GLOBAL"}}
         
         === databricks-cli auth with a valid cached token: the event is uploaded
         
        @@ -26,14 +18,7 @@
         Installed 1 skill.
         
         >>> print_telemetry
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STARTED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"FLAGS_VALIDATED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"SCOPE_RESOLVED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"AGENTS_SELECTED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"PLAN_CONFIRMED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"SKILLS_INSTALLED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
         {"command":"aitools_install","exit_code":0,"aitools_install_event":{"agents":["CLAUDE_CODE"],"scope":"GLOBAL","error_category":"TYPE_UNSPECIFIED"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"FINISHED","outcome":"SUCCESS","interactive":false,"install_mode":"SKILLS_ONLY","agents":["CLAUDE_CODE"],"scope":"GLOBAL"}}
         
         === databricks-cli auth with an expired cached token: no refresh, nothing is uploaded
         
        @@ -44,6 +28,22 @@
         Installed 1 skill.
         
         >>> print_requests.py //oidc/v1/token //telemetry-ext
        +{
        +  "method": "POST",
        +  "path": "/oidc/v1/token",
        +  "raw_body": "client_id=databricks-cli&grant_type=refresh_token&refresh_token=test-refresh-token"
        +}
        +{
        +  "method": "POST",
        +  "path": "/telemetry-ext",
        +  "body": {
        +    "uploadTime": [UNIX_TIME_MILLIS],
        +    "items": [],
        +    "protoLogs": [
        +      "{\"frontend_log_event_id\":\"[UUID][0]\",\"entry\":{\"databricks_cli_log\":{\"execution_context\":{\"cmd_exec_id\":\"[UUID][1]\",\"version\":\"[CLI_VERSION]\",\"command\":\"aitools_install\",\"operating_system\":\"linux\",\"execution_time_ms\":6,\"exit_code\":0},\"aitools_install_event\":{\"agents\":[\"CLAUDE_CODE\"],\"scope\":\"GLOBAL\",\"error_category\":\"TYPE_UNSPECIFIED\"}}}}"
        +    ]
        +  }
        +}
         
         >>> gron.py [TEST_TMP_DIR]/home/.databricks/token-cache.json
        -json.tokens.test-profile.access_token = "expired-access-token";
        +json.tokens.test-profile.access_token = "oauth-token";
        
FAIL

latest (v1.26.0):

acceptance_test.go:1213: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:38871" ("[DATABRICKS_HOST]")
    acceptance_test.go:1213: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:38871" ("[DATABRICKS_HOST]")
    acceptance_test.go:1345: Diff:
        --- cmd/aitools/telemetry/install/output.txt
        +++ /tmp/TestAcceptcmdaitoolstelemetryinstall3197830777/002/output.txt
        @@ -8,14 +8,7 @@
         Installed 1 skill.
         
         >>> print_telemetry
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STARTED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"FLAGS_VALIDATED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"SCOPE_RESOLVED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"AGENTS_SELECTED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"PLAN_CONFIRMED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"SKILLS_INSTALLED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
         {"command":"aitools_install","exit_code":0,"aitools_install_event":{"agents":["CLAUDE_CODE"],"scope":"GLOBAL","error_category":"TYPE_UNSPECIFIED"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"FINISHED","outcome":"SUCCESS","interactive":false,"install_mode":"SKILLS_ONLY","agents":["CLAUDE_CODE"],"scope":"GLOBAL"}}
         
         === databricks-cli auth with a valid cached token: the event is uploaded
         
        @@ -26,14 +18,7 @@
         Installed 1 skill.
         
         >>> print_telemetry
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STARTED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"FLAGS_VALIDATED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"SCOPE_RESOLVED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"AGENTS_SELECTED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"PLAN_CONFIRMED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"SKILLS_INSTALLED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
         {"command":"aitools_install","exit_code":0,"aitools_install_event":{"agents":["CLAUDE_CODE"],"scope":"GLOBAL","error_category":"TYPE_UNSPECIFIED"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"FINISHED","outcome":"SUCCESS","interactive":false,"install_mode":"SKILLS_ONLY","agents":["CLAUDE_CODE"],"scope":"GLOBAL"}}
         
         === databricks-cli auth with an expired cached token: no refresh, nothing is uploaded
         
        @@ -44,6 +28,22 @@
         Installed 1 skill.
         
         >>> print_requests.py //oidc/v1/token //telemetry-ext
        +{
        +  "method": "POST",
        +  "path": "/oidc/v1/token",
        +  "raw_body": "client_id=databricks-cli&grant_type=refresh_token&refresh_token=test-refresh-token"
        +}
        +{
        +  "method": "POST",
        +  "path": "/telemetry-ext",
        +  "body": {
        +    "uploadTime": [UNIX_TIME_MILLIS],
        +    "items": [],
        +    "protoLogs": [
        +      "{\"frontend_log_event_id\":\"[UUID][0]\",\"entry\":{\"databricks_cli_log\":{\"execution_context\":{\"cmd_exec_id\":\"[UUID][1]\",\"version\":\"[CLI_VERSION]\",\"command\":\"aitools_install\",\"operating_system\":\"linux\",\"execution_time_ms\":11,\"exit_code\":0},\"aitools_install_event\":{\"agents\":[\"CLAUDE_CODE\"],\"scope\":\"GLOBAL\",\"error_category\":\"TYPE_UNSPECIFIED\"}}}}"
        +    ]
        +  }
        +}
         
         >>> gron.py [TEST_TMP_DIR]/home/.databricks/token-cache.json
        -json.tokens.test-profile.access_token = "expired-access-token";
        +json.tokens.test-profile.access_token = "oauth-token";
        
FAIL
TestAccept/cmd/aitools/telemetry/install-failure: branch ✅ | main (8b4f052) ❌ | latest (v1.26.0) ❌

main (8b4f052):

acceptance_test.go:1213: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:36947" ("[DATABRICKS_HOST]")
    acceptance_test.go:1213: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:36947" ("[DATABRICKS_HOST]")
    acceptance_test.go:1345: Diff:
        --- cmd/aitools/telemetry/install-failure/output.txt
        +++ /tmp/TestAcceptcmdaitoolstelemetryinstall-failure1248573910/002/output.txt
        @@ -8,12 +8,6 @@
         Error: skill "nonexistent" not found
         
         >>> print_telemetry
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STARTED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"FLAGS_VALIDATED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"SCOPE_RESOLVED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"AGENTS_SELECTED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"PLAN_CONFIRMED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"FINISHED","outcome":"FAILURE","error_category":"SKILL_NOT_FOUND","interactive":false,"install_mode":"SKILLS_ONLY","agents":["CURSOR"],"scope":"GLOBAL"}}
         {"command":"aitools_install","exit_code":1,"aitools_install_event":{"agents":["CURSOR"],"scope":"GLOBAL","error_category":"SKILL_NOT_FOUND"}}
         
         === a manifest the server does not have is MANIFEST_NOT_FOUND
        @@ -25,12 +19,6 @@
         Error: skills manifest at databricks-agent-skills@missing-ref: not found
         
         >>> print_telemetry
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STARTED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"FLAGS_VALIDATED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"SCOPE_RESOLVED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"AGENTS_SELECTED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"PLAN_CONFIRMED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"FINISHED","outcome":"FAILURE","error_category":"MANIFEST_NOT_FOUND","interactive":false,"install_mode":"SKILLS_ONLY","agents":["CURSOR"],"scope":"GLOBAL"}}
         {"command":"aitools_install","exit_code":1,"aitools_install_event":{"agents":["CURSOR"],"scope":"GLOBAL","error_category":"MANIFEST_NOT_FOUND"}}
         
         === any other HTTP error status from the manifest fetch is SKILLS_FETCH_FAILED
        @@ -42,13 +30,7 @@
         Error: failed to fetch manifest: HTTP 500
         
         >>> print_telemetry
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STARTED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"FLAGS_VALIDATED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"SCOPE_RESOLVED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"AGENTS_SELECTED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"PLAN_CONFIRMED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"FINISHED","outcome":"FAILURE","error_category":"SKILLS_FETCH_FAILED","interactive":false,"install_mode":"SKILLS_ONLY","agents":["CURSOR"],"scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":1,"aitools_install_event":{"agents":["CURSOR"],"scope":"GLOBAL","error_category":"SKILLS_FETCH_FAILED"}}
        +{"command":"aitools_install","exit_code":1,"aitools_install_event":{"agents":["CURSOR"],"scope":"GLOBAL","error_category":"UNCATEGORIZED"}}
         
         === a named agent skipped for the scope takes its category from the agent's outcome
         
        @@ -57,10 +39,4 @@
         Error: Cursor: does not support project-scoped skills
         
         >>> print_telemetry
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STARTED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"FLAGS_VALIDATED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"SCOPE_RESOLVED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"PROJECT"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"AGENTS_SELECTED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"PROJECT"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"PLAN_CONFIRMED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"PROJECT"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"FINISHED","outcome":"FAILURE","error_category":"UNSUPPORTED_SCOPE","interactive":false,"install_mode":"SKILLS_ONLY","agents":["CURSOR"],"scope":"PROJECT"}}
         {"command":"aitools_install","exit_code":1,"aitools_install_event":{"agents":["CURSOR"],"scope":"PROJECT","error_category":"TYPE_UNSPECIFIED","agent_results":[{"agent":"CURSOR","error_category":"UNSUPPORTED_SCOPE"}]}}

latest (v1.26.0):

acceptance_test.go:1213: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:40937" ("[DATABRICKS_HOST]")
    acceptance_test.go:1213: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:40937" ("[DATABRICKS_HOST]")
    acceptance_test.go:1345: Diff:
        --- cmd/aitools/telemetry/install-failure/output.txt
        +++ /tmp/TestAcceptcmdaitoolstelemetryinstall-failure1617174388/002/output.txt
        @@ -8,12 +8,6 @@
         Error: skill "nonexistent" not found
         
         >>> print_telemetry
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STARTED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"FLAGS_VALIDATED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"SCOPE_RESOLVED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"AGENTS_SELECTED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"PLAN_CONFIRMED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"FINISHED","outcome":"FAILURE","error_category":"SKILL_NOT_FOUND","interactive":false,"install_mode":"SKILLS_ONLY","agents":["CURSOR"],"scope":"GLOBAL"}}
         {"command":"aitools_install","exit_code":1,"aitools_install_event":{"agents":["CURSOR"],"scope":"GLOBAL","error_category":"SKILL_NOT_FOUND"}}
         
         === a manifest the server does not have is MANIFEST_NOT_FOUND
        @@ -25,12 +19,6 @@
         Error: skills manifest at databricks-agent-skills@missing-ref: not found
         
         >>> print_telemetry
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STARTED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"FLAGS_VALIDATED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"SCOPE_RESOLVED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"AGENTS_SELECTED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"PLAN_CONFIRMED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"FINISHED","outcome":"FAILURE","error_category":"MANIFEST_NOT_FOUND","interactive":false,"install_mode":"SKILLS_ONLY","agents":["CURSOR"],"scope":"GLOBAL"}}
         {"command":"aitools_install","exit_code":1,"aitools_install_event":{"agents":["CURSOR"],"scope":"GLOBAL","error_category":"MANIFEST_NOT_FOUND"}}
         
         === any other HTTP error status from the manifest fetch is SKILLS_FETCH_FAILED
        @@ -42,13 +30,7 @@
         Error: failed to fetch manifest: HTTP 500
         
         >>> print_telemetry
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STARTED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"FLAGS_VALIDATED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"SCOPE_RESOLVED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"AGENTS_SELECTED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"PLAN_CONFIRMED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"FINISHED","outcome":"FAILURE","error_category":"SKILLS_FETCH_FAILED","interactive":false,"install_mode":"SKILLS_ONLY","agents":["CURSOR"],"scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":1,"aitools_install_event":{"agents":["CURSOR"],"scope":"GLOBAL","error_category":"SKILLS_FETCH_FAILED"}}
        +{"command":"aitools_install","exit_code":1,"aitools_install_event":{"agents":["CURSOR"],"scope":"GLOBAL","error_category":"UNCATEGORIZED"}}
         
         === a named agent skipped for the scope takes its category from the agent's outcome
         
        @@ -57,10 +39,4 @@
         Error: Cursor: does not support project-scoped skills
         
         >>> print_telemetry
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STARTED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"FLAGS_VALIDATED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"SCOPE_RESOLVED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"PROJECT"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"AGENTS_SELECTED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"PROJECT"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"PLAN_CONFIRMED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"PROJECT"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"FINISHED","outcome":"FAILURE","error_category":"UNSUPPORTED_SCOPE","interactive":false,"install_mode":"SKILLS_ONLY","agents":["CURSOR"],"scope":"PROJECT"}}
         {"command":"aitools_install","exit_code":1,"aitools_install_event":{"agents":["CURSOR"],"scope":"PROJECT","error_category":"TYPE_UNSPECIFIED","agent_results":[{"agent":"CURSOR","error_category":"UNSUPPORTED_SCOPE"}]}}
TestAccept/cmd/aitools/telemetry/update-uninstall: branch ✅ | main (8b4f052) ❌ | latest (v1.26.0) ❌

main (8b4f052):

acceptance_test.go:1213: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:37329" ("[DATABRICKS_HOST]")
    acceptance_test.go:1213: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:37329" ("[DATABRICKS_HOST]")
    acceptance_test.go:1345: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1345
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1086
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:642
        	Error:      	Not equal: 
        	            	expected: "\n=== update: STARTED, FINISHED with SUCCESS\n\n>>> [CLI] aitools update --scope=global\nAlready up to date.\n\n>>> print_telemetry\n{\"command\":\"aitools_update\",\"exit_code\":0,\"aitools_command_event\":{\"command\":\"UPDATE\",\"stage\":\"STARTED\",\"interactive\":false}}\n{\"command\":\"aitools_update\",\"exit_code\":0,\"aitools_command_event\":{\"command\":\"UPDATE\",\"stage\":\"FINISHED\",\"outcome\":\"SUCCESS\",\"interactive\":false,\"scope\":\"GLOBAL\"}}\n\n=== uninstall: STARTED, FINISHED with SUCCESS\n\n>>> [CLI] aitools uninstall --scope=global\nUninstalled 1 skill.\n\n>>> print_telemetry\n{\"command\":\"aitools_uninstall\",\"exit_code\":0,\"aitools_command_event\":{\"command\":\"UNINSTALL\",\"stage\":\"STARTED\",\"interactive\":false}}\n{\"command\":\"aitools_uninstall\",\"exit_code\":0,\"aitools_command_event\":{\"command\":\"UNINSTALL\",\"stage\":\"FINISHED\",\"outcome\":\"SUCCESS\",\"interactive\":false,\"scope\":\"GLOBAL\"}}\n"
        	            	actual  : "\n=== update: STARTED, FINISHED with SUCCESS\n\n>>> [CLI] aitools update --scope=global\nAlready up to date.\n\n>>> print_telemetry\n\n=== uninstall: STARTED, FINISHED with SUCCESS\n\n>>> [CLI] aitools uninstall --scope=global\nUninstalled 1 skill.\n\n>>> print_telemetry\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -7,4 +7,2 @@
        	            	 >>> print_telemetry
        	            	-{"command":"aitools_update","exit_code":0,"aitools_command_event":{"command":"UPDATE","stage":"STARTED","interactive":false}}
        	            	-{"command":"aitools_update","exit_code":0,"aitools_command_event":{"command":"UPDATE","stage":"FINISHED","outcome":"SUCCESS","interactive":false,"scope":"GLOBAL"}}
        	            	 
        	            	@@ -16,4 +14,2 @@
        	            	 >>> print_telemetry
        	            	-{"command":"aitools_uninstall","exit_code":0,"aitools_command_event":{"command":"UNINSTALL","stage":"STARTED","interactive":false}}
        	            	-{"command":"aitools_uninstall","exit_code":0,"aitools_command_event":{"command":"UNINSTALL","stage":"FINISHED","outcome":"SUCCESS","interactive":false,"scope":"GLOBAL"}}
        	            	 
        	Test:       	TestAccept/cmd/aitools/telemetry/update-uninstall
        	Messages:   	cmd/aitools/telemetry/update-uninstall/output.txt vs /tmp/TestAcceptcmdaitoolstelemetryupdate-uninstall2050213570/002/output.txt
    acceptance_test.go:1123: 
        LOG.install: Installing Databricks skills for Cursor...
        LOG.install: Using skills version test-ref
        LOG.install: Fetching skills manifest...
        LOG.install: Installed 1 skill.
    acceptance_test.go:1123: LOG.install-telemetry: {"command":"aitools_install","exit_code":0,"aitools_install_event":{"agents":["CURSOR"],"scope":"GLOBAL","error_category":"TYPE_UNSPECIFIED"}}

latest (v1.26.0):

acceptance_test.go:1213: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:34527" ("[DATABRICKS_HOST]")
    acceptance_test.go:1213: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:34527" ("[DATABRICKS_HOST]")
    acceptance_test.go:1345: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1345
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1086
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:642
        	Error:      	Not equal: 
        	            	expected: "\n=== update: STARTED, FINISHED with SUCCESS\n\n>>> [CLI] aitools update --scope=global\nAlready up to date.\n\n>>> print_telemetry\n{\"command\":\"aitools_update\",\"exit_code\":0,\"aitools_command_event\":{\"command\":\"UPDATE\",\"stage\":\"STARTED\",\"interactive\":false}}\n{\"command\":\"aitools_update\",\"exit_code\":0,\"aitools_command_event\":{\"command\":\"UPDATE\",\"stage\":\"FINISHED\",\"outcome\":\"SUCCESS\",\"interactive\":false,\"scope\":\"GLOBAL\"}}\n\n=== uninstall: STARTED, FINISHED with SUCCESS\n\n>>> [CLI] aitools uninstall --scope=global\nUninstalled 1 skill.\n\n>>> print_telemetry\n{\"command\":\"aitools_uninstall\",\"exit_code\":0,\"aitools_command_event\":{\"command\":\"UNINSTALL\",\"stage\":\"STARTED\",\"interactive\":false}}\n{\"command\":\"aitools_uninstall\",\"exit_code\":0,\"aitools_command_event\":{\"command\":\"UNINSTALL\",\"stage\":\"FINISHED\",\"outcome\":\"SUCCESS\",\"interactive\":false,\"scope\":\"GLOBAL\"}}\n"
        	            	actual  : "\n=== update: STARTED, FINISHED with SUCCESS\n\n>>> [CLI] aitools update --scope=global\nAlready up to date.\n\n>>> print_telemetry\n\n=== uninstall: STARTED, FINISHED with SUCCESS\n\n>>> [CLI] aitools uninstall --scope=global\nUninstalled 1 skill.\n\n>>> print_telemetry\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -7,4 +7,2 @@
        	            	 >>> print_telemetry
        	            	-{"command":"aitools_update","exit_code":0,"aitools_command_event":{"command":"UPDATE","stage":"STARTED","interactive":false}}
        	            	-{"command":"aitools_update","exit_code":0,"aitools_command_event":{"command":"UPDATE","stage":"FINISHED","outcome":"SUCCESS","interactive":false,"scope":"GLOBAL"}}
        	            	 
        	            	@@ -16,4 +14,2 @@
        	            	 >>> print_telemetry
        	            	-{"command":"aitools_uninstall","exit_code":0,"aitools_command_event":{"command":"UNINSTALL","stage":"STARTED","interactive":false}}
        	            	-{"command":"aitools_uninstall","exit_code":0,"aitools_command_event":{"command":"UNINSTALL","stage":"FINISHED","outcome":"SUCCESS","interactive":false,"scope":"GLOBAL"}}
        	            	 
        	Test:       	TestAccept/cmd/aitools/telemetry/update-uninstall
        	Messages:   	cmd/aitools/telemetry/update-uninstall/output.txt vs /tmp/TestAcceptcmdaitoolstelemetryupdate-uninstall1243998857/002/output.txt
    acceptance_test.go:1123: 
        LOG.install: Installing Databricks skills for Cursor...
        LOG.install: Using skills version test-ref
        LOG.install: Fetching skills manifest...
        LOG.install: Installed 1 skill.
    acceptance_test.go:1123: LOG.install-telemetry: {"command":"aitools_install","exit_code":0,"aitools_install_event":{"agents":["CURSOR"],"scope":"GLOBAL","error_category":"TYPE_UNSPECIFIED"}}
TestAccept/cmd/aitools/telemetry/workspace-unreachable: branch ✅ | main (8b4f052) ❌ | latest (v1.26.0) ❌

main (8b4f052):

acceptance_test.go:1213: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:38629" ("[DATABRICKS_HOST]")
    acceptance_test.go:1213: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:38629" ("[DATABRICKS_HOST]")
    acceptance_test.go:1345: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1345
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1086
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:642
        	Error:      	Not equal: 
        	            	expected: "\n=== install: the background auth resolution prints nothing\n\n>>> [CLI] aitools install --skills-only --scope=global --agents=cursor\nInstalling Databricks skills for Cursor...\nUsing skills version test-ref\nFetching skills manifest...\nInstalled 1 skill.\n\n=== update: the background auth resolution prints nothing\n\n>>> [CLI] aitools update --scope=global\nAlready up to date.\n\n=== uninstall: the background auth resolution prints nothing\n\n>>> [CLI] aitools uninstall --scope=global\nUninstalled 1 skill.\n"
        	            	actual  : "\n=== install: the background auth resolution prints nothing\n\n>>> [CLI] aitools install --skills-only --scope=global --agents=cursor\nWarn: [hostmetadata] failed to fetch host metadata for https://unreachable.test, will skip for 1m0s\nInstalling Databricks skills for Cursor...\nUsing skills version test-ref\nFetching skills manifest...\nInstalled 1 skill.\n\n=== update: the background auth resolution prints nothing\n\n>>> [CLI] aitools update --scope=global\nAlready up to date.\n\n=== uninstall: the background auth resolution prints nothing\n\n>>> [CLI] aitools uninstall --scope=global\nUninstalled 1 skill.\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -4,2 +4,3 @@
        	            	 >>> [CLI] aitools install --skills-only --scope=global --agents=cursor
        	            	+Warn: [hostmetadata] failed to fetch host metadata for https://unreachable.test, will skip for 1m0s
        	            	 Installing Databricks skills for Cursor...
        	Test:       	TestAccept/cmd/aitools/telemetry/workspace-unreachable
        	Messages:   	cmd/aitools/telemetry/workspace-unreachable/output.txt vs /tmp/TestAcceptcmdaitoolstelemetryworkspace-unreachable3179399290/002/output.txt

latest (v1.26.0):

acceptance_test.go:1213: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:39055" ("[DATABRICKS_HOST]")
    acceptance_test.go:1213: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:39055" ("[DATABRICKS_HOST]")
    acceptance_test.go:1345: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1345
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1086
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:642
        	Error:      	Not equal: 
        	            	expected: "\n=== install: the background auth resolution prints nothing\n\n>>> [CLI] aitools install --skills-only --scope=global --agents=cursor\nInstalling Databricks skills for Cursor...\nUsing skills version test-ref\nFetching skills manifest...\nInstalled 1 skill.\n\n=== update: the background auth resolution prints nothing\n\n>>> [CLI] aitools update --scope=global\nAlready up to date.\n\n=== uninstall: the background auth resolution prints nothing\n\n>>> [CLI] aitools uninstall --scope=global\nUninstalled 1 skill.\n"
        	            	actual  : "\n=== install: the background auth resolution prints nothing\n\n>>> [CLI] aitools install --skills-only --scope=global --agents=cursor\nWarn: [hostmetadata] failed to fetch host metadata for https://unreachable.test, will skip for 1m0s\nInstalling Databricks skills for Cursor...\nUsing skills version test-ref\nFetching skills manifest...\nInstalled 1 skill.\n\n=== update: the background auth resolution prints nothing\n\n>>> [CLI] aitools update --scope=global\nAlready up to date.\n\n=== uninstall: the background auth resolution prints nothing\n\n>>> [CLI] aitools uninstall --scope=global\nUninstalled 1 skill.\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -4,2 +4,3 @@
        	            	 >>> [CLI] aitools install --skills-only --scope=global --agents=cursor
        	            	+Warn: [hostmetadata] failed to fetch host metadata for https://unreachable.test, will skip for 1m0s
        	            	 Installing Databricks skills for Cursor...
        	Test:       	TestAccept/cmd/aitools/telemetry/workspace-unreachable
        	Messages:   	cmd/aitools/telemetry/workspace-unreachable/output.txt vs /tmp/TestAcceptcmdaitoolstelemetryworkspace-unreachable1262876323/002/output.txt
TestAccept/cmd/version: branch ✅ | main (8b4f052) ❌ | latest (v1.26.0) ❌

main (8b4f052):

acceptance_test.go:1345: Diff:
        --- cmd/version/output.txt
        +++ /tmp/TestAcceptcmdversion3740701937/002/output.txt
        @@ -24,7 +24,6 @@
         >>> [CLI] version --check
         Databricks CLI v[CLI_VERSION]
         This is a development build; skipping the update check.
        -Tip: coding agents detected without Databricks skills. Run 'databricks aitools install' to install them.
         
         === version --check --output json with a coding agent prints no tip

latest (v1.26.0):

acceptance_test.go:1345: Diff:
        --- cmd/version/output.txt
        +++ /tmp/TestAcceptcmdversion2071503413/002/output.txt
        @@ -8,7 +8,7 @@
         
         >>> [CLI] version --check
         Databricks CLI v[CLI_VERSION]
        -This is a development build; skipping the update check.
        +Could not reach GitHub to check for a newer version. See https://github.com/databricks/cli/releases for the latest release.
         
         === version --check --output json
         
        @@ -16,15 +16,14 @@
         {
           "current_version": "[CLI_VERSION]",
           "update_available": false,
        -  "development_build": true
        +  "check_failed": true
         }
         
         === version --check with a coding agent and no Databricks tooling
         
         >>> [CLI] version --check
         Databricks CLI v[CLI_VERSION]
        -This is a development build; skipping the update check.
        -Tip: coding agents detected without Databricks skills. Run 'databricks aitools install' to install them.
        +Could not reach GitHub to check for a newer version. See https://github.com/databricks/cli/releases for the latest release.
         
         === version --check --output json with a coding agent prints no tip
         
        @@ -32,7 +25,7 @@
         {
           "current_version": "[CLI_VERSION]",
           "update_available": false,
        -  "development_build": true
        +  "check_failed": true
         }
         
         === version --check from Claude Code prints only the root hint
        @@ -40,4 +33,4 @@
         >>> [CLI] version --check
         Databricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install
         Databricks CLI v[CLI_VERSION]
        -This is a development build; skipping the update check.
        +Could not reach GitHub to check for a newer version. See https://github.com/databricks/cli/releases for the latest release.

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Nov 11, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

✅ Integration test report

Commit: 2adef03

Run: 43205953065

Env 🔄​flaky 💚​RECOVERED ✅​pass 🙈​skip Time
💚​ aws linux-2core-8gb 1 270 27 7:15
💚​ aws-windows-latest-4core-16gb 1 272 25 5:11
💚​ azure linux-2core-8gb 1 269 27 7:38
🔄​ azure-windows-latest-4core-16gb 2 1 269 25 6:19
💚​ gcp linux-2core-8gb 1 270 27 8:00
💚​ gcp-windows-latest-4core-16gb 1 272 25 5:14
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🔄​ TestFsCpDirToDirWithOverwriteFlag ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p
🔄​ TestFsCpDirToDirWithOverwriteFlag/uc-volumes_to_uc-volumes ✅​p ✅​p ✅​p 🔄​f ✅​p ✅​p
Top 6 slowest tests (at least 2 minutes):
duration env testname
3:58 aws linux-2core-8gb TestAccept
3:57 azure linux-2core-8gb TestAccept
3:57 gcp linux-2core-8gb TestAccept
3:52 gcp-windows-latest-4core-16gb TestAccept
3:50 azure-windows-latest-4core-16gb TestAccept
3:41 aws-windows-latest-4core-16gb TestAccept

@anton-107
anton-107 force-pushed the aitools/drop-proceed-confirm branch from 593c4fe to deca41f Compare November 11, 2026 16:53
@anton-107
anton-107 force-pushed the aitools/skills-failure-keeps-plugins branch from e15f2f9 to 0fb3e5c Compare November 11, 2026 16:53
@anton-107
anton-107 force-pushed the aitools/drop-proceed-confirm branch from deca41f to e980d56 Compare November 11, 2026 16:55
@anton-107
anton-107 force-pushed the aitools/skills-failure-keeps-plugins branch from 0fb3e5c to 520f504 Compare November 11, 2026 16:55
@anton-107
anton-107 force-pushed the aitools/drop-proceed-confirm branch from e980d56 to e155c53 Compare November 11, 2026 16:57
@anton-107
anton-107 force-pushed the aitools/skills-failure-keeps-plugins branch from 520f504 to 6709c89 Compare November 11, 2026 16:57
@anton-107
anton-107 force-pushed the aitools/drop-proceed-confirm branch from e155c53 to a791ac7 Compare November 11, 2026 19:34
@anton-107
anton-107 force-pushed the aitools/skills-failure-keeps-plugins branch from 6709c89 to e7d2703 Compare November 11, 2026 19:34
@anton-107
anton-107 force-pushed the aitools/drop-proceed-confirm branch from a791ac7 to a45b450 Compare November 11, 2026 19:42
@anton-107
anton-107 force-pushed the aitools/skills-failure-keeps-plugins branch 2 times, most recently from 912057b to 5f9958b Compare November 11, 2026 20:03
@anton-107
anton-107 force-pushed the aitools/drop-proceed-confirm branch from a45b450 to eb589c2 Compare November 11, 2026 20:07
@anton-107
anton-107 force-pushed the aitools/skills-failure-keeps-plugins branch from 5f9958b to 6f53d25 Compare November 11, 2026 20:07
@anton-107
anton-107 force-pushed the aitools/drop-proceed-confirm branch from eb589c2 to df49877 Compare November 11, 2026 21:04
@anton-107
anton-107 force-pushed the aitools/skills-failure-keeps-plugins branch from 6f53d25 to bced029 Compare November 11, 2026 21:04
simonfaltum
simonfaltum reviewed Nov 11, 2026
View reviewed changes

@simonfaltum simonfaltum left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated review from simonfaltum's review agent (pr-review skill) at bced029. Each finding was verified against the code; simonfaltum may follow up.

Not approving yet: part of the automated review did not complete.

This PR lets databricks aitools install keep installing plugins for plugin agents such as Claude Code when the skills-group install fails, and still returns the skills error as the command's error. The change is small, focused and green in CI. One non-blocking issue: after a skills failure, recording the plugin now advances the shared state.Release, so aitools version and, with a pinned ref, aitools update can report stale skills as up to date. The other comments are small nits: tests for the joined-error paths, a stale test comment, and the changelog wording.

Comments outside the changed lines

  • [nit] cmd/aitools/install_test.go:531: This comment says the skills-group path "returns before appending any per-agent outcome", but this PR removed that early return. The test still passes because a Cursor-only plan has no plugin items. Suggest rewording it, for example: "Cursor is skills-only, so this fails in the skills-group path, which records no per-agent outcome."

Comment thread cmd/aitools/install.go Outdated
@@ -630,7 +631,7 @@ func executePlan(ctx context.Context, src installer.ManifestSource, plan []agent
}
if len(records) > 0 {
if err := recordPluginInstallsFn(ctx, opts.Scope, records, ref); err != nil {

@simonfaltum simonfaltum Nov 11, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nice to have] Now that the plugin step runs after a skills failure, RecordPluginInstalls (libs/aitools/installer/plugin.go:487) still sets state.Release to the new ref, even though InstallSkillsForAgents returned before it saved state. The skills on disk and in state.Skills stay at the old release. Example: Cursor's skills are at v0.2.5, a v0.2.6 install fails during the skills fetch, and the Claude Code plugin is recorded. The state then says v0.2.6, aitools version reports "up to date" (cmd/aitools/version.go:132), and with a pinned ref (cli-compat pin or DATABRICKS_SKILLS_REF) aitools update stops at explicit && state.Release == latestTag (libs/aitools/installer/update.go:101) and prints "Already up to date." without updating Cursor's skills. Before this PR, the early return meant nothing was written after a skills failure. Suggest leaving Release unchanged when recording plugins after a skills failure (for example a flag or a separate record call), and adding a test that checks the saved state after the mixed failure.

@anton-107 anton-107 Nov 11, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. When the skills install failed, executePlan now records the plugins with an empty ref, and RecordPluginInstalls leaves state.Release unchanged for an empty ref (doc comment updated). TestExecutePlanSkillsFailureStillInstallsPlugins now runs the real RecordPluginInstalls against a temp HOME seeded with a v0.2.5 state, and checks that the saved state has the Claude Code plugin and Release still v0.2.5. Without the fix the test gets v0.2.6. A fresh install whose skills step fails gets Release empty, which matches having no skills installed.

Commit: cad6ebe

Comment thread cmd/aitools/install.go Outdated
ref, err := installer.GetPluginSkillsRef(ctx)
if err != nil {
return outcomes, err
return outcomes, errors.Join(skillsErr, err)

@simonfaltum simonfaltum Nov 11, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] The new errors.Join(skillsErr, err) returns (ref lookup here, state write at L634) have no tests. No test stubs recordPluginInstallsFn to fail, and the new test pins DATABRICKS_SKILLS_REF, so the ref lookup always succeeds. A case where the skills install and recordPluginInstallsFn both fail would pin down the joined message and the category picked for it. classifyInstallError ranks errors by type, not by order, so a typed plugin-side error such as StateError can outrank an untyped skills error.

@anton-107 anton-107 Nov 11, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added TestExecutePlanSkillsAndStateWriteFailuresAreJoined: an untyped skills error, and recordPluginInstallsFn returning a StateError. It pins the joined message (skills error first, then the state error), errors.Is for both, and the category STATE_INVALID, so the type ranking you describe is now explicit in a test. I didn't add a separate test for the ref-lookup path: it uses the same errors.Join(skillsErr, err) return, and making GetPluginSkillsRef fail would mean adding a new seam just for this test.

Commit: cad6ebe

@anton-107 anton-107 Nov 12, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update after rebasing onto the current #7478: the ref-lookup failure no longer returns an error there (the plugin version is recorded as unknown), so the only joined-error path left is the state write, and TestExecutePlanSkillsAndStateWriteFailuresAreJoined covers it. PR description updated to match.

Commit: 4f38e11

Comment thread .nextchanges/cli/aitools-install-skills-failure-keeps-plugins.md Outdated
@@ -0,0 +1 @@
* `databricks aitools install` still installs plugins for the other coding agents when installing skill files fails. ([#7479](https://github.com/databricks/cli/pull/7479))

@simonfaltum simonfaltum Nov 11, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] The pr-checklist (.agents/skills/pr-checklist/SKILL.md:75) asks for an imperative verb, or the command in the present tense with "now". "still installs" reads as if the behavior didn't change. Suggest: "* databricks aitools install now installs plugins for the other coding agents even when installing skill files fails. (#7479)"

@anton-107 anton-107 Nov 11, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Applied your wording, PR link kept: "databricks aitools install now installs plugins for the other coding agents even when installing skill files fails."

Commit: cad6ebe

@anton-107
anton-107 force-pushed the aitools/skills-failure-keeps-plugins branch from bced029 to cad6ebe Compare November 11, 2026 22:09
@anton-107
anton-107 force-pushed the aitools/drop-proceed-confirm branch from 24a70e8 to bf2f7d3 Compare November 11, 2026 22:27
@anton-107
anton-107 force-pushed the aitools/skills-failure-keeps-plugins branch from fade60a to e3135af Compare November 11, 2026 22:27
@anton-107
anton-107 force-pushed the aitools/drop-proceed-confirm branch from bf2f7d3 to a358695 Compare November 11, 2026 22:32
@anton-107
anton-107 force-pushed the aitools/skills-failure-keeps-plugins branch from e3135af to 5d98a15 Compare November 11, 2026 22:32
@anton-107
anton-107 force-pushed the aitools/drop-proceed-confirm branch from a358695 to 627fc32 Compare November 11, 2026 22:39
@anton-107
anton-107 force-pushed the aitools/skills-failure-keeps-plugins branch from 5d98a15 to 12aee78 Compare November 11, 2026 22:39
@anton-107
anton-107 force-pushed the aitools/drop-proceed-confirm branch from 627fc32 to 7aa11b6 Compare November 11, 2026 22:59
@anton-107
anton-107 force-pushed the aitools/skills-failure-keeps-plugins branch from 12aee78 to 08b09de Compare November 11, 2026 22:59
@anton-107
anton-107 force-pushed the aitools/drop-proceed-confirm branch 2 times, most recently from 473c8b4 to 1353f9b Compare November 12, 2026 00:27
@anton-107
anton-107 force-pushed the aitools/skills-failure-keeps-plugins branch from 08b09de to 4f38e11 Compare November 12, 2026 00:40
simonfaltum
simonfaltum reviewed Nov 12, 2026
View reviewed changes

@simonfaltum simonfaltum left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated review from simonfaltum's review agent (pr-review skill) at 4f38e11. Each finding was verified against the code; simonfaltum may follow up.

Not approving yet: part of the automated review did not complete.

This PR lets databricks aitools install keep installing plugins for agents like Claude Code when the skills-group install fails. The skills error stays the command's error, and the plugin is recorded without advancing state.Release. The change is small and well tested, and CI is green. The feedback from the earlier review round is all addressed. One non-blocking issue remains: on a machine with skills from an install that predates state tracking, a failed mixed install now writes a plugin-only state file, after which aitools update silently skips those skills.

Comment thread cmd/aitools/install.go
if skillsErr != nil {
releaseRef = ""
}
if err := recordPluginInstallsFn(ctx, opts.Scope, records, releaseRef); err != nil {

@simonfaltum simonfaltum Nov 12, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nice to have] On a machine with raw skills from an install that predates state tracking (no .state.json), a skills failure now leads RecordPluginInstalls to create a state file holding only the plugin and an empty Skills map. After that, aitools update treats the scope as plugin-only (state == nil || len(state.Skills) > 0, cmd/aitools/update.go:171) and exits 0 with no output. Before this PR it printed "found skills from a previous install without state tracking; run 'databricks aitools install' to refresh before updating", so Cursor's legacy skills now silently stop being updated. The new state also turns off the legacyInstall guards in InstallSkillsForAgents (libs/aitools/installer/installer.go:367-373): the block on --skills installs and the ban on the embedded-manifest fallback.

Example: legacy skills on disk, aitools install --agents cursor,claude-code --scope global hits a manifest 503, then aitools update does nothing. I checked this with a scratch test: with the base install.go, update returns the legacy error; at this head it returns nil and never calls updateSkillsFn. A plugin-only install already has this gap, but with this PR a run that was asked to install skills for Cursor, and failed, ends up there too.

Suggest not creating a new state file from the skills-failure path when raw skills are already on disk without state (for example, an option on RecordPluginInstalls backed by hasLegacyInstall), or running the legacy check in update when the state has plugins but no skills. A test that starts from a legacy skills dir with no state file would pin this down.

@anton-107 anton-107 Nov 12, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed with your first suggestion. New installer.HasUntrackedSkills(ctx, scope) reports skills on disk from an install that predates state tracking (global scope, no state file, hasLegacyInstall). When the skills install failed and that is true, executePlan installs the plugins but doesn't call RecordPluginInstalls, so no plugin-only state file is created. aitools update then still returns the "found skills from a previous install without state tracking" error, and the --skills block and the embedded-manifest fallback ban in InstallSkillsForAgents stay in effect. The plugin is recorded by the next install whose skills step succeeds.

New test TestExecutePlanSkillsFailureKeepsUntrackedSkillsVisible starts from a legacy skills dir with no state file and a manifest 503. It checks that the plugin is installed, that no state file exists afterwards, and that HasUntrackedSkills is still true. Without the guard, it fails on the plugin-only state file.

I left the pre-existing plugin-only gap (a successful plugin-only install on a machine with legacy skills) unchanged, because it's outside this PR's change.

Commit: d0d5fc2

@anton-107
anton-107 force-pushed the aitools/skills-failure-keeps-plugins branch from 4f38e11 to d0d5fc2 Compare November 12, 2026 08:36
@anton-107
anton-107 force-pushed the aitools/drop-proceed-confirm branch from 1353f9b to 172757e Compare November 12, 2026 08:47
@anton-107
anton-107 force-pushed the aitools/skills-failure-keeps-plugins branch from d0d5fc2 to 55882e3 Compare November 12, 2026 08:48
@anton-107
anton-107 force-pushed the aitools/drop-proceed-confirm branch from 172757e to da4d9e2 Compare November 12, 2026 08:50
@anton-107
anton-107 force-pushed the aitools/skills-failure-keeps-plugins branch from 55882e3 to 942cec5 Compare November 12, 2026 08:51
@anton-107
anton-107 force-pushed the aitools/drop-proceed-confirm branch from da4d9e2 to efae3db Compare November 12, 2026 09:00
@anton-107
anton-107 force-pushed the aitools/skills-failure-keeps-plugins branch from 942cec5 to ddd4225 Compare November 12, 2026 09:00
@anton-107
anton-107 force-pushed the aitools/drop-proceed-confirm branch from efae3db to 6aa9729 Compare November 12, 2026 09:06
@anton-107
anton-107 force-pushed the aitools/skills-failure-keeps-plugins branch from ddd4225 to 45b26f8 Compare November 12, 2026 09:06
@anton-107
anton-107 force-pushed the aitools/drop-proceed-confirm branch 2 times, most recently from f5bf3ea to ac0065e Compare November 12, 2026 09:19
@anton-107
anton-107 force-pushed the aitools/skills-failure-keeps-plugins branch from 45b26f8 to a3e42ce Compare November 12, 2026 09:30
@anton-107
anton-107 force-pushed the aitools/drop-proceed-confirm branch from ac0065e to 2a658e6 Compare November 12, 2026 10:01
@anton-107
anton-107 force-pushed the aitools/skills-failure-keeps-plugins branch from a3e42ce to cad677d Compare November 12, 2026 10:01
@anton-107
anton-107 force-pushed the aitools/drop-proceed-confirm branch from 2a658e6 to 51b22e3 Compare November 12, 2026 11:02
@anton-107 Isaac
aitools: keep installing plugins when the skills install fails
2adef03 
`databricks aitools install` no longer stops when installing skill files fails. Agents that get a plugin (Claude Code) don't depend on the skill files, so their plugin is still installed. The skills error is still the command's error. The plugin is recorded in the install state without advancing `state.Release`, because the skill files on disk are still at the old release; otherwise `aitools version` and a pinned `aitools update` would report them as up to date. If skills from an install that predates state tracking are on disk with no state file, the plugin is not recorded at all, so `aitools update` and the install's legacy checks still see those skills.

```diff
 executePlan
   install skills for skills agents
-    on error: return error            # plugin agents never attempted
+    on error: remember skillsErr
   install plugins for plugin agents
-    on state write error: return err
+    on state write error: return errors.Join(skillsErr, err)
+    record plugins; if skillsErr, keep state.Release   # skills on disk are still at the old release
+    if skillsErr and untracked legacy skills on disk: don't create a plugin-only state file
+  if skillsErr: return skillsErr      # stays the top-level error
   if per-agent errors: return agentErrors
```

Changelog fragment: `.nextchanges/cli/aitools-install-skills-failure-keeps-plugins.md`.

Co-authored-by: Isaac <no-reply@databricks.com>
@anton-107
anton-107 force-pushed the aitools/skills-failure-keeps-plugins branch from cad677d to 2adef03 Compare November 12, 2026 11:02

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@simonfaltum simonfaltum simonfaltum approved these changes
@lennartkats-db lennartkats-db Awaiting requested review from lennartkats-db lennartkats-db is a code owner
@misha-db misha-db Awaiting requested review from misha-db misha-db is a code owner automatically assigned from databricks/eng-deco-ax
@parthban-db parthban-db Awaiting requested review from parthban-db parthban-db was automatically assigned from databricks/eng-deco-platform

Assignees

No one assigned

Labels

ai-auto PRs made by AI loops & factories,

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@anton-107 @eng-dev-ecosystem-bot @simonfaltum