Skip to content

aitools: mention databricks aitools install in version --check - #7471

Open
anton-107 wants to merge 1 commit into
aitools/command-funnel-telemetryfrom
aitools/version-check-install-hint
Open

anton-107 wants to merge 1 commit into
aitools/command-funnel-telemetryfrom
aitools/version-check-install-hint

Conversation

@anton-107

@anton-107 anton-107 commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #7470.

Stacked on: aitools: add funnel telemetry to update, list, version and uninstall

This change is independent of the telemetry PRs below it in the stack; it touches cmd/version and adds one helper to libs/aitools/agents/recommend.go.

Summary

databricks version --check (text output) is followed on stderr by the apps init skills tip when a coding agent is detected without Databricks tooling. Stdout keeps only the update-check result:

 Databricks CLI v0.245.0
 You're on the latest version.
+Tip: coding agents detected without Databricks skills. Run 'databricks aitools install' to install them.

The same tip follows "A new version is available ...", the download link, "Could not reach GitHub ..." and the development-build line. It comes from agents.MaybeVersionCheckTip (next to PrintSkillsTip) and is not printed:

  • when an agent has the databricks plugin, either in its own manifest or recorded in the CLI install state (installer.InstalledTools, which covers Codex CLI and GitHub Copilot), or Databricks skills in the shared store, in the agent's own skills dir, or in the project dirs up to the repository root ((*Agent).DatabricksTooling);
  • when Claude Code drives the CLI: agents.MaybeHint already covers that caller with its per-session throttle, so a command prints at most one install hint;
  • on Databricks Runtime (like the other version notices), and for --output json.

It logs no hint telemetry, because databricks version never resolves auth and the event would not be uploaded. Changelog fragment: .nextchanges/cli/version-check-aitools-hint.md.

Evidence

TestMaybeVersionCheckTip in libs/aitools/agents/recommend_test.go covers the shown case and each hidden case: no agent, plugin in the agent manifest, plugin recorded in the install state, global skills, project skills, project skills at the repository root when run from a subdirectory, skills only in the agent's own skills dir, Databricks Runtime, Claude Code caller. TestRenderUpdateCheckAIToolsTip in cmd/version/version_test.go checks that the tip goes to stderr (development builds included), that a Codex plugin recorded in .state.json suppresses it end to end, and that --output json prints none. acceptance/cmd/version runs the real command: a committed codex-home/.codex shows the tip on stderr, --output json does not, and CLAUDECODE=1 prints only the root MaybeHint line. TestUpdateCheckTemplate checks that stdout is unchanged for all update-check outcomes.

  • Before (each gate in MaybeVersionCheckTip disabled in turn, tests kept): go test ./libs/aitools/agents ./cmd/version → --- FAIL for the matching cases (recorded plugin, project and agent skills, Claude Code caller, Databricks Runtime)
  • After (this branch): ok github.com/databricks/cli/libs/aitools/agents, ok github.com/databricks/cli/cmd/version, go test ./acceptance -run TestAccept/cmd/version → ok

Merge Danger

Door: two-way

Adds one stderr line after the text-mode update check; reverting removes it.

Blast Radius: small

  • Stdout of version --check is unchanged; the extra line is on stderr. --output json is unchanged.
  • databricks version without --check is unchanged.
  • No new network calls, waits or exit-code changes. The tip check reads local agent config dirs and the install state files.
  • Tested on Linux only, not on Windows or macOS.

This pull request and its description were written by Isaac.

@anton-107
anton-107 requested review from a team as code owners November 11, 2026 14:25
@anton-107 anton-107 added the ai-auto PRs made by AI loops & factories, label Nov 11, 2026
@anton-107
anton-107 requested a review from rugpanov November 11, 2026 14:25
@anton-107 anton-107 added the ai-auto PRs made by AI loops & factories, label Nov 11, 2026
@anton-107
anton-107 force-pushed the aitools/command-funnel-telemetry branch from a5568f8 to 2b4b8c6 Compare November 11, 2026 14:32
@anton-107
anton-107 requested review from a team and lennartkats-db as code owners November 11, 2026 14:32
@anton-107
anton-107 force-pushed the aitools/version-check-install-hint branch from 2e276b6 to 848f252 Compare November 11, 2026 14:32
@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Commit: 66114a45f | Workflow run

Regression test report

Selected 12 changed tests (limit=20, 0 not selected)

12 tests: regression=8 unreleased=0 coverage=4 failing=0
test branch main (8b4f052) latest (v1.26.0)
TestAccept/cmd/aitools/install-host-metadata ✅ ❌ ❌
TestAccept/cmd/aitools/list/explicit-ref ✅ ✅ ✅
TestAccept/cmd/aitools/list/latest ✅ ✅ ✅
TestAccept/cmd/aitools/list/latest-lookup-fails ✅ ✅ ✅
TestAccept/cmd/aitools/list/pinned ✅ ✅ ✅
TestAccept/cmd/aitools/telemetry/apps-init-tip ✅ ❌ ❌
TestAccept/cmd/aitools/telemetry/hint ✅ ❌ ❌
TestAccept/cmd/aitools/telemetry/install ✅ ❌ ❌
TestAccept/cmd/aitools/telemetry/install-failure ✅ ❌ ❌
TestAccept/cmd/aitools/telemetry/update-uninstall ✅ ❌ ❌
TestAccept/cmd/aitools/telemetry/workspace-unreachable ✅ ❌ ❌
TestAccept/cmd/version ✅ ❌ ❌
TestAccept/cmd/aitools/install-host-metadata: branch ✅ | main (8b4f052) ❌ | latest (v1.26.0) ❌

main (8b4f052):

acceptance_test.go:1213: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:45857" ("[DATABRICKS_HOST]")
    acceptance_test.go:1345: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1345
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1086
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:642
        	Error:      	Not equal: 
        	            	expected: "\n=== workspace unreachable: the background auth resolution prints nothing\n\n>>> [CLI] aitools install --skills-only --scope=global --agents=claude-code\nInstalling Databricks skills for Claude Code...\nUsing skills version test-ref\nFetching skills manifest...\nInstalled 1 skill.\n\n=== --debug still shows the hostmetadata line, at debug level\nDebug: [hostmetadata] failed to fetch host metadata for https://unreachable.test\n"
        	            	actual  : "\n=== workspace unreachable: the background auth resolution prints nothing\n\n>>> [CLI] aitools install --skills-only --scope=global --agents=claude-code\nWarn: [hostmetadata] failed to fetch host metadata for https://unreachable.test, will skip for 1m0s\nInstalling Databricks skills for Claude Code...\nUsing skills version test-ref\nFetching skills manifest...\nInstalled 1 skill.\n\n=== --debug still shows the hostmetadata line, at debug level\n\nExit code: 1\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -4,2 +4,3 @@
        	            	 >>> [CLI] aitools install --skills-only --scope=global --agents=claude-code
        	            	+Warn: [hostmetadata] failed to fetch host metadata for https://unreachable.test, will skip for 1m0s
        	            	 Installing Databricks skills for Claude Code...
        	            	@@ -10,3 +11,4 @@
        	            	 === --debug still shows the hostmetadata line, at debug level
        	            	-Debug: [hostmetadata] failed to fetch host metadata for https://unreachable.test
        	            	 
        	            	+Exit code: 1
        	            	+
        	Test:       	TestAccept/cmd/aitools/install-host-metadata
        	Messages:   	cmd/aitools/install-host-metadata/output.txt vs /tmp/TestAcceptcmdaitoolsinstall-host-metadata2050066103/002/output.txt

latest (v1.26.0):

acceptance_test.go:1213: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:46645" ("[DATABRICKS_HOST]")
    acceptance_test.go:1345: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1345
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1086
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:642
        	Error:      	Not equal: 
        	            	expected: "\n=== workspace unreachable: the background auth resolution prints nothing\n\n>>> [CLI] aitools install --skills-only --scope=global --agents=claude-code\nInstalling Databricks skills for Claude Code...\nUsing skills version test-ref\nFetching skills manifest...\nInstalled 1 skill.\n\n=== --debug still shows the hostmetadata line, at debug level\nDebug: [hostmetadata] failed to fetch host metadata for https://unreachable.test\n"
        	            	actual  : "\n=== workspace unreachable: the background auth resolution prints nothing\n\n>>> [CLI] aitools install --skills-only --scope=global --agents=claude-code\nWarn: [hostmetadata] failed to fetch host metadata for https://unreachable.test, will skip for 1m0s\nInstalling Databricks skills for Claude Code...\nUsing skills version test-ref\nFetching skills manifest...\nInstalled 1 skill.\n\n=== --debug still shows the hostmetadata line, at debug level\n\nExit code: 1\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -4,2 +4,3 @@
        	            	 >>> [CLI] aitools install --skills-only --scope=global --agents=claude-code
        	            	+Warn: [hostmetadata] failed to fetch host metadata for https://unreachable.test, will skip for 1m0s
        	            	 Installing Databricks skills for Claude Code...
        	            	@@ -10,3 +11,4 @@
        	            	 === --debug still shows the hostmetadata line, at debug level
        	            	-Debug: [hostmetadata] failed to fetch host metadata for https://unreachable.test
        	            	 
        	            	+Exit code: 1
        	            	+
        	Test:       	TestAccept/cmd/aitools/install-host-metadata
        	Messages:   	cmd/aitools/install-host-metadata/output.txt vs /tmp/TestAcceptcmdaitoolsinstall-host-metadata376885661/002/output.txt
TestAccept/cmd/aitools/telemetry/apps-init-tip: branch ✅ | main (8b4f052) ❌ | latest (v1.26.0) ❌

main (8b4f052):

acceptance_test.go:1213: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:37797" ("[DATABRICKS_HOST]")
    acceptance_test.go:1213: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:37797" ("[DATABRICKS_HOST]")
    acceptance_test.go:1345: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1345
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1086
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:642
        	Error:      	Not equal: 
        	            	expected: "\n=== apps init with flags prints the tip: one hint-shown event per hint printed\n\n>>> [CLI] apps init --template ./template --skip-install --name app-tip\nDatabricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install\n✔ Creating project\n\n✔ Project created successfully!\n\n  Location: [TEST_TMP_DIR]/app-tip\n  Files: 3\n\nTip: coding agents detected without Databricks skills. Run 'databricks aitools install' to install them.\n\n>>> print_telemetry\n{\"command\":\"apps_init\",\"exit_code\":0,\"aitools_hint_event\":{\"hint_kind\":\"INSTALL_TEXT\"}}\n{\"command\":\"apps_init\",\"exit_code\":0,\"aitools_hint_event\":{\"hint_kind\":\"APPS_INIT_TIP\"}}\n\n=== plugin installed: no hint, no tip, no event\n\n>>> [CLI] apps init --template ./template --skip-install --name app-plugin\n✔ Creating project\n\n✔ Project created successfully!\n\n  Location: [TEST_TMP_DIR]/app-plugin\n  Files: 3\n\n\n>>> print_telemetry\n"
        	            	actual  : "\n=== apps init with flags prints the tip: one hint-shown event per hint printed\n\n>>> [CLI] apps init --template ./template --skip-install --name app-tip\nDatabricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install\n✔ Creating project\n\n✔ Project created successfully!\n\n  Location: [TEST_TMP_DIR]/app-tip\n  Files: 3\n\nTip: coding agents detected without Databricks skills. Run 'databricks aitools install' to install them.\n\n>>> print_telemetry\n\n=== plugin installed: no hint, no tip, no event\n\n>>> [CLI] apps init --template ./template --skip-install --name app-plugin\n✔ Creating project\n\n✔ Project created successfully!\n\n  Location: [TEST_TMP_DIR]/app-plugin\n  Files: 3\n\nTip: coding agents detected without Databricks skills. Run 'databricks aitools install' to install them.\n\n>>> print_telemetry\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -15,4 +15,2 @@
        	            	 >>> print_telemetry
        	            	-{"command":"apps_init","exit_code":0,"aitools_hint_event":{"hint_kind":"INSTALL_TEXT"}}
        	            	-{"command":"apps_init","exit_code":0,"aitools_hint_event":{"hint_kind":"APPS_INIT_TIP"}}
        	            	 
        	            	@@ -28,2 +26,3 @@
        	            	 
        	            	+Tip: coding agents detected without Databricks skills. Run 'databricks aitools install' to install them.
        	            	 
        	Test:       	TestAccept/cmd/aitools/telemetry/apps-init-tip
        	Messages:   	cmd/aitools/telemetry/apps-init-tip/output.txt vs /tmp/TestAcceptcmdaitoolstelemetryapps-init-tip3578967814/002/output.txt
    acceptance_test.go:1123: LOG.apps-init: 
    acceptance_test.go:1123: LOG.apps-init-plugin:

latest (v1.26.0):

acceptance_test.go:1213: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:35671" ("[DATABRICKS_HOST]")
    acceptance_test.go:1213: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:35671" ("[DATABRICKS_HOST]")
    acceptance_test.go:1345: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1345
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1086
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:642
        	Error:      	Not equal: 
        	            	expected: "\n=== apps init with flags prints the tip: one hint-shown event per hint printed\n\n>>> [CLI] apps init --template ./template --skip-install --name app-tip\nDatabricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install\n✔ Creating project\n\n✔ Project created successfully!\n\n  Location: [TEST_TMP_DIR]/app-tip\n  Files: 3\n\nTip: coding agents detected without Databricks skills. Run 'databricks aitools install' to install them.\n\n>>> print_telemetry\n{\"command\":\"apps_init\",\"exit_code\":0,\"aitools_hint_event\":{\"hint_kind\":\"INSTALL_TEXT\"}}\n{\"command\":\"apps_init\",\"exit_code\":0,\"aitools_hint_event\":{\"hint_kind\":\"APPS_INIT_TIP\"}}\n\n=== plugin installed: no hint, no tip, no event\n\n>>> [CLI] apps init --template ./template --skip-install --name app-plugin\n✔ Creating project\n\n✔ Project created successfully!\n\n  Location: [TEST_TMP_DIR]/app-plugin\n  Files: 3\n\n\n>>> print_telemetry\n"
        	            	actual  : "\n=== apps init with flags prints the tip: one hint-shown event per hint printed\n\n>>> [CLI] apps init --template ./template --skip-install --name app-tip\nDatabricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install\n✔ Creating project\n\n✔ Project created successfully!\n\n  Location: [TEST_TMP_DIR]/app-tip\n  Files: 3\n\nTip: coding agents detected without Databricks skills. Run 'databricks aitools install' to install them.\n\n>>> print_telemetry\n\n=== plugin installed: no hint, no tip, no event\n\n>>> [CLI] apps init --template ./template --skip-install --name app-plugin\n✔ Creating project\n\n✔ Project created successfully!\n\n  Location: [TEST_TMP_DIR]/app-plugin\n  Files: 3\n\nTip: coding agents detected without Databricks skills. Run 'databricks aitools install' to install them.\n\n>>> print_telemetry\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -15,4 +15,2 @@
        	            	 >>> print_telemetry
        	            	-{"command":"apps_init","exit_code":0,"aitools_hint_event":{"hint_kind":"INSTALL_TEXT"}}
        	            	-{"command":"apps_init","exit_code":0,"aitools_hint_event":{"hint_kind":"APPS_INIT_TIP"}}
        	            	 
        	            	@@ -28,2 +26,3 @@
        	            	 
        	            	+Tip: coding agents detected without Databricks skills. Run 'databricks aitools install' to install them.
        	            	 
        	Test:       	TestAccept/cmd/aitools/telemetry/apps-init-tip
        	Messages:   	cmd/aitools/telemetry/apps-init-tip/output.txt vs /tmp/TestAcceptcmdaitoolstelemetryapps-init-tip4181968450/002/output.txt
    acceptance_test.go:1123: LOG.apps-init: 
    acceptance_test.go:1123: LOG.apps-init-plugin:
TestAccept/cmd/aitools/telemetry/hint: branch ✅ | main (8b4f052) ❌ | latest (v1.26.0) ❌

main (8b4f052):

acceptance_test.go:1213: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:40309" ("[DATABRICKS_HOST]")
    acceptance_test.go:1213: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:40309" ("[DATABRICKS_HOST]")
    acceptance_test.go:1345: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1345
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1086
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:642
        	Error:      	Not equal: 
        	            	expected: "\n=== hint printed on an authenticated command: one hint-shown event is uploaded\n\n>>> [CLI] jobs list\nDatabricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install\n\n>>> print_telemetry\n{\"command\":\"jobs_list\",\"exit_code\":0,\"aitools_hint_event\":{\"hint_kind\":\"INSTALL_TEXT\"}}\n\n=== hint printed on a command that resolves no auth: nothing is uploaded\n\n>>> [CLI] version\nDatabricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install\nDatabricks CLI v[CLI_VERSION]\n\n>>> print_telemetry\n\n=== plugin installed: no hint, no event\n\n>>> [CLI] jobs list\n\n>>> print_telemetry\n"
        	            	actual  : "\n=== hint printed on an authenticated command: one hint-shown event is uploaded\n\n>>> [CLI] jobs list\nDatabricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install\n\n>>> print_telemetry\n\n=== hint printed on a command that resolves no auth: nothing is uploaded\n\n>>> [CLI] version\nDatabricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install\nDatabricks CLI v[CLI_VERSION]\n\n>>> print_telemetry\n\n=== plugin installed: no hint, no event\n\n>>> [CLI] jobs list\n\n>>> print_telemetry\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -7,3 +7,2 @@
        	            	 >>> print_telemetry
        	            	-{"command":"jobs_list","exit_code":0,"aitools_hint_event":{"hint_kind":"INSTALL_TEXT"}}
        	            	 
        	Test:       	TestAccept/cmd/aitools/telemetry/hint
        	Messages:   	cmd/aitools/telemetry/hint/output.txt vs /tmp/TestAcceptcmdaitoolstelemetryhint3004612483/002/output.txt

latest (v1.26.0):

acceptance_test.go:1213: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:32819" ("[DATABRICKS_HOST]")
    acceptance_test.go:1213: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:32819" ("[DATABRICKS_HOST]")
    acceptance_test.go:1345: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1345
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1086
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:642
        	Error:      	Not equal: 
        	            	expected: "\n=== hint printed on an authenticated command: one hint-shown event is uploaded\n\n>>> [CLI] jobs list\nDatabricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install\n\n>>> print_telemetry\n{\"command\":\"jobs_list\",\"exit_code\":0,\"aitools_hint_event\":{\"hint_kind\":\"INSTALL_TEXT\"}}\n\n=== hint printed on a command that resolves no auth: nothing is uploaded\n\n>>> [CLI] version\nDatabricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install\nDatabricks CLI v[CLI_VERSION]\n\n>>> print_telemetry\n\n=== plugin installed: no hint, no event\n\n>>> [CLI] jobs list\n\n>>> print_telemetry\n"
        	            	actual  : "\n=== hint printed on an authenticated command: one hint-shown event is uploaded\n\n>>> [CLI] jobs list\nDatabricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install\n\n>>> print_telemetry\n\n=== hint printed on a command that resolves no auth: nothing is uploaded\n\n>>> [CLI] version\nDatabricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install\nDatabricks CLI v[CLI_VERSION]\n\n>>> print_telemetry\n\n=== plugin installed: no hint, no event\n\n>>> [CLI] jobs list\n\n>>> print_telemetry\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -7,3 +7,2 @@
        	            	 >>> print_telemetry
        	            	-{"command":"jobs_list","exit_code":0,"aitools_hint_event":{"hint_kind":"INSTALL_TEXT"}}
        	            	 
        	Test:       	TestAccept/cmd/aitools/telemetry/hint
        	Messages:   	cmd/aitools/telemetry/hint/output.txt vs /tmp/TestAcceptcmdaitoolstelemetryhint161196894/002/output.txt
TestAccept/cmd/aitools/telemetry/install: branch ✅ | main (8b4f052) ❌ | latest (v1.26.0) ❌

main (8b4f052):

acceptance_test.go:1213: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:42937" ("[DATABRICKS_HOST]")
    acceptance_test.go:1213: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:42937" ("[DATABRICKS_HOST]")
    acceptance_test.go:1345: Diff:
        --- cmd/aitools/telemetry/install/output.txt
        +++ /tmp/TestAcceptcmdaitoolstelemetryinstall970946208/002/output.txt
        @@ -8,14 +8,7 @@
         Installed 1 skill.
         
         >>> print_telemetry
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STARTED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"FLAGS_VALIDATED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"SCOPE_RESOLVED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"AGENTS_SELECTED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"PLAN_CONFIRMED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"SKILLS_INSTALLED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
         {"command":"aitools_install","exit_code":0,"aitools_install_event":{"agents":["CLAUDE_CODE"],"scope":"GLOBAL","error_category":"TYPE_UNSPECIFIED"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"FINISHED","outcome":"SUCCESS","interactive":false,"install_mode":"SKILLS_ONLY","agents":["CLAUDE_CODE"],"scope":"GLOBAL"}}
         
         === databricks-cli auth with a valid cached token: the event is uploaded
         
        @@ -26,14 +18,7 @@
         Installed 1 skill.
         
         >>> print_telemetry
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STARTED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"FLAGS_VALIDATED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"SCOPE_RESOLVED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"AGENTS_SELECTED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"PLAN_CONFIRMED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"SKILLS_INSTALLED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
         {"command":"aitools_install","exit_code":0,"aitools_install_event":{"agents":["CLAUDE_CODE"],"scope":"GLOBAL","error_category":"TYPE_UNSPECIFIED"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"FINISHED","outcome":"SUCCESS","interactive":false,"install_mode":"SKILLS_ONLY","agents":["CLAUDE_CODE"],"scope":"GLOBAL"}}
         
         === databricks-cli auth with an expired cached token: no refresh, nothing is uploaded
         
        @@ -44,6 +28,22 @@
         Installed 1 skill.
         
         >>> print_requests.py //oidc/v1/token //telemetry-ext
        +{
        +  "method": "POST",
        +  "path": "/oidc/v1/token",
        +  "raw_body": "client_id=databricks-cli&grant_type=refresh_token&refresh_token=test-refresh-token"
        +}
        +{
        +  "method": "POST",
        +  "path": "/telemetry-ext",
        +  "body": {
        +    "uploadTime": [UNIX_TIME_MILLIS],
        +    "items": [],
        +    "protoLogs": [
        +      "{\"frontend_log_event_id\":\"[UUID][0]\",\"entry\":{\"databricks_cli_log\":{\"execution_context\":{\"cmd_exec_id\":\"[UUID][1]\",\"version\":\"[CLI_VERSION]\",\"command\":\"aitools_install\",\"operating_system\":\"linux\",\"execution_time_ms\":12,\"exit_code\":0},\"aitools_install_event\":{\"agents\":[\"CLAUDE_CODE\"],\"scope\":\"GLOBAL\",\"error_category\":\"TYPE_UNSPECIFIED\"}}}}"
        +    ]
        +  }
        +}
         
         >>> gron.py [TEST_TMP_DIR]/home/.databricks/token-cache.json
        -json.tokens.test-profile.access_token = "expired-access-token";
        +json.tokens.test-profile.access_token = "oauth-token";

latest (v1.26.0):

acceptance_test.go:1213: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:46787" ("[DATABRICKS_HOST]")
    acceptance_test.go:1213: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:46787" ("[DATABRICKS_HOST]")
    acceptance_test.go:1345: Diff:
        --- cmd/aitools/telemetry/install/output.txt
        +++ /tmp/TestAcceptcmdaitoolstelemetryinstall3856072538/002/output.txt
        @@ -8,14 +8,7 @@
         Installed 1 skill.
         
         >>> print_telemetry
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STARTED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"FLAGS_VALIDATED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"SCOPE_RESOLVED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"AGENTS_SELECTED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"PLAN_CONFIRMED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"SKILLS_INSTALLED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
         {"command":"aitools_install","exit_code":0,"aitools_install_event":{"agents":["CLAUDE_CODE"],"scope":"GLOBAL","error_category":"TYPE_UNSPECIFIED"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"FINISHED","outcome":"SUCCESS","interactive":false,"install_mode":"SKILLS_ONLY","agents":["CLAUDE_CODE"],"scope":"GLOBAL"}}
         
         === databricks-cli auth with a valid cached token: the event is uploaded
         
        @@ -26,14 +18,7 @@
         Installed 1 skill.
         
         >>> print_telemetry
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STARTED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"FLAGS_VALIDATED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"SCOPE_RESOLVED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"AGENTS_SELECTED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"PLAN_CONFIRMED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"SKILLS_INSTALLED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
         {"command":"aitools_install","exit_code":0,"aitools_install_event":{"agents":["CLAUDE_CODE"],"scope":"GLOBAL","error_category":"TYPE_UNSPECIFIED"}}
        -{"command":"aitools_install","exit_code":0,"aitools_command_event":{"command":"INSTALL","stage":"FINISHED","outcome":"SUCCESS","interactive":false,"install_mode":"SKILLS_ONLY","agents":["CLAUDE_CODE"],"scope":"GLOBAL"}}
         
         === databricks-cli auth with an expired cached token: no refresh, nothing is uploaded
         
        @@ -44,6 +28,22 @@
         Installed 1 skill.
         
         >>> print_requests.py //oidc/v1/token //telemetry-ext
        +{
        +  "method": "POST",
        +  "path": "/oidc/v1/token",
        +  "raw_body": "client_id=databricks-cli&grant_type=refresh_token&refresh_token=test-refresh-token"
        +}
        +{
        +  "method": "POST",
        +  "path": "/telemetry-ext",
        +  "body": {
        +    "uploadTime": [UNIX_TIME_MILLIS],
        +    "items": [],
        +    "protoLogs": [
        +      "{\"frontend_log_event_id\":\"[UUID][0]\",\"entry\":{\"databricks_cli_log\":{\"execution_context\":{\"cmd_exec_id\":\"[UUID][1]\",\"version\":\"[CLI_VERSION]\",\"command\":\"aitools_install\",\"operating_system\":\"linux\",\"execution_time_ms\":14,\"exit_code\":0},\"aitools_install_event\":{\"agents\":[\"CLAUDE_CODE\"],\"scope\":\"GLOBAL\",\"error_category\":\"TYPE_UNSPECIFIED\"}}}}"
        +    ]
        +  }
        +}
         
         >>> gron.py [TEST_TMP_DIR]/home/.databricks/token-cache.json
        -json.tokens.test-profile.access_token = "expired-access-token";
        +json.tokens.test-profile.access_token = "oauth-token";
        
FAIL
TestAccept/cmd/aitools/telemetry/install-failure: branch ✅ | main (8b4f052) ❌ | latest (v1.26.0) ❌

main (8b4f052):

acceptance_test.go:1213: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:33745" ("[DATABRICKS_HOST]")
    acceptance_test.go:1213: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:33745" ("[DATABRICKS_HOST]")
    acceptance_test.go:1345: Diff:
        --- cmd/aitools/telemetry/install-failure/output.txt
        +++ /tmp/TestAcceptcmdaitoolstelemetryinstall-failure3431447327/002/output.txt
        @@ -8,12 +8,6 @@
         Error: skill "nonexistent" not found
         
         >>> print_telemetry
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STARTED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"FLAGS_VALIDATED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"SCOPE_RESOLVED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"AGENTS_SELECTED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"PLAN_CONFIRMED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"FINISHED","outcome":"FAILURE","error_category":"SKILL_NOT_FOUND","interactive":false,"install_mode":"SKILLS_ONLY","agents":["CURSOR"],"scope":"GLOBAL"}}
         {"command":"aitools_install","exit_code":1,"aitools_install_event":{"agents":["CURSOR"],"scope":"GLOBAL","error_category":"SKILL_NOT_FOUND"}}
         
         === a manifest the server does not have is MANIFEST_NOT_FOUND
        @@ -25,12 +19,6 @@
         Error: skills manifest at databricks-agent-skills@missing-ref: not found
         
         >>> print_telemetry
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STARTED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"FLAGS_VALIDATED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"SCOPE_RESOLVED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"AGENTS_SELECTED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"PLAN_CONFIRMED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"FINISHED","outcome":"FAILURE","error_category":"MANIFEST_NOT_FOUND","interactive":false,"install_mode":"SKILLS_ONLY","agents":["CURSOR"],"scope":"GLOBAL"}}
         {"command":"aitools_install","exit_code":1,"aitools_install_event":{"agents":["CURSOR"],"scope":"GLOBAL","error_category":"MANIFEST_NOT_FOUND"}}
         
         === any other HTTP error status from the manifest fetch is not classified yet (the error is untyped)
        @@ -42,12 +30,6 @@
         Error: failed to fetch manifest: HTTP 500
         
         >>> print_telemetry
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STARTED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"FLAGS_VALIDATED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"SCOPE_RESOLVED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"AGENTS_SELECTED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"PLAN_CONFIRMED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"FINISHED","outcome":"FAILURE","error_category":"UNCATEGORIZED","interactive":false,"install_mode":"SKILLS_ONLY","agents":["CURSOR"],"scope":"GLOBAL"}}
         {"command":"aitools_install","exit_code":1,"aitools_install_event":{"agents":["CURSOR"],"scope":"GLOBAL","error_category":"UNCATEGORIZED"}}
         
         === a named agent skipped for the scope takes its category from the agent's outcome
        @@ -57,10 +39,4 @@
         Error: Cursor: does not support project-scoped skills
         
         >>> print_telemetry
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STARTED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"FLAGS_VALIDATED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"SCOPE_RESOLVED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"PROJECT"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"AGENTS_SELECTED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"PROJECT"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"PLAN_CONFIRMED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"PROJECT"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"FINISHED","outcome":"FAILURE","error_category":"UNSUPPORTED_SCOPE","interactive":false,"install_mode":"SKILLS_ONLY","agents":["CURSOR"],"scope":"PROJECT"}}
         {"command":"aitools_install","exit_code":1,"aitools_install_event":{"agents":["CURSOR"],"scope":"PROJECT","error_category":"TYPE_UNSPECIFIED","agent_results":[{"agent":"CURSOR","error_category":"UNSUPPORTED_SCOPE"}]}}

latest (v1.26.0):

acceptance_test.go:1213: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:42483" ("[DATABRICKS_HOST]")
    acceptance_test.go:1213: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:42483" ("[DATABRICKS_HOST]")
    acceptance_test.go:1345: Diff:
        --- cmd/aitools/telemetry/install-failure/output.txt
        +++ /tmp/TestAcceptcmdaitoolstelemetryinstall-failure3197712493/002/output.txt
        @@ -8,12 +8,6 @@
         Error: skill "nonexistent" not found
         
         >>> print_telemetry
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STARTED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"FLAGS_VALIDATED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"SCOPE_RESOLVED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"AGENTS_SELECTED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"PLAN_CONFIRMED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"FINISHED","outcome":"FAILURE","error_category":"SKILL_NOT_FOUND","interactive":false,"install_mode":"SKILLS_ONLY","agents":["CURSOR"],"scope":"GLOBAL"}}
         {"command":"aitools_install","exit_code":1,"aitools_install_event":{"agents":["CURSOR"],"scope":"GLOBAL","error_category":"SKILL_NOT_FOUND"}}
         
         === a manifest the server does not have is MANIFEST_NOT_FOUND
        @@ -25,12 +19,6 @@
         Error: skills manifest at databricks-agent-skills@missing-ref: not found
         
         >>> print_telemetry
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STARTED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"FLAGS_VALIDATED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"SCOPE_RESOLVED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"AGENTS_SELECTED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"PLAN_CONFIRMED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"FINISHED","outcome":"FAILURE","error_category":"MANIFEST_NOT_FOUND","interactive":false,"install_mode":"SKILLS_ONLY","agents":["CURSOR"],"scope":"GLOBAL"}}
         {"command":"aitools_install","exit_code":1,"aitools_install_event":{"agents":["CURSOR"],"scope":"GLOBAL","error_category":"MANIFEST_NOT_FOUND"}}
         
         === any other HTTP error status from the manifest fetch is not classified yet (the error is untyped)
        @@ -42,12 +30,6 @@
         Error: failed to fetch manifest: HTTP 500
         
         >>> print_telemetry
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STARTED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"FLAGS_VALIDATED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"SCOPE_RESOLVED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"AGENTS_SELECTED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"PLAN_CONFIRMED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"GLOBAL"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"FINISHED","outcome":"FAILURE","error_category":"UNCATEGORIZED","interactive":false,"install_mode":"SKILLS_ONLY","agents":["CURSOR"],"scope":"GLOBAL"}}
         {"command":"aitools_install","exit_code":1,"aitools_install_event":{"agents":["CURSOR"],"scope":"GLOBAL","error_category":"UNCATEGORIZED"}}
         
         === a named agent skipped for the scope takes its category from the agent's outcome
        @@ -57,10 +39,4 @@
         Error: Cursor: does not support project-scoped skills
         
         >>> print_telemetry
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STARTED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"FLAGS_VALIDATED","interactive":false,"install_mode":"SKILLS_ONLY"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"SCOPE_RESOLVED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"PROJECT"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"AGENTS_SELECTED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"PROJECT"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"STEP_COMPLETED","step":"PLAN_CONFIRMED","interactive":false,"install_mode":"SKILLS_ONLY","scope":"PROJECT"}}
        -{"command":"aitools_install","exit_code":1,"aitools_command_event":{"command":"INSTALL","stage":"FINISHED","outcome":"FAILURE","error_category":"UNSUPPORTED_SCOPE","interactive":false,"install_mode":"SKILLS_ONLY","agents":["CURSOR"],"scope":"PROJECT"}}
         {"command":"aitools_install","exit_code":1,"aitools_install_event":{"agents":["CURSOR"],"scope":"PROJECT","error_category":"TYPE_UNSPECIFIED","agent_results":[{"agent":"CURSOR","error_category":"UNSUPPORTED_SCOPE"}]}}
TestAccept/cmd/aitools/telemetry/update-uninstall: branch ✅ | main (8b4f052) ❌ | latest (v1.26.0) ❌

main (8b4f052):

acceptance_test.go:1213: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:39633" ("[DATABRICKS_HOST]")
    acceptance_test.go:1213: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:39633" ("[DATABRICKS_HOST]")
    acceptance_test.go:1345: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1345
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1086
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:642
        	Error:      	Not equal: 
        	            	expected: "\n=== update: STARTED, FINISHED with SUCCESS\n\n>>> [CLI] aitools update --scope=global\nAlready up to date.\n\n>>> print_telemetry\n{\"command\":\"aitools_update\",\"exit_code\":0,\"aitools_command_event\":{\"command\":\"UPDATE\",\"stage\":\"STARTED\",\"interactive\":false}}\n{\"command\":\"aitools_update\",\"exit_code\":0,\"aitools_command_event\":{\"command\":\"UPDATE\",\"stage\":\"FINISHED\",\"outcome\":\"SUCCESS\",\"interactive\":false,\"scope\":\"GLOBAL\"}}\n\n=== uninstall: STARTED, FINISHED with SUCCESS\n\n>>> [CLI] aitools uninstall --scope=global\nUninstalled 1 skill.\n\n>>> print_telemetry\n{\"command\":\"aitools_uninstall\",\"exit_code\":0,\"aitools_command_event\":{\"command\":\"UNINSTALL\",\"stage\":\"STARTED\",\"interactive\":false}}\n{\"command\":\"aitools_uninstall\",\"exit_code\":0,\"aitools_command_event\":{\"command\":\"UNINSTALL\",\"stage\":\"FINISHED\",\"outcome\":\"SUCCESS\",\"interactive\":false,\"scope\":\"GLOBAL\"}}\n"
        	            	actual  : "\n=== update: STARTED, FINISHED with SUCCESS\n\n>>> [CLI] aitools update --scope=global\nAlready up to date.\n\n>>> print_telemetry\n\n=== uninstall: STARTED, FINISHED with SUCCESS\n\n>>> [CLI] aitools uninstall --scope=global\nUninstalled 1 skill.\n\n>>> print_telemetry\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -7,4 +7,2 @@
        	            	 >>> print_telemetry
        	            	-{"command":"aitools_update","exit_code":0,"aitools_command_event":{"command":"UPDATE","stage":"STARTED","interactive":false}}
        	            	-{"command":"aitools_update","exit_code":0,"aitools_command_event":{"command":"UPDATE","stage":"FINISHED","outcome":"SUCCESS","interactive":false,"scope":"GLOBAL"}}
        	            	 
        	            	@@ -16,4 +14,2 @@
        	            	 >>> print_telemetry
        	            	-{"command":"aitools_uninstall","exit_code":0,"aitools_command_event":{"command":"UNINSTALL","stage":"STARTED","interactive":false}}
        	            	-{"command":"aitools_uninstall","exit_code":0,"aitools_command_event":{"command":"UNINSTALL","stage":"FINISHED","outcome":"SUCCESS","interactive":false,"scope":"GLOBAL"}}
        	            	 
        	Test:       	TestAccept/cmd/aitools/telemetry/update-uninstall
        	Messages:   	cmd/aitools/telemetry/update-uninstall/output.txt vs /tmp/TestAcceptcmdaitoolstelemetryupdate-uninstall288056345/002/output.txt
    acceptance_test.go:1123: 
        LOG.install: Installing Databricks skills for Cursor...
        LOG.install: Using skills version test-ref
        LOG.install: Fetching skills manifest...
        LOG.install: Installed 1 skill.
    acceptance_test.go:1123: LOG.install-telemetry: {"command":"aitools_install","exit_code":0,"aitools_install_event":{"agents":["CURSOR"],"scope":"GLOBAL","error_category":"TYPE_UNSPECIFIED"}}

latest (v1.26.0):

acceptance_test.go:1213: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:36567" ("[DATABRICKS_HOST]")
    acceptance_test.go:1213: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:36567" ("[DATABRICKS_HOST]")
    acceptance_test.go:1345: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1345
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1086
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:642
        	Error:      	Not equal: 
        	            	expected: "\n=== update: STARTED, FINISHED with SUCCESS\n\n>>> [CLI] aitools update --scope=global\nAlready up to date.\n\n>>> print_telemetry\n{\"command\":\"aitools_update\",\"exit_code\":0,\"aitools_command_event\":{\"command\":\"UPDATE\",\"stage\":\"STARTED\",\"interactive\":false}}\n{\"command\":\"aitools_update\",\"exit_code\":0,\"aitools_command_event\":{\"command\":\"UPDATE\",\"stage\":\"FINISHED\",\"outcome\":\"SUCCESS\",\"interactive\":false,\"scope\":\"GLOBAL\"}}\n\n=== uninstall: STARTED, FINISHED with SUCCESS\n\n>>> [CLI] aitools uninstall --scope=global\nUninstalled 1 skill.\n\n>>> print_telemetry\n{\"command\":\"aitools_uninstall\",\"exit_code\":0,\"aitools_command_event\":{\"command\":\"UNINSTALL\",\"stage\":\"STARTED\",\"interactive\":false}}\n{\"command\":\"aitools_uninstall\",\"exit_code\":0,\"aitools_command_event\":{\"command\":\"UNINSTALL\",\"stage\":\"FINISHED\",\"outcome\":\"SUCCESS\",\"interactive\":false,\"scope\":\"GLOBAL\"}}\n"
        	            	actual  : "\n=== update: STARTED, FINISHED with SUCCESS\n\n>>> [CLI] aitools update --scope=global\nAlready up to date.\n\n>>> print_telemetry\n\n=== uninstall: STARTED, FINISHED with SUCCESS\n\n>>> [CLI] aitools uninstall --scope=global\nUninstalled 1 skill.\n\n>>> print_telemetry\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -7,4 +7,2 @@
        	            	 >>> print_telemetry
        	            	-{"command":"aitools_update","exit_code":0,"aitools_command_event":{"command":"UPDATE","stage":"STARTED","interactive":false}}
        	            	-{"command":"aitools_update","exit_code":0,"aitools_command_event":{"command":"UPDATE","stage":"FINISHED","outcome":"SUCCESS","interactive":false,"scope":"GLOBAL"}}
        	            	 
        	            	@@ -16,4 +14,2 @@
        	            	 >>> print_telemetry
        	            	-{"command":"aitools_uninstall","exit_code":0,"aitools_command_event":{"command":"UNINSTALL","stage":"STARTED","interactive":false}}
        	            	-{"command":"aitools_uninstall","exit_code":0,"aitools_command_event":{"command":"UNINSTALL","stage":"FINISHED","outcome":"SUCCESS","interactive":false,"scope":"GLOBAL"}}
        	            	 
        	Test:       	TestAccept/cmd/aitools/telemetry/update-uninstall
        	Messages:   	cmd/aitools/telemetry/update-uninstall/output.txt vs /tmp/TestAcceptcmdaitoolstelemetryupdate-uninstall690950254/002/output.txt
    acceptance_test.go:1123: 
        LOG.install: Installing Databricks skills for Cursor...
        LOG.install: Using skills version test-ref
        LOG.install: Fetching skills manifest...
        LOG.install: Installed 1 skill.
    acceptance_test.go:1123: LOG.install-telemetry: {"command":"aitools_install","exit_code":0,"aitools_install_event":{"agents":["CURSOR"],"scope":"GLOBAL","error_category":"TYPE_UNSPECIFIED"}}
TestAccept/cmd/aitools/telemetry/workspace-unreachable: branch ✅ | main (8b4f052) ❌ | latest (v1.26.0) ❌

main (8b4f052):

acceptance_test.go:1213: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:41429" ("[DATABRICKS_HOST]")
    acceptance_test.go:1213: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:41429" ("[DATABRICKS_HOST]")
    acceptance_test.go:1345: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1345
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1086
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:642
        	Error:      	Not equal: 
        	            	expected: "\n=== install: the background auth resolution prints nothing\n\n>>> [CLI] aitools install --skills-only --scope=global --agents=cursor\nInstalling Databricks skills for Cursor...\nUsing skills version test-ref\nFetching skills manifest...\nInstalled 1 skill.\n\n=== update: the background auth resolution prints nothing\n\n>>> [CLI] aitools update --scope=global\nAlready up to date.\n\n=== uninstall: the background auth resolution prints nothing\n\n>>> [CLI] aitools uninstall --scope=global\nUninstalled 1 skill.\n"
        	            	actual  : "\n=== install: the background auth resolution prints nothing\n\n>>> [CLI] aitools install --skills-only --scope=global --agents=cursor\nWarn: [hostmetadata] failed to fetch host metadata for https://unreachable.test, will skip for 1m0s\nInstalling Databricks skills for Cursor...\nUsing skills version test-ref\nFetching skills manifest...\nInstalled 1 skill.\n\n=== update: the background auth resolution prints nothing\n\n>>> [CLI] aitools update --scope=global\nAlready up to date.\n\n=== uninstall: the background auth resolution prints nothing\n\n>>> [CLI] aitools uninstall --scope=global\nUninstalled 1 skill.\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -4,2 +4,3 @@
        	            	 >>> [CLI] aitools install --skills-only --scope=global --agents=cursor
        	            	+Warn: [hostmetadata] failed to fetch host metadata for https://unreachable.test, will skip for 1m0s
        	            	 Installing Databricks skills for Cursor...
        	Test:       	TestAccept/cmd/aitools/telemetry/workspace-unreachable
        	Messages:   	cmd/aitools/telemetry/workspace-unreachable/output.txt vs /tmp/TestAcceptcmdaitoolstelemetryworkspace-unreachable4170265492/002/output.txt

latest (v1.26.0):

acceptance_test.go:1213: Substituted DATABRICKS_CLI_GITHUB_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:41615" ("[DATABRICKS_HOST]")
    acceptance_test.go:1213: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:41615" ("[DATABRICKS_HOST]")
    acceptance_test.go:1345: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1345
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1086
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:642
        	Error:      	Not equal: 
        	            	expected: "\n=== install: the background auth resolution prints nothing\n\n>>> [CLI] aitools install --skills-only --scope=global --agents=cursor\nInstalling Databricks skills for Cursor...\nUsing skills version test-ref\nFetching skills manifest...\nInstalled 1 skill.\n\n=== update: the background auth resolution prints nothing\n\n>>> [CLI] aitools update --scope=global\nAlready up to date.\n\n=== uninstall: the background auth resolution prints nothing\n\n>>> [CLI] aitools uninstall --scope=global\nUninstalled 1 skill.\n"
        	            	actual  : "\n=== install: the background auth resolution prints nothing\n\n>>> [CLI] aitools install --skills-only --scope=global --agents=cursor\nWarn: [hostmetadata] failed to fetch host metadata for https://unreachable.test, will skip for 1m0s\nInstalling Databricks skills for Cursor...\nUsing skills version test-ref\nFetching skills manifest...\nInstalled 1 skill.\n\n=== update: the background auth resolution prints nothing\n\n>>> [CLI] aitools update --scope=global\nAlready up to date.\n\n=== uninstall: the background auth resolution prints nothing\n\n>>> [CLI] aitools uninstall --scope=global\nUninstalled 1 skill.\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -4,2 +4,3 @@
        	            	 >>> [CLI] aitools install --skills-only --scope=global --agents=cursor
        	            	+Warn: [hostmetadata] failed to fetch host metadata for https://unreachable.test, will skip for 1m0s
        	            	 Installing Databricks skills for Cursor...
        	Test:       	TestAccept/cmd/aitools/telemetry/workspace-unreachable
        	Messages:   	cmd/aitools/telemetry/workspace-unreachable/output.txt vs /tmp/TestAcceptcmdaitoolstelemetryworkspace-unreachable1983775359/002/output.txt
TestAccept/cmd/version: branch ✅ | main (8b4f052) ❌ | latest (v1.26.0) ❌

main (8b4f052):

acceptance_test.go:1345: Diff:
        --- cmd/version/output.txt
        +++ /tmp/TestAcceptcmdversion4063098404/002/output.txt
        @@ -24,7 +24,6 @@
         >>> [CLI] version --check
         Databricks CLI v[CLI_VERSION]
         This is a development build; skipping the update check.
        -Tip: coding agents detected without Databricks skills. Run 'databricks aitools install' to install them.
         
         === version --check --output json with a coding agent prints no tip

latest (v1.26.0):

acceptance_test.go:1345: Diff:
        --- cmd/version/output.txt
        +++ /tmp/TestAcceptcmdversion1135123406/002/output.txt
        @@ -8,7 +8,7 @@
         
         >>> [CLI] version --check
         Databricks CLI v[CLI_VERSION]
        -This is a development build; skipping the update check.
        +Could not reach GitHub to check for a newer version. See https://github.com/databricks/cli/releases for the latest release.
         
         === version --check --output json
         
        @@ -16,15 +16,14 @@
         {
           "current_version": "[CLI_VERSION]",
           "update_available": false,
        -  "development_build": true
        +  "check_failed": true
         }
         
         === version --check with a coding agent and no Databricks tooling
         
         >>> [CLI] version --check
         Databricks CLI v[CLI_VERSION]
        -This is a development build; skipping the update check.
        -Tip: coding agents detected without Databricks skills. Run 'databricks aitools install' to install them.
        +Could not reach GitHub to check for a newer version. See https://github.com/databricks/cli/releases for the latest release.
         
         === version --check --output json with a coding agent prints no tip
         
        @@ -32,7 +25,7 @@
         {
           "current_version": "[CLI_VERSION]",
           "update_available": false,
        -  "development_build": true
        +  "check_failed": true
         }
         
         === version --check from Claude Code prints only the root hint
        @@ -40,4 +33,4 @@
         >>> [CLI] version --check
         Databricks skills are not installed. To work with Databricks reliably, first run: databricks aitools install
         Databricks CLI v[CLI_VERSION]
        -This is a development build; skipping the update check.
        +Could not reach GitHub to check for a newer version. See https://github.com/databricks/cli/releases for the latest release.

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Collaborator

✅ Integration test report

Commit: 66114a4

Run: 43200321431

Env 💚​RECOVERED ✅​pass 🙈​skip Time
💚​ aws linux-2core-8gb 1 270 27 12:04
💚​ aws-windows-latest-4core-16gb 1 272 25 9:52
💚​ azure linux-2core-8gb 1 269 27 8:57
💚​ azure-windows-latest-4core-16gb 1 271 25 8:49
💚​ gcp linux-2core-8gb 1 270 27 13:13
💚​ gcp-windows-latest-4core-16gb 1 272 25 14:55
Test Name aws linux-2core-8gb aws-windows-latest-4core-16gb azure linux-2core-8gb azure-windows-latest-4core-16gb gcp linux-2core-8gb gcp-windows-latest-4core-16gb
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
Top 10 slowest tests (at least 2 minutes):
duration env testname
4:10 gcp-windows-latest-4core-16gb TestFilerRecursiveDelete/dbfs
3:58 azure-windows-latest-4core-16gb TestAccept
3:57 gcp-windows-latest-4core-16gb TestAccept
3:45 aws linux-2core-8gb TestAccept
3:28 aws-windows-latest-4core-16gb TestAccept
3:20 gcp linux-2core-8gb TestAccept
3:13 azure linux-2core-8gb TestAccept
3:06 gcp-windows-latest-4core-16gb TestFsCpDirToDirWithOverwriteFlag/dbfs_to_dbfs
2:42 gcp-windows-latest-4core-16gb TestFsCpFileToDirWithOverwriteFlag/dbfs_to_dbfs
2:04 gcp-windows-latest-4core-16gb TestFsCpDirToDirFileNotOverwritten/dbfs_to_dbfs

@simonfaltum simonfaltum left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated review from simonfaltum's review agent (pr-review skill) at 848f252. Each finding was verified against the code; simonfaltum may follow up.

Not approving yet: 1 open design question.

This PR adds a line pointing to databricks aitools install to the text output of databricks version --check on every outcome except development builds. JSON output and plain version are unchanged. The template renders correctly in every case, the tests and changelog fragment look right, and CI is green. Two things are open: whether this hint should be gated like the other aitools install suggestions (it also shows to users who already installed and to users with no coding agent), and moving it from stdout to stderr like the other hints. There is also a small note on the PR description format.

Questions

  1. Should this line be shown unconditionally? The other places that suggest databricks aitools install all check first: agents.MaybeHint (libs/aitools/agents/hint.go), RecommendSkillsInstall (libs/aitools/agents/recommend.go) and aitools version. They stay quiet when the plugin or skills are already installed, or when no coding agent is present, and the other version notices skip Databricks Runtime. With this change, version --check also tells users who already ran aitools install to set it up, and it shows on DBR. Users with no coding agent are sent to a command that replies 'No supported coding agents found on PATH'. Claude Code sessions without tooling get this line plus the separate MaybeHint line on stderr. Options: (a) keep it unconditional and explain in the description why it differs from the other hints, or (b) route it through the libs/aitools/agents hint helpers so it shows only when an agent is detected and no Databricks plugin or skills are present.

Comments outside the changed lines

  • [nit] .github/PULL_REQUEST_TEMPLATE.md: The description uses ## Summary, ## Evidence and ## Merge Danger. .agents/skills/pr-checklist/SKILL.md (PR description section) asks for the .github/PULL_REQUEST_TEMPLATE.md headings (## Changes, ## Why, ## Tests) in that order, with no invented sections. As a result, the description has no ## Why explaining the reason for adding this hint. Could you move the content under the template headings (Summary → Changes, Evidence → Tests) and add a short Why?

Comment thread cmd/version/version.go Outdated
{{green "You're on the latest version."}}
{{- end}}
{{- if not .DevelopmentBuild}}
Set up Databricks AI tools for your coding agents: databricks aitools install

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nice to have] The hint is part of updateCheckTemplate, so cmdio.RenderWithTemplate writes it to stdout together with the version status. Every other aitools install hint (agents.MaybeHint, PrintSkillsTip) goes to stderr through cmdio.LogString. As a result, a script or agent reading the text result, for example databricks version --check | tail -n1 or a check that the last line is "You're on the latest version.", now gets the aitools line instead, and redirecting stderr can't remove it. Suggest dropping the line from the template and printing it after the render with cmdio.LogString(ctx, ...) (still skipped for development builds), so stdout keeps only the update-check result.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 90c8726: the line is no longer part of updateCheckTemplate. renderUpdateCheck renders the result and then prints the line with cmdio.LogString (stderr), only in text mode and not for development builds, so stdout keeps only the update-check result. TestRenderUpdateCheckAIToolsHint covers the stdout/stderr split, the dev-build skip and --output json.

@anton-107
anton-107 force-pushed the aitools/version-check-install-hint branch from 848f252 to 90c8726 Compare November 11, 2026 16:53
@anton-107
anton-107 force-pushed the aitools/command-funnel-telemetry branch from 2b4b8c6 to bbc2b80 Compare November 11, 2026 17:27
@anton-107
anton-107 force-pushed the aitools/version-check-install-hint branch from 90c8726 to 125c8ca Compare November 11, 2026 17:28
@anton-107
anton-107 force-pushed the aitools/command-funnel-telemetry branch from bbc2b80 to 6cd41f6 Compare November 11, 2026 19:28
@anton-107
anton-107 force-pushed the aitools/version-check-install-hint branch 2 times, most recently from 0148734 to 7d1dfc8 Compare November 11, 2026 19:59
@anton-107

Copy link
Copy Markdown
Contributor Author

Re Question 1: went with (b) in 7d1dfc8. version --check now prints the databricks aitools install line (on stderr, text mode only) only when agents.HasDatabricksToolingInstalled reports a coding agent detected without the databricks plugin or Databricks skills, the same check the apps init tip uses. Like the other version notices it is skipped on Databricks Runtime (dbr.RunsOnRuntime), and it is still skipped for development builds. TestRenderUpdateCheckAIToolsHint covers the shown case and the hidden cases: no agent, plugin installed, skills installed, Databricks Runtime, development build, --output json. The changelog fragment and the description are updated to match.

@simonfaltum simonfaltum left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated review from simonfaltum's review agent (pr-review skill) at 7d1dfc8. Each finding was verified against the code; simonfaltum may follow up.

Not approving yet: 1 blocking issue(s), tagged [blocking]; 1 open design question.

This PR prints a databricks aitools install suggestion on stderr after databricks version --check when a coding agent's config dir is found without Databricks tooling. Stdout and JSON output are unchanged, and the tests pass. The check misses plugin installs for Codex CLI and GitHub Copilot, which are recorded only in the CLI's install state, so those users keep seeing the line after they install. That needs fixing before merge. Claude Code sessions also get this line in addition to the existing root hint. There is also an open question about whether this hint should go through the shared hint helpers.

Questions

  1. This adds a second install-hint path in cmd/version alongside agents.MaybeHint and PrintSkillsTip. Unlike MaybeHint, it shows to anyone who has a coding agent's config directory, including people at a terminal. It shows on every version --check run, with no session throttle, CI suppression or opt-out, and it uses new wording, although hint.go notes that hintText was tuned for agents. Is that difference intended? Options: (a) keep it in cmd/version and explain in the description why its audience and frequency differ; (b) move it into libs/aitools/agents as a helper next to PrintSkillsTip, sharing wording, throttle and dedupe with MaybeHint; (c) drop it and rely on MaybeHint, extending that to other agents if the coverage matters.

Agreeing with existing threads

Comment thread cmd/version/version.go Outdated
if dbr.HasDetection(ctx) && dbr.RunsOnRuntime(ctx) {
return false
}
return !agents.HasDatabricksToolingInstalled(ctx)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[blocking] HasDatabricksToolingInstalled only recognizes a plugin through an agent's own manifest reader, and only Claude Code has one. For Codex CLI and GitHub Copilot, databricks aitools install installs the plugin by default, records it only in ~/.databricks/aitools/skills/.state.json and writes no skill dirs. So after a successful install, every version --check still tells the user to run databricks aitools install. I reproduced this with a release build: with ~/.codex and a recorded Codex plugin, databricks aitools version lists Plugin (Codex CLI, global, user scope): v0.2.6 and version --check still prints the line. Could you also count recorded plugin installs? For example, return !agents.HasDatabricksToolingInstalled(ctx) && len(installer.InstalledTools(ctx)) == 0, since InstalledTools already merges the manifests with the recorded state. Please add a test case for a recorded Codex plugin.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in b53927b. The tip now also stays quiet when installer.InstalledTools reports a databricks plugin, so plugin installs recorded only in the CLI install state (Codex CLI, GitHub Copilot) count. The check lives in agents.MaybeVersionCheckTip, which takes installer.InstalledTools as a parameter because libs/aitools/installer imports libs/aitools/agents. Tests: TestRenderUpdateCheckAIToolsTip/codex_plugin_recorded_in_the_install_state (cmd/version) writes ~/.codex plus a recorded Codex plugin in .state.json and expects no tip. TestMaybeVersionCheckTip/plugin_recorded_in_install_state covers the same gate in the agents package.

Comment thread cmd/version/version.go Outdated
Comment on lines +49 to +50
if output == flags.OutputText && !result.DevelopmentBuild && showAIToolsHint(ctx) {
cmdio.LogString(ctx, aitoolsHint)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nice to have] In a Claude Code session without the Databricks tooling, agents.MaybeHint (root pre-run, cmd/root/root.go:95) already prints its install recommendation for version. version --check now adds a second, differently worded line for the same command. MaybeHint shows at most once per hour per session, but this line repeats on every check in the session. Agents are sent to this command by UpdateCLIAdvice (libs/genie/agentstream/renderer.go:33). I reproduced both lines with a release build and CLAUDECODE=1. Could you skip this line when MaybeHint covers the caller (Claude Code as the agent provider), for example with a small exported helper in libs/aitools/agents, and add a test case for it?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in b53927b. agents.MaybeVersionCheckTip returns early when Claude Code is the agent provider, because agents.MaybeHint covers that caller in the root pre-run hook with its per-session throttle. A command now prints at most one install hint. Test: TestMaybeVersionCheckTip/claude_code_caller_is_left_to_MaybeHint.

Comment thread cmd/version/version.go Outdated
if dbr.HasDetection(ctx) && dbr.RunsOnRuntime(ctx) {
return false
}
return !agents.HasDatabricksToolingInstalled(ctx)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nice to have] HasDatabricksSkillsInstalled only looks in $HOME/.databricks/aitools/skills and the legacy dir. After databricks aitools install --scope project for a skills-only agent (Kiro, Goose, Pi, Gemini), running version --check in that project still says to install. I reproduced this with ~/.kiro and project-scoped databricks-apps skills under .databricks/aitools/skills/ and .kiro/skills/. Consider also returning false when agents.DetectProjectInstalled(cwd) finds skills in the current project.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in b53927b. The tip also stays quiet when agents.DetectProjectInstalled(cwd) finds Databricks skills in the current project. Test: TestMaybeVersionCheckTip/project_skills sets up ~/.kiro and .kiro/skills/databricks-apps in the working directory and expects no tip. Project-scoped plugin records are already covered by installer.InstalledTools, which reads both the global and the project install state.

Comment thread cmd/version/version_test.go Outdated
home := t.TempDir()
t.Setenv("HOME", home)
t.Setenv("USERPROFILE", home)
for _, name := range []string{"CLAUDE_CONFIG_DIR", "PI_CODING_AGENT_DIR", "GEMINI_CLI_HOME", "APPDATA", "XDG_CONFIG_HOME", "GOOSE_PATH_ROOT"} {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] setupAgentHome hard-codes the config-dir override env vars, duplicating clearAgentDirOverrides in libs/aitools/installer/installer_test.go. If an agent adds a new override, this list goes stale and the "no agent detected" case can start reading the host's real agent dir. testutil.SetAgentEnv(t, "HOME", home, "USERPROFILE", home) (internal/testutil/agent.go, already used in cmd/aitools/funnel_test.go) clears the whole environment and avoids that.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in b53927b. setupAgentHome is gone. Both tests use testutil.SetAgentEnv(t, "HOME", home, "USERPROFILE", home) and t.Chdir(t.TempDir()), so neither the host's agent dirs nor the repo's project dirs leak in.

@anton-107
anton-107 force-pushed the aitools/version-check-install-hint branch from 7d1dfc8 to b53927b Compare November 11, 2026 23:37
@anton-107

Copy link
Copy Markdown
Contributor Author

Re Question 1 (second review): went with (b) in b53927b. The line moved into agents.MaybeVersionCheckTip in libs/aitools/agents/recommend.go, next to PrintSkillsTip, and now prints the same text as the apps init tip (skillsTipText) on stderr. I didn't reuse hintText: its comment says the wording is tuned for agents, while version --check is also run by people at a terminal. To keep one install hint per command, it skips Claude Code callers, whom MaybeHint already covers with its per-session throttle. It adds no throttle or opt-out of its own. It is still skipped on Databricks Runtime, for development builds and for --output json, and when the databricks plugin (agent manifest or CLI install state) or Databricks skills (global or current project) are present. It logs no hint telemetry: databricks version never resolves auth, so per logHintShown the event would not be uploaded. Changelog fragment and description are updated.

@anton-107
anton-107 force-pushed the aitools/command-funnel-telemetry branch from 6cd41f6 to 8643a8e Compare November 11, 2026 23:50
@anton-107
anton-107 force-pushed the aitools/version-check-install-hint branch from b53927b to 14f43cd Compare November 11, 2026 23:50
@anton-107
anton-107 force-pushed the aitools/command-funnel-telemetry branch from 8643a8e to 4653c00 Compare November 11, 2026 23:57
@anton-107
anton-107 force-pushed the aitools/version-check-install-hint branch from 14f43cd to 8e754f4 Compare November 11, 2026 23:58
@anton-107
anton-107 force-pushed the aitools/command-funnel-telemetry branch from 4653c00 to cf15bb3 Compare November 12, 2026 00:05
@anton-107
anton-107 force-pushed the aitools/version-check-install-hint branch from 8e754f4 to 14cb376 Compare November 12, 2026 00:05
@anton-107
anton-107 force-pushed the aitools/command-funnel-telemetry branch from cf15bb3 to 9dac556 Compare November 12, 2026 00:15
@anton-107
anton-107 force-pushed the aitools/version-check-install-hint branch from 14cb376 to d03528e Compare November 12, 2026 00:15
@anton-107
anton-107 force-pushed the aitools/command-funnel-telemetry branch from 9dac556 to e5b0edd Compare November 12, 2026 00:27
@anton-107
anton-107 force-pushed the aitools/version-check-install-hint branch from d03528e to 0aed2cb Compare November 12, 2026 00:27

@simonfaltum simonfaltum left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated review from simonfaltum's review agent (pr-review skill) at 0aed2cb. Each finding was verified against the code; simonfaltum may follow up.

Not approving yet: 1 open design question.

This PR prints the apps init skills tip on stderr after text-mode databricks version --check when a coding agent is detected without Databricks tooling. It skips Claude Code callers, Databricks Runtime, development builds and JSON output, and stdout is unchanged. Every point from the earlier review rounds is fixed and CI is green. The remaining findings are non-blocking: the tip still appears in a project subdirectory or when skills live only in an agent's own skills dir, the dev-build skip leaves it without acceptance coverage, the doc comment is long, and the description doesn't use the template. One question is left: should apps init share the new suppression checks in this PR or in a follow-up?

Questions

  1. MaybeVersionCheckTip now also counts plugins recorded in the CLI install state and project-scoped skills. But apps init prints the same tip text and still decides with HasDatabricksToolingInstalled alone (cmd/apps/init.go:1916). So a Codex CLI or GitHub Copilot user who installed the plugin through databricks aitools install gets no tip from version --check but still sees it from apps init. Is that split intended? Options: (a) keep this PR scoped to version --check and align apps init in a follow-up; (b) move these checks into the shared predicate here, for example with a pluginRecorded-style parameter like the one (*Agent).DatabricksTooling takes. Option (b) makes both commands agree and drops the []string callback.

Agreeing with existing threads

Comment thread libs/aitools/agents/recommend.go Outdated
if HasDatabricksToolingInstalled(ctx) || len(installedPlugins(ctx)) > 0 {
return
}
if cwd, err := os.Getwd(); err == nil && len(DetectProjectInstalled(cwd)) > 0 {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nice to have] The new checks look only at the exact working directory (DetectProjectInstalled(cwd)) and, for global skills, only at the shared ~/.databricks/aitools/skills store. So the tip can claim skills are missing when the agent already loads them. Example: with ~/.kiro and a project install at the repo root (<repo>/.kiro/skills/databricks-apps, as aitools install --scope project writes), databricks version --check stays quiet in <repo> but prints the tip from <repo>/src. The same happens when the skills exist only in an agent's own skills dir, for example ~/.cursor/skills/databricks-apps written by another installer. I reproduced both with a release-versioned build of this commit. (*Agent).DatabricksTooling in libs/aitools/agents/calling.go already handles both cases: it checks a.SkillsDir(ctx) and walks projectDirs(cwd) up to the repository root. Could the project check walk projectDirs(resolvePath(cwd)) and also check each detected agent's own SkillsDir, or reuse DatabricksTooling for the detected agents? A test case run from a project subdirectory would cover it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 9dd6d21. After the existing checks, MaybeVersionCheckTip now asks each registry agent's DatabricksTooling. That covers the agent's own skills dir (a.SkillsDir) and walks projectDirs(resolvePath(cwd)) up to the repository root. Plugins recorded in the install state are already handled by installedPlugins before that loop, so the loop passes a pluginRecorded that returns false. The old cwd-only DetectProjectInstalled(cwd) check is gone, since the walk covers it. New cases in TestMaybeVersionCheckTip: project skills at the repository root run from <repo>/src (with a .git at the root), and skills only in ~/.cursor/skills/databricks-apps. All three skills cases fail if the loop is removed.

Comment thread cmd/version/version.go Outdated
if err != nil {
return err
}
if output == flags.OutputText && !result.DevelopmentBuild {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nice to have] The tip is skipped for development builds, and the comment doesn't say why. Acceptance binaries are always development builds, so this condition is why no acceptance test can show the new stderr line. .agents/rules/testing.md asks for acceptance tests first for user-visible cmd/... output. The unit tests call renderUpdateCheck directly, so they don't cover the RunE wiring (root.OutputType(cmd)) or how the tip interacts with the root pre-run MaybeHint line. Acceptance runs already set HOME to an empty temp dir (acceptance/internal/prepare_server.go), so dropping !result.DevelopmentBuild wouldn't change the existing acceptance/cmd/version output. Suggest dropping it (the tip is about agent tooling, not the release check). Then add cases to acceptance/cmd/version using sethome with a committed home/.codex (tip on stderr) and CLAUDECODE=1 (only the root hint). If dev builds should stay quiet, a one-line comment saying why would be enough.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dropped the skip in 9dd6d21. It didn't stand for anything about agent tooling, so development builds now get the tip as well. New cases in acceptance/cmd/version: sethome ./codex-home with a committed .codex shows the tip on stderr after the dev-build line; --output json in the same home prints no tip; sethome ./claude-home with CLAUDECODE=1 prints only the root MaybeHint line. The existing cases are unchanged, because the default acceptance HOME has no agent dirs. TestRenderUpdateCheckAIToolsTip now expects the tip for a development build too.

Comment on lines +25 to +42
// MaybeVersionCheckTip prints skillsTipText after `databricks version --check`
// when a coding agent is detected without Databricks tooling. It prints
// nothing:
// - when Claude Code drives the CLI: MaybeHint covers that caller in the
// root pre-run hook, with its per-session throttle, so a command prints at
// most one install hint;
// - on Databricks Runtime, like the other version notices;
// - when no agent is detected, a detected agent has the databricks plugin in
// its own manifest, or Databricks skills are installed globally
// (HasDatabricksToolingInstalled);
// - when installedPlugins reports a databricks plugin recorded in the CLI
// install state (Codex CLI and GitHub Copilot record plugin installs only
// there). Callers pass installer.InstalledTools, which this package cannot
// import;
// - when Databricks skills are installed in the current project.
//
// It logs no telemetry: `databricks version` never resolves auth, so the event
// would not be uploaded (see logHintShown).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] This 18-line doc comment lists every early return, which .agents/rules/comments.md asks to avoid ("A doc comment gives the caller the contract and any surprising behavior, not a line-by-line re-spec"), and the list will drift as the gates change. Suggest keeping only what the code can't say: Claude Code is skipped because MaybeHint covers it, installedPlugins is a parameter because this package can't import installer, and no telemetry is logged because version never resolves auth.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shortened in 9dd6d21. The doc comment now covers only the three points you listed: Claude Code is skipped because MaybeHint covers it, installedPlugins is a parameter because this package can't import the installer, and no telemetry is logged because version never resolves auth. The checks themselves are in the code and the test table.

@anton-107
anton-107 force-pushed the aitools/version-check-install-hint branch from 0aed2cb to 9dd6d21 Compare November 12, 2026 09:34
@anton-107

Copy link
Copy Markdown
Contributor Author

Re Question 1 (third review): (a). This PR stays scoped to version --check, and apps init is left unchanged. Moving these checks into the shared predicate would change when apps init shows its tip and prompt, which is a behavior change for another command. Aligning the two is better done in a separate PR.

@anton-107
anton-107 force-pushed the aitools/command-funnel-telemetry branch from e5b0edd to e1c95af Compare November 12, 2026 09:57
@anton-107
anton-107 force-pushed the aitools/version-check-install-hint branch from 9dd6d21 to 7e2cf0e Compare November 12, 2026 09:57
@anton-107
anton-107 force-pushed the aitools/command-funnel-telemetry branch from e1c95af to 1a67570 Compare November 12, 2026 10:20
@anton-107
anton-107 force-pushed the aitools/version-check-install-hint branch from 7e2cf0e to 2920789 Compare November 12, 2026 10:20
@anton-107
anton-107 force-pushed the aitools/command-funnel-telemetry branch from 1a67570 to f379423 Compare November 12, 2026 10:23
`databricks version --check` (text output) is followed by the `apps init` skills tip on stderr when a coding agent is detected without Databricks tooling. Stdout keeps only the update-check result:

```diff
 Databricks CLI v0.245.0
 You're on the latest version.
+Tip: coding agents detected without Databricks skills. Run 'databricks aitools install' to install them.
```

The tip comes from `agents.MaybeVersionCheckTip`. It is not printed when an agent has the databricks plugin (in its own manifest or recorded in the CLI install state) or Databricks skills (in the shared store, in the agent's own skills dir, or in the project dirs up to the repository root), on Databricks Runtime, or when Claude Code drives the CLI (`agents.MaybeHint` already covers that caller). Changelog fragment: `.nextchanges/cli/version-check-aitools-hint.md`.

Co-authored-by: Isaac <no-reply@databricks.com>
@anton-107
anton-107 force-pushed the aitools/version-check-install-hint branch from 2920789 to 66114a4 Compare November 12, 2026 10:24

@simonfaltum simonfaltum left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Approval from simonfaltum's review agent at 66114a4, on Simon's decision.

The design questions from the earlier automated reviews no longer block this PR: your answers are accepted, and any still-open ones are non-blocking. The changes since the last automated review (0aed2cb → 66114a4) address its comments.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai-auto PRs made by AI loops & factories,

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants