Skip to content

Support asynchronous timeseries and table column deletion - #19186

Draft
Caideyipi wants to merge 15 commits into
apache:masterfrom
Caideyipi:feature/async-schema-deletion
Draft

Caideyipi wants to merge 15 commits into
apache:masterfrom
Caideyipi:feature/async-schema-deletion

Conversation

@Caideyipi

@Caideyipi Caideyipi commented Nov 9, 2026 •

Copy link
Copy Markdown
Collaborator

Description

Tree-model DELETE/DROP TIMESERIES and table-model ALTER TABLE ... DROP COLUMN return after durable admission of a background procedure. Callers can inspect tree deletion with SHOW DELETE TIMESERIES PROGRESS and column deletion with information_schema.drop_table_column_procedures, then wait for completion before dependent DDL.

  • Add a bounded, configurable tree-deletion scheduler that controls queue length, the minimum interval between execution starts, and task concurrency.
  • Preserve query-ID idempotency, expose task IDs and progress, and keep terminal progress separate from durable finalization. Return clear admission messages for asynchronous table and column deletion.
  • Persist admission and completion transitions, isolate stored procedure state, and retain same-database DDL reservations through database deletion/recreation and recovery. Keep tree/table database namespaces separate and protect database configuration from being overwritten during recreation.
  • Expose column-deletion procedure state, phase, errors, and mod-file counters using the same semantics as DROP TABLE. Share counter aggregation and persistence between table and column deletion.
  • Disable RPC deadlines for background tree, column, and table deletion work. Keep cache-invalidation broadcasts bounded and restore pooled clients to their original timeout after success or failure.
  • Update integration-test helpers and dependent tests to wait for durable deletion completion, including JDBC and Session paths.

Tree-model asynchronous deletion configuration

Configure these ConfigNode settings in iotdb-system.properties. The execution interval can be changed at runtime; queue capacity and task concurrency take effect after a ConfigNode restart.

Configuration Default Meaning
timeseries_deletion_queue_capacity 1024 Maximum number of tree-model timeseries deletion tasks waiting in the queue.
timeseries_deletion_execution_interval_ms 1000 (1 second) Minimum interval between starting/dequeuing tree-model timeseries deletion tasks.
timeseries_deletion_max_concurrent_tasks 1 Maximum number of tree-model timeseries deletion tasks executing concurrently.

The execution interval applies when tasks leave the queue to start execution. Enqueue requests are not throttled by this interval. With the defaults, up to 1024 tasks can wait, one task executes at a time, and successive task starts are at least 1 second apart.

Example: submit tree deletions and track completion

CREATE DATABASE root.async_delete_demo;
CREATE TIMESERIES root.async_delete_demo.d1.s1 WITH DATATYPE=DOUBLE, ENCODING=GORILLA;
CREATE TIMESERIES root.async_delete_demo.d1.s2 WITH DATATYPE=DOUBLE, ENCODING=GORILLA;

DELETE TIMESERIES root.async_delete_demo.d1.s1;
DELETE TIMESERIES root.async_delete_demo.d1.s2;

SHOW DELETE TIMESERIES PROGRESS;

Each DELETE TIMESERIES returns after its task has been durably admitted, with a task/procedure ID in the response message. It does not wait for deletion to finish. The two requests can be submitted back-to-back; with the default scheduler settings, their tasks execute one at a time and start at least 1 second apart.

SHOW DELETE TIMESERIES PROGRESS returns ProcedureId, QueryId, Paths, State, Progress, and ErrorMessage. Match each returned procedure ID and poll until its State is SUCCESS before recreating the corresponding timeseries or issuing dependent DDL. FINALIZING still means completion is being persisted; ROLLEDBACK indicates failure and should be checked with ErrorMessage.

For example, after the task for root.async_delete_demo.d1.s1 reports SUCCESS, the series can be recreated:

CREATE TIMESERIES root.async_delete_demo.d1.s1 WITH DATATYPE=DOUBLE, ENCODING=GORILLA;

Example: track column deletion

After submitting ALTER TABLE ... DROP COLUMN, query its procedure:

SELECT * FROM information_schema.drop_table_column_procedures
WHERE database = 'example_db'
  AND table_name = 'example_table'
  AND column_name = 'reading';

The columns are database, table_name, column_name, procedure_id, state, progress, error_message, written_mod_file_count, and remaining_mod_file_count. Wait for state = 'SUCCESS' before dependent DDL. Completed procedures remain visible for the existing procedure-result retention period.

The mod-file counters use the same aggregation as information_schema.drop_table_procedures: one executing replica per Region contributes, so replicas do not multiply counts. Unavailable counts remain NULL; attribute deletion and field deletion without data finish with 0 written and 0 remaining. Column progress is persisted and older serialized column procedures remain readable. DESCRIBE ... DETAILS continues to show the column's DELETING status while deletion is in progress.

Validation

  • Latest RPC-deadline follow-up at 2686b541cf22021dae6514dd3c5fa3c5f2836676: clean English ConfigNode dependency-closure compilation and all 3 request-manager tests passed. These exercise deadline removal and pooled-client timeout restoration on success/error for 7 deletion request types, plus bounded tree/table/column cache invalidation. Spotless and git diff --check passed.

  • The column-progress runtime and Chinese compilation results below are from 5a50867adb08f638f2d5c7a74532045bf8d97af7; those checks were not rerun for the RPC-deadline follow-up.

  • Column-progress update at 5a50867adb08f638f2d5c7a74532045bf8d97af7: 32 targeted unit tests passed with no failures, errors, or skips, covering procedure persistence/backward compatibility, running and retained task results, counter aggregation, view compatibility, and DataRegion column-deletion mod files.

  • Both new column-progress ITs passed in 1C1D. A clean English integration-test dependency-closure verify -pl integration-test -am -P with-integration-tests,TableClusterIT build passed all 3 cases in the complete mod-progress IT class, with no failures, errors, or skips. Reports and node logs confirm the 1C3D topology and multi-node query verification.

  • Clean Chinese test compilation of ConfigNode and its IoTDB dependency closure passed. Spotless, git diff --check, message-key/placeholder parity, and integration-test discovery checks passed.

  • Earlier tree-deletion validation is historical: the preceding implementation had 56 targeted unit-test and 8 targeted IT passes, including 1C1D/1C3D and Memory/PBTree coverage. The selected audit IT was skipped by its existing class-level @Ignore; its body did not run. These results were not rerun for this column-progress update.

  • Verification reused third-party dependency caches and disabled Maven extensions/build caching. The relevant IoTDB dependency closures were rebuilt from source. Checkstyle was skipped and Enforcer remained at the repository default skipped setting.

  • Not run: full-reactor English/Chinese builds, complete distribution packaging, the remaining affected IT matrix (including the Session-path test update), the audit IT body, and real-cluster leader-switch race scenarios.

This PR has:

  • completed independent self-review of the latest full diff.
  • added unit tests or modified existing tests to cover new code paths.
  • added integration tests.
  • been tested in a test IoTDB cluster, with the snapshot and scope limits described above.

The column-progress update is a scoped fix. The PR remains a draft pending a fresh independent review of the complete latest diff.

Key changed classes

ProcedureManager, ProcedureExecutor, DeleteTimeSeriesProcedure, DeleteTimeSeriesProcedureScheduler, ConfigProcedureStore, DropTableColumnProcedure, TableDeletionModProgress, information-schema suppliers, SQL parsing/progress RPC classes, and integration-test completion helpers.

@Caideyipi
Caideyipi marked this pull request as draft November 9, 2026 08:11
@Caideyipi
Caideyipi marked this pull request as ready for review November 9, 2026 08:17
@Caideyipi

Copy link
Copy Markdown
Collaborator Author

CI reported an error in IoTDBTableDDLHAIT.testHAWithOneDataNodeIsDown while the test waited for asynchronous timeseries deletion. The progress query used the cluster-wide result wrapper, which also queried the DataNode intentionally stopped by this test. The test now polls deletion progress through a single-node statement on its existing live connection.

Validation passed on Windows 11 with JDK 17: mvn -T 2 -Ddevelocity.off=true clean verify -DskipUTs -Dit.test=IoTDBTableDDLHAIT#testHAWithOneDataNodeIsDown -DfailIfNoTests=false -Dfailsafe.failIfNoSpecifiedTests=false -pl integration-test -am -PTableClusterIT -P with-integration-tests (1 test, 0 failures, 0 errors, 0 skipped). mvn spotless:apply -pl integration-test -P with-integration-tests and git diff --check also passed.

Updated from source head 3e66ecaf42e763ae42fc653dd27cf548fcbb9352; final commit: 971feafe81b9dec11a115f36cedfb8b715aa3aa2. Both the source branch and PR head match the final commit.

@Caideyipi
Caideyipi marked this pull request as draft November 11, 2026 03:29
@Caideyipi

Caideyipi commented Nov 11, 2026 •

Copy link
Copy Markdown
Collaborator Author

Scoped re-review: column deletion progress.

Confirmed and fixed the missing observability for ALTER TABLE ... DROP COLUMN: previously DESCRIBE ... DETAILS exposed only DELETING, without a procedure list, execution phase, errors, or mod-file counts.

The new information_schema.drop_table_column_procedures table reports database/table/column, procedure ID, state, phase, errors, and written/remaining mod-file counts. It shares aggregation and persistence with DROP TABLE: one executing replica per Region contributes, unknown counts stay NULL, and completed procedures remain visible under the existing retention policy. Attribute and empty-table column deletion finish with 0/0 mod files. The RPC path, information-table routing, admission messages, backward-compatible persistence, and regression coverage are included.

Column-progress validation at 5a50867adb08f638f2d5c7a74532045bf8d97af7:

  • 32 targeted unit tests passed, with no failures, errors, or skips.
  • Both new column-progress ITs passed in 1C1D, with no failures, errors, or skips.
  • Clean English verify -pl integration-test -am -P with-integration-tests,TableClusterIT passed all 3 mod-progress IT cases, with no failures, errors, or skips. Reports and node logs confirm 1C3D and multi-node query verification.
  • Clean Chinese test compilation of ConfigNode and its IoTDB dependency closure, Spotless, git diff --check, message-key/placeholder parity, and IT discovery passed.
  • Full-reactor builds, complete distribution packaging, the remaining affected IT matrix, and leader-switch race scenarios were not run. Earlier tree-deletion results in the description are historical.

Reviewed source head: 971feafe81b9dec11a115f36cedfb8b715aa3aa2.
Column-progress local commit: 5a50867adb08f638f2d5c7a74532045bf8d97af7.
Confirmed source-branch and PR head after the column-progress update: 5a50867adb08f638f2d5c7a74532045bf8d97af7 (both matched that column-progress commit).

This was a scoped fix round, not an independent review of the full PR. The PR remains a draft pending a fresh independent review of the complete latest diff. Remote CI was not awaited.

Requested follow-up: internal deletion RPC deadlines

The tree/column deletion requests still used the default client deadline. They now use the same temporary -1 timeout policy as DROP TABLE: tree-schema blacklist construction/rollback, timeseries data/schema deletion, and field/attribute column deletion. The original timeout is restored when the pooled client returns after success or error. Cache-invalidation broadcasts retain their bounded timeout, as for DROP TABLE.

Validation at 2686b541cf22021dae6514dd3c5fa3c5f2836676: clean English ConfigNode and transitive IoTDB dependency-closure compilation passed (29 reactor modules), with all 3 request-manager test methods passing and no failures, errors, or skips. They exercise 7 deletion request types, both default/explicit timeout arguments, success/error restoration, and bounded tree/table/column cache invalidation. Spotless and git diff --check passed. ITs, Chinese compilation, complete regression, full-reactor builds, and distribution packaging were not rerun for this deadline-only follow-up; the column-progress results above remain tied to their recorded snapshot.

Follow-up source baseline: 5a50867adb08f638f2d5c7a74532045bf8d97af7.
Final local and confirmed remote source-branch/PR head: 2686b541cf22021dae6514dd3c5fa3c5f2836676.

The PR remains a draft pending a fresh independent review of the complete latest diff. This requested follow-up is not another independent review round. Remote CI was not awaited.

Comment on lines +2424 to +2460
final String paths = "[" + sql.substring(sql.lastIndexOf(' ') + 1) + "]";
// Poll the ConfigNode directly so diagnostics do not add SQL audit records to this test.
try (SyncConfigNodeIServiceClient client =
(SyncConfigNodeIServiceClient) EnvFactory.getEnv().getLeaderConfigNodeConnection()) {
final long previousProcedureId =
client.showDeleteTimeSeriesProcedures().getProcedureInfoList().stream()
.filter(p -> paths.equals(p.getPathPatterns()))
.mapToLong(p -> p.getProcedureId())
.max()
.orElse(-1L);
statement.execute(sql);
await()
.atMost(60, TimeUnit.SECONDS)
.pollInSameThread()
.pollInterval(20, TimeUnit.MILLISECONDS)
.until(
() -> {
final java.util.Optional<
org.apache.iotdb.confignode.rpc.thrift.TDeleteTimeSeriesProcedureInfo>
procedure =
client.showDeleteTimeSeriesProcedures().getProcedureInfoList().stream()
.filter(p -> paths.equals(p.getPathPatterns()))
.filter(p -> p.getProcedureId() > previousProcedureId)
.max(java.util.Comparator.comparingLong(p -> p.getProcedureId()));
if (!procedure.isPresent()) {
return false;
}
if ("ROLLEDBACK".equals(procedure.get().getState())) {
throw new AssertionError(
"Timeseries deletion procedure "
+ procedure.get().getProcedureId()
+ " rolled back: "
+ procedure.get().getErrorMessage());
}
return "SUCCESS".equals(procedure.get().getState());
});
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is TestUtils.deleteTimeSeries not applied here.
And mind the import.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test deliberately keeps progress polling out of SQL. TestUtils.deleteTimeSeries submits through a Session and then polls SHOW DELETE TIMESERIES PROGRESS, which would add audit events and change the exact audit-log expectations of this test. The test keeps its JDBC submission and uses direct ConfigNode RPC for diagnostics; the imports are now explicit.

Comment on lines +68 to +109
@Test
public void testColumnDeletionWaitsForFinalization() {
final AtomicInteger probes = new AtomicInteger();
final ResultSet absentColumn =
(ResultSet)
Proxy.newProxyInstance(
ResultSet.class.getClassLoader(),
new Class<?>[] {ResultSet.class},
(proxy, method, args) -> {
if ("next".equals(method.getName())) {
return false;
}
if ("close".equals(method.getName())) {
return null;
}
throw new UnsupportedOperationException(method.getName());
});
final Statement statement =
(Statement)
Proxy.newProxyInstance(
Statement.class.getClassLoader(),
new Class<?>[] {Statement.class},
(proxy, method, args) -> {
if ("executeQuery".equals(method.getName())) {
return absentColumn;
}
if ("execute".equals(method.getName())) {
assertEquals(
"ALTER TABLE test.t DROP COLUMN IF EXISTS \"removed\"\"column\"", args[0]);
if (probes.incrementAndGet() == 1) {
throw new SQLException(
"deletion still finalizing",
null,
TSStatusCode.OVERLAP_WITH_EXISTING_TASK.getStatusCode());
}
return false;
}
throw new UnsupportedOperationException(method.getName());
});
TestUtils.waitForColumnDeletion(statement, "test.t", "removed\"column");
assertEquals(2, probes.get());
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does not seem to be an integration test.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. The test only used proxy Statement/ResultSet objects and never exercised an IoTDB environment, so I removed it from the integration-test class. The real table-deletion IT now exercises TestUtils.dropColumn against IoTDB.

Comment on lines +141 to +168
Awaitility.await()
.pollInterval(200, TimeUnit.MILLISECONDS)
.atMost(60, TimeUnit.SECONDS)
.until(
() -> {
boolean present = false;
try (ResultSet result =
statement.executeQuery("DESCRIBE " + database + ".sensor DETAILS")) {
while (result.next()) {
present |= "changing".equals(result.getString(1));
}
}
if (!present) {
return true;
}
// A failed task retains its pre-delete marker. Retrying the drop after restoring
// the RPC completes it; a still-running task rejects the overlapping request.
try {
update("ALTER TABLE sensor DROP COLUMN changing");
} catch (SQLException e) {
assertTrue(
e.getMessage(),
e.getErrorCode() == TSStatusCode.OVERLAP_WITH_EXISTING_TASK.getStatusCode()
|| e.getErrorCode() == TSStatusCode.COLUMN_NOT_EXISTS.getStatusCode());
}
return false;
});
TestUtils.waitForColumnDeletion(statement, database + ".sensor", "changing");

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are the two waits duplicated?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The waits cover different outcomes. The first loop belongs to the injected-failure recovery scenario: it retries while the column remains and confirms schema removal after the RPC is restored. waitForColumnDeletion then waits for the asynchronous procedure to finish durable finalization before dependent DDL. The visible schema check overlaps, but the completion probe does not.

Comment on lines +1415 to +1416
statement.execute("ALTER TABLE t DROP COLUMN attr");
TestUtils.waitForColumnDeletion(progress, "async_drop_column_db.t", "attr");

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not TestUtils.dropColumn?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed to TestUtils.dropColumn, which submits the DDL and waits for completion through the shared helper. The IT covers field and attribute deletion, plus a quoted column name.

Comment on lines +827 to +832
public void setTimeseriesDeletionExecutionIntervalMs(long intervalMs) {
if (intervalMs <= 0) {
throw new IllegalArgumentException(CommonMessages.SIZE_MUST_BE_POSITIVE);
}
timeseriesDeletionExecutionIntervalMs = intervalMs;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

<= 0 means no wait

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated: zero and negative values normalize to 0, which the scheduler treats as no start interval. The properties template documents this, and tests cover both values plus runtime reload.

Comment on lines +205 to +210
if (prefix.hasWildcard()) {
remaining.appendPathPattern(path);
continue;
}
final Map<String, TDatabaseSchema> matched =
manager.getClusterSchemaManager().getMatchedDatabaseSchemasByPrefix(prefix);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May still delete a database if it is fully covered by the prefix.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. A literal prefix ending in ** still matches full tree-model databases and routes those through the database-schema procedure path, with the template check retained and schemas recreated by the durable procedure. Patterns that do not cover a whole database continue through per-timeseries deletion.

Comment on lines +465 to +480
final String[] databaseNodes = databasePath.getNodes();
for (PartialPath path : getPatternTree().getAllPathPatterns()) {
if (!PathPatternUtil.isMultiLevelMatchWildcard(path.getMeasurement())
|| path.getNodeLength() <= 1) {
continue;
}
final PartialPath prefix =
new PartialPath(Arrays.copyOf(path.getNodes(), path.getNodeLength() - 1));
if (!prefix.hasWildcard()
&& prefix.getNodeLength() <= databaseNodes.length
&& Arrays.equals(
prefix.getNodes(), Arrays.copyOf(databaseNodes, prefix.getNodeLength()))) {
return true;
}
}
return false;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to search the database nodes directly in the pattern tree and check whether the matching node is a "**" or contains a child of "**", without constructing the intermediate paths?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The async procedure keeps the schema manager prefix lookup. It strips only terminal ** to form the literal prefix and calls getMatchedDatabaseSchemasByPrefix; it does not synthesize intermediate paths. Prefixes containing wildcards stay on the normal pattern-tree path. This preserves the indexed lookup and its matching semantics.

Comment on lines +483 to 486
public String getDeleteProgress() {
final DeleteTimeSeriesState state = getCurrentState();
return isFinished() || state == null ? getState().name() : state.name();
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A detailed progress assembling dropping table?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Progress is phase-based rather than a percentage: the procedure exposes queued/running/finalizing status and phases such as preparation, blacklist construction, cache/data/schema deletion, and database recreation. Region deletion does not supply reliable per-series completion counts, so a percentage would be misleading.

Comment on lines +91 to +102
private List<Procedure<ConfigNodeProcedureEnv>> copyDeletionProcedures(
List<Procedure<ConfigNodeProcedureEnv>> procedures) {
final List<Procedure<ConfigNodeProcedureEnv>> copies = new ArrayList<>(procedures.size());
for (Procedure<ConfigNodeProcedureEnv> procedure : procedures) {
try {
copies.add(copyDeletionProcedure(procedure));
} catch (IOException e) {
throw new IllegalStateException(ConfigNodeMessages.LOAD_PROCEDURE_WAL_FAILED, e);
}
}
return copies;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a detailed explaination about why copying is necessary.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment now explains the ownership boundary: ProcedureInfo retains the last consensus-applied instance, while the executor mutates its working instance before the next consensus write. The serialized copy keeps uncommitted state changes from appearing durable when a write fails, protecting retries and recovery.

Comment on lines +2389 to +2393
# Minimum interval between starting tree-model timeseries deletion tasks, in milliseconds.
# Measured when tasks leave the queue to start execution; submissions are not rate-limited.
# effectiveMode: restart
# Datatype: long, greater than 0
timeseries_deletion_execution_interval_ms=1000

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make it hot-reload

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The execution interval is hot-reloadable: configuration updates the ProcedureExecutor scheduler and wakes workers waiting on the interval. Tests cover applying a runtime change, waking an existing wait after an interval decrease, and keeping the scheduler in sync when a later setting fails validation.

@Caideyipi

Copy link
Copy Markdown
Collaborator Author

Independent review: two-patch scope

This round reviewed only 5a50867adb08f638f2d5c7a74532045bf8d97af7 (column-deletion procedure, execution phase, and mod-file progress) and 2686b541cf22021dae6514dd3c5fa3c5f2836676 (internal deletion RPC timeout policy), against 971feafe81b9dec11a115f36cedfb8b715aa3aa2. The reviewed PR/source head was 2686b541cf22021dae6514dd3c5fa3c5f2836676. Final local commit, confirmed source-branch head, and confirmed PR head are all 1af99c45596be2b983b7cc9057b376b3c5dad5d5. The PR remains Draft.

Finding (P2, fixed): In AsyncRequestManager.java:230 and CnToDnInternalServiceAsyncRequestManager.java:588, a synchronous exception while dispatching a deletion RPC after its deadline was disabled could return the pooled client without restoring its configured timeout. The next borrower could then inherit the unbounded timeout for unrelated requests. The pre-dispatch cleanup now restores the original timeout before returning the client, and a regression test covers this failure path.

No other confirmed issues were found in the two-patch scope. The review checked progress aggregation and per-Region replica deduplication, procedure/task correlation under retries and replica changes, persistence and older serialized state, view-column compatibility, information-schema registration/RPC/routing/result fields, error and terminal-state reporting, long-running deletion RPC deadlines, bounded cache-invalidation broadcasts, and shared DROP TABLE behavior.

A separate user-requested change in the final update makes timeseries_deletion_execution_interval_ms hot reloadable and wakes a waiting deletion scheduler. It is outside the two-patch review conclusion and is included in the next independent review scope.

Verification: On final candidate 1af99c45596be2b983b7cc9057b376b3c5dad5d5, a clean English ConfigNode dependency-closure test run passed all four focused test classes: 12 tests, 0 failures, 0 errors, 0 skipped. Spotless apply for ConfigNode and node-commons passed with no formatting changes. At 5a50867adb08f638f2d5c7a74532045bf8d97af7, the earlier focused tests, 1C1D/1C3D ITs, and English/Chinese dependency-closure compiles passed; at 2686b541cf22021dae6514dd3c5fa3c5f2836676, the clean English dependency-closure compile and focused unit tests passed. Those results belong to their stated snapshots.

The 1C1D/1C3D ITs, Chinese compile, full build, and full regression were not rerun on the final candidate. A separate clean remote build attempt could not resolve Maven Surefire from Maven Central and did not execute tests. Remote CI was not awaited. A fresh independent review is required after this behavior fix; the PR remains Draft.

@Caideyipi

Copy link
Copy Markdown
Collaborator Author

Independent re-review completed in this session (one round). Three P2 behavioral findings were fixed and pushed. The PR remains Draft pending a fresh independent re-review.

  • Comparison baseline: 971feafe81b9dec11a115f36cedfb8b715aa3aa2.
  • Reviewed PR/source head: 1af99c45596be2b983b7cc9057b376b3c5dad5d5.
  • Final local commit and confirmed source-branch/PR head: a4ff94d3fe86e9df85828f61dc5c4b3c9f54e686.

Original column-progress and RPC changes

  1. P2: column deletion reported a terminal result before durable completion. While terminal consensus persistence was being retried, the public progress listing could already expose SUCCESS or ROLLEDBACK. The listing now reports RUNNABLE / FINALIZING until the executor retains the completed result, preserving the error and mod-file counters. Regression coverage checks both terminal outcomes, including actual blocked persistence paths. See ProcedureManager.java.
  2. P2: completing tasks could disappear from a progress query. The lazy active-task stream was consumed after the completed-task snapshot had already been taken. Completion between those reads could therefore miss both collections. Column and shared DROP TABLE listings now use the existing eager active-first snapshot helper, retaining procedure-ID deduplication. A deterministic regression exercises both listings. See ProcedureManager.java.

The previous timeout-restoration repair was independently checked in dispatch cleanup, callbacks, retries and replica switching. The synchronous-dispatch regression now asserts the original timeout inside returnClient, so restoration after pool return cannot satisfy it. It passed without changing the starting production timeout implementation. Cache-invalidation broadcasts continue to use finite RPC timeouts.

The requested diff and relevant call contexts were also checked for Region/replica counting, NULL versus zero counters, procedure/task association, concurrent and late responses, persistence and old serialization, view-column compatibility, information-schema registration/query paths, and shared table-deletion behavior. No additional confirmed finding was identified in those paths during this round.

Additional deletion-interval hot reload

  1. P2: a later configuration error could leave the scheduler using the old interval. The descriptor can apply the interval before another setting fails validation. Previously, that exception skipped the scheduler update; retrying the same interval could skip it again. The old comparison was also captured before the configuration-file lock. Configuration application and scheduler reconciliation are now serialized, with reconciliation in finally against the effective in-memory value. Tests exercise the public configuration API, a later parse failure, same-value retry, invalid nonpositive values, waiting-worker wakeup, and interval changes while a task is already active. See ConfigManager.java.

Verification

  • Reproduced premature column terminal display and the hot-reload mismatch on the starting production source with regression tests: 46 tests, four expected assertion failures, no errors/skips. The snapshot race was separately reproduced before its repair.

  • Built the English ConfigNode dependency closure from clean source. After the last ConfigNode-only edit, cleaned and rebuilt ConfigNode and reused the unchanged dependencies from that exact-source build. Final-source results: 47 tests, zero failures, errors or skips:

    Test class Tests
    CnToDnInternalServiceAsyncRequestManagerTest 4
    TimeseriesDeletionExecutionIntervalHotReloadTest 2
    AsyncTimeSeriesDeletionTest 17
    ConfigManagerTimeseriesDeletionIntervalReloadTest 3
    DropTableColumnProcedureTest 5
    DropTableProcedureTest 10
    DeleteTimeSeriesProcedureSchedulerTest 6
  • ConfigNode Spotless apply and git diff --check passed. New tests added in this session ran remotely; they were not run locally. An initial regression attempt stopped at a wrong import in a newly written test; that import was corrected before the reported test executions.

  • ITs, Chinese compilation, a full build and the complete regression suite were not run on the final commit. Historical results from earlier snapshots are not counted as final-commit reruns. Remote CI was not awaited.

  • Immediately before pushing, the PR was still Draft and both PR/source heads matched the reviewed SHA. The update was guarded against a changed source ref. After pushing to Caideyipi/iotdb:feature/async-schema-deletion, both heads were confirmed equal to the final local commit above; Draft status was retained.

@Caideyipi

Copy link
Copy Markdown
Collaborator Author

Independent re-review completed in this session (one round). One P2 behavioral finding was repaired. The PR remains Draft pending a fresh independent re-review.

  • Comparison baseline: 971feafe81b9dec11a115f36cedfb8b715aa3aa2.
  • Reviewed PR/source head: a4ff94d3fe86e9df85828f61dc5c4b3c9f54e686.
  • Final local commit and confirmed source-branch/PR head: ddaaaed71d92e8f8cc15e5491e892b8df530f849.

Original column-progress and RPC changes

  1. P2: a transition during column-progress listing could bypass the durable-completion guard. The listing read state first and progress later. If execution entered SUCCESS or ROLLEDBACK between those reads, the earlier nonterminal state bypassed the FINALIZING guard while the later read exposed terminal progress, even though terminal persistence could still be retrying. The repair captures progress before checking terminal state and derives both terminal display fields from that checked state. The new deterministic regression exercises both terminal outcomes before and after executor completion. See ProcedureManager.java and AsyncTimeSeriesDeletionTest.java.

The full current diff and relevant callers were inspected, including synchronous dispatch exceptions and timeout restoration before pool return, success/error callbacks, retries, replica switching and finite cache-invalidation broadcasts. Region deduplication, executing-replica counts, unknown versus zero values, procedure/task IDs, concurrent and late responses, persistence and old formats, view-column compatibility, information-schema registration/query/filtering, terminal errors, and the shared DROP TABLE progress helper and eager snapshots were checked. No additional confirmed finding was identified in those paths. The earlier timeout, snapshot and persistence repairs and their regression logic were checked independently.

Additional deletion-interval hot reload

No new confirmed finding. Checked configuration applicability and positive-value validation, serialized configuration application, reconciliation in finally after later validation errors and same-value retries, executor initialization/reload synchronization, waiting-worker wakeup, interval increases/decreases, and preservation of active-task concurrency slots.

Verification

  • The new regression reproduced the issue on the reviewed production source: one test, one expected assertion failure (FINALIZING versus SUCCESS), no errors/skips.

  • Built the required TsFile source and the 29-module English ConfigNode dependency closure from clean source. After the ConfigNode-only repair, cleaned/rebuilt ConfigNode and reused only unchanged dependencies from that exact-source build. Final-source English results: 50 tests, zero failures, errors or skips:

    Selection Tests
    CnToDnInternalServiceAsyncRequestManagerTest 4
    TimeseriesDeletionExecutionIntervalHotReloadTest 2
    AsyncTimeSeriesDeletionTest 18
    ConfigManagerTimeseriesDeletionIntervalReloadTest 3
    DropTableColumnProcedureTest 5
    DropTableProcedureTest 10
    DeleteTimeSeriesProcedureSchedulerTest 6
    DropViewColumnProcedureTest 1
    DataRegionTest#testDropColumnModFileProgress 1
  • Clean Chinese test-compile of the complete ConfigNode dependency closure passed on the final source. ConfigNode Spotless apply and git diff --check passed.

  • Executable tests and compilation ran remotely; no local tests were run. No cluster ITs, full distribution build or complete regression suite were run on the final source. The starting-source dependency installation also ran one pre-existing EnvScriptIT successfully; that is not counted as a final-source IT rerun. Historical results are not treated as reruns. Remote CI was not awaited.

  • Immediately before pushing, the PR remained Draft and both PR/source heads matched the reviewed SHA. The push to Caideyipi/iotdb:feature/async-schema-deletion was guarded against a changed source ref. After pushing, both heads were confirmed equal to the final local commit above, with Draft status retained.

@Caideyipi

Copy link
Copy Markdown
Collaborator Author

Independent Review Summary

Compared with base 971feafe81b9dec11a115f36cedfb8b715aa3aa2; reviewed PR head ddaaaed71d92e8f8cc15e5491e892b8df530f849.

Found one test-logic issue at DeleteTimeSeriesProcedureSchedulerTest.java:93. The latch was released before the worker entered scheduler.poll(), so the short future timeout could expire before polling began and let the test pass without exercising a waiting worker. The test now waits until the worker is TIMED_WAITING and confirms the future is still incomplete before reloading the interval.

No source-behavior issue was confirmed in the complete PR diff.

Final local commit and confirmed source/PR head: a4894c01a9fc43e32a3c364e09752b95d47ce6a8.

Verification: Spotless apply and git diff --check passed. The focused scheduler test did not execute on the final commit: remote Maven builds stopped before compilation because org.apache:apache:pom:33 was unavailable in the dependency cache and Maven Central was unreachable. No local tests were run; earlier results at the reviewed SHA are not counted as validation of the final commit.

@Caideyipi
Caideyipi marked this pull request as ready for review November 12, 2026 03:04
@Caideyipi
Caideyipi marked this pull request as draft November 17, 2026 03:11
@Caideyipi

Copy link
Copy Markdown
Collaborator Author

Addressed the 12 inline comments in a1cb725: reused the shared column-drop helper and removed the proxy-only IT; kept direct ConfigNode RPC polling in the audit test so diagnostics do not add audit events; documented non-positive execution intervals and covered runtime reload; separated table-task conflicts from tree deletion; retained the whole-database prefix optimization and template validation; and clarified why stored procedure copies are required. Phase-based progress and the two-stage column wait remain intentional because they track different procedure phases and completion conditions.

Validation: Spotless and git diff --check passed; 10 targeted ConfigNode unit tests passed; IoTDBTableIT#testAsyncDropColumn passed under both TableSimpleIT and TableClusterIT (one test each). Remote Maven attempts were blocked because Maven Central was unreachable, and the offline retry lacked the Checkstyle plugin. The local commit and PR head are a1cb725. The PR remains Draft pending a fresh independent review.

@Caideyipi

Copy link
Copy Markdown
Collaborator Author

Fixed the failing AsyncTimeSeriesDeletionTest.testFullDatabaseDeletionReservesTableAndDatabaseChanges case.

checkDuplicateTableTask now treats a pending full-database tree timeseries deletion as a conflict for table-model operations in the same database, including while the procedure is finalizing. The correlated DELETE_DATABASE_PROCEDURE from that deletion may continue, while another table operation with the same query ID remains blocked.

Validation on 8325dc3: a clean ConfigNode dependency-closure run passed 18 tests, with 0 failures, errors, or skips. Spotless and git diff --check passed. The remote attempt could not reach compilation because the required TsFile snapshot was absent from that host cache; the matching snapshot source was built locally and the focused test passed locally.

Starting PR head: a1cb725. Final local, source-branch, and PR head: 8325dc3. The PR remains Draft pending an independent re-review of the full diff.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants