Skip to content

[PERF] queue_job: declare the indexes as models.Index and drop the one on channel - #1034

Open
moylop260 wants to merge 1 commit into
OCA:19.0from
vauxoo-dev:drop-covered-channel-index-19.0
Open

moylop260 wants to merge 1 commit into
OCA:19.0from
vauxoo-dev:drop-covered-channel-index-19.0

Conversation

@moylop260

@moylop260 moylop260 commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

queue.job.channel has index=True, so the ORM creates queue_job__channel_index. The composite index on (channel, date_done, date_created) has channel as its leading column, so the single column index is covered by it. PgHero reports it as such. On a real database it is 64 MB that every job insert and state change maintains and vacuum visits.

This PR does two things:

  1. Removes index=True from channel. Existing databases keep the old index (the ORM only logs "Keep unexpected index" when a field stops asking for one), so it has to be dropped by hand there.
  2. Moves the two indexes that init() created with create_index to models.Index attributes, the way 19.0 declares them, with the names they already have. Index.apply_to_database rebuilds an index whose comment differs from its definition, so migrations/19.0.2.5.0/pre-migration.py gives the two existing indexes the comment the ORM expects. The directory name assumes the next minor bump; rename it if the bot bumps another version, otherwise the first update rebuilds the two indexes once.

Index that replaces the removed one

Measured: the same query with and without the index

EXPLAIN (ANALYZE, VERBOSE, BUFFERS) SELECT id FROM <table> WHERE <column> = <value> on the staging copy of 2026-11-09 (PostgreSQL 18.6, work_mem 16MB, a database of about 200 GB). Each plan is run twice inside a transaction and the second is kept. "Without" is a DROP INDEX of the removed index in the same transaction (rolled back afterwards). Two values per column: the most frequent one and one of median frequency. enable_seqscan=off was set so that every plan uses an index, otherwise the small tables answer with a sequential scan in both cases. Buffers (shared hit + read) are the number to compare; the milliseconds move with the cache.

Field Index removed Replaced by (columns) Definition MB Most frequent value: rows, buffers with -> without, ms with -> without Median value: rows, buffers with -> without, ms with -> without Verdict
queue.job.channel queue_job__channel_index queue_job_channel_date_done_date_created_index (channel, date_done, date_created) queue_job.py#L162 66.9 3209529 rows, 863143 -> 878710, 8228.6 -> 13160.7 786686 rows, 416475 -> 703278, 1298.8 -> 1779.7 wider index reads more

Not confirmed as unnecessary. The narrow index reads fewer pages when the lookup is by that column alone and unselective:

  • mail.followers.res_model = <most frequent model> (about 8M rows): 125,235 buffers with the simple index, 3,718,346 without it (2.5 s -> 7.6 s forced; 3.1 s -> 5.6 s when the planner chooses). With a selective value (249 rows) both cost the same.
  • queue.job.channel: the most frequent value (3.2M rows) costs the same (863,143 vs 878,710 buffers), a median one (787K rows) reads 416,475 buffers with the simple index and 703,278 without it.

The other 26 fields cost the same or within a few pages; the "wider index reads more" mark on the small tables (account.journal.company_id, portal_donor.card.type.name, ...) is a difference of tens of pages on tables that answer in less than 1 ms.

Checked with a real -u all

On a clone of a migrated database (about 200 GB, queue_job with 3.2M rows), with the comments set by hand to the same text as the pre-migration (the manifest version was not bumped, so the script itself did not run): -u all ended with RC=0 and odoo.schema at DEBUG logged no index event for queue_job, so neither index was rebuilt. A second -u all logged no index creation or drop at all. The OCA test suite was not run.

@OCA-git-bot

Copy link
Copy Markdown
Contributor

Hi @guewen, @sbidoul,
some modules you are maintaining are being modified, check this out!

@moylop260
moylop260 force-pushed the drop-covered-channel-index-19.0 branch 2 times, most recently from 304cf60 to a14107d Compare November 10, 2026 07:44
…e on channel

The two indexes created in init() become models.Index attributes, the way
19.0 declares them, with the names they already have. models.Index
compares the comment of the existing index with its definition and
rebuilds it when they differ, so a pre-migration gives the existing
indexes the comment the ORM expects and the first update does not
rebuild them on a big queue_job table.

queue.job.channel has index=True, which makes the ORM create
queue_job__channel_index. The composite index on
(channel, date_done, date_created) has channel as its leading column.
New databases never create the single column index. Existing ones keep
it, because the ORM only logs "Keep unexpected index" when a field stops
asking for one.
@moylop260 moylop260 changed the title [PERF] queue_job: drop the index on channel, the composite index covers it [PERF] queue_job: declare the indexes as models.Index and drop the one on channel Nov 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants