chatwoot/spec/models
Vishnu Narayanan fed32d5964
fix(perf): force better index for pending_reponses longest first sort (#14291)
## Description

Fix a Postgres planner trap on the "Pending Response: Longest first"
sort that causes the conversation list to hang on busy accounts.

The current `sort_on_waiting_since` generates query with `ORDER BY
waiting_since ASC NULLS LAST, created_at ASC`. That order-by is exactly
the shape of the single-column `index_conversations_on_waiting_since`
btree, so the planner picks a forward index walk thinking `LIMIT 25`
will stop early. In practice the per-account matches are spread along
the global waiting_since timeline, so the scan reads tens of millions of
rows from other accounts and discards them via the filter before
producing any results which in turn causes the requests to time out and
the conversation list spinner never resolves.

DESC direction and every other sort (`priority`, `created_at`,
`last_activity_at`) are unaffected. They fall through to
`conv_acid_inbid_stat_asgnid_idx` (account-scoped composite), which is
the right index for this access pattern.

This change leads the ORDER BY with the expression `(waiting_since IS
NULL)`, which no column-only btree can satisfy. The planner falls back
to the same account-scoped index used by every other sort, and sorts in
memory. Same logical NULLS LAST output for both directions; no behavior
change for users.

Fixes CW-6965

## Type of change

- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality not to work as expected)
- [ ] This change requires a documentation update

## How Has This Been Tested?

- [x] Existing specs pass
- [x] Added new specs to cover NULL case
- [x] Verify results and order for old and new query in prod
- [x] Tested in prod since staging data was not sufficient

`EXPLAIN (ANALYZE, BUFFERS)` for the same query (status=0, ASC, LIMIT
25) on a representative production account, before vs after:

| Metric | Before | After |
| --- | --- | --- |
| Execution time | 34,679 ms | 0.71 ms |
| Rows discarded by filter | 36,906,962 | 0 |
| Shared buffer hits | 12,699,924 | 11 |
| Blocks read from disk | 851,226 | 112 |
| I/O read time | 17,785 ms | 0.3 ms |
| Pages dirtied / written | 98 / 31,043 | 0 / 0 |

Verified on two production accounts: identical row IDs in identical
order between the old and new ORDER BY for both ASC and DESC. A
NULL-bucket regression spec was added covering ASC/DESC tail ordering
when some conversations have a null `waiting_since`.

Roughly `49,000×` faster on this query (34,679 ms → 0.71 ms), and
trivially less I/O and buffer pressure on the cluster while it runs.

## Checklist:

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [ ] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules

Co-authored-by: Tanmay Deep Sharma <32020192+tds-1@users.noreply.github.com>
2026-04-28 18:50:29 +07:00
..
channel fix: call authorization_error! on IMAP auth failures (#13560) (revert) (#13671) 2026-02-26 18:45:18 -08:00
concerns fix: Skip email rate limiting for self-hosted instances (#13915) 2026-03-26 18:06:10 +05:30
enterprise/audit feat: Add conversation delete feature (#11677) 2025-06-05 15:53:17 -05:00
integrations feat: new Captain Editor (#13235) 2026-01-21 13:39:07 +05:30
.keep Fix url in emails, add frontendURL helper (#19) 2019-08-25 19:59:28 +05:30
account_spec.rb fix: validate support_email format and handle parse errors in mailer (#13958) 2026-04-13 19:06:06 +07:00
account_user_spec.rb chore: Add permissions to auth data (#9695) 2024-06-28 12:52:48 -07:00
agent_bot_inbox_spec.rb chore: Enable the new Rubocop rules (#7122) 2023-05-19 14:37:10 +05:30
agent_bot_spec.rb fix(agent-bots): destroy permissibles on AgentBot deletion and skip orphans in index (#14273) 2026-04-27 19:17:32 +05:30
application_record_external_credentials_encryption_spec.rb feat: Secure external credentials with database encryption (#12648) 2025-10-13 18:05:12 +05:30
article_spec.rb chore: allow article to create without content (#14007) 2026-04-09 10:40:37 +05:30
assignment_policy_spec.rb feat: Add assignment policies controllers with jbuilder views (#12199) 2025-08-18 19:15:21 -07:00
attachment_spec.rb fix: scope external_url override to Instagram DM conversations only (#13982) 2026-04-02 19:56:23 +05:30
automation_rule_spec.rb fix: add explicit remove assignment actions to macros and automations (#12172) 2026-04-16 15:57:41 +05:30
campaign_spec.rb chore: Add validations to campaign model 2025-03-19 17:29:08 -07:00
category_spec.rb chore: Enable the new Rubocop rules (#7122) 2023-05-19 14:37:10 +05:30
contact_inbox_spec.rb Revert "chore: Upgrade Rails to 7.2.2 and update Gemfile dependencies (#11037)" 2026-02-03 21:09:42 -08:00
contact_spec.rb fix: [CW-6931] Harden external downloads against SSRF [avatar from url job] (#14153) 2026-04-24 18:59:45 +05:30
conversation_participants_spec.rb chore: Enable the new Rubocop rules (#7122) 2023-05-19 14:37:10 +05:30
conversation_spec.rb fix(perf): force better index for pending_reponses longest first sort (#14291) 2026-04-28 18:50:29 +07:00
csat_survey_response_spec.rb chore: Enable the new Rubocop rules (#7122) 2023-05-19 14:37:10 +05:30
custom_attribute_definition_spec.rb feat: harden filter service 2026-03-09 21:19:20 +05:30
data_import_spec.rb chore: Add delay before running dataimport job (#8039) 2023-10-03 22:18:57 -07:00
folder_spec.rb chore: Enable the new Rubocop rules (#7122) 2023-05-19 14:37:10 +05:30
inbox_member_spec.rb chore: Enable the new Rubocop rules (#7122) 2023-05-19 14:37:10 +05:30
inbox_spec.rb feat: remove colon and semicolons when sanitizing inbox name (#11889) 2025-07-08 09:41:40 +05:30
installation_config_spec.rb fix: Annotaterb model annotation incomplete migration (#13132) 2026-03-25 17:51:06 -07:00
label_spec.rb chore: Enable the new Rubocop rules (#7122) 2023-05-19 14:37:10 +05:30
macro_spec.rb test: reset Current.user after spec (#8123) 2023-10-17 18:06:54 +05:30
mention_spec.rb chore: Enable the new Rubocop rules (#7122) 2023-05-19 14:37:10 +05:30
message_spec.rb fix: Send raw content in webhook payloads instead of channel-rendered markdown (#13896) 2026-03-25 16:56:22 +04:00
note_spec.rb feat(ee): Add Captain features (#10665) 2025-01-14 16:15:47 -08:00
notification_setting_spec.rb Feature: Add new notification settings for user (#569) 2020-02-29 20:41:09 +05:30
notification_spec.rb feat: speed up circleci and github actions (#12849) 2025-11-19 15:32:48 +05:30
platform_app_permissible_spec.rb feat: Add Platform APIs (#1456) 2021-01-14 20:35:22 +05:30
platform_app_spec.rb chore: Limit objects returned by conversation API (#2721) 2021-07-31 21:19:42 +05:30
portal_spec.rb feat: Add draft status for help center locales (#13768) 2026-03-17 12:45:54 +04:00
related_category_spec.rb chore: Enable the new Rubocop rules (#7122) 2023-05-19 14:37:10 +05:30
reporting_event_spec.rb chore: Enable the new Rubocop rules (#7122) 2023-05-19 14:37:10 +05:30
reporting_events_rollup_spec.rb feat(rollup): add models and write path [1/3] (#13796) 2026-03-19 13:12:36 +05:30
team_member_spec.rb chore: Enable the new Rubocop rules (#7122) 2023-05-19 14:37:10 +05:30
team_spec.rb feat: invalidate cache after inbox members or team members update (#10869) 2025-02-20 21:28:38 -08:00
user_spec.rb fix: linear and user association spec (#13056) 2025-12-12 18:53:26 +05:30
webhook_spec.rb feat: add per-webhook secret with backfill migration (#13573) 2026-02-26 17:26:12 +05:30
working_hour_spec.rb fix: Update timezone to get wday from working_hours (#5605) 2022-10-12 14:32:54 -07:00