chatwoot/app/models
Vishnu Narayanan fed32d5964
fix(perf): force better index for pending_reponses longest first sort (#14291)
## Description

Fix a Postgres planner trap on the "Pending Response: Longest first"
sort that causes the conversation list to hang on busy accounts.

The current `sort_on_waiting_since` generates query with `ORDER BY
waiting_since ASC NULLS LAST, created_at ASC`. That order-by is exactly
the shape of the single-column `index_conversations_on_waiting_since`
btree, so the planner picks a forward index walk thinking `LIMIT 25`
will stop early. In practice the per-account matches are spread along
the global waiting_since timeline, so the scan reads tens of millions of
rows from other accounts and discards them via the filter before
producing any results which in turn causes the requests to time out and
the conversation list spinner never resolves.

DESC direction and every other sort (`priority`, `created_at`,
`last_activity_at`) are unaffected. They fall through to
`conv_acid_inbid_stat_asgnid_idx` (account-scoped composite), which is
the right index for this access pattern.

This change leads the ORDER BY with the expression `(waiting_since IS
NULL)`, which no column-only btree can satisfy. The planner falls back
to the same account-scoped index used by every other sort, and sorts in
memory. Same logical NULLS LAST output for both directions; no behavior
change for users.

Fixes CW-6965

## Type of change

- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality not to work as expected)
- [ ] This change requires a documentation update

## How Has This Been Tested?

- [x] Existing specs pass
- [x] Added new specs to cover NULL case
- [x] Verify results and order for old and new query in prod
- [x] Tested in prod since staging data was not sufficient

`EXPLAIN (ANALYZE, BUFFERS)` for the same query (status=0, ASC, LIMIT
25) on a representative production account, before vs after:

| Metric | Before | After |
| --- | --- | --- |
| Execution time | 34,679 ms | 0.71 ms |
| Rows discarded by filter | 36,906,962 | 0 |
| Shared buffer hits | 12,699,924 | 11 |
| Blocks read from disk | 851,226 | 112 |
| I/O read time | 17,785 ms | 0.3 ms |
| Pages dirtied / written | 98 / 31,043 | 0 / 0 |

Verified on two production accounts: identical row IDs in identical
order between the old and new ORDER BY for both ASC and DESC. A
NULL-bucket regression spec was added covering ASC/DESC tail ordering
when some conversations have a null `waiting_since`.

Roughly `49,000×` faster on this query (34,679 ms → 0.71 ms), and
trivially less I/O and buffer pressure on the cluster while it runs.

## Checklist:

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [ ] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules

Co-authored-by: Tanmay Deep Sharma <32020192+tds-1@users.noreply.github.com>
2026-04-28 18:50:29 +07:00
..
channel feat: sign webhooks for API channel and agentbots (#13892) 2026-04-06 15:28:25 +05:30
concerns fix(perf): force better index for pending_reponses longest first sort (#14291) 2026-04-28 18:50:29 +07:00
integrations Revert "chore: Upgrade Rails to 7.2.2 and update Gemfile dependencies (#11037)" 2026-02-03 21:09:42 -08:00
access_token.rb Feature: Access tokens for API access (#604) 2020-03-11 00:02:15 +05:30
account_user.rb feat: Add migration files for assignment v2 (#12147) 2025-08-11 21:44:38 -07:00
account.rb feat: onboarding account details with enriched data [UPM-17][UPM-18] (#13979) 2026-04-28 10:35:51 +05:30
agent_bot_inbox.rb Feature: Access tokens for API access (#604) 2020-03-11 00:02:15 +05:30
agent_bot.rb fix(agent-bots): destroy permissibles on AgentBot deletion and skip orphans in index (#14273) 2026-04-27 19:17:32 +05:30
application_record.rb chore: fix sla email notifications (#9192) 2024-04-04 21:16:49 +05:30
article.rb chore: allow article to create without content (#14007) 2026-04-09 10:40:37 +05:30
assignment_policy.rb chore(annotations): sync model annotations with current schema (#12245) 2025-08-20 20:23:42 +02:00
attachment.rb fix: scope external_url override to Instagram DM conversations only (#13982) 2026-04-02 19:56:23 +05:30
automation_rule.rb fix: standardize contact company field on company_name (#14099) 2026-04-27 18:43:26 +05:30
campaign.rb feat: WhatsApp campaigns (#11910) 2025-07-16 09:04:02 +05:30
canned_response.rb chore: Apply fixes for items in rubocop_todo [CW-1806] (#8864) 2024-02-07 13:36:04 +04:00
category.rb feat: Add draft status for help center locales (#13768) 2026-03-17 12:45:54 +04:00
contact_inbox.rb fix: Change contact_inboxes.source_id to text column (#12882) 2025-11-17 16:09:36 +05:30
contact.rb fix: prevent deserialization error on deletion (#13264) 2026-01-14 18:00:12 +05:30
conversation_participant.rb chore: Add controllers for conversation participants (#6462) 2023-02-15 16:33:31 -08:00
conversation.rb fix: optimize message query with account_id filter (#13759) 2026-03-10 16:46:20 -07:00
csat_survey_response.rb fix: Annotaterb model annotation incomplete migration (#13132) 2026-03-25 17:51:06 -07:00
custom_attribute_definition.rb fix: standardize contact company field on company_name (#14099) 2026-04-27 18:43:26 +05:30
custom_filter.rb chore: Increase custom filter limit from 50 to 1000 per user (#12603) 2025-10-06 10:41:26 -07:00
dashboard_app.rb fix: validate url for Dashboard Apps [CW-2979] (#8736) 2024-01-18 17:48:30 +05:30
data_import.rb chore: Add delay before running dataimport job (#8039) 2023-10-03 22:18:57 -07:00
email_template.rb chore: upgrade ruby version to 3.4.4 (#11524) 2025-05-21 19:40:07 +05:30
folder.rb feat: Portal endpoint (#4633) 2022-05-16 13:59:59 +05:30
inbox_assignment_policy.rb feat: Add assignment policies controllers with jbuilder views (#12199) 2025-08-18 19:15:21 -07:00
inbox_member.rb feat: auditlog for team and inbox member updates (#7516) 2023-08-15 19:55:19 -07:00
inbox.rb fix: sanitize parentheses from email From header to prevent SMTP 553 errors (#14075) 2026-04-21 11:30:20 +05:30
installation_config.rb fix: Annotaterb model annotation incomplete migration (#13132) 2026-03-25 17:51:06 -07:00
integrations.rb Feature: Slack integration (#783) 2020-06-12 23:12:47 +05:30
jsonb_attributes_length_validator.rb Fix: added validation for custom and additional attribute (#4260) 2022-03-24 15:38:28 +05:30
kbase.rb Feature: Knowledge Base APIs (#1002) 2020-09-26 02:32:34 +05:30
label.rb feat: multiple UX improvements to labels (#7358) 2023-06-25 18:49:49 +05:30
macro.rb fix: add explicit remove assignment actions to macros and automations (#12172) 2026-04-16 15:57:41 +05:30
mention.rb fix: Notification page breakages (#5236) 2022-08-10 13:46:46 +02:00
message.rb fix: strip markdown hard-break backslashes from webhook payloads (#13950) 2026-04-16 13:19:35 +05:30
note.rb feat(ee): Add Captain features (#10665) 2025-01-14 16:15:47 -08:00
notification_setting.rb fix: Specify external db with non-standard port (#2711) 2021-07-28 19:36:51 +05:30
notification_subscription.rb fix: Change the column identifier from string to text to avoid overflow (#9073) 2024-03-07 11:13:01 +05:30
notification.rb fix: pass serialized data in notification.deleted event to avoid Deserialisation (#13061) 2026-01-12 13:15:40 +05:30
platform_app_permissible.rb fix: SuperAdmin Improvements (#3733) 2022-01-11 19:00:00 -08:00
platform_app.rb Chore: Replaced dependent destroy with dependent destroy_async in all models (#3249) 2021-11-18 10:32:29 +05:30
portal.rb feat: Add draft status for help center locales (#13768) 2026-03-17 12:45:54 +04:00
related_category.rb feat: CRUD operation for associated articles to current article (#4912) 2022-07-04 20:29:44 +05:30
reporting_event.rb fix: Annotaterb model annotation incomplete migration (#13132) 2026-03-25 17:51:06 -07:00
reporting_events_rollup.rb fix: Annotaterb model annotation incomplete migration (#13132) 2026-03-25 17:51:06 -07:00
super_admin.rb feat: Add company model and API with tests (#12548) 2025-10-08 07:53:43 -07:00
team_member.rb feat: auditlog for team and inbox member updates (#7516) 2023-08-15 19:55:19 -07:00
team.rb feat: invalidate cache after inbox members or team members update (#10869) 2025-02-20 21:28:38 -08:00
user.rb fix: Add validation to the name attribute in user (#10805) 2026-04-27 15:47:11 +05:30
webhook.rb feat: sign webhooks for API channel and agentbots (#13892) 2026-04-06 15:28:25 +05:30
working_hour.rb chore: Replace deprecated functions (#5611) 2022-10-12 14:55:59 -07:00