12 Commits

Author	SHA1	Message	Date
Vinay Keerthi	59cbf57e20	feat: Advanced Search Backend (#12917) ... ## Description Implements comprehensive search functionality with advanced filtering capabilities for Chatwoot (Linear: CW-5956). This PR adds: 1. **Time-based filtering** for contacts and conversations (SQL-based search) 2. **Advanced message search** with multiple filters (OpenSearch/Elasticsearch-based) - **`from` filter**: Filter messages by sender (format: `contact:42` or `agent:5`) - **`inbox_id` filter**: Filter messages by specific inbox - **Time range filters**: Filter messages using `since` and `until` parameters (Unix timestamps in seconds) - **90-day limit enforcement**: Automatically limits searches to the last 90 days to prevent performance issues The implementation extends the existing `Enterprise::SearchService` module for advanced features and adds time filtering to the base `SearchService` for SQL-based searches. ## API Documentation ### Base URL All search endpoints follow this pattern: ``` GET /api/v1/accounts/{account_id}/search/{resource} ``` ### Authentication All requests require authentication headers: ``` api_access_token: YOUR_ACCESS_TOKEN ``` --- ## 1. Search All Resources **Endpoint:** `GET /api/v1/accounts/{account_id}/search` Returns results from all searchable resources (contacts, conversations, messages, articles). ### Parameters | Parameter | Type | Description | Required | |-----------|------|-------------|----------| | `q` | string | Search query | Yes | | `page` | integer | Page number (15 items per page) | No | | `since` | integer | Unix timestamp (contacts/conversations only) | No | | `until` | integer | Unix timestamp (contacts/conversations only) | No | ### Example Request ```bash curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search?q=customer" \ -H "api_access_token: YOUR_ACCESS_TOKEN" ``` ### Example Response ```json { "payload": { "contacts": [...], "conversations": [...], "messages": [...], "articles": [...] } } ``` --- ## 2. Search Contacts **Endpoint:** `GET /api/v1/accounts/{account_id}/search/contacts` Search contacts by name, email, phone number, or identifier with optional time filtering. ### Parameters | Parameter | Type | Description | Required | |-----------|------|-------------|----------| | `q` | string | Search query | Yes | | `page` | integer | Page number (15 items per page) | No | | `since` | integer | Unix timestamp - filter by last_activity_at | No | | `until` | integer | Unix timestamp - filter by last_activity_at | No | ### Example Requests **Basic search:** ```bash curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/contacts?q=john" \ -H "api_access_token: YOUR_ACCESS_TOKEN" ``` **Search contacts active in the last 7 days:** ```bash SINCE=$(date -v-7d +%s) curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/contacts?q=john&since=${SINCE}" \ -H "api_access_token: YOUR_ACCESS_TOKEN" ``` **Search contacts active between 30 and 7 days ago:** ```bash SINCE=$(date -v-30d +%s) UNTIL=$(date -v-7d +%s) curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/contacts?q=john&since=${SINCE}&until=${UNTIL}" \ -H "api_access_token: YOUR_ACCESS_TOKEN" ``` ### Example Response ```json { "payload": { "contacts": [ { "id": 42, "email": "john@example.com", "name": "John Doe", "phone_number": "+1234567890", "identifier": "user_123", "additional_attributes": {}, "created_at": 1701234567 } ] } } ``` --- ## 3. Search Conversations **Endpoint:** `GET /api/v1/accounts/{account_id}/search/conversations` Search conversations by display ID, contact name, email, phone number, or identifier with optional time filtering. ### Parameters | Parameter | Type | Description | Required | |-----------|------|-------------|----------| | `q` | string | Search query | Yes | | `page` | integer | Page number (15 items per page) | No | | `since` | integer | Unix timestamp - filter by last_activity_at | No | | `until` | integer | Unix timestamp - filter by last_activity_at | No | ### Example Requests **Basic search:** ```bash curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/conversations?q=billing" \ -H "api_access_token: YOUR_ACCESS_TOKEN" ``` **Search conversations active in the last 24 hours:** ```bash SINCE=$(date -v-1d +%s) curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/conversations?q=billing&since=${SINCE}" \ -H "api_access_token: YOUR_ACCESS_TOKEN" ``` **Search conversations from last month:** ```bash SINCE=$(date -v-30d +%s) UNTIL=$(date +%s) curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/conversations?q=billing&since=${SINCE}&until=${UNTIL}" \ -H "api_access_token: YOUR_ACCESS_TOKEN" ``` ### Example Response ```json { "payload": { "conversations": [ { "id": 123, "display_id": 45, "inbox_id": 1, "status": "open", "messages": [...], "meta": {...} } ] } } ``` --- ## 4. Search Messages (Advanced) **Endpoint:** `GET /api/v1/accounts/{account_id}/search/messages` Advanced message search with multiple filters powered by OpenSearch/Elasticsearch. ### Prerequisites - OpenSearch/Elasticsearch must be running (`OPENSEARCH_URL` env var configured) - Account must have `advanced_search` feature flag enabled - Messages must be indexed in OpenSearch ### Parameters | Parameter | Type | Description | Required | |-----------|------|-------------|----------| | `q` | string | Search query | Yes | | `page` | integer | Page number (15 items per page) | No | | `from` | string | Filter by sender: `contact:{id}` or `agent:{id}` | No | | `inbox_id` | integer | Filter by specific inbox ID | No | | `since` | integer | Unix timestamp - searches from this time (max 90 days ago) | No | | `until` | integer | Unix timestamp - searches until this time | No | ### Important Notes - **90-Day Limit**: If `since` is not provided, searches default to the last 90 days - If `since` exceeds 90 days, returns `422` error: "Search is limited to the last 90 days" - All time filters use message `created_at` timestamp ### Example Requests **Basic message search:** ```bash curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund" \ -H "api_access_token: YOUR_ACCESS_TOKEN" ``` **Search messages from a specific contact:** ```bash curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&from=contact:42" \ -H "api_access_token: YOUR_ACCESS_TOKEN" ``` **Search messages from a specific agent:** ```bash curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&from=agent:5" \ -H "api_access_token: YOUR_ACCESS_TOKEN" ``` **Search messages in a specific inbox:** ```bash curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&inbox_id=3" \ -H "api_access_token: YOUR_ACCESS_TOKEN" ``` **Search messages from the last 7 days:** ```bash SINCE=$(date -v-7d +%s) curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&since=${SINCE}" \ -H "api_access_token: YOUR_ACCESS_TOKEN" ``` **Search messages between specific dates:** ```bash SINCE=$(date -v-30d +%s) UNTIL=$(date -v-7d +%s) curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&since=${SINCE}&until=${UNTIL}" \ -H "api_access_token: YOUR_ACCESS_TOKEN" ``` **Combine all filters:** ```bash SINCE=$(date -v-14d +%s) curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&from=contact:42&inbox_id=3&since=${SINCE}" \ -H "api_access_token: YOUR_ACCESS_TOKEN" ``` **Attempt to search beyond 90 days (returns error):** ```bash SINCE=$(date -v-120d +%s) curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/messages?q=refund&since=${SINCE}" \ -H "api_access_token: YOUR_ACCESS_TOKEN" ``` ### Example Response (Success) ```json { "payload": { "messages": [ { "id": 789, "content": "I need a refund for my purchase", "message_type": "incoming", "created_at": 1701234567, "conversation_id": 123, "inbox_id": 3, "sender": { "id": 42, "type": "contact" } } ] } } ``` ### Example Response (90-day limit exceeded) ```json { "error": "Search is limited to the last 90 days" } ``` **Status Code:** `422 Unprocessable Entity` --- ## 5. Search Articles **Endpoint:** `GET /api/v1/accounts/{account_id}/search/articles` Search help center articles by title or content. ### Parameters | Parameter | Type | Description | Required | |-----------|------|-------------|----------| | `q` | string | Search query | Yes | | `page` | integer | Page number (15 items per page) | No | ### Example Request ```bash curl -X GET "https://app.chatwoot.com/api/v1/accounts/1/search/articles?q=installation" \ -H "api_access_token: YOUR_ACCESS_TOKEN" ``` ### Example Response ```json { "payload": { "articles": [ { "id": 456, "title": "Installation Guide", "slug": "installation-guide", "portal_slug": "help", "account_id": 1, "category_name": "Getting Started", "status": "published", "updated_at": 1701234567 } ] } } ``` --- ## Technical Implementation ### SQL-Based Search (Contacts, Conversations, Articles) - Uses PostgreSQL `ILIKE` queries by default - Optional GIN index support via `search_with_gin` feature flag for better performance - Time filtering uses `last_activity_at` for contacts/conversations - Returns paginated results (15 per page) ### Advanced Search (Messages) - Powered by OpenSearch/Elasticsearch via Searchkick gem - Requires `OPENSEARCH_URL` environment variable - Requires `advanced_search` account feature flag - Enforces 90-day lookback limit via `Limits::MESSAGE_SEARCH_TIME_RANGE_LIMIT_DAYS` - Validates inbox access permissions before filtering - Returns paginated results (15 per page) --- ## Type of change - [x] New feature (non-breaking change which adds functionality) - [x] Enhancement (improves existing functionality) --- ## How Has This Been Tested? ### Unit Tests - **Contact Search Tests**: 3 new test cases for time filtering (`since`, `until`, combined) - **Conversation Search Tests**: 3 new test cases for time filtering - **Message Search Tests**: 10+ test cases covering: - Individual filters (`from`, `inbox_id`, time range) - Combined filters - Permission validation for inbox access - Feature flag checks - 90-day limit enforcement - Error handling for exceeded time limits ### Test Commands ```bash # Run all search controller tests bundle exec rspec spec/controllers/api/v1/accounts/search_controller_spec.rb # Run search service tests (includes enterprise specs) bundle exec rspec spec/services/search_service_spec.rb ``` ### Manual Testing Setup A rake task is provided to create 50,000 test messages across multiple inboxes: ```bash # 1. Create test data bundle exec rake search:setup_test_data # 2. Start OpenSearch mise elasticsearch-start # 3. Reindex messages rails runner "Message.search_index.import Message.all" # 4. Enable feature flag rails runner "Account.first.enable_features('advanced_search')" # 5. Test via API or Rails console ``` --- ## Checklist - [x] My code follows the style guidelines of this project - [x] I have performed a self-review of my code - [x] I have commented on my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation (this PR description) - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] Any dependent changes have been merged and published in downstream modules --- ## Additional Notes ### Requirements - **OpenSearch/Elasticsearch**: Required for advanced message search - Set `OPENSEARCH_URL` environment variable - Example: `export OPENSEARCH_URL=http://localhost:9200` - **Feature Flags**: - `advanced_search`: Account-level flag for message advanced search - `search_with_gin` (optional): Account-level flag for GIN-based SQL search ### Performance Considerations - 90-day limit prevents expensive long-range queries on large datasets - GIN indexes recommended for high-volume search on SQL-based resources - OpenSearch/Elasticsearch provides faster full-text search for messages ### Breaking Changes - None. All new parameters are optional and backward compatible. ### Frontend Integration - Frontend PR tracking advanced search UI will consume these endpoints - Time range pickers should convert JavaScript `Date` to Unix timestamps (seconds) - Date conversion: `Math.floor(date.getTime() / 1000)` ### Error Handling - Invalid `from` parameter format is silently ignored (filter not applied) - Time range exceeding 90 days returns `422` with error message - Missing `q` parameter returns `422` (existing behavior) - Unauthorized inbox access is filtered out (no error, just excluded from results) --------- Co-authored-by: iamsivin <iamsivin@gmail.com> Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com> Co-authored-by: Pranav <pranav@chatwoot.com> Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>	2026-01-07 15:30:49 +05:30
Sivin Varghese	dedb0426fb	chore: Improve pagination with compact number formatting and pluralization (#12962)	2025-11-27 10:32:34 +05:30
Shivam Mishra	7b20ecd27b	chore: Revert pagination formatting and pluralization (#12954)	2025-11-25 21:36:13 +05:30
Sivin Varghese	26778156dc	chore: Improve pagination with compact number formatting and pluralization (#12921) ... # Pull Request Template ## Description This PR enhances the pagination component with standardized number formatting and improved i18n handling. **Includes:** * Added `formatCompactNumber` and `formatFullNumber` helpers using `Intl.NumberFormat`. * `< 1,000`: show exact value (e.g., `999`) * `1,000–999,999`: show compact format (`1k`, `1k+`) * `1,000,000+`: show in millions with one decimal (e.g., `1.2M`) * Updated `PaginationFooter` to use the new formatters for all displayed numbers. * Added proper pluralization to pagination i18n strings. Fixes https://linear.app/chatwoot/issue/CW-5999/better-display-of-numbers ## Type of change - [x] Bug fix (non-breaking change which fixes an issue) ## How Has This Been Tested? ### Screenshoots **Before** <img width="991" height="69" alt="image" src="https://github.com/user-attachments/assets/9fcf8baa-ae32-4a8a-85b0-24002fd863db" /> <img width="991" height="69" alt="image" src="https://github.com/user-attachments/assets/3d7138b7-133e-4ae6-b55f-67eff73ff1cc" /> <img width="991" height="69" alt="image" src="https://github.com/user-attachments/assets/1bbf7070-0681-492d-9308-a33874052d28" /> <img width="991" height="69" alt="image" src="https://github.com/user-attachments/assets/4e441672-26aa-4e66-965e-9edb807eaa72" /> <img width="991" height="69" alt="image" src="https://github.com/user-attachments/assets/11836702-1b74-4834-8932-31c20adc2db8" /> <img width="991" height="69" alt="image" src="https://github.com/user-attachments/assets/d37971bc-09af-4238-8601-ccc2ae69dbe7" /> **After** <img width="991" height="69" alt="image" src="https://github.com/user-attachments/assets/8eaf2a23-beea-486b-b555-37f8b36ab904" /> <img width="991" height="69" alt="image" src="https://github.com/user-attachments/assets/f44f508a-e39d-45cb-afd8-98deb26920f8" /> <img width="991" height="69" alt="image" src="https://github.com/user-attachments/assets/d3b90711-bd7e-44ee-8bb3-48e45b799420" /> <img width="991" height="69" alt="image" src="https://github.com/user-attachments/assets/30dca6cd-f2be-4dcb-8596-924326ebf8c0" /> <img width="991" height="69" alt="image" src="https://github.com/user-attachments/assets/58896699-1f05-46c9-88cb-908318e71476" /> <img width="991" height="69" alt="image" src="https://github.com/user-attachments/assets/ea0d91b0-077b-4d72-81a7-d38d17742da6" /> ## Checklist: - [x] My code follows the style guidelines of this project - [x] I have performed a self-review of my code - [x] I have commented on my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] Any dependent changes have been merged and published in downstream modules	2025-11-24 17:49:24 -08:00
Sivin Varghese	d5286c9535	fix: Installation name not showing (#12096)	2025-08-06 13:11:22 +05:30
Sivin Varghese	b429ce0ad5	fix: Prevent template variables from becoming links (#10725) ... # Pull Request Template ## Description **Issue** This PR fixes template variables in messages (e.g., {{customer.name}}) that were being incorrectly converted to clickable links by the `MessageFormatter's linkify` functionality. This caused formatting issues and broken links. **Solution** Added a `linkify` parameter to `MessageFormatter` to optionally disable link conversion ## Type of change - [x] Bug fix (non-breaking change which fixes an issue) ## How Has This Been Tested? **Screenshots** **Before** <img width="1012" alt="image" src="https://github.com/user-attachments/assets/70abb238-b4d9-439d-9e51-c7513cf482fb" /> **After** <img width="1012" alt="image" src="https://github.com/user-attachments/assets/387acb74-674e-4b26-85cc-2d7190d256b1" /> ## Checklist: - [x] My code follows the style guidelines of this project - [x] I have performed a self-review of my code - [x] I have commented on my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] Any dependent changes have been merged and published in downstream modules	2025-01-23 18:18:14 +05:30
Sivin Varghese	579efd933b	feat(v4): Update the campaigns page design (#10371) ... <img width="1439" alt="Screenshot 2024-10-30 at 8 58 12 PM" src="https://github.com/user-attachments/assets/26231270-5e73-40fb-9efa-c661585ebe7c"> Fixes https://linear.app/chatwoot/project/campaign-redesign-f82bede26ca7/overview --------- Co-authored-by: Pranav <pranavrajs@gmail.com> Co-authored-by: Shivam Mishra <scm.mymail@gmail.com>	2024-10-31 11:57:13 +05:30
Shivam Mishra	42f6621afb	feat: Vite + vue 3 💚 (#10047) ... Fixes https://github.com/chatwoot/chatwoot/issues/8436 Fixes https://github.com/chatwoot/chatwoot/issues/9767 Fixes https://github.com/chatwoot/chatwoot/issues/10156 Fixes https://github.com/chatwoot/chatwoot/issues/6031 Fixes https://github.com/chatwoot/chatwoot/issues/5696 Fixes https://github.com/chatwoot/chatwoot/issues/9250 Fixes https://github.com/chatwoot/chatwoot/issues/9762 --------- Co-authored-by: Pranav <pranavrajs@gmail.com> Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com>	2024-10-02 00:36:30 -07:00
Muhsin Keloth	185a122ec7	fix: Contact filter attribute groups (#10080)	2024-09-06 13:41:48 +05:30
Fayaz Ahmed	3b5f5b41ad	chore: Replace campaign mixin with composable [CW-3463] (#9987) ... # Pull Request Template ## Description Repalces campaignMixin and its usage with the new useCampaign mixin Fixes https://linear.app/chatwoot/issue/CW-3463/rewrite-campaignmixin-mixin-to-a-composable --------- Co-authored-by: Shivam Mishra <scm.mymail@gmail.com> Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>	2024-08-28 00:53:18 +05:30
Fayaz Ahmed	fe5670832a	chore: Replace filtersMixin with useFilter composable [CW-3466] (#10036) ... Co-authored-by: Shivam Mishra <scm.mymail@gmail.com> Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com>	2024-08-27 13:50:25 +05:30
Fayaz Ahmed	f82ec3b885	chore: Repalce message formatter mixin with useMessageFormatter [CW-3470] (#9986) ... # Pull Request Template ## Description Replaced the old messageFormatterMixin with a useMessageFormatter composable	2024-08-27 08:06:51 +05:30