mirror of
https://github.com/chatwoot/chatwoot.git
synced 2026-06-04 21:02:35 +08:00
4371793741
## Description Inbound email attachments are stored with `file_type: 'file'` regardless of their actual MIME type. As a result, image screenshots shared by customers via email are not exposed to Captain V2's multimodal pipeline — `Captain::OpenAiMessageBuilderService#attachment_parts` selects images via `attachments.where(file_type: :image)` and emits a placeholder `"User has shared an attachment"` text part instead of an `image_url` part. The model never gets the image, so Captain keeps asking the customer to retype information that is already visible in the screenshot. This PR makes the email mailbox derive `file_type` from the blob's `content_type` using the existing shared `FileTypeHelper`, matching how every other inbound channel (`twilio`, `sms`, `telegram`, `line`, `tiktok`, `twitter`, `messenger`) and `MessageBuilder` already classify attachments. Fixes #14448 ## Type of change - [x] Bug fix (non-breaking change which fixes an issue) ## How Has This Been Tested? Reproduced and verified on a self-hosted production instance: 1. Real customer reply via email with a PNG screenshot of an in-app error. Before: ```ruby a = Message.find(<id>).attachments.first a.file_type # => "file" a.file.blob.content_type # => "image/png" Captain::OpenAiMessageBuilderService.new(message: a.message).generate_content # => [{type: 'text', text: '...'}, # {type: 'text', text: 'User has shared an attachment'}] ❌ no image_url ``` Captain reply: "Please copy and paste the full error text…" (model never saw the image). 2. After the patch + force-recreate, same conversation: ```ruby a.file_type # => "image" Captain::OpenAiMessageBuilderService.new(message: a.message).generate_content # => [{type: 'text', text: '...'}, # {type: 'image_url', image_url: {url: 'https://.../<blob>.png'}}] ✅ ``` Captain reply now correctly references the on-screen error text from the screenshot via the multimodal vision path — no more deflection. 3. Regression sanity-check on non-image attachments (PDF / Office docs): `file_type` falls through to `:file`, behavior unchanged. ## Notes for self-hosted operators Existing email image attachments in the DB will still have `file_type: 'file'`. A one-shot backfill is straightforward and safe (no data loss, only metadata): ```ruby Attachment.joins(message: :conversation) .where(messages: { content_type: 'incoming_email' }) .where(file_type: 'file') .find_each do |a| next unless a.file.attached? ct = a.file.blob.content_type.to_s next unless ct.start_with?('image/', 'audio/', 'video/') new_type = ct.start_with?('image/') ? :image : (ct.start_with?('video/') ? :video : :audio) a.update_columns(file_type: Attachment.file_types[new_type]) end ``` ## Checklist - [x] My code follows the style guidelines of this project - [x] I have performed a self-review of my code - [x] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective — happy to add a `mailbox_helper_spec` example for `process_regular_attachments` if maintainers prefer; existing specs in that file focus on inline-image handling. --------- Co-authored-by: Aakash Bakhle <48802744+aakashb95@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| imap | ||
| application_mailbox.rb | ||
| default_mailbox.rb | ||
| incoming_email_validity_helper.rb | ||
| mailbox_helper.rb | ||
| mailbox_inline_attachment_helper.rb | ||
| reply_mailbox.rb | ||