chatwoot/lib/captain/csat_utility_analysis_service.rb
Muhsin Keloth 6be95e79f8
Some checks failed
Frontend Lint & Test / test (push) Has been cancelled
Publish Chatwoot EE docker images / build (linux/amd64, ubuntu-latest) (push) Has been cancelled
Publish Chatwoot EE docker images / build (linux/arm64, ubuntu-22.04-arm) (push) Has been cancelled
Publish Chatwoot CE docker images / build (linux/amd64, ubuntu-latest) (push) Has been cancelled
Publish Chatwoot CE docker images / build (linux/arm64, ubuntu-22.04-arm) (push) Has been cancelled
Run Chatwoot CE spec / lint-backend (push) Has been cancelled
Run Chatwoot CE spec / lint-frontend (push) Has been cancelled
Run Chatwoot CE spec / frontend-tests (push) Has been cancelled
Run Chatwoot CE spec / backend-tests (0, 16) (push) Has been cancelled
Run Chatwoot CE spec / backend-tests (1, 16) (push) Has been cancelled
Run Chatwoot CE spec / backend-tests (10, 16) (push) Has been cancelled
Run Chatwoot CE spec / backend-tests (11, 16) (push) Has been cancelled
Run Chatwoot CE spec / backend-tests (12, 16) (push) Has been cancelled
Run Chatwoot CE spec / backend-tests (13, 16) (push) Has been cancelled
Run Chatwoot CE spec / backend-tests (14, 16) (push) Has been cancelled
Run Chatwoot CE spec / backend-tests (15, 16) (push) Has been cancelled
Run Chatwoot CE spec / backend-tests (2, 16) (push) Has been cancelled
Run Chatwoot CE spec / backend-tests (3, 16) (push) Has been cancelled
Run Chatwoot CE spec / backend-tests (4, 16) (push) Has been cancelled
Run Chatwoot CE spec / backend-tests (5, 16) (push) Has been cancelled
Run Chatwoot CE spec / backend-tests (6, 16) (push) Has been cancelled
Run Chatwoot CE spec / backend-tests (7, 16) (push) Has been cancelled
Run Chatwoot CE spec / backend-tests (8, 16) (push) Has been cancelled
Run Chatwoot CE spec / backend-tests (9, 16) (push) Has been cancelled
Publish Chatwoot EE docker images / merge (push) Has been cancelled
Publish Chatwoot CE docker images / merge (push) Has been cancelled
feat(csat): Add WhatsApp utility template analyzer with rewrite guidance (#13575)
CSAT templates for WhatsApp are submitted as Utility, but Meta may
reclassify them as Marketing based on content, which can significantly
increase messaging costs.
This PR introduces a Captain-powered CSAT template analyzer for
WhatsApp/Twilio WhatsApp that predicts utility fit, explains likely
risks, and suggests safer rewrites before submission. The flow is manual
(button-triggered), Captain-gated, and applies rewrites only on explicit
user action. It also updates UX copy to clearly set expectations: the
system submits as Utility, Meta makes the final categorization decision.

Fixes
https://linear.app/chatwoot/issue/CW-6424/ai-powered-whatsapp-template-classifier-for-csat-submissions


https://github.com/user-attachments/assets/8fd1d6db-2f91-447c-9771-3de271b16fd9
2026-02-24 15:11:04 +04:00

67 lines
1.7 KiB
Ruby

class Captain::CsatUtilityAnalysisService < Captain::BaseTaskService
pattr_initialize [:account!, :message!, { button_text: nil, language: 'en', baseline: {} }]
def perform
api_response = make_api_call(
model: GPT_MODEL,
messages: [
{ role: 'system', content: system_prompt },
{ role: 'user', content: message }
]
)
return api_response if api_response[:error]
build_result(api_response[:message])
end
private
def build_result(response_message)
parsed = parse_json_response(response_message)
return { error: 'Invalid LLM response format' } if parsed.blank?
core_result(parsed).merge(message: response_message)
end
def core_result(parsed)
{
classification: normalize_classification(parsed['classification']),
optimized_message: parsed['optimized_message'].presence || baseline[:optimized_message]
}
end
def system_prompt
template = prompt_from_file('csat_utility_analysis')
Liquid::Template.parse(template).render(prompt_variables)
end
def prompt_variables
{
'message' => message.to_s,
'button_text' => button_text.to_s,
'language' => language.to_s,
'baseline_classification' => baseline[:classification].to_s
}
end
def parse_json_response(content)
raw = content.to_s.strip
json = raw.match(/```json\s*(.*?)\s*```/m)&.captures&.first || raw
JSON.parse(json)
rescue JSON::ParserError
nil
end
def normalize_classification(value)
normalized = value.to_s.upcase
return normalized if %w[LIKELY_UTILITY LIKELY_MARKETING UNCLEAR].include?(normalized)
baseline[:classification].presence || 'UNCLEAR'
end
def event_name
'csat_utility_analysis'
end
end