mirror of
https://github.com/chatwoot/chatwoot.git
synced 2026-07-01 21:03:46 +08:00
ef54f07d5b
## Description Implements company backfill migration infrastructure for existing contacts. This is **Part 1 of 2** for the company model production rollout as described in [CW-5726](https://linear.app/chatwoot/issue/CW-5726/company-model-setting-it-up-on-production). Creates jobs and services to associate existing contacts with companies based on their email domains, filtering out free email providers (gmail, yahoo, etc.) and disposable addresses. **What's included:** - Business email detector service with ValidEmail2 (uses `disposable_domain?` to avoid DNS lookups) - Per-account batch job to process contacts for one account - Orchestrator job to iterate all accounts - Rake task: `bundle exec rake companies:backfill` ~~*NOTE*: I'm using a hard-coded approach to determine if something is a "business" email by filtering out emails that are usually personal. I've also added domains that are common to some of our customers' regions. This should be simpler. I looked into `Valid_Email2` and I couldn't find anything to dictate whether an email is a personal email or a business one. I don't think the approach used in the frontend is valid here.~~ UPDATE: Using `email_provider_info` gem instead. **Pending - Part 2 (separate PR):** Real-time company creation for new contacts ## Type of change - [x] New feature (non-breaking change which adds functionality) ## How Has This Been Tested? ```bash # Run all new tests bundle exec rspec spec/enterprise/services/companies/business_email_detector_service_spec.rb \\ spec/enterprise/jobs/migration/company_account_batch_job_spec.rb \\ spec/enterprise/jobs/migration/company_backfill_job_spec.rb # Run RuboCop bundle exec rubocop enterprise/app/services/companies/business_email_detector_service.rb \\ enterprise/app/jobs/migration/company_account_batch_job.rb \\ enterprise/app/jobs/migration/company_backfill_job.rb \\ lib/tasks/companies.rake ``` **Performance optimization:** - Uses `disposable_domain?` instead of `disposable?` to avoid DNS MX lookups (discovered via tcpdump analysis - `disposable?` was making network calls for every email, causing 100x slowdown) ## Checklist: - [x] My code follows the style guidelines of this project - [x] I have performed a self-review of my code - [x] I have commented on my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] Any dependent changes have been merged and published in downstream modules --------- Co-authored-by: Sojan Jose <sojan@pepalo.com> |
||
|---|---|---|
| .. | ||
| 20230426130150_init_schema.rb | ||
| 20230503101201_create_sla_policies.rb | ||
| 20230509101256_add_index_to_messages_created_at.rb | ||
| 20230510060828_add_index_to_messages_for_reportings.rb | ||
| 20230510113208_set_default_empty_string_for_contact_name.rb | ||
| 20230515051424_update_article_image_keys.rb | ||
| 20230523104139_add_partial_index_for_resolved_contacts.rb | ||
| 20230525085402_add_custom_sender_name_toggle.rb | ||
| 20230608040738_add_processed_message_content_to_messages.rb | ||
| 20230612103936_add_index_to_reporting_events.rb | ||
| 20230614044633_add_sender_name_to_in.rb | ||
| 20230620132319_add_sla_policy_to_conversations.rb | ||
| 20230620212340_add_waiting_since_to_conversations.rb | ||
| 20230706090122_sentiment_column_to_messages.rb | ||
| 20230714054138_add_api_key_sid_to_twilio_sms.rb | ||
| 20230727065605_add_partial_index_contact_account_id.rb | ||
| 20230801180936_update_default_status_in_hooks.rb | ||
| 20230905060223_add_regex_to_custom_attribute_definition.rb | ||
| 20231011041615_ensure_message_private_not_null.rb | ||
| 20231013072802_add_icon_to_categories.rb | ||
| 20231114111614_change_feature_flags_to_bigint_in_accounts.rb | ||
| 20231129091149_add_snoozed_until_to_notifications.rb | ||
| 20231201014644_remove_notifications_with_message_primary_actor.rb | ||
| 20231211010807_add_cached_labels_list.rb | ||
| 20231219000743_re_run_cache_label_job.rb | ||
| 20231219073832_add_last_activity_at_to_notifications.rb | ||
| 20231223033019_refactor_sla_policy_columns.rb | ||
| 20231223040257_create_applied_slas.rb | ||
| 20240124054340_add_contact_type_to_contacts.rb | ||
| 20240124084032_add_middle_name_and_last_name_to_contacts.rb | ||
| 20240129080827_add_location_and_country_code_to_contacts.rb | ||
| 20240131040316_remove_imap_inbox_syned_at_from_channel_email.rb | ||
| 20240207103014_change_applied_sla_sla_status_to_enum.rb | ||
| 20240213131252_add_blocked_to_contacts.rb | ||
| 20240215065844_add_meta_to_notifications.rb | ||
| 20240216055809_add_unique_index_to_applied_slas.rb | ||
| 20240306201954_change_identifier_type_in_notifications_subscriptions.rb | ||
| 20240319062553_create_sla_events.rb | ||
| 20240322071629_convert_cached_label_list_to_text.rb | ||
| 20240415210313_add_channel_web_widget_to_portals.rb | ||
| 20240515201632_index_improvements_conversations_contacts.rb | ||
| 20240516003531_add_index_for_message_type_account_date.rb | ||
| 20240726220747_add_custom_roles.rb | ||
| 20240923215335_add_locale_to_article.rb | ||
| 20241217041352_fix_old_audio_alert_data.rb | ||
| 20250104200055_create_captain_tables.rb | ||
| 20250104210328_remove_robin_tables.rb | ||
| 20250105001414_add_status_to_captain_documents.rb | ||
| 20250105005821_remove_not_null_from_captain_documents.rb | ||
| 20250107030743_add_config_to_captain_assistant.rb | ||
| 20250108031358_create_captain_inbox.rb | ||
| 20250108211541_remove_index_from_captain_assistants.rb | ||
| 20250116000103_add_status_to_captain_assistant_responses.rb | ||
| 20250116061033_convert_document_to_polymorphic_association.rb | ||
| 20250207040150_add_meta_to_attachment.rb | ||
| 20250228185548_add_internal_attributes_to_accounts.rb | ||
| 20250315202035_add_index_to_articles.rb | ||
| 20250326034635_add_instagram_channel.rb | ||
| 20250402233933_remove_portal_members.rb | ||
| 20250410061725_convert_csml_bots_to_webhook_bots.rb | ||
| 20250416182131_flip_chatwoot_v4_default_feature_flag_installation_config.rb | ||
| 20250421082927_add_settings_column_to_account.rb | ||
| 20250421085134_update_auto_resolve_to_mminutes.rb | ||
| 20250512231036_create_copilot_threads.rb | ||
| 20250512231037_create_copilot_messages.rb | ||
| 20250514045638_add_csat_config_to_inboxes.rb | ||
| 20250523024825_remove_uuid_from_copilot_threads.rb | ||
| 20250523024826_remove_user_id_from_copilot_messages.rb | ||
| 20250523031839_change_message_type_to_integer_in_copilot_messages.rb | ||
| 20250620120000_create_channel_voice.rb | ||
| 20250627195529_add_index_to_messages.rb | ||
| 20250709102213_add_template_params_to_campaigns.rb | ||
| 20250710145708_create_captain_scenarios.rb | ||
| 20250714104358_add_response_guidelines_and_guardrails_to_captain_assistants.rb | ||
| 20250722083820_add_ssl_settings_to_portals.rb | ||
| 20250722152516_add_index_on_contact_type_and_account_id_to_contacts.rb | ||
| 20250805082345_add_metadata_to_captain_documents.rb | ||
| 20250805160307_add_notifications_performance_index.rb | ||
| 20250806140000_create_assignment_policies.rb | ||
| 20250806140001_create_inbox_assignment_policies.rb | ||
| 20250806140002_create_agent_capacity_policies.rb | ||
| 20250806140003_create_inbox_capacity_limits.rb | ||
| 20250806140004_add_agent_capacity_policy_to_account_users.rb | ||
| 20250806140005_create_leaves.rb | ||
| 20250808123008_add_feature_citation_to_assistant_config.rb | ||
| 20250820130619_add_two_factor_to_users.rb | ||
| 20250822061042_add_content_templates_to_twilio_sms.rb | ||
| 20250825070005_create_account_saml_settings.rb | ||
| 20250826000000_drop_telegram_bots.rb | ||
| 20250916024703_add_allowed_domains_to_channel_widgets.rb | ||
| 20250917012759_add_verified_to_channel_email.rb | ||
| 20250929105219_create_companies.rb | ||
| 20250929132305_add_company_to_contacts.rb | ||
| 20251003091242_create_captain_custom_tools.rb | ||
| 20251021082242_add_unique_index_to_companies_domain.rb | ||
| 20251022152158_add_index_to_conversations_identifier.rb | ||