CodeCow/chatwoot
1
0
Fork 0
mirror of https://github.com/chatwoot/chatwoot.git synced 2026-06-04 21:02:35 +08:00
Code Issues Actions 4 Packages Projects Releases Wiki Activity
debug/wider-generation
chatwoot/Gemfile
Shivam Mishra 3d20a7b049
feat: generate Help Center for Onboarding (#14370) 
## Manually triggering help center generation

Open a Rails console (`bundle exec rails console`):

```ruby
account = Account.find(<ACCOUNT_ID>)
user    = account.users.first

# Optional: refresh brand info from the customer's website
domain = 'example.com'
result = WebsiteBrandingService.new("noreply@#{domain}").perform
account.update!(
  name: result[:title].presence || account.name,
  custom_attributes: account.custom_attributes.merge('website' => domain, 'brand_info' => result)
)

# Optional: wipe existing portals so a fresh one is created
account.portals.destroy_all

Onboarding::HelpCenterCreationService.new(account, user).perform
```

Sidekiq must be running — articles are written by
`Onboarding::HelpCenterArticleGenerationJob`. Avoid running on
production; generation calls the LLM provider.


### Generation flow (Happy Path) 

```mermaid
sequenceDiagram
    autonumber

    participant Kickoff as HelpCenterCreationService
    participant DB as DB
    participant GenJob as HelpCenterArticleGenerationJob
    participant Curator as HelpCenterCurator
    participant Firecrawl as Firecrawl
    participant CuratorLLM as Curation LLM
    participant Redis as Redis Progress
    participant WriterJob as HelpCenterArticleWriterJob
    participant Builder as HelpCenterArticleBuilder
    participant WriterLLM as Writer LLM
    participant Cable as ActionCable

    Kickoff->>DB: Create portal for account<br/>homepage_link=https://chatwoot.com
    Kickoff->>DB: Attach brand logo if available
    Kickoff->>GenJob: Enqueue generation job<br/>account_id, portal_id, user_id, generation_id

    GenJob->>Curator: Curate help center plan
    Curator->>Firecrawl: map https://chatwoot.com<br/>search: docs help support faq
    Firecrawl-->>Curator: Return discovered links
    Curator->>CuratorLLM: Select categories + article plans<br/>from discovered links only
    CuratorLLM-->>Curator: Return categories, articles, allowed_urls

    GenJob->>DB: Create portal categories
    GenJob->>GenJob: Stamp articles with category_id
    GenJob->>GenJob: Filter article URLs against allowed_urls
    GenJob->>GenJob: Drop articles with no category<br/>or no approved source URLs

    GenJob->>Redis: Start progress<br/>status=generating, total=N, finished=0

    loop For each approved article
      GenJob->>WriterJob: Enqueue writer job<br/>title, category_id, approved URLs
    end

    par Writer jobs run independently
      WriterJob->>Builder: Build article from approved URLs
      Builder->>Firecrawl: batch_scrape approved URLs
      Firecrawl-->>Builder: Return Markdown source pages
      Builder->>WriterLLM: Rewrite sources into one article
      WriterLLM-->>Builder: Return title, description, Markdown content
      Builder->>DB: Create draft portal article<br/>meta.source_urls
      WriterJob->>Redis: Increment finished count
      WriterJob->>Cable: Broadcast help_center.article_generated
    end

    WriterJob->>Redis: If finished >= total<br/>mark status=completed
    WriterJob->>Cable: Broadcast help_center.generation_completed
```

### Redis State Management

```mermaid
 stateDiagram-v2
    [*] --> active_pointer_set
    active_pointer_set --> generating: generation job creates valid plan
    active_pointer_set --> skipped: curation skipped/failed

    generating --> generating: each writer job increments finished
    generating --> completed: finished == total
    generating --> ignored_completion: generation_id superseded

    skipped --> [*]
    completed --> [*]
    ignored_completion --> [*]
```
2026-05-21 16:25:01 +05:30

281 lines
7.1 KiB
Ruby
Raw Permalink Blame History

source 'https://rubygems.org'
ruby '3.4.4'
##-- base gems for rails --##
gem 'rack-cors', '2.0.0', require: 'rack/cors'
gem 'rails', '~> 7.1'
# Reduces boot times through caching; required in config/boot.rb
gem 'bootsnap', require: false
##-- rails application helper gems --##
gem 'acts-as-taggable-on'
gem 'attr_extras'
gem 'browser'
gem 'hashie'
gem 'jbuilder'
gem 'kaminari'
gem 'responders', '>= 3.1.1'
gem 'rest-client'
gem 'telephone_number'
gem 'time_diff'
gem 'tzinfo-data'
gem 'valid_email2'
gem 'email-provider-info'
gem 'gemoji'
# compress javascript config.assets.js_compressor
gem 'uglifier'
##-- used for single column multiple binary flags in notification settings/feature flagging --##
gem 'flag_shih_tzu'
# Random name generator for user names
gem 'haikunator'
# Template parsing safely
gem 'liquid'
# Parse Markdown to HTML
gem 'commonmarker'
# Validate Data against JSON Schema
gem 'json_schemer'
# used in swagger build
gem 'json_refs'
# Rack middleware for blocking & throttling abusive requests
gem 'rack-attack', '>= 6.7.0'
# a utility tool for streaming, flexible and safe downloading of remote files
gem 'down'
# SSRF-safe URL fetching
gem 'ssrf_filter', '~> 1.5'
# authentication type to fetch and send mail over oauth2.0
gem 'gmail_xoauth'
# Lock net-smtp to 0.3.4 to avoid issues with gmail_xoauth2
gem 'net-smtp', '~> 0.3.4'
# Prevent CSV injection
gem 'csv-safe'
##-- for active storage --##
gem 'aws-sdk-s3', require: false
# original gem isn't maintained actively
# we wanted updated version of faraday which is a dependency for slack-ruby-client
gem 'azure-storage-blob', git: 'https://github.com/chatwoot/azure-storage-ruby', branch: 'chatwoot', require: false
gem 'google-cloud-storage', '>= 1.48.0', require: false
gem 'image_processing'
##-- for actionmailbox --##
gem 'aws-actionmailbox-ses', '~> 0'
##-- gems for database --#
gem 'groupdate'
gem 'pg'
gem 'redis'
gem 'redis-namespace'
# super fast record imports in bulk
gem 'activerecord-import'
gem 'searchkick'
gem 'opensearch-ruby'
gem 'faraday_middleware-aws-sigv4'
##--- gems for server & infra configuration ---##
gem 'dotenv-rails', '>= 3.0.0'
gem 'foreman'
gem 'puma'
gem 'vite_rails'
# metrics on heroku
gem 'barnes'
##--- gems for authentication & authorization ---##
gem 'devise', '>= 4.9.4'
gem 'devise-secure_password', git: 'https://github.com/chatwoot/devise-secure_password', branch: 'chatwoot'
gem 'devise_token_auth', '>= 1.2.3'
gem 'rails-i18n', '~> 7.0'
# two-factor authentication
gem 'devise-two-factor', '>= 5.0.0'
# authorization
gem 'jwt'
gem 'pundit'
# super admin
gem 'administrate', '>= 0.20.1'
gem 'administrate-field-active_storage', '>= 1.0.3'
gem 'administrate-field-belongs_to_search', '>= 0.9.0'
##--- gems for pubsub service ---##
# https://karolgalanciak.com/blog/2019/11/30/from-activerecord-callbacks-to-publish-slash-subscribe-pattern-and-event-driven-design/
gem 'wisper', '2.0.0'
##--- gems for channels ---##
gem 'facebook-messenger'
gem 'line-bot-api'
gem 'twilio-ruby'
# twitty will handle subscription of twitter account events
# gem 'twitty', git: 'https://github.com/chatwoot/twitty'
gem 'twitty', '~> 0.1.5'
# facebook client
gem 'koala'
# slack client
gem 'slack-ruby-client', '~> 2.7.0'
# for dialogflow integrations
gem 'google-cloud-dialogflow-v2', '>= 0.24.0'
gem 'grpc'
# Translate integrations
# 'google-cloud-translate' gem depends on faraday 2.0 version
# this dependency breaks the slack-ruby-client gem
gem 'google-cloud-translate-v3', '>= 0.7.0'
##-- apm and error monitoring ---#
# loaded only when environment variables are set.
# ref application.rb
gem 'datadog', '~> 2.0', require: false
gem 'elastic-apm', require: false
gem 'newrelic_rpm', require: false
gem 'newrelic-sidekiq-metrics', '>= 1.6.2', require: false
gem 'scout_apm', require: false
gem 'sentry-rails', '>= 5.19.0', require: false
gem 'sentry-ruby', require: false
gem 'sentry-sidekiq', '>= 5.19.0', require: false
##-- background job processing --##
gem 'sidekiq', '~> 7.3', '>= 7.3.1'
# We want cron jobs
gem 'sidekiq-cron', '>= 2.4.0'
# for sidekiq healthcheck
gem 'sidekiq_alive'
##-- Push notification service --##
gem 'fcm'
gem 'web-push', '>= 3.0.1'
##-- geocoding / parse location from ip --##
# http://www.rubygeocoder.com/
gem 'geocoder'
# to parse maxmind db
gem 'maxminddb'
# to create db triggers
gem 'hairtrigger'
gem 'procore-sift'
# parse email
gem 'email_reply_trimmer'
gem 'html2text'
# to calculate working hours
gem 'working_hours'
# full text search for articles
gem 'pg_search'
# Subscriptions, Billing
gem 'stripe', '~> 18.0'
## - helper gems --##
## to populate db with sample data
gem 'faker'
# Include logrange conditionally in intializer using env variable
gem 'lograge', '~> 0.14.0', require: false
# worked with microsoft refresh token
gem 'omniauth-oauth2'
gem 'audited', '~> 5.4', '>= 5.4.1'
# need for google auth
gem 'omniauth', '>= 2.1.2'
gem 'omniauth-saml'
gem 'omniauth-google-oauth2', '>= 1.1.3'
gem 'omniauth-rails_csrf_protection', '~> 1.0', '>= 1.0.2'
## Gems for reponse bot
# adds cosine similarity to postgres using vector extension
gem 'neighbor'
gem 'pgvector'
# Convert Website HTML to Markdown
gem 'reverse_markdown'
gem 'iso-639'
gem 'ruby-openai'
gem 'ai-agents', '>= 0.10.0'
# TODO: Move this gem as a dependency of ai-agents
gem 'ruby_llm', '>= 1.14.1'
gem 'ruby_llm-schema'
gem 'cld3', '~> 3.7'
# OpenTelemetry for LLM observability
gem 'opentelemetry-sdk'
gem 'opentelemetry-exporter-otlp'
gem 'shopify_api'
gem 'firecrawl-sdk', '~> 1.0', require: 'firecrawl'
### Gems required only in specific deployment environments ###
##############################################################
group :production do
# we dont want request timing out in development while using byebug
gem 'rack-timeout'
# for heroku autoscaling
gem 'judoscale-rails', require: false
gem 'judoscale-sidekiq', require: false
end
group :development do
gem 'annotaterb'
gem 'bullet'
gem 'letter_opener'
gem 'scss_lint', require: false
gem 'web-console', '>= 4.2.1'
# When we want to squash migrations
gem 'squasher'
# profiling
gem 'rack-mini-profiler', '>= 3.2.0', require: false
gem 'stackprof'
# Should install the associated chrome extension to view query logs
gem 'meta_request', '>= 0.8.3'
gem 'tidewave'
end
group :test do
# fast cleaning of database
gem 'database_cleaner'
# mock http calls
gem 'webmock'
# test profiling
gem 'test-prof'
gem 'simplecov_json_formatter', require: false
end
group :development, :test do
gem 'active_record_query_trace'
##--- gems for debugging and error reporting ---##
# static analysis
gem 'brakeman'
gem 'bundle-audit', require: false
gem 'byebug', platform: :mri
gem 'climate_control'
gem 'debug', '~> 1.8'
gem 'factory_bot_rails', '>= 6.4.3'
gem 'listen'
gem 'mock_redis'
gem 'pry-rails'
gem 'rspec_junit_formatter'
gem 'rspec-rails', '>= 6.1.5'
gem 'rubocop', require: false
gem 'rubocop-performance', require: false
gem 'rubocop-rails', require: false
gem 'rubocop-rspec', require: false
gem 'rubocop-factory_bot', require: false
gem 'seed_dump'
gem 'shoulda-matchers'
gem 'simplecov', '>= 0.21', require: false
gem 'skooma'
gem 'spring'
gem 'spring-watcher-listen'
end
Reference in New Issue View Git Blame Copy Permalink