LogStruct LogoLogStructby DocSpring

Configuration

LogStruct works without any configuration. To change its defaults, create config/initializers/logstruct.rb. This page lists every setting.

A typical initializer looks like this:

# config/initializers/logstruct.rb
LogStruct.configure do |config|
  # Filter these keys too, on top of LogStruct's defaults and Rails' filter_parameters
  config.filters.filter_keys += [:date_of_birth, :account_number]

  # Keep the same email hashes when secret_key_base changes, and in every app that logs the
  # same people. Without it, the key is derived from secret_key_base.
  config.filters.hash_salt = Rails.application.credentials.logstruct_hash_key

  # Add the signed-in user to request lines (see append_info_to_payload below)
  config.integrations.lograge_custom_options = ->(event, _options) {
    {user_id: event.payload[:user_id]}
  }

  # Log SQL queries slower than 250 ms
  config.integrations.enable_sql_logging = true
  config.integrations.sql_slow_query_threshold = 250.0

  # Log the account and organization a mail was sent for
  config.integrations.actionmailer_id_mapping = {account: :account_id, organization: :organization_id}
end

Rails passes each request's instrumentation payload to Lograge, so a controller can add fields for lograge_custom_options to read:

app/controllers/application_controller.rb
class ApplicationController < ActionController::Base
  def append_info_to_payload(payload)
    super
    payload[:user_id] = current_user&.id
  end
end
Configure LogStruct in config/initializers (or earlier, in config/application.rb or config/environments/*.rb). LogStruct sets up its logger and integrations after the initializers run. A setting changed after the app has booted only affects what LogStruct reads as it writes each line, such as the filters and limits.

Enabling LogStruct

By default LogStruct is on:

  • In production, for the processes it recognizes: Puma, Unicorn, Thin, Falcon, rails server, a Sidekiq server and good_job start.
  • In test, for every process except the console, locally and on CI. Keeping test close to production catches logging problems (a value that can't be written, a broken integration) before they reach production.

It's off in development, in consoles, and in production Rake tasks, rails runner and job runners it doesn't recognize (such as Solid Queue's bin/jobs, Resque or Delayed Job), which keep Rails' usual logger. When LogStruct runs has the details.

Overriding the Defaults

These settings decide whether LogStruct is on, from highest precedence to lowest:

  1. The LOGSTRUCT_ENABLED environment variable
    true, t, yes, y or 1 turns LogStruct on in any process and environment, consoles included. Any other value turns it off.
    # See the JSON lines in development, or in a console or Rake task
LOGSTRUCT_ENABLED=true bin/rails server
LOGSTRUCT_ENABLED=true bin/rails console
LOGSTRUCT_ENABLED=true bin/rails db:migrate

# Turn LogStruct on for a job runner it doesn't recognize
LOGSTRUCT_ENABLED=true bin/jobs

# Turn it off in tests
LOGSTRUCT_ENABLED=false bin/rails test
  2. config.enabled in an initializer
    It has no environment check of its own: it applies to every process that loads the initializer, in every environment. Wrap it in a condition:
    LogStruct.configure do |config|
  # config.enabled applies to every process that loads this initializer, in every
  # environment, so limit it to the environment you mean
  config.enabled = true if Rails.env.development?
end
  3. config.enabled_environments
    The environments where LogStruct switches on for the processes it recognizes (default [:test, :production]).

Environment Configuration

LogStruct.configure do |config|
  # The environments where LogStruct switches on for the processes it recognizes (the default)
  config.enabled_environments = [:test, :production]

  # The environments where the LogProduction and ReportProduction error modes raise instead
  # of logging, and where LogStruct also writes Rails' log file (the default)
  config.local_environments = [:development, :test]
end

In the environments that aren't local (production, staging and any other), LogStruct logs to STDOUT only. In local environments it also writes Rails' log file (log/development.log, log/test.log). In test it writes only the log file, so test output stays clean. With RAILS_LOG_TO_STDOUT set, every environment logs to STDOUT only. See Logging to STDOUT.

Logger Settings

When LogStruct is on in development, it writes the same JSON as production, so you can check your lines locally. For colored, pretty-printed lines instead, set prefer_json_in_development to false. These are the logger settings and their defaults:

LogStruct.configure do |config|
  # In development (when LogStruct is on there), write the same JSON as production (the
  # default). Set it to false for colored, pretty-printed lines instead.
  config.prefer_json_in_development = true
  config.enable_color_output = true

  # The colors for those lines. Types you leave out keep their default color.
  config.color_map = {key: :yellow, string: :green, number: :blue, bool: :magenta, nil: :red, name: :cyan}

  # Leave out the lines from ActionView, ActionController::RoutingError and
  # ActiveRecord::SchemaMigration (off by default)
  config.filter_noisy_loggers = false
end

Integration Configuration

Each integration can be turned off. The integrations page describes the lines each one writes.

LogStruct.configure do |config|
  # Turn integrations on or off. These are the defaults: every integration is on, except SQL
  # logging. An integration does nothing unless its gem is loaded.
  config.integrations.enable_actionmailer = true
  config.integrations.enable_active_model_serializers = true
  config.integrations.enable_activejob = true
  config.integrations.enable_activestorage = true
  config.integrations.enable_ahoy = true
  config.integrations.enable_carrierwave = true
  config.integrations.enable_dotenv = true
  config.integrations.enable_goodjob = true
  config.integrations.enable_host_authorization = true
  config.integrations.enable_lograge = true
  config.integrations.enable_rack_error_handler = true
  config.integrations.enable_shrine = true
  config.integrations.enable_sidekiq = true
  config.integrations.enable_puma = true
  config.integrations.enable_sorbet_error_handlers = true
  # Logs queries slower than sql_slow_query_threshold, in place of Rails' own SQL lines
  config.integrations.enable_sql_logging = false
end

The integrations' other settings, with their defaults:

LogStruct.configure do |config|
  # Add fields to request lines (see "Custom Lograge Options" below). Default: nil
  config.integrations.lograge_custom_options = ->(event, _options) { {user_id: event.payload[:user_id]} }

  # The mailer instance variables whose IDs are logged on mail lines. Default:
  config.integrations.actionmailer_id_mapping = {account: :account_id, user: :user_id}

  # With SQL logging on, log queries that take at least this many milliseconds (0 or nil logs
  # every query). Default: 100.0
  config.integrations.sql_slow_query_threshold = 100.0

  # Log SQL queries' bind values, filtered by column name. Default: false in production, true
  # elsewhere
  config.integrations.sql_log_bind_params = !Rails.env.production?
end

Filtering Sensitive Data

Every setting for filtering keys and scrubbing strings, with the defaults where the example doesn't change them:

LogStruct.configure do |config|
  # Filter more keys, as well as the defaults (password, token, secret, credentials, auth, ...).
  # Keys match by their words, so :account_number also filters accountNumber and
  # Account-Number, but not longer keys like account_number_last4.
  config.filters.filter_keys += [:account_number, :routing_number]

  # Filter these keys too, and log a hash of their values, so lines about the same value can
  # be found (default: email, email_address)
  config.filters.filter_keys_with_hashes += [:username]

  # Sensitive data in every string
  config.filters.credit_card_numbers = true  # Filter credit card numbers
  config.filters.email_addresses = true      # Filter email addresses
  config.filters.ip_addresses = false        # Filter IP addresses (off by default)
  config.filters.mac_addresses = false       # Filter MAC addresses (off by default)
  config.filters.phone_numbers = true        # Filter phone numbers
  config.filters.ssns = true                 # Filter social security numbers
  config.filters.url_passwords = true        # Filter passwords in URLs
  config.filters.url_query_params = true     # Filter sensitive params in URLs (?token=...)
  config.filters.authorization_credentials = true # Bearer tokens and Basic credentials in text

  # More rules for key names, beside the default pattern for sensitive keys
  config.filters.filter_matchers << LogStruct::ConfigStruct::FilterMatcher.new(
    callable: ->(key, _value) { key.start_with?("internal_") },
    label: "internal prefix"
  )

  # Rules that need a key's path from the line's top-level field, or the parameters around it.
  # Rails' dotted filter_parameters (like "card.number") become path matchers at boot.
  # uses_root: false for a rule that doesn't read the parameters, so they aren't copied for it.
  config.filters.path_matchers << LogStruct::ConfigStruct::PathFilterMatcher.new(
    callable: ->(path, _value, _root) { path.last(2) == ["bank", "number"] },
    label: "bank account numbers",
    uses_root: false
  )

  # The HMAC key for hashed values. Default: a key derived from the Rails app's secret_key_base
  config.filters.hash_salt = ENV["LOGSTRUCT_HASH_KEY"]

  # How many hex characters of each hash to log (default: 16)
  config.filters.hash_length = 16
end
Add to the default lists with += (as above). Assigning a new list replaces the defaults. The Filtering Sensitive Data page explains what each setting matches.

Limits

LogStruct writes only part of very large values, so a huge request body or Hash can't slow down the thread that writes your logs. Long strings are scrubbed before they're cut.

LogStruct.configure do |config|
  # How much of each logged value is written (the defaults)
  config.limits.max_hash_keys = 100           # The rest are counted in _more_keys
  config.limits.max_array_items = 10          # Followed by "... and N more items"
  config.limits.max_backtrace_lines = 20      # Arrays that look like backtraces
  config.limits.max_string_bytes = 32_768     # Longer strings are cut, and say how long they were
end

Error Handling Configuration

LogStruct sorts the errors it handles into four groups, each with a mode. The modes are:

  • Ignore: do nothing.
  • Log: write an error line.
  • Report: write an error line and report the error.
  • LogProduction and ReportProduction: log, or log and report, outside the local_environments; raise in them (development and test by default).
  • Raise: raise the error everywhere, without logging or reporting it.

The groups and their defaults:

LogStruct.configure do |config|
  modes = config.error_handling_modes

  # The defaults:
  # Sorbet type errors: raised in development and test, logged and reported in production
  modes.type_checking_errors = LogStruct::ErrorHandlingMode::ReportProduction
  # Errors inside LogStruct: raised in development and test, logged in production
  modes.logstruct_errors = LogStruct::ErrorHandlingMode::LogProduction
  # CSRF and IP spoofing errors: logged and reported (Rails still responds as usual)
  modes.security_errors = LogStruct::ErrorHandlingMode::Report
  # Errors LogStruct's integrations pass on, such as a mailer's delivery error: raised
  modes.standard_errors = LogStruct::ErrorHandlingMode::Raise
end

A raised error reaches your app as usual, so your error reporter still sees it. Reports go to the first of Sentry, Bugsnag, Rollbar and Honeybadger that your app loads; with none of them, LogStruct logs the error instead. To use another service, set a custom error reporting handler.

ActionMailer ID Mapping

config.integrations.actionmailer_id_mapping lists the mailer instance variables whose IDs are added to mail lines, so you can see which account or user an email was sent for. With the default mapping ({account: :account_id, user: :user_id}), this mailer's lines get account_id and user_id fields:

class UserMailer < ApplicationMailer
  def welcome_email(user, account)
    @user = user        # user_id
    @account = account  # account_id
    mail(to: user.email, subject: "Welcome")
  end
end

Custom Lograge Options

Request lines include the method, path, format, controller, action, status, durations (duration_ms, view, db), params (as Rails filtered them), source_ip, host, user_agent, referer (without its query string), content_type and accept, and req_id. To add your own fields:

LogStruct.configure do |config|
  # Add fields to every request line: return a Hash of fields, or change options in place.
  # Fields that are nil are left out.
  config.integrations.lograge_custom_options = ->(event, _options) {
    {user_id: event.payload[:user_id], account_id: event.payload[:account_id]}
  }
end

The fields are added at the top level of the request line, and filtered like everything else.

With config.exceptions_app = routes, Rails renders an error page by sending the request through your routes again as GET /500 (or /404, /422). When an action raises, the request has one line, the action's, with the error's status: the error page logs none. When the error comes before any action (a routing error), the error page's line is the request's line, with the request's own method and path. When a middleware raises after the action has finished, the request has two lines: the action's, with the status the action returned, and the error page's, with the error's status and the request's method and path.

Custom String Scrubbing

To scrub values the built-in scrubbers don't know about:

# Called with every string LogStruct writes, after its own scrubbers. Return the scrubbed string.
LogStruct.configure do |config|
  config.string_scrubbing_handler = ->(value) {
    # Replace numbers that look like bank account numbers
    value.gsub(/\b\d{10,12}\b/, "[BANK_ACCOUNT]")
  }
end

Custom Error Reporting

If LogStruct doesn't support your error reporting service, register a handler. (Or send a PR!)

LogStruct.configure do |config|
  # Called for each error LogStruct reports, in place of Sentry, Bugsnag, Rollbar or Honeybadger.
  # The context is filtered and scrubbed like a log line.
  config.error_reporting_handler = ->(error, context, source) {
    ErrorTracker.notify(error, context: context, source: source.serialize)
  }
end

If the handler raises (a timeout calling your service), LogStruct logs its error as a logstruct error line, and the error it was reporting is handled as it would have been.

When your app loads more than one of the supported services, choose one:

# LogStruct reports to the first of Sentry, Bugsnag, Rollbar and Honeybadger that's loaded
# (with none, it logs the error). To choose one when your app loads several:
LogStruct::MultiErrorReporter.reporter = :bugsnag

Sorbet Integration

LogStruct sets Sorbet's runtime error handlers, so a type error anywhere in your app or its gems is raised in development and test, and logged and reported in production while your app carries on (error_handling_modes.type_checking_errors). Without LogStruct, Sorbet raises them everywhere. The handlers replace any your app set before LogStruct's initializer ran.

config.integrations.enable_sorbet_error_handlers = true

# This configures the following error handlers for Sorbet:
# - T::Configuration.inline_type_error_handler
# - T::Configuration.call_validation_error_handler
# - T::Configuration.sig_builder_error_handler
# - T::Configuration.sig_validation_error_handler
Edit this page on GitHub