Logging levels (severities)
INFO
The INFO
severity is used for informational messages that do not
require action.
WARNING
The WARNING
severity is used for situations which may require special handling,
where normal operation is expected to resume automatically.
ERROR
The ERROR
severity is used for situations that require special handling,
where normal operation could not proceed as expected.
Other operations can continue mostly unaffected.
FATAL
The FATAL
severity is used for situations that require an immedate, hard
server shutdown. A report is also sent to telemetry if telemetry
is enabled.
Logging channels
DEV
The DEV
channel is used during development to collect log
details useful for troubleshooting that fall outside the
scope of other channels. It is also the default logging
channel for events not associated with a channel.
This channel is special in that there are no constraints as to
what may or may not be logged on it. Conversely, users in
production deployments are invited to not collect DEV
logs in
centralized logging facilities, because they likely contain
sensitive operational data.
See Configure logs.
OPS
The OPS
channel is used to report "point" operational events,
initiated by user operators or automation:
- Operator or system actions on server processes: process starts, stops, shutdowns, crashes (if they can be logged), including each time: command-line parameters, current version being run
- Actions that impact the topology of a cluster: node additions, removals, decommissions, etc.
- Job-related initiation or termination
- Cluster setting changes
- Zone configuration changes
HEALTH
The HEALTH
channel is used to report "background" operational
events, initiated by CockroachDB or reporting on automatic processes:
- Current resource usage, including critical resource usage
- Node-node connection events, including connection errors and gossip details
- Range and table leasing events
- Up- and down-replication, range unavailability
STORAGE
The STORAGE
channel is used to report low-level storage
layer events (RocksDB/Pebble).
SESSIONS
The SESSIONS
channel is used to report client network activity when enabled via
the server.auth_log.sql_connections.enabled
and/or
server.auth_log.sql_sessions.enabled
cluster setting:
- Connections opened/closed
- Authentication events: logins, failed attempts
- Session and query cancellation
This is typically configured in "audit" mode, with event numbering and synchronous writes.
SQL_SCHEMA
The SQL_SCHEMA
channel is used to report changes to the
SQL logical schema, excluding privilege and ownership changes
(which are reported separately on the PRIVILEGES
channel) and
zone configuration changes (which go to the OPS
channel).
This includes:
- Database/schema/table/sequence/view/type creation
- Adding/removing/changing table columns
- Changing sequence parameters
SQL_SCHEMA
events generally comprise changes to the schema that affect the
functional behavior of client apps using stored objects.
USER_ADMIN
The USER_ADMIN
channel is used to report changes
in users and roles, including:
- Users added/dropped
- Changes to authentication credentials (e.g., passwords, validity, etc.)
- Role grants/revocations
- Role option grants/revocations
This is typically configured in "audit" mode, with event numbering and synchronous writes.
PRIVILEGES
The PRIVILEGES
channel is used to report data
authorization changes, including:
- Privilege grants/revocations on database, objects, etc.
- Object ownership changes
This is typically configured in "audit" mode, with event numbering and synchronous writes.
SENSITIVE_ACCESS
The SENSITIVE_ACCESS
channel is used to report SQL
data access to sensitive data:
- Data access audit events (when table audit is enabled via ALTER TABLE ... EXPERIMENTAL_AUDIT)
- Data access audit events (when role-based audit is enabled via
sql.log.user_audit
cluster setting) - SQL statements executed by users with the admin role
- Operations that write to system tables
This is typically configured in "audit" mode, with event numbering and synchronous writes.
SQL_EXEC
The SQL_EXEC
channel is used to report SQL execution on
behalf of client connections:
- Logical SQL statement executions (when enabled via the
sql.log.all_statements.enabled
cluster setting) - uncaught Go panic errors during the execution of a SQL statement.
SQL_PERF
The SQL_PERF
channel is used to report SQL executions
that are marked as "out of the ordinary"
to facilitate performance investigations.
This includes the SQL "slow query log".
Arguably, this channel overlaps with SQL_EXEC
.
However, we keep both channels separate for backward compatibility
with versions prior to v21.1, where the corresponding events
were redirected to separate files.
SQL_INTERNAL_PERF
The SQL_INTERNAL_PERF
channel is like the SQL_PERF
channel, but is aimed at
helping developers of CockroachDB itself. It exists as a separate
channel so as to not pollute the SQL_PERF
logging output with
internal troubleshooting details.
TELEMETRY
The TELEMETRY
channel reports telemetry events. Telemetry events describe
feature usage within CockroachDB and anonymizes any application-
specific data.
KV_DISTRIBUTION
The KV_DISTRIBUTION
channel is used to report data distribution events, such as moving
replicas between stores in the cluster, or adding (removing) replicas to
ranges.
STRUCTURED_EVENTS
The STRUCTURED_EVENTS
channel is used to send log events with messages containing structured
JSON, which can be consumed externally to power o11y features.