What's New in v19.2

On this page

On this page, you can read about changes and find downloads for all production and testing releases of CockroachDB v19.2.

• For key feature enhancements in v19.2 and other upgrade considerations, refer to the notes for v19.2.0.
• For details about release types, naming, and licensing, refer to the Releases page.
• Be sure to also review the Release Support Policy.
• After downloading a supported CockroachDB binary, learn how to install CockroachDB or upgrade your cluster.

Get future release notes emailed to you:

Thanks!

v19.2.12

Release Date: January 19, 2021

Downloads

Docker image

$ docker pull cockroachdb/cockroach:v19.2.12

Bug fixes

• Fixed a panic in protobuf decoding. #58876
• Fixed a bug where the age function did not normalize the duration for large day or H:M:S values in the same way PostgreSQL does. #55528

v19.2.11

Release Date: October 12, 2020

Downloads

Docker image

$ docker pull cockroachdb/cockroach:v19.2.11

General changes

• Links to unimplemented issues that are returned in error messages now use the CockroachLabs redirect/short-link server. #53955

SQL language changes

• Reduced memory used by scans of tables containing JSON data. #53319

Bug fixes

• The Cluster Node Map and the Debug Page that lists cluster localities are now available again to non-admin users. This fixes a bug introduced in v19.2.3. #53332
• Fixed a bug that could cause backfills to fail with a vague TransactionStatus error as they are completing. #53765
• Fixed a bug that could cause CockroachDB to crash when evaluating queries with window functions with the GROUPS mode of framing when OFFSET FOLLOWING boundary was used and when the offset was a very large value such that it could result in an integer overflow. #53758
• Fixed a bug that could cause CockroachDB to crash when using a RANGE-mode window function with an offset in some cases (e.g, OVER (PARTITION BY b ORDER BY a RANGE 1 PRECEDING). #54074
• Adding a NOT NULL constraint no longer fails with the error check ... does not exist. #54290
• A NOT VALID foreign key constraint no longer fails with the internal error table descriptor is not valid: duplicate constraint name. #54290
• Fixed a bug that could cause columns used in an index containing foreign key columns as a prefix to lead to all of the index columns being set to NULL or the default value on CASCADE. #54592
• Fixed a rare bug that could lead to index backfills failing on transaction restarts. #54860

Contributors

This release includes 18 merged PRs by 11 authors.

v19.2.10

Release Date: August 24, 2020

Downloads

Docker image

$ docker pull cockroachdb/cockroach-unstable:v19.2.10

Bug fixes

• Fixed "column not in input" internal error in some corner cases. #50860
• Previously, CockroachDB could crash when internal memory accounting hit a discrepancy. Now it will report an error instead. #51015
• Fixed incorrect results in some cases involving joins on interleaved tables with limits. #51435
• Previously, CockroachDB could hit an internal error when executing regexp_replace builtin. This has been fixed. #51348
• Previously, CockroachDB could hit a "command is too large" error when performing an UPSERT operation with many values. Internally, we attempt to perform such operation by splitting it into "batches", but the batching mechanism was broken. This has been fixed. #51612
• CockroachDB previously could crash on some queries with merge joins. This has been fixed. #52074
• Fixed an internal error that could occur when an aggregate function argument contained a correlated subquery with another aggregate function referencing the outer scope. This now returns an appropriate user-friendly error, "aggregate function calls cannot be nested". #52141
• Previously, subtracting months from a TIMESTAMP / TIMESTAMPTZ or DATE whose date value is greater than 28 could subtract a further year off. This has been fixed. #52155
• Large write requests no longer have a chance of erroneously throwing a "transaction with sequence has a different value" error. #52268
• Previously, CockroachDB could return incorrect results on query that encountered a ReadWithinUncertaintyInterval error. This has been fixed. #52252
• Previously, CockroachDB would erroneously restart the execution of empty, unclosed portals after they had been fully exhausted. This has been fixed. #52444
• Previously, executing a large number of statements in a transaction without committing could crash a CockroachDB server. This has been fixed. #52403

Performance improvements

• Queries no longer block during planning if cached table statistics have become stale and the new statistics have not yet been loaded. Instead, the stale statistics are used for planning until the new statistics have been loaded. This improves performance because it prevents latency spikes that may occur if there is a delay in loading the new statistics. #52189

Contributors

This release includes 24 merged PRs by 9 authors.

v19.2.9

Release Date: July 6, 2020

Downloads

Docker image

$ docker pull cockroachdb/cockroach-unstable:v19.2.9

General changes

• Schema changes are now logged in greater detail. #50375

Bug fixes

• Fixed a bug that occurred during incremental backups with revision history on a database (or full cluster) when a table in the database being backed up was dropped. When other tables were later created, dropped, or altered, future incremental backups would return an error. #50208
• Fixed a bug in TRUNCATE that could leave tables in a state where they could not be renamed. #50714 #50756#50756]

Contributors

This release includes 4 merged PRs by 3 authors.

v19.2.8

Release Date: June 29, 2020

Downloads

Docker image

$ docker pull cockroachdb/cockroach-unstable:v19.2.8

Security updates

• HTTP endpoints beginning with /debug/ now require a valid admin login session. #50489

SQL language changes

• The pg_database table in pg_catalog no longer requires privileges on any database in order for the data to be visible. #48766

Bug fixes

• Fixed a bug where cockroach dump on a table with collated strings would omit the collation clause for the data insertion statements. #48833
• Manually writing a NULL value into the system.users table for the "hashedPassword" column will no longer cause a server crash during user authentication. #48837
• Fixed a memory leak which could affect changefeeds performing scans of large tables. #49162
• Previously, when the value passed to --drain-wait was very small, but non-zero, cockroach quit in certain cases would not proceed to perform a hard shutdown. This has been corrected. This bug was present in v19.1.9, v19.2.7, and v20.1.1. #49364
• Previously, some benign errors were reported as unexpected internal errors by the vectorized execution engine. This is now fixed. #49535
• Fixed a bug causing file descriptors to be leaked during GSS authentication. #49655
• Casting to width-limited strings now works correctly for strings containing Unicode characters. #50160
• Fixed cases in which casting a string to a width-limited string array was not truncating the string. #50169
• Fixed a RocksDB bug that could result in inconsistencies in rare circumstances. #50498

Performance improvements

• Improved the optimizer's estimation of the selectivity of some filters involving a disjunction (OR) of predicates over multiple columns. This results in more accurate cardinality estimation and enables the optimizer to choose better query plans in some cases. #50473

Build changes

• Release Docker images are now built on Debian 9.12. #50481

Doc updates

• Updated guidance on node decommissioning. #7304
• Renamed "whitelist/blacklist" terminology to "allowlist/blocklist". #7535
• Updated the Releases navigation in the sidebar to expose the latest Production and Testing releases. #7550
• Fixed scrollbar visibility on Chrome. #7487

v19.2.7

Release Date: May 20, 2020

This page lists additions and changes in v19.2.7 since v19.2.6.

• For a comprehensive summary of features in v19.2, see the v19.2 GA release notes.
• To upgrade to v19.2, see Upgrade to CockroachDB v19.2

Downloads

Docker image

$ docker pull cockroachdb/cockroach:v19.2.7

Backward-incompatible changes

• Prior to this patch, the phase of server shutdown responsible for range lease transfers to other nodes would give up after 10000 attempts of transferring replica leases away, regardless of the value of server.shutdown.lease_transfer_wait. The limit of 10,000 attempts has been removed, so that now only the maximum duration server.shutdown.lease_transfer_wait applies. #47691
• The textual error and warning messages displayed by cockroach quit under various circumstances have been updated. The message ok still indicates that the operation has likely succeeded. #47691
• cockroach quit now prints out progress details on its standard error stream, even when --logtostderr is not specified. Previously, nothing was printed on standard error. Scripts that wish to ignore this output can redirect the standard error stream. #47691
• The copy of system and crdb_internal tables extracted by cockroach debug zip is now written using the TSV format (inside the ZIP file), instead of an ASCII-art table as previously. #48098

General changes

• Prior to this patch, the phase of server shutdown responsible for range lease transfers to other nodes had a hard timeout of 5 seconds. This timeout is now configurable with the new cluster setting server.shutdown.lease_transfer_wait. #47691

SQL language changes

• Columns in the process of being added or removed to a table are now always set to their default or computed value if another transaction concurrently inserts, updates, or upserts a row. This fixes an issue where a column being backfilled would not be properly set by concurrent transactions. #47411
• The pg_collation, pg_proc, and pg_type tables in pg_catalog no longer require privileges on any database in order for the data to be visible. #48081
• Histogram collection with CREATE STATISTICS is no longer supported on columns with type ARRAY. Only row count, distinct count, and null count are collected for ARRAY-type columns. #48344
• CockroachDB now correctly reports the type length for the CHAR data type. #48643

Command-line changes

• The time that cockroach quit waits client-side for the node to drain (i.e., remove existing clients and push range leases away) is now configurable with the flag --drain-wait. The default value for --drain-wait is 10 minutes. Note that separate server-side timeouts still apply. Check the server.shutdown.* cluster settings for details. #47691
• It is now possible to drain a node without shutting down the process, using cockroach node drain. This makes it easier to integrate CockroachDB with service managers and orchestration. It is safe to issue cockroach node drain and then separately stop the service via a process manager or orchestrator. Without this new command, there is a risk of misconfiguring the service manager to auto-restart the node after it shuts down via cockroach quit. cockroach node drain also recognizes the new --drain-wait flag. #47691
• The commands cockroach quit and cockroach node drain now report a "work remaining" metric on their standard error stream. The value reduces until it reaches 0 to indicate that the graceful shutdown has completed server-side. #47691
• To support clusters that have been configured to use a cluster name, client commands such as cockroach init and cockroach quit now support the --cluster-name and --disable-cluster-name-verification flags. Previously it was impossible to run such commands against nodes configured with the --cluster-name flag. #48017
• cockroach debug zip now properly skips over fully decommissioned nodes. #48095
• cockroach debug zip now tries multiple times to retrieve data using SQL if it encounters retry errors. #48098
• cockroach debug zip now supports two command-line parameters: --nodes and --exclude-nodes. When specified, they control which nodes are inspected when gathering the data. This makes it possible to focus on a group of nodes of interest in a large cluster, or to exclude nodes that cockroach debug zip would have trouble reaching otherwise. Both flags accept a list of individual node IDs or ranges of node IDs, e.g., --nodes=1,10,13-15. #48098

Bug fixes

• Previously, CockroachDB could incorrectly consider a non-NULL value from an interleaved parent table to be NULL when the interleaved child has a NULL value in the row with the corresponding index key. This bug has been fixed. #47104
• A data race on AST nodes for SELECT statements that include a WINDOW definition has been fixed. Previously, these queries could have returned incorrect results. #47176
• Previously, a panic could occur when SHOW RANGES or SHOW RANGE FOR ROW was called with a virtual table. This bug has been fixed. #47516
• Fixed incorrect results that could occur when casting negative intervals or timestamps to type decimal. #47521
• Previously, when the query was executed with the row-by-row engine, CockroachDB could incorrectly return no output instead of returning a parsing error in queries with count(*). This bug has been fixed. #47486
• Previously, CockroachDB incorrectly released the memory used in hash aggregation operations, which could lead to node crashes. This bug has been fixed. #47519
• Previously, PARTITION BY and ORDER BY columns in window specifications lost qualifications when used inside views. This bug has been fixed. #47716
• Fixed a bug where target column specifications in IMPORT INTO were ignored. #47796
• Columns named with reserved keywords and/or other special characters are now correctly handled. #47796
• CockroachDB will now avoid producing a severe "internal error" upon certain privilege check failures with pg_catalog built-in functions. #48243
• Fixed a bug causing cascading deletes to fail when deleting thousands of rows across interleaved tables. #48287
• Fixed a rare bug that could lead to data corruption or data loss if a replica was both the source of a snapshot, was being concurrently removed from the range, and if certain specific conditions exist inside RocksDB. #48319
• Fixed a rare bug where stats were not automatically generated for a new table. #48028
• Fixed a rare bug where stats refreshes could fail. CockroachDB now triggers stats refreshing after completing an import job, after the transaction finishes the job commits. #48028
• Fixed an error that occurred when statistics collection was explicitly requested on a column with type ARRAY. #48344
• Fixed a rare bug causing a range to deadlock and all the writes to the respective range to timeout. #48304
• Fixed a bug causing HTTP requests to fail with error 503 "transport: authentication handshake failed: io: read/write on closed pipe" and continue failing until the node was restarted. #48482
• cockroach debug zip can now successfully avoid out-of-memory errors when extracting very large system or crdb_internal tables. #48098
• cockroach debug zip will now properly report an error if an error is encountered while writing the end of the output ZIP file. #48098
• Fixed a bug that caused vectorized queries on composite data types to occasionally return invalid data. #48464
• Previously, SHOW STATISTICS USING JSON output contained incorrect single quotes for strings with spaces inside histograms. This bug has been fixed. #48545
• Fixed a bug preventing timestamps from being closed, which could result in failed follower reads or failure to observe resolved timestamps in changefeeds. #48683
• Fixed a bug that falsely indicated that kv.closed_timestamp.max_behind_nanos was almost always growing. #48690
• CockroachDB now prevents dropping databases that contain tables that are currently offline due to IMPORT or RESTORE. Previously, dropping a database in this state could lead to a corrupted schema, which prevented running backups. #48612
• Fixed a bug that caused cockroach dump to erroneously include an additional CREATE UNIQUE INDEX "primary" ... INTERLEAVE IN PARENT statement in the dump output on tables with interleaved primary keys. This bug made it impossible to re-import dumped data without manual editing. #48777
• Fixed a bug that caused SHOW CREATE statements to sometimes show a partitioning step for an index that was previously dropped. #48767
• Fixed a bug that caused changefeeds that were created with a cursor, and that encounter a retryable error before writing their first checkpoint, to perform full-table backfills upon restart. #48752
• Fixed a bug that could cause canceled changefeeds to not stop. #48752

Performance improvements

• Fixed a bug in the histogram filtering logic of the optimizer which was causing inaccurate cardinality estimates for queries with equality predicates on UUIDs and strings, as well as some other types. This bug has existed since histograms were first introduced into the optimizer in the v19.2.0 release. Fixing it improves the optimizer's cardinality estimates and results in better query plans in some cases. #48625
• Histograms used by the optimizer for query planning now have more accurate row counts per histogram bucket, particularly for columns that have many NULL values. This results in better plans in some cases. #48645
• Fixed a performance inefficiency in the vectorized execution engine. This fix speeds up all queries that are executed with the vectorized engine, with most noticeable gains on the queries that output many rows. #48733

v19.2.6

Release Date: April 13, 2020

This page lists additions and changes in v19.2.6 since v19.2.5.

• For a comprehensive summary of features in v19.2, see the v19.2 GA release notes.
• To upgrade to v19.2, see Upgrade to CockroachDB v19.2

Downloads

Docker image

$ docker pull cockroachdb/cockroach:v19.2.6

Backward-incompatible changes

• cockroach init now waits for server readiness and thus no longer fails when a mistaken server address is provided. #46477

Security updates

• The non-authenticated /health HTTP endpoint was previously exposing the private IP address of the node, which can be privileged information in some deployments. This has been corrected. Deployments using automation to retrieve a node build details and address details should use /_status/details/local instead and use a valid admin authentication cookie. #46477
• All users can now view any comments on any object (bypassing other privileges), but modifying comments require write privilege on the target object. Previously, any user could modify any database/table/view/index comment via direct SQL updates to system.comments. This was unintended and a form of privilege escalation, and is now prevented. The privileges required for the COMMENT statement, pg_description, col_description(), obj_description(), and shobj_description() are operating as in PostgreSQL and are unaffected by this change. #46833

SQL language changes

• The type checking code now prefers aggregate overloads with string inputs if there are multiple possible candidates due to arguments of unknown type. #46807
• Added an unimplemented error when attempting to ADD CONSTRAINT with the EXCLUDE USING syntax. #46910
• Added support for CREATE INDEX .... INCLUDE (col1, col2, ...), which is an alias that PostgreSQL uses that is analogous to CockroachDB's STORING (col1, col2, ...) syntax. #46910
• Added support for parsing the REINDEX syntax, which results in an unimplemented error that explains that REINDEXing is not required in CockroachDB. #46910
• CREATE INDEX CONCURRENTLY and DROP INDEX CONCURRENTLY are now parsed but return errors. #46805
• Improved the error message for unsupported interactions between correlated subqueries and WITH clauses. #46971

Command-line changes

• The cockroach init command now waits until the node at the provided server address is ready to accept initialization. This also waits for network readiness. This makes it easier to implement initialization scripts by removing the need for a loop. In addition, implementing such a loop is operationally unsafe and is not recommended. #46477
• CockroachDB will now dump the stacks of all goroutines upon receiving SIGQUIT prior to terminating. This feature is intended for use while troubleshooting misbehaving nodes. #46478
• The cockroach start and cockroach start-single-node commands no longer initiate a 1-minute hard shutdown countdown after a request to gracefully terminate. This means that graceful shutdowns are now free to take longer than one minute. It also means that deployments where a maximum shutdown time must be enforced must now use a service manager that is suitably configured to do so. #46483
• The cockroach debug zip command now creates valid zip files even if some of its requests encounter an error. #46635

Admin UI changes

• Metrics relating to SQL transaction restarts and rollbacks are now properly captured and exported. #46272
• The display options are now saved separately for each authenticated user. Note: When upgrading to a version with this change, all current display customizations for admin users are lost. #46479

Bug fixes

• Fixed a bug where pg_catalog.pg_indexes showed the wrong index definition for inverted indexes. #46564
• Fixed a performance bug where TRUNCATE would take 2*num columns round trips. This could lead to slow truncate performance in distributed clusters. #46648
• Fixed an internal error that could occur in the optimizer when a WHERE filter contained at least one correlated subquery and one non-correlated subquery. #46167
• CockroachDB no longer returns an internal error in some cases when running queries with CASE, AND, or OR operators via the vectorized execution engine. #46088
• Fixed a bug where various session variables could not be set to the values "on" or "off", only true or false. #46219
• Accesses to /health using a non-root authentication token no longer hang when a node is currently under load, or if a system range is unavailable. #46477
• Fixed a race in the vectorized execution engine. #46486
• The cockroach debug zip command now properly collects heap profiles. #46482
• The goroutine dump facility now functions properly when logging to files is disabled, e.g., via --log-dir= or --logtostderr. #46482
• Fixed a bug where the vectorized engine could sometimes give an incorrect result when reading from interleaved parents or children. #46495
• Fixed an internal error that could occur when an aggregate inside the right-hand side of a LATERAL join was scoped at the level of the left-hand side. #46544
• Fixed an error that incorrectly occurred when an aggregate was used inside the WHERE or ON clause of a subquery but was scoped at an outer level of the query. #46544
• Added support for queries with qualified stars that refer to tables in outer scopes. #46538
• Fixed an incorrect "no data source matches prefix" error in some cases involving subqueries that use views. #46537
• Expressions of the form NOT(a && b) no longer fail. #46539
• Fixed a bug where operations on an index that contained a collated string in descending order would fail.#46578
• Fixed a bug where the vectorized engine would throw an internal error when executing a query that utilized an inverted index. #46506
• Fixed an internal error that could happen during planning when a column with a histogram was filtered with a predicate of a different data type. #46667
• CockroachDB no longer incorrectly accounts for some RAM usage when computing aggregate functions. #46622
• Fixed an incorrect query result that could occur when a scalar aggregate was called with a null input. #46807
• Benign "outbox next" errors are now only logged when log verbosity is set to 1 or greater. #46837
• Fixed incorrect result with count(*) when grouping on constant columns. #46894
• Customizations of the Admin UI are once again properly saved across sessions. #46479
• Fixed a rare bug causing transactions that have performed schema changes to deadlock after they restart. #46904

Performance improvements

• Less memory is loaded into memory for crdb_internal.jobs. #46452
• Virtual tables that access all table descriptors now make fewer round trips. #46950

Contributors

This release includes 42 merged PRs by 16 authors.

v19.2.5

Release Date: March 23, 2020

This page lists additions and changes in v19.2.5 since v19.2.4.

• For a comprehensive summary of features in v19.2, see the v19.2 GA release notes.
• To upgrade to v19.2, see Upgrade to CockroachDB v19.2

Downloads

Docker image

$ docker pull cockroachdb/cockroach:v19.2.5

Security updates

• Non-licensed users are now able to add more principals to the special superuser role/group admin. Creation of additional roles is still a licensed feature. #45395

SQL language changes

• The SQL:2008 syntax OFFSET <x> ROWS and FETCH FIRST <y> ROWS ONLY now accepts parameter values. #45166
• It is now possible to use GRANT and REVOKE to add users to the admin role without a valid license. This change aims to enable use of the Admin UI and other privileged features without a license. #45395

Command-line changes

• cockroach debug zip now includes secondary log files (e.g., the RocksDB logs) in the main log directory. Log files in alternate log directories, if different from the main log directory, are not included. #45202
• cockroach debug zip now displays its progress differently on the terminal. #45207
• cockroach debug zip now properly supports special characters in database and table names. #45207
• cockroach debug zip will now apply the --timeout parameters to the SQL queries it performs. Previously, there was no timeout, which could cause cockroach debug zip to hang. #45207
• cockroach debug zip is now able to tolerate more forms of cluster unavailability. In case system ranges are unavailable, we recommend running cockroach debug zip on each node's address, to maximize useful data collection. #45207
• The debug.zip file generated by cockroach debug zip now contains hex a representation of the marshaled jobs payload, progress, and table descriptors. #46040

Admin UI changes

• The "Log file list" endpoint now includes secondary log files (e.g., the RocksDB logs) in the main log directory. Log files in alternate log directories, if different from the main log directory, are not included. #45202

Bug fixes

• Fixed a "cannot map variable" error that could occur in rare cases involving joins. #44859
• Fixed a bug where DROP INDEX jobs waiting for GC TTL might be erroneously deleted early. Usually a DROP INDEX job would appear as "waiting for GC TTL" to indicate that the data was still on disk. However, this bug might have caused the job to be deleted before the data was actually removed from disk. #44873
• Fixed occasional "concurrent map write" crash. #44874
• Previously, CockroachDB would return an internal error when a substring function with non-INT8 start and length arguments was executed with the vectorized execution engine. This is now fixed. #44902
• Fixed incorrect deduplication of impure expressions (e.g., gen_random_uuid) in projections and default values. #44915
• Previously, CockroachDB could return an internal error on the queries that return INT columns when the default integer size has been changed. This is now fixed. #44980
• Previously, CockroachDB would return an internal error when a merge join operation was performed with the vectorized execution engine in the case when two sides of the join had comparable but different types in the equality columns (e.g., INT2 on the left and INT4 on the right). This is now fixed. #45032
• Previously, CockroachDB could crash when running EXPLAIN (VEC) in some edge cases. Now, an internal error is returned instead. #45004
• Fixed an internal error that could occur when NULLIF was called with one null argument. #45390
• Some vectorized execution plans that used lookup joins with DECIMALs would previously return incorrect results. This is now fixed. #45548
• AbortSpan records are now cleaned up more aggressively when it is known to be safe. #45553
• It is now possible to create inverted indexes on columns whose names are mixed-case. #45660
• Fixed a bug causing distributed deadlock detection between transactions to stall and fail to resolve a deadlock. #45664
• Fixed a bug where large ranges with slow send rates would hit the timeout in several storage system queues by making the timeout dynamic based on the current rate limit and the size of the data being sent. This affects several storage system queues: the Raft snapshot queue, the replication queue, and the merge queue. #44952
• Previously, drivers that did not truncate trailing zeroes for decimals in the binary format end up having inaccuracies of up to 10^4 during the decode step. Trailing zeroes are now truncated as appropriate. This fixes known incorrect decoding cases with Postgrex in Elixir. #45670
• Range garbage collection will now trigger based on a large abort span, adding defense-in-depth against ranges growing large and eventually unstable. #45744
• Previously, an internal error could occur in CockroachDB when executing queries that contained unordered synchronizers with the vectorized execution engine. This is now fixed. #45739
• Previously, CockroachDB would crash when using a window function with the RANGE mode of framing an offset PRECEDING or offset FOLLOWING boundary where the single column in an ORDER BY clause was also in the PARTITION BY clause. This is now fixed. #45805
• Previously, CockroachDB could crash when computing window functions with the RANGE mode of framing when one of the bounds was either offset PRECEDING or offset FOLLOWING and there were NULL values in the single column of an ORDER BY clause. Additionally, RANGE mode bounds 0 PRECEDING and 0 FOLLOWING could have been handled incorrectly. This is now fixed. #45805
• When considering if a job should be orphaned, CockroachDB used to take the conservative approach when a descriptor ID pointing to non-existent descriptor was found. This caused jobs to hang and be garbage collected. We now disregard these IDs when considering if a job still has work to do. #45895
• The --set parameter for cockroach sql and cockroach demo is now able to properly override all client-side options, as advertised. #46130
• Expected errors from the vectorized execution engine are no longer mistakenly annotated as unexpected errors. #46092

Performance improvements

• Execution plans are now faster in some cases that involve IN/NOT IN with an empty tuple, or = ANY with an empty array. #45213
• Improved the selectivity estimation of some predicates containing OR, leading to better plan selection by the optimizer, in some cases. #45972
• Improved cardinality estimation in the optimizer for relations with a small number of rows. This may lead to the optimizer choosing a better query plan. #45972

Contributors

This release includes 36 merged PRs by 15 authors.

v19.2.4

Release Date: February 11, 2020

This page lists additions and changes in v19.2.4 since v19.2.3.

• For a comprehensive summary of features in v19.2, see the v19.2 GA release notes.
• To upgrade to v19.2, see Upgrade to CockroachDB v19.2

Downloads

Docker image

$ docker pull cockroachdb/cockroach:v19.2.4

Security updates

• Previous versions of CockroachDB were incorrectly enabling non-admin SQL users to use the statements details in the Admin UI and the HTTP endpoint /_status/statements. This information is sensitive because the endpoint does not hide data that the requester does not have privilege over. This has been corrected by requiring a SQL admin user to access the statements details page and the HTTP endpoint. #44354

Bug fixes

• Fixed a bug where CockroachDB would return an internal error when the substring function with a negative length was executed via the vectorized engine. CockroachDB now returns a regular query error on executing the function. #44629
• Fixed "no output column equivalent to.." and "column not in input" errors in some cases involving DISTINCT ON and ORDER BY. #44598
• Fixed "expected constant FD to be strict" internal error. #44604
• Fixed possibly incorrect query results in various cornercases, especially when SELECT DISTINCT is used. #44604
• Fixed a bug where running a query with the LIKE operator using the custom ESCAPE symbol when the pattern contained Unicode characters could result in an internal error in CockroachDB. #44648
• CockroachDB no longer repeatedly looks for non-existing jobs, which may cause high memory usage, when cleaning up schema changes. #44698
• Fixed "no indexes" internal error in some cases when we GROUP BY on a virtual table. #44723
• Fixed invalid query results in some corner cases where part of a WHERE clause is incorrectly discarded. #44749
• Fixed a typechecking error where BETWEEN would sometimes allow boundary expressions of a different type. #44810
• CASE operators with an unknown WHEN type no longer return an error. #44818

Contributors

This release includes 15 merged PRs by 10 authors.

v19.2.3

Release Date: February 3, 2020

This page lists additions and changes in v19.2.3 since v19.2.2.

• For a comprehensive summary of features in v19.2, see the v19.2 GA release notes.
• To upgrade to v19.2, see Upgrade to CockroachDB v19.2

Downloads

Docker image

$ docker pull cockroachdb/cockroach:v19.2.3

Security updates

• SQL admins can now use the new command cockroach auth-session login to create authentication tokens with an arbitrary expiration date. Operators should be careful to monitor system.web_sessions and enforce policy-mandated expirations using either SQL queries or the new command cockroach auth-session logout. #44110

Enterprise edition changes

• CockroachDB is now more resilient to connection/network errors when reading data from HTTP servers. CockroachDB now attempts to resume the download after such errors. #43687

SQL language changes

• extract() now supports millennium, century, decade, isoyear, isodow, and julian for date, timestamp, and timestamptz. DATE_TRUNC now supports millennium, century, and decade for date, timestamp, and timestamptz. #42957
• The output of EXPLAIN now shows joins where there are no equality columns as "cross" instead of "hash". Cross joins can be very expensive and should be avoided. #43239
• Previously, we error-coded duplicate foreign key constraint names with 42830 or 23503. We changed this to be 42710 to be in line with PostgreSQL. #43425
• Vectorized queries that execute only on supported types, even if those types form part of a table with unsupported types are now run through the vectorized engine. This would previously fall back to the row-by-row execution engine. #44014
• SHOW RANGES now shows locality information consistent with the range descriptor when node ID and store ID do not match. #44312
• Previously, CockroachDB would return an internal error when using SET TRACING with any type other than STRING. Now it will return a regular query error. Additionally, boolean arguments are now supported in SET TRACING, and true is mapped to on mode of tracing whereas false is mapped to off. #44347

Command-line changes

• Telemetry is now recorded for whenever the command cockroach demo is used. #43819
• Three new CLI commands, cockroach auth-session login, cockroach auth-session list, and cockroach auth-session logout, are now provided to facilitate the management of web sessions. The command auth-session login also produces a HTTP cookie which can be used by non-interactive HTTP-based database management tools. It also can generate such a cookie for the root user, who would not otherwise be able to do so using a web browser. #44110

Admin UI changes

• We previously introduced a fix on the Admin UI to prevent non-admin users from executing queries. However, this inadvertently caused certain pages requiring table details not to display. This issue has now been resolved. #44191

Bug fixes

• Fixed a bug during planning for some queries that could cause an infinite loop and prevent the query from being cancelled. #43082
• Some incorrect issue links referenced by error hints have been corrected. #43233
• EXPLAIN can now be used with statements that use AS OF SYSTEM TIME. #43303
• This bug may cause jobs to be left indefinitely in PENDING state and never run. #43421
• Fixed a bug that prevented error reporting while generating shell autocomplete scripts. #43507
• Fixed a bug that prevented error reporting while reading from crdb_internal.gossip_nodes which could lead to lost writes or other undefined behavior. #43502
• Fixed a bug that prevented error reporting while generating plan diagrams rather than silently reporting no result. #43506
• Fixed a bug where an internal error could occur when CASE operator operating on distinct although compatible types was executed via the vectorized engine. For example, a query similar to SELECT CASE WHEN false THEN 0:::INT2 ELSE 1:::INT8 END previously would error out. #43566
• Fixed a bug that prevented error reporting while looking up table descriptors during schema changes and other transactional interactions with schema which could cause lost writes or other undefined behavior. #43503
• Fixed a bug in the parsing logic for server.host_based_authentication.configuration, where both single-character strings, and quoted strings containing spaces and separated by commas, were not properly parsed. #43714
• Fixed a bug that prevented error reporting while failing to parse URIs for multi-locality backup. #43501
• It is now possible to perform ALTER COLUMN SET/DROP NULL on multiple (different) columns of the same table inside a single transaction. #43645
• CockroachDB now properly rejects ALTER COLUMN DROP NOT NULL on a column that is part of the primary key. #43645
• Fixed a bug where errors from JSON containment operations were silently ignored rather than being returned. #43505
• Previously, CockroachDB could incorrectly report that some values were NULL when, in fact, they weren't. This could occur only when vectorized execution engine was used with vectorize=experimental_on and now has been fixed. #43823
• Fixed a bug where CockroachDB returned incorrect results for some aggregate functions when used as window functions with non-default window frame. Note that MIN, MAX, SUM, AVG, as well as "pure" window functions (i.e., non-aggregates) were not affected. #43854
• Fixed a bug where CockroachDB could, in some cases, return an internal error when running a query with a CAST operation (:::) if vectorized execution engine is used. #43890
• A SQL row write that is re-issued after already succeeding will no longer throw a duplicate key error when the previous write in its transaction deleted the row. #43937
• On Linux machines, CockroachDB now respects the available memory limit set by the cgroup limits which apply to the cockroach process. #43860
• Fixed a bug in which a query shutdown mechanism could fail to fully cleanup the infrastructure when the query was executed via the vectorized engine and the query plan contained wrapped row-by-row processors. #43944
• Fixed a bug introduced in 19.2 that would allow foreign keys to use a unique index on the referenced columns that indexed more columns than were included in the columns used in the foreign key constraint, which allows potentially violating uniqueness in the referenced columns themselves. #43796
• A benign error previously logged at the ERROR level is now logged at the INFO level behind a verbosity(2) flag. This error might have been observed as context canceled: readerCtx in Inbox stream handler. #44037
• Fixed a bug where an internal error could occur when a query with an aggregate function MIN or MAX was executed via the vectorized engine when the input column was either INT2 or INT4 type. #44055
• cockroach zip now emits the goroutine file in the proper sub-directory when the corresponding call fails with an error. #44065
• cockroach debug zip is again able to operate correctly and continue to iterate over all nodes if one of the nodes does not deliver its goroutine dumps. It would previously prematurely and incorrectly stop with an incomplete dump; this was a regression introduced in 19.2. #44065
• CDC is no longer susceptible to a bug where a resolved timestamp might be published before all events that precede it have been published in the presence of a range merge. #44077
• Converted a panic in golang.org/x/text/language/tags.go when using collated strings to an error. #44117
• CockroachDB now properly supports using --url with query options (e.g., application_name) but without specifying sslmode. The default of sslmode=disable is assumed in that case. #44115
• Removed statistics information from backup jobs' payload information to avoid excessive memory utilization when issuing commands such as SHOW JOBS. #44190
• Fixed a bug where CockroachDB could crash in special circumstances when vectorized execution engine is used (it was more likely to happen if vectorize=experimental_on setting was used). #44196
• Fixed a planning bug related to FULL joins between single-row relations. #44228
• Fixed CopyFrom requires empty destination internal error. #44245
• Fixed a bug where multiple nodes attempted to populate the results for CREATE TABLE ... AS leading to duplicate rows. #44300
• Fixed internal error when mixed types are used with BETWEEN. #44319
• Fixed a bug where CockroachDB would return an internal error when a query with CASE operator that returns only NULL values was executed via the vectorized engine. #44353
• Fixed an internal error in the planner when table statistics were collected manually using CREATE STATISTICS for different columns at different times. #44442
• Fixed a bug that prevented admin users from using BACKUP, RESTORE, and IMPORT. #44456
• Fixed incorrect plans in very rare cases involving filters that aren't constant folded in the optimizer but that can be evaluated statically when running a given query. #44419

Performance improvements

• Adjusted the optimizer's cost of lookup join when the lookup columns aren't a key in the table. This will now cause some queries to switch to using a hash or merge join instead of a lookup join, improving performance in most cases. #43059
• Improved the estimated row count for some lookup joins during planning, leading to better plans in some cases. #43364
• A transaction running into multiple intents from an abandoned conflicting transaction now cleans them up more efficiently. #43588
• Histograms are now collected automatically for all boolean columns, resulting in better query plans in some cases. For tables that aren't being modified frequently, it might be necessary to run CREATE STATISTICS manually to see the benefit. #44171

Allocator

• The kv.allocator.range_rebalance_threshold setting, which controls how far away from the mean a store's range count must be before it is considered for rebalance, is now subject to a 2-replica minimum. If, for example, the mean number of replicas per store is 5.6 and the setting is 5%, the store will not be considered for rebalance unless the number of replicas is lesser than 3 or greater than 8. Previously, the bounds would have been 5 and 6. #44359

Doc updates

• Added a Django app development tutorial. #6359 #6365
• Updated the Hibernate app development tutorial with client-side retry logic. #5760
• Documented how to use keyset pagination to iterate through query results. #6114
• Improved the GSSAPI authentication instructions for configuring Active Directory and MIT and for configuring the client. #6115
• Expanded the Kubernetes tutorial to show how to use a custom CA instead of Kubernetes built-in CA when using manual configs. #6232
• Updated the Kubernetes tutorial for compatibility with Helm 3.0. #6121
• Added language-specific connection strings to the instructions on connecting to a CockroachCloud cluster. #6077
• Added Docker as a download option on the full release notes list. #5792
• Updated the IMPORT documentation with an example usage of DELIMITED with escaping, a note about DEFAULT values, and an explanation of the strict_quotes option. #6244
• Added an FAQ on why Cockroach Labs changed the license for CockroachDB. #6154
• Corrected the description of the possible result of clock skew outside the configured clock offset bounds. #6329
• Expanded the data types overview to indicate whether or not a type supports vectorized execution. #6327

v19.2.2

Release Date: December 16, 2019

This page lists additions and changes in v19.2.2 since v19.2.1.

• For a comprehensive summary of features in v19.2, see the v19.2 GA release notes.
• To upgrade to v19.2, see Upgrade to CockroachDB v19.2

Downloads

Docker image

$ docker pull cockroachdb/cockroach:v19.2.2

Security updates

• CockroachDB previously allowed non-authenticated access to privileged HTTP endpoints like /_admin/v1/events, which operate using root user permissions and can thus access (and sometimes modify) any and all data in the cluster. This security vulnerability has been patched by disallowing non-authenticated access to these endpoints and restricting access to admin users only.

  Note:

  Users who have built monitoring automation using these HTTP endpoints must modify their automation to work using an HTTP session token for an admin user.

• Some Admin UI screens (e.g., Jobs) were previously incorrectly displayed using root user permissions, regardless of the logged-in user's credentials. This enabled insufficiently privileged users to access privileged information. This security vulnerability has been patched by using the credentials of the logged-in user to display all Admin UI screens.

• Privileged HTTP endpoints and certain Admin UI screens require an admin user. However, root is disallowed from logging in via HTTP and it is not possible to create additional admin accounts without an Enterprise license. This is further discussed here and will be addressed in an upcoming patch revision.

  Note:

  Users without an Enterprise license can create an additional admin user using a temporary evaluation license, until an alternative is available. A user created this way will persist beyond the license expiry.

• Some Admin UI screens currently display an error or a blank page when viewed by a non-admin user (e.g., Table Details). This is a known limitation mistakenly introduced by the changes described above. This situation is discussed further here and will be addressed in an upcoming patch revision. The list of UI pages affected includes but is not limited to:

  • Job details
  • Database details
  • Table details
  • Zone configurations
  Note:

  Users can access these Admin UI screens using an admin user until a fix is available.

The list of HTTP endpoints affected by the first change above includes:

General changes

• Improved the consistency checker's log output. #42149
• Nodes that have been terminated as the result of a failed consistency check now refuse to restart, making it more likely that the operator notices that there is a persistent issue in a timely manner. #42149
• Client usernames can now be defined to start with a digit; in particular, all-digit usernames are now permitted. #42989
• To curb the spread of corrupted data, outliers are now terminated when the replicas within a range are corrupted. Previously, the leaseholder replica would terminate, regardless of which replicas disagreed with each other. #42149

Enterprise edition changes

• RESTORE now supports the restoration of empty databases. #42733

SQL language changes

• Added the ability to run avg() over intervals. #42679
• Some columns (typinput, typoutput, typreceive, typsend, typmodin, typmodout, typanalyze) in pg_type were incorrectly typed as OID instead of REGPROC. This issue has been resolved. #42784
• It is now valid for SELECT and HAVING to refer to ungrouped columns in the special case when the grouped columns contain the primary key of the table containing the ungrouped column. #42694
• It is now supported to specify selection target aliases as GROUP BY columns. Note that the FROM columns take precedence over the aliases, which are only used if there is no column with that name in the current scope. #42694
• CockroachDB will now report a more accurate error message, hint, and error code if an error is encountered while adding a new column. #42530

Admin UI changes

• Certain web UI pages (like the list of databases or tables) now restrict their content to match the privileges of the logged-in user. #42726
• The event log now presents all cluster settings changes, unredacted, when an admin user uses the page. #42726
• Customization of the UI by users is now only properly saved if the user has write privilege to system.ui (i.e., is an admin user). Also, all authenticated users share the same customizations. This is a known limitation and should be lifted in a future version. #42726
• Access to table statistics are temporarily blocked from access by non-admin users until further notice, for security reasons. #42726
• Certain debug pages have been blocked from non-admin users for security reasons. #42726

Bug fixes

• The TIMESTAMP data type is now fully supported by the vectorized engine. Previously, an "unsupported type" error was thrown. #42576
• Previously, casting the same type into two or more different precisions/widths from a table in the same SELECT query would result in only the first precision specified. For example, SELECT a::decimal(10, 3), a::decimal(10, 1) FROM t would return both results as a::decimal(10, 3). This PR fixes that behavior. #42578
• Fixed a bug that would produce a spurious failure with the error message "incompatible COALESCE expressions" when adding or validating MATCH FULL foreign key constraints involving composite keys with columns of differing types. #42651
• It is now possible to transfer range leases to lagging replicas. #42761
• Long running transactions which attempt to TRUNCATE can now be pushed and will commit in cases where they previously failed or retried forever. #42772
• Fixed a case in which CockroachDB incorrectly determined that a query (or part of a query) that contains an IS NULL constraint on a unique index column returns at most one row, possibly ignoring a LIMIT 1 clause. #42791
• ALTER INDEX IF EXISTS no longer fails when using an unqualified index name that does not match any existing index. Now it is a no-op. #42838
• Fixed a bug with incorrect handling of Top K sort by the vectorized engine when K is greater than 1024. #42843
• Prevent internal error in some cases when a NULL literal is passed to the OVERLAPS operator #42878
• Fixed a bug that caused rare crashes when using built-in functions. #42674
• Other callers to acquireNodeLease now do not get erroneously cancelled just because the context of the first caller was cancelled. #42893
• Changefeeds now emit backfill row updates for a dropped column when the table descriptor drops that column. #42669
• Tables that are being restored or imported are no longer considered valid targets in backups and changefeeds. #42735
• Fixed a bug where selecting columns by forcing an INTERLEAVING index would return an error instead of returning the correct results. #42901
• CockroachDB now prevents a number of panics from the SQL layer caused by an invalid range split. These would usually manifest with messages containing encoding errors like "found null on not null column". #42859
• Fixed the row count estimate during query planning for some queries with multiple predicates in which the selectivity of one predicate was calculated using a histogram. #42962
• Previously, when executing a built-in function via the vectorized engine that returns an output type unsupported by the vectorized engine, CockroachDB could return an error instead of falling back to the row-by-row execution engine. This is now fixed. #42990
• Empty arrays are now correctly encoded and decoded over the binary protocol. #42974
• Fixed some existing caching issues surrounding role memberships, where users could sometimes see out-of-date role membership information. #43004
• Previously, attempting to parse 0000-01-01 00:00 when involving time did not work as pgdate does not understand 0000 as a year. This is now fixed. #42918
• The allocator now considers stores with very close diversity scores equal (all other things being the same) and doesn't attempt to rebalance. #43077
• The allocator now considers the new store being added when looking for a target in the case of a rebalance. #43077
• CockroachDB did not previously handle date casts from TIMESTAMP/TIMESTAMPZ with time attached for times before the UNIX epoch correctly. For example, '1969-12-30 01:00:00'::timestamp would round to '1969-12-31' instead of '1969-12-30'. This is now fixed. #43022
• CockroachDB is now less likely to hang in an inconvenient or inoperative state if it attempts to access an external HTTP server that is blocked or overloaded. Nodes failing to shut down with cockroach quit were a symptom of this bug. #42539

v19.2.1

Release Date: November 25, 2019

This page lists additions and changes in v19.2.1 since v19.2.0.

• For a comprehensive summary of features in v19.2, see the v19.2 GA release notes.
• To upgrade to v19.2, see Upgrade to CockroachDB v19.2

Downloads

Docker image

$ docker pull cockroachdb/cockroach:v19.2.1

SQL language changes

• Dropping a unique index that was created with CREATE UNIQUE INDEX no longer requires the CASCADE option. #42413
• The vectorized execution engine now supports the UUID and TIMESTAMP data types. #42414 #42416
• The vectorized execution engine now supports IS NULL and IS NOT NULL operators. #42433

Command-line changes

• When restarting a node, CockroachDB now provides a hint to ensure that all cockroach start commands include the --join flag. #42325

Bug fixes

• Reduced the likelihood of out-of-memory errors during histogram collection. #42370
• Fixed an out-of-memory error that could occur when collecting statistics on tables with a string index column. #42380
• Fixed some casts from OID to TEXT. #42257
• OUTER HASH joins are now executed correctly via the vectorized engine when NULL values are present. #42438
• Fixed a bug in the crdb_internal.encode_key() built-in function that could cause SHOW RANGE FOR ROW statements on tables with dropped indexes to return incorrect results or an error. #42461
• Prevented memory leaks when counting rows during BACKUP. #42535

Performance improvements

• The ./cockroach debug check-store command is now faster. #42150
• Improved statistics estimation during query planning for columns with many NULL values. #42436

Contributors

This release includes 12 merged PRs by 8 authors.

v19.2.0

Release Date: November 12, 2019

With the release of CockroachDB v19.2, we’ve made a variety of performance, resiliency, and usability improvements. Check out a comprehensive summary of the most significant user-facing changes and then upgrade to CockroachDB v19.2. You can also read more about these changes in the v19.2 blog post or watch our 19.2 release demo and overview.

Downloads

Docker image

$ docker pull cockroachdb/cockroach:v19.2.0

Summary

This section summarizes the most significant user-facing changes in v19.2.0. For a complete list of features and changes, including bug fixes and performance improvements, see the release notes for previous testing releases.

• CockroachCloud
• Core features
• Enterprise features
• Backward-incompatible changes
• Known limitations
• Documentation

CockroachCloud

CockroachCloud will soon offer a completely self-service account creation and management workflow as well as single-region cluster creation in AWS and GCP using a credit card. Sign up for the beta program.

Core features

These features are freely available in the core version and do not require an enterprise license.

Enterprise features

These features require an enterprise license. Register for a 30-day trial license here, or consider testing enterprise features locally using the cockroach demo CLI command, which starts an in-memory CockroachDB cluster with a temporary enterprise license pre-loaded.

Backward-incompatible changes

Before upgrading to CockroachDB v19.2.0, be sure to review the following backward-incompatible changes and adjust your application as necessary.

• The cockroach zone CLI command has been removed. Use the equivalent CONFIGURE ZONE and SHOW ZONE CONFIGURATIONS SQL statements for managing replication zones instead.

• The IMPORT SQL statement no longer accepts quotes inside unquoted CSV fields.

• CONFIGURE ZONE SQL statements now fail if the user does not have sufficient privileges. If the target is a system range, the system database, or a table in the system database, the user must have an admin role. For all other databases and tables, the user must have the CREATE privilege on the target database or table.

  This change might be backward-incompatible for users running scripted CONFIGURE ZONE statements with restricted permissions. To add the necessary permissions, use GRANT <privileges> or GRANT <roles> as a user with an admin role. For example, to grant a user the admin role, run GRANT admin TO <user>. To grant the CREATE privilege on a database or table, run GRANT CREATE ON [DATABASE | TABLE] <name> TO <user>.

• INT and FLOAT columns of less than the max width will now be returned as their own type via the binary protocol. For example, an int4 column will be returned in 32 bits over the pgwire binary protocol instead of 64 bits.

• The changefeed.min_high_water metric has been removed from the Prometheus Endpoint. For detecting high changefeed latency, use the changefeed.max_behind_nanos metric instead.

Known limitations

For information about new and unresolved limitations in CockroachDB v19.2, with suggested workarounds where applicable, see Known Limitations.

Documentation

v19.2.0-rc.4

Release Date: November 7, 2019

Downloads

Docker image

$ docker pull cockroachdb/cockroach-unstable:v19.2.0-rc.4

SQL language changes

• CockroachDB now uses the zig-zag join algorithm only in cases where it can benefit performance. #42186
• Added the sql.defaults.zigzag_join.enabled cluster setting, which can be used to control the default value of the enable_zigzag_join session variable and, thus, whether zig-zag joins are allowed. #42182
• The error message generated when a transaction containing DDL is both partially committed and partially rolled back (XXAOO) now contains a link to GitHub issue #42061 where this situation is discussed further. #42090

Bug fixes

• CockroachDB now supports GSSAPI authentication from Postgres 11.5 and 12.0 clients. #42189
• Fixed a "cannot map variable to an indexed var" error for certain queries against partitioned tables. #42183
• Fixed "output columnTypes unset after planning" and "inconsistent post-processing" errors for certain queries involving UNION ALL. #42184
• Fixed an "invalid indexIdx" error for certain queries involving zig-zag joins. #42182

Contributors

This release includes 6 merged PRs by 3 authors.

v19.2.0-rc.3

Release Date: November 4, 2019

Downloads

Docker image

$ docker pull cockroachdb/cockroach-unstable:v19.2.0-rc.3

General changes

• Go 1.12.12 is now required to build CockroachDB from source. #41994

SQL language changes

• Changed the data type of the system.reports_meta.generated and system.replication_constraint_stats.violation_start columns from TIMESTAMP to TIMESTAMPTZ. #41915

Bug fixes

• Fixed a rare corruption bug when files are ingested that do not overlap with the in-memory buffer and compaction has fallen behind. #42009
• CockroachDB no longer erroneously transfers leases to replicas that are in the process of being removed, which could lead to ranges being unavailable due to an invalid lease. #42079

Doc updates

• Added a CockroachDB Performance page describing the performance profiles you can expect (scaling, throughout, and latency), based on Cockroach Labs' extensive testing using industry-standard benchmarks like TPC-C and Sysbench. #5595
• Documented how to run the built-in movr workload. #5713
• Documented how to show comments for virtual tables, including links to further documentation. #5695
• Refreshed the Deploy a Python To-Do App with Flask, Kubernetes, and CockroachCloud. #5694

Contributors

This release includes 4 merged PRs by 3 authors.

v19.2.0-rc.2

Release Date: October 28, 2019

Downloads

Docker image

$ docker pull cockroachdb/cockroach-unstable:v19.2.0-rc.2

General changes

• Go 1.12.10 is now required to build CockroachDB from source. #41647
• Named ports in URLs are no longer supported due to a URL parsing security fix in Go 1.12.8. Use port numbers instead. #41647

SQL language changes

• Mutations under UNION or UNION ALL are once again supported. #41500
• Added new skip and nullif options for IMPORT ... DELIMITED DATA, which ignore the specified number of lines at the beginning of the file and force all empty string fields to be inserted as NULL in the imported table, respectively. #41656
• It is no longer possible to use IMPORT INTO with an interleaved table. This restriction is temporary and will be lifted in a future release. #41651
• IMPORT is no longer allowed during a rolling upgrade. #41733

Bug fixes

• Fixed multiple bugs relating to COPY FROM PARENT in replication zones. #41677
• The system.replication_stats table no longer incorrectly shows some ranges belonging to table partitions as over-replicated. #41700
• Fixed a bug that resulted in some queries returning an "index join must be against primary index" error. #41723
• Fixed a bug where backfills could lead to certain changefeed row updates never being emitted. #41721
• Fixed a bug that could cause changefeed row updates to be emitted at a timestamp less than or equal to an already forwarded resolved timestamp. #41714
• Fixed a bug in cloud storage sink file naming for changefeeds that could violate ordering in the presence of schema changes. #41693

Performance improvements

• Increased BACKUP parallelism in some cases. #41729
• Improved the performance of generating the system.replication_* tables for large clusters. #41757
• Improved performance of bulk ingestion. #41768

Doc updates

• Clarified that to upgrade to the next major version upgrade path for rolling upgrades, a cluster must be on a production release of the previous version, as opposed to a testing release (alpha/beta). #5644
• Documented the experimental SHOW RANGE FOR ROW statement, which shows information about a range for a particular row of data. This information is useful for verifying how SQL data maps to underlying ranges, and where the replicas for a range are located. #5612
• Improved the usability of the tutorial on deploying a Python To-Do app with Flask, Kubernetes, and CockroachCloud. #5694
• Updated Helm-specific instructions for Kubernetes version 1.16.0 and fixed various errors. #5657, #5659
• Documented the FOR UPDATE locking clause, which is supported in selection queries for database migration compatibility. #5658
• Documented using timezone() and AT TIME ZONE functions to convert a TIMESTAMPTZ into a TIMESTAMP, or vice versa, in a specified timezone. #5614
• Clarified the required privileges for the CONFIGURE ZONE statement. #5609

Contributors

This release includes 32 merged PRs by 19 authors.

v19.2.0-rc.1

Release Date: October 21, 2019

Downloads

Docker image

$ docker pull cockroachdb/cockroach-unstable:v19.2.0-rc.1

SQL language changes

• INTERVAL(6) is now supported as an alias for INTERVAL, for compatibility with third-party tools. #41447
• Added the pg_available_extensions table, for compatibility with third-party tools. #41447
• The pg_attrdef() and pg_get_indexdef() built-in functions no longer include type annotations on default expressions, and pg_get_indexdef() now includes the index method in the returned CREATE INDEX statement, for compatibility with third-party tools. #41447

Bug fixes

• The generate_subscripts() built-in function now behaves properly on int2vector and oidvector types. #41447
• Fixed a bug causing the cluster_logical_timestamp() built-in function to sometimes return incorrect results. #41439
• Prevented an unlikely race condition where a WAL pending recycling could be deleted, leading to a crash. This bug was present in past 19.2 alphas/betas only, not in any earlier release. #41523

Performance improvements

• Metadata queries that use the 'string'::regclass form are now much more efficient in the presence of a large number of tables. #41447
• The pg_get_coldesc() built-in function now retrieves comments more efficiently. #41447

Doc updates

• Clarified epoch-based vs. expiration-based leases in the Replication Layer architecture documentation. #5578
• Updated the list of Enterprise features. #5599
• Documented the new --geo-partitioned-replicas flag of the cockroach demo command, which starts a 9-node in-memory cluster with the Geo-Partitioned Replicas data topology applied to the movr database. This lets you quickly simulate one of the most effective data topology patterns for getting low latency in a multi-region cluster. #5587
  

Contributors

This release includes 12 merged PRs by 10 authors.

v19.2.0-beta.20191014

Release Date: October 14, 2019

Downloads

Docker image

$ docker pull cockroachdb/cockroach-unstable:v19.2.0-beta.20191014

SQL language changes

• The new SHOW RANGE ... FOR ROW statement identifies the range containing a specific row of a table or secondary index. This is particularly useful for validating the location of a specific piece of data when using partitions to control data location for improved performance or GDPR compliance. Note that this new statement is experimental and its interface and output are subject to change. #40954
• The SHOW PARTITIONS statement now displays inherited as well as explicitly set zone configurations for partitions. Also, it is now possible to retrieve full zone configurations via the full_config_sql column in the crdb_internal.zones table, and the new zone_id and subzone_id columns in the crdb_internal.partitions table point to partitions' corresponding zone configurations in crdb_internal.zones. #40493

Command-line changes

• When vmodule is set to exec_logs=2 to log queries executed by a node, CockroachDB-generated statements are no longer logged alongside client-generated statements. To log CockroachDB-generated statements as well, you must now use exec_logs=3. #41149

Bug fixes

• Fixed a bug that would cause queries combining SQL mutation statements in sub-queries inside UNION to return invalid results or to modify the wrong rows if the transaction is forced to retry. #41307
• Fixed a bug that prevented changes to the kv.bulk_io_write.concurrent_addsstable_requests cluster setting from taking effect. #41195
• Fixed a memory leak that affected secondary logging (SQL audit logs, statement execution, and RocksDB logging). #41231
• Fixed a bug where zone configuration changes on tables with existing index zone configurations would not take effect unless the num_replicas field was also set. #41261
• Fixed a bug where zone configuration changes on secondary indexes wouldn't perform setting verification, such as constraints with lease preferences or setting only one of min_range_size and max_range size. #41216
• Fixed a rare data corruption bug in RocksDB caused by newer Linux kernels' handling of i_generation on certain filesystems. #41346

Performance improvements

• Stabilized performance of IMPORT and RESTORE by changing the default value of the rocksdb.ingest_backpressure.pending_compaction_threshold cluster setting from 64 GiB to 2 GiB. #41263

Doc updates

• Expanded the architecture documentation to describe the new Parallel Commits protocol, which cuts the commit latency of a transaction in half, from two rounds of consensus down to one. #5394
• The tutorial on getting Low Latency Reads and Writes in a Multi-Region Cluster now features two of the most important multi-region data topologies, Geo-Partitioned Replicas and Duplicate Indexes, to dramatically reduce the impact of network latency. #5558
• Documented the new cockroach start-single-node cli command, which starts a single-node cluster with replication disabled for quick SQL testing or app development. #5445
• Documented how to use SPLIT AT ... WITH EXPIRATION SQL statement to set an expiration on a range split as well as how to remove a split enforcement via UNSPLIT AT. #5360
• Updated hardware recommendations in the Production Checklist. #5472
• Clarified the benefit of merge joins over hash joins. #5464
• Updated the recommendations for improving IMPORT performance. #5452
• Documented the ALTER COLUMN ... SET NOT NULL SQL statement, which adds a non-null constraint to an existing column. #5456
• Documented how to use the pg_collation_for() built-in function, or its alternative syntax form COLLATION FOR, to return the locale name of a collated string. #5576
• Documented how to use SHOW BACKUP SCHEMAS SQL statement to list the schemas of enterprise backups alongside their contents. #5455
• Updated the SQL statements that are allowed as row sources for table expressions.
• Documented how to use the SHOW JOBS WHEN COMPLETED SQL statement to list only just that have completed. #5395
• Documented the skip_missing_views option for the RESTORE SQL statement, which restores a table with a view but does not restore the view's dependencies.
• Documented the --display-every and --display-format flags for the cockroach workload cli command, which control the frequency and format of per-operation statistics. #5424
• Documented the naming requirements for CSR names when orchestrating CockroachDB with Kubernetes. #5422
• Documented the --watch flag for the cockroach sql cli command, which repeats the SQL statements specified with --execute or -e until a SQL error occurs or the process is terminated. #5543
• Documented how to use the JDBC driver to retrieve large data sets in chunks using cursors. #5544
• Clarified that CHANGEFEED cursors must be in quotes. #5529
• Documented how to show the locality of a node using the crdb_internal.locality_value() built-in function. #5540
• Documented how to reference a common table expression (CTE) multiple times in a single query, using a WITH operator. #5485

Contributors

This release includes 57 merged PRs by 23 authors.

v19.2.0-beta.20190930

Release Date: September 30, 2019

In addition to bug fixes and various general, enterprise, SQL, and command-line enhancements, this release includes several major highlights:

• Locality-aware backup and restore: You can now back up or restore a subset of a table's data according to zone configurations (e.g., send your EU backup partition to an EU cloud storage location).
• Import performance improvements: Internal improvements have been made to IMPORT to enable more efficient loading of large data sets.
• Show partitions using SQL: CockroachDB now supports the SHOW PARTITIONS statement, which can be used to identify partitions. It also enhances SHOW CREATE TABLE to show constraints as described.
• Vectorized Execution: Queries are now automatically faster for users. This work expands upon the SQL vectorized execution prototype, and allows it to be used in production. Note that this does not cover the long-tail of features necessary to use vectorization on all queries. Vectorized execution is turned on by default for a subset of queries and defaults back to DistSQL execution for unsupported queries.
• Improve Streaming Behavior on JDBC Driver: PostgreSQL wire protocol cursors have been implemented. As a result, PGJDBC is able to use streaming behavior to return pagination-related results. Limitations still apply. For more information, see Build a Java App with CockroachDB.
• Improve partitioning user experience: CockroachDB now prevents users from specifying a non-partitioned secondary index on a partitioned table by default to improve performance. Users will have the ability to override this default as it may be necessary in some rare use cases. Additionally, users can now specify the same name for the partition for tables and for all corresponding indexes.
• Improve setting of zone configs via SQL: ALTER TABLE now returns an error when users select conflicting or impossible zone constraints due to user error.
• Make it easy to focus queries on partitioned data: CockroachDB now automatically limits a query to only the applicable partition without having to specify the particular partition for a geo-distributed table. This prevents the query from conducting a full table scan.
• Better insight into constraint conformance: You can now tell if your cluster is in a healthy state based on the status of conformance to constraints like zone configurations, lease preferences, and replica diversification.

Downloads

Docker image

$ docker pull cockroachdb/cockroach-unstable:v19.2.0-beta.20190930

General changes

• Replicas are now added using a Raft learner and go through the normal Raft snapshot process to catch them up, eliminating technical debt. No user-facing changes are expected. #39034
• Atomic replication changes are now enabled by default. #40464
• RocksDB INFO logs are now stored with CockroahDB debug logs. #40681

Backward-incompatible changes

• Quotes are no longer accepted inside unquoted CSV fields. #40424
• CONFIGURE ZONE statements now fail if the user does not have sufficient privileges. If the target is a system range, the system database, or a table in the system database, the user must have an admin role. For all other databases and tables, the user must have the CREATE privilege on the target database or table. This change might be backward-incompatible for users running scripted CONFIGURE ZONE statements with restricted permissions. To add the necessary permissions, use GRANT <privileges> or GRANT <roles> as a user with an admin role. For example, to grant a user the admin role, run GRANT admin TO <user>. To grant the CREATE privilege on a database or table, run GRANT CREATE ON [DATABASE | TABLE] <name> TO <user>. #40879

Enterprise edition changes

• Added support for client certificates for Kafka changefeed authentication. #39832
• Added support for locality-aware backups across multiple backup destinations. #39250
• Added support for displaying creation statements of relations stored in a backups. #39323
• Table statistics for the tables that are being backed up are now also backed up. #39644
• Corrected behavior for unmatched quotes in enclosed fields. #40960
• Added auth=implicit setting, which removes access key requirement when listing S3 files for bulk I/O operations. #40836

SQL language changes

• Zone configurations now error out when they conflict with each other. #39081
• Renamed SHOW EXPERIMENTAL_RANGES to SHOW RANGES command, as it is no longer an experimental feature. #39153
• Added the SHOW RANGES FROM DATABASE command. #39194
• Added partial support for row limits during portal execution in pgwire. #39085
• Added the SHOW LOCALITY command. #39058
• Added the SHOW PARTITIONS command. #39053
• CREATE VIEW now supports all queries supported by the optimizer, including those containing correlated subqueries. #39169
• Mutations are no longer allowed inside views. #39169
• Added the COLLATION FOR syntax and pg_collation_for() built-in function. #39198
• Implemented the timezone() built-in and AT TIME ZONE syntax, which allows converting a timestamptz into a timezone at a specified timezone, and converting a timestamp into a timestamptz at a specified timezone. #39186
• The new vectorize_row_count_threshold cluster setting controls when the vectorized execution engine is used. #39329
• Added a VEC option to the EXPLAIN command, which shows detailed plan information for vectorized plans. #39280
• Partition names can now be reused between different indexes on the same table. #39332
• Columns without an explicit default value now support foreign keys with the SET DEFAULT action, in the same way as the SET NULL and SET DEFAULT NULL cases. #39136
• Added SHOW JOB <jobid>, SHOW JOB <a_expr>, and SHOW JOBS <selectclause>. #39324
• Added SHOW JOB WHEN COMPLETE <a_expr> and SHOW JOBS WHEN COMPLETE <selectclause>. #39457
• IMPORT INTO now cleans up any imported rows if it fails. #39459
• Updated SHOW CREATE to display information about partition and table zone configurations. #39102
• Explicitly closing portals over the wire after partial use is now supported. #39639
• Added progress tracking for new implementation of IMPORT. #39646
• CockroachDB now supports an optional frame exclusion clause in the specification of window frames. #28262
• Enabled collection of histograms by default during automatic statistics collection. #39743
• The force_retry() function, which previously required admin privileges, is now available to all users to test their transaction retry loops. #39246
• When DROP CONSTRAINT is executed in a transaction on a validated constraint, it no longer takes effect immediately, and subsequent writes will now fail if they violate that constraint. This is to enforce consistency when dropping constraints asynchronously in the schema changer. #39648
• The ALTER PARTITION statement now supports applying a zone configuration to all the partitions of a table and its indexes that share the same partition name. The syntax for this is ALTER PARTITION <partition name> OF INDEX <table name>@*. #39750
• CockroachDB now supports the FOR UPDATE modifier on SELECT statements for compatibility with third-party tools. It has no effect because SERIALIZABLE transactions do not require explicit write locks to maintain consistency. However, this means that SELECT FOR UPDATE cannot be used as an advisory exclusive lock in CockroachDB, as used by some tools.#40206
• SHOW PARTITIONS now shows the full zone configuration statement used to configure the partition. #39814
• Added the pg_my_temp_schema() built-in for Postgres compatibility. #40194
• Added the overlaps (&&), contains (@>), and contained_by (<@) operators for arrays. #40194
• Added pg_get_function_identity_arguments() built-in. #40194
• Added the width_bucket() builtin function. #39263
• Added range size information to SHOW RANGES. #40281
• Added replica locality information to SHOW RANGES. #39813
• Reordered columns in SHOW RANGES output #40501
• SHOW CREATE TABLE now warns users if they have a partitioned table that does not contain zone configurations #40475
• Added the replication_constraint_stats, replication_stats, and replication_critical_localities system tables. These tables contain information about replication status, constraint conformance, and critical localities. #40625
• Common table expressions that are used a single time no longer create an optimization fence. #40673
• EXPLAIN now has additional output that shows if a query will run with the vectorized execution engine, and if the query will be distributed across nodes. #40652
• Column references are no longer allowed in ROWS and RANGE clauses in window functions. #40832
• Added the pg_function_is_visible and pg_get_function_result built-in functions. #40902
• Added parsing for FOR UPDATE [ OF tables ] form. #40962
• Mutations under UNION or UNION ALL are now disallowed; WITH should be used on top of the union operation instead. This restriction is temporary and will be lifted in a future release. #40975
• Added support for the syntax for oid wrapper arrays, like REGCLASS[]. #40949
• EXPLAIN ANALYSE can now be used as an alias to EXPLAIN ANALYZE. #41090
• SHOW PARTITIONS now displays inherited zone configurations. #40493
• Added the zone_id and subzone_id columns to crdb_internal.partitions. These columns form a link to the zone configurations in crdb_internal.zones that apply to the partitions. #40493
• Renamed the config_yaml, config_sql, and config_proto columns in crdb_internal.zones to raw_config_yaml, raw_config_sql, and raw_config_proto. #40493
• Added the columns full_config_sql and full_config_yaml to thecrdb_internal.zones table. #40493
• Added FOR ROW clause to SHOW RANGES to show range information for a particular row. #40954

Command-line changes

• cockroach demo now accepts a --nodes flag, which permits customization of the size of the demo cluster. The default remains at 1. #39088
• cockroach demo now accepts a --demo-locality flag, which makes it possible to set node localities. #39454
• cockroach demo without an argument is now equivalent to cockroach demo movr. The previous behavior (no pre-defined dataset) is still available via cockroach demo --empty. #39298
• cockroach start now writes the client URL to the file specified via --listen-url-file as soon as the server is ready to accept connections. This also happens before the server detaches from the terminal when --background is specified. #39300
• Using cockroach start without --join is now deprecated and this mode of execution will be removed in a later version of CockroachDB. Consider using cockroach start-single-node instead or combine cockroach start with cockroach init. #28495
• The advisory/informative check performed by cockroach start on the validity of addresses contained in the node certificate is now relaxed to focus on the advertised node address, and to tolerate cases when the cert contains an IP address but a hostname is specified as advertised address. #39166
• Added locality information to the output of cockroach node status. #39059
• Removed the deprecated cockroach zone CLI commands. Please use the equivalent SHOW ZONE CONFIGURATIONS and CONFIGURE ZONE commands in a SQL client. #39177
• Added a new cockroach start-single-node command to start single-node clusters with replication disabled. #28495
• The cockroach user sub-commands are now deprecated. Users and roles can be managed using SQL statements instead. This functionality will be removed in the next release. #39297
• cockroach start now accepts a --cluster-name flag. If specified, it must match when a node connects to a cluster. This feature is meant for use in combination with the cluster ID, which is still checked in any case, to protect newly created nodes (that do not have a cluster ID yet) from joining the wrong cluster. To introduce a cluster name into an existing cluster without one, the new flag can be temporarily paired with --disable-cluster-name-verification. #39270
• cockroach start now accepts a --sql-addr flag, which makes it possible to accept connections by clients on a separate TCP address and/or port number from the one used for intra-cluster (node-node) connections. This is aimed to enable firewalling client traffic from server traffic. #39305
• cockroach workload run and cockroach workload init now support loading initial table data using IMPORT. #35312
• cockroach workload now supports additional command-line parameters to customize the output, to facilitate the integration with third-party testing tools:
  • For tools that wish to observe the metrics more frequently than every second, a new flag --display-every is now supported, which can be used to specify the period between metric reports. This applies to both the JSON and textual output.
  • For tools that require a different output format than the default, a new --display-format argument is supported. For now only the formats "simple" (original output format) and "incremental-json" (RFC3339 timestamps, no summary row) are supported. #37929
• Added d, dt, du, l metacommands to the cockroach sql interactive shell. #39141
• Added default cluster locality topologies for multi-node cockroach demo clusters. #40221
• Errors in cockroach demo are no longer logged to the command-line interface. #40196
• Added d client-side command with no arguments as an alias for dt, for more ease of adoption by psql users. #40309
• Added a MovR workload to run with cockroach demo. #40271
• Added location information for default localities to populate Admin UI for cockroach demo. #40229
• cockroach demo now attempts to contact a license server to obtain a temporary license for demo usage. #40273
• cockroach demo now enables telemetry for demo clusters. You can opt out of telemetry by setting the COCKROACH_SKIP_ENABLING_DIAGNOSTIC_REPORTING environment variable. For more information, see Diagnostics Reporting. #40273
• Deprecated the --init flag on the workload run command. Use workload init instead. #40426
• Added support for a new DELIMITED data format to the IMPORT statement. #40482
• Added an option for cockroach demo to automatically apply the geo-partitioned replicas topology to the movr dataset using the --geo-partitioned-replicas flag. #40355
• cockroach sql now supports the --watch flag. When used to specified a duration, the client will repeat the statement(s) specified by -e until an error occurs. This is intended for simple monitoring scenarios during development and testing. #40594
• Added support for redirecting license acquisition to a replacement server (e.g., a suitably configured HTTP proxy), with the environment variable COCKROACH_DEMO_LICENSE_URL. This enables cockroach demo with enterprise features in firewalled network environments. #41029

Admin UI changes

• Metrics about p90 and p99 percentiles of per-application average transaction time are now exposed in the Admin UI. #39429

Bug fixes

• Fixed a potential infinite loop in queries involving reverse scans. #39084
• Upgraded the grpc library to fix connection state management bug. #39041
• Properly initialize proposal quota tracking to prevent quota leak which can hang imports or other AddSSTable operations. #39135
• crdb_internal.ranges can now be used inside views. Note that such views can become invalid in future releases if crdb_internal.ranges changes. #39195
• Prevented panic due to recording into finished tracing spans caused by acknowledging an earlier failed re-proposal when a later re-proposal exists. #39203
• CockroachDB now properly computes the default port number if an IPv6 address is provided without port number with --locality-advertise-addr. #39334
• Unary negatives in constant arithmetic expressions are no longer ignored. #39245
• pg_catalog now shows correct type lengths for INT and FLOAT. #39337
• Fixed a rare panic (message: "raft group deleted") that could occur during splits. #39571
• Fixed a panic due to incorrect type inference of some ARRAY(...) expressions. #39469
• Zone configurations now propagate to non-gossiped system tables. #39638
• cockroach dump now works properly when handling INTERLEAVED tables, printing them outside of CREATE TABLE statements. #39486
• Fixed internal errors generated during execution of some complicated cases of correlated subqueries. #39818
• Using ORDER BY inside aggregation functions is now fully supported. #40242
• Lookup joins now spill to disk if the index lookups return more rows than can be stored in memory. #40208
• Fixed an internal planning error that could occur when a DISTINCT or GROUP BY expression was contained in a subquery. #40468
• Fixed an optimizer panic when building array access expressions. #40469
• Fixed faulty error when trying to delete a table and an unrelated index in the same transaction. #40516
• ordinal_position in information_schema.columns now matches pg_attribute.attnum after a column is dropped. #40485
• Fixed fatal error with message replica descriptor of local store not found. #39796
• Intents in a read's uncertainty interval are now considered as uncertain as committed values. This removes the potential for stale reads when a causally dependent transaction runs into unresolved intents from a causal ancestor. #40600
• Fixed a cluster migration bug that could occur in clusters running a mixture of v19.1 and v19.2. This bug was signaled by messages of the form: X at key /Table/54 failed: unexpected value: ... Affected clusters should be updated to 19.2 or, if 19.1 is preferred, recreated from backup. #40464
• Fixed behavior of SERIAL columns on mixed-case tables when experimental_serial_normalization=sql_sequence. #40965
• width_bucket function no longer errors on 0-length arrays #40626
• Fixed an error that could occur when the optimizer creates a plan with a lookup semi- or anti-join nested inside another join. #40669
• Fixed a memory corruption error in the optimizer that could cause stack overflow. #40644
• The cockroach commands are now able to connect to a server with the environment variable COCKROACH_URL. #40824
• Index hints used with DELETE and UPDATE are now taken into consideration. #40715
• Fixed an issue with creating table indexes when the server is running on Windows and the store is on a file system that does not support hard links (e.g., FAT32 or network share) #40875
• Restore correct result generation for queries with index disjunctions on tables with multiple column families. #40898
• Fixed a stack overflow that could occur via use of WITH. #40937
• Detailed crash reports now always appear in the main log file. If SQL audit reporting or statement logging was activated, crash reports (i.e., "panic messages") could previously be reported in the wrong file. #40941
• Users must now have admin privileges to cancel, pause, or resume jobs. #40978
• CockroachDB now properly removes excess secondary log files (SQL audit logging, statement execution logging, and RocksDB events). #41020
• CockroachDB now avoids the internal re-use of Replica objects to fix the following crashes:
  • #38772 "found rXXX:{-} [, next=0, gen=0?] in place of the RHS"
  • #39796 "replica descriptor of local store not found in right hand side of split"
  • #40470 "split trigger found right-hand side with tombstone"
  • #40257 "snapshot widens existing replica, but no replica exists for subsumed key" #40892
• Fixed panic in cockroach workload movr run. #41072
• Resolved an issue with server hanging on cockroach quit on Windows. #40994
• The cockroach zip, cockroach node and cockroach user cockroach commands now work properly when the defaultdb database is manually dropped and the connection URL does not specify a database. cockroach user is deprecated in 19.2. #41121
• cockroach workload init ycsb now works with --data-loader=import. #41071
• Fixed bug where zone configurations on indexes could leak into configurations on partitions. [#41089][#41089
• Statements inside [ ... ] can no longer refer to outer columns or common table expressions. #41158
• The SQL for non-SELECT common table expressions now shows up in EXPLAIN output. #41159
• Fixed bug that prevented changes to kv.bulk_io_write.concurrent_addsstable_requests from taking effect. #41195
• Fixed a memory leak that affected secondary logging (SQL audit logs, statement execution, and RocksDB logging). #41231

Performance improvements

• Sped up file-writing during bulk-ingestion. #37528
• Snapshots sent between replicas are now applied more performantly and use less memory. #38932
• Raft entries no longer wait to be applied to the RocksDB storage engine before signaling their success to clients, they now only wait until they are committed in their Raft log. #38954
• Adjusted load-based splitting QPS threshold to avoid over-splitting. #39687
• IMPORT now writes unsorted data directly, reducing IMPORT's overhead (but potentially increasing RocksDB compaction overhead). #39487
• Improved performance for mutation statements with RETURNING that are not inside an explicit transaction. #40871

Doc updates

• Added an Oracle migration guide, which guides users through the process of converting their schema and then exporting their data so they can load it into CockroachDB via IMPORT. #5902
• Added an overview of MovR, CockroachDB's fictional vehicle-sharing dataset and application, and updated several SQL pages and examples to use the built-in MovR dataset, for example, Learn CockroachDB SQL. #5075, #5216
• Added a tutorial on how to use EXPLAIN to identify and resolve common SQL performance problems. #5178

v19.2.0-alpha.20190805

Release Date: August 5, 2019

Downloads

Docker image

$ docker pull cockroachdb/cockroach-unstable:v19.2.0-alpha.20190805

Enterprise edition changes

• Added a skip_missing_views option for RESTORE to skip restoring views which cannot be restored because their dependencies are not being restored at the same time. #38597

SQL language changes

• Added support for a table reader that performs a loose index scan over the underlying table. The index scan table reader uses information about the index being scanned to skip unnecessary rows while scanning the table, allowing for some optimizations to be used for some types of queries. #38216
• Enabled automatic detection of compression schemes on filenames in IMPORT that have parameters after the path. #38521
• Added check_constraints table to the information_schema database. #38545
• Added vectorized support for the IN selection operator. #38546
• Made some pgwire error codes for binary parameter decoding better match Postgres. #38365
• Added support for projection of the IN operator. #38613
• EXPLAIN returns more information about lookup joins. #38748
• The first statement of a transaction will no longer return a transaction retry error if it is an UPDATE or DELETE (this was already true for INSERT). #38668
• Non-fatal errors updating jobs while dropping a table are now ignored. #38617
• Added support for NOT VALID option for Foreign Key constraints. #38663
• Added cluster setting sql.stats.histogram_collection.enabled to enable collection of histograms during statistics collection. #38812
• Common Table Expressions (CTEs) may now be referenced from multiple locations in a query. #38670
• CREATE TABLE AS now allows user-defined primary keys and column families. #38904, #38589
• EXPLAIN (OPT,ENV) now returns a URL with the data encoded in the fragment portion. Opening the URL shows a page with the decoded data. Note that the data is processed in the local browser session and is never sent out. #38973

Command-line changes

• cockroach dump will now more clearly refer to issue 37768 when it encounters a table with no visible columns, which (currently) cannot be dumped successfully. #38725
• CockroachDB now requires square brackets to specify IPv6 addresses in --join, --host, --listen-addr and similar flags, for example --listen-addr=[::1]. The syntax without square brackets had been deprecated since CockroachDB 2.1. #38924

Admin UI changes

• Dashboard graphs now offer a "Last 2 days" option in the time selector. #38852
• Corrected the units displayed for the MVCC Intent Age and the MVCC GC Bytes Age fields in the Range Status page. #38283

Bug fixes

• The ALTER PARTITION x OF INDEX y CONFIGURE ZONE USING syntax no longer erroneously modifies the entire index's zone configuration. #38423
• Fixed an issue that prevented restoring some backups if they included tables that were partitioned by columns of a certain types while also interleaved by child tables. #38341
• Restoring multi-line CLI histories in the SQL shell no longer produces terminal anomalies when the cursor is on the last line of the screen. #38505
• Fixed a panic where generators in an aggregate function were not appropriately rejected. #38554
• Handled database_name and composite primary key for UNSPLIT ALL. #38338
• Ranges consisting of only one row (and historical versions of that row) are now correctly up-replicating. #38529
• Nodes that have been down now recover more quickly when they rejoin, assuming they weren't down for much more than the value of the server.time_until_store_dead cluster setting (which defaults to 5 minutes). #38484
• cockroach sqlfmt no longer strips ORDER BY from aggregates. #38623
• Corrected issue that caused certain plans that contained both offsets and limits to return more rows than desired. #38667
• Fixed a bug that prevented inverted indexes from being created on JSON columns containing NULL values. #38738
• Transaction retries in schema changes are again handled correctly. #38668
• On macOS, the server process can once again start in environments where the kern.maxfilesperproc sysctl has been lowered. #38472
• Fixed a bug where dropping an index that a foreign key depends on could cause an inconsistent table state if the schema change were rolled back. #38783
• Fixed an issue where Cost-based optimizer-planned upserts that used lookup join that were run during column mutations on the table being upserted into could cause crashes or other issues. #38637
• Fixed incorrect results, or "unordered span" errors, in some cases involving exclusive inequalities with non-numeric types. #38880
• Prevented OOM conditions during lookup joins between tables with a very large n:1 relationship. #38837
• sum_int() now errors on overflow instead of silently wrapping. #38921
• Fixed bug in the optimizer causing a bad index for lookup join in some cases. #39016
• Backfills are no longer performed when a user explicitly sets the default value of a new column to be NULL. #38910

Performance improvements

• Introduced new concurrent Raft proposal buffer, which increases the degree of write concurrency supported on a single range. #38343
• Grow stack prior to decoding grpc BatchRequest messages. #38107
• Batched raft entry application and coalesce writes to applied state for the batch. #38568
• Improved performance of the BACKUP process. #37887

Build changes

• Building CockroachDB from source now requires node version 10. This was previously implicitly required. Now the requirement is explicit. #38955
• Building CockroachDB from source now requires Go 1.12 or higher. #38379

Doc updates

• Expanded single-region Kubernetes tutorial to cover running on Amazon’s hosted EKS. #5063
• Build an App with CockroachDB examples now include C++ and Clojure application examples for secure clusters. In addition, the client-side transaction logic has been updated in the Java, Python, and Go code examples. #5028, #5084, #5085, #5088
• Updated Common Errors document with specific error strings to aid user searches. #5033
• Documented a known limitation about hanged requests to a restarted node that needs Raft snapshots. #5018
• Updated the BACKUP and RESTORE docs to specify that the system.users table is not included with a backup. To restore the table, you must explicitly include it. #5072

v19.2.0-alpha.20190701

Release Date: July 1, 2019

This release contains enhancements and bug fixes to the 19.2.0 series, including:

• Experimental support for using IMPORT INTO to ingest CSV data into an existing table. Usage: IMPORT INTO t(a, b) CSV DATA ('gs://acme-co/employees.csv').
• Adding non-null constraints to existing columns with ALTER TABLE ... SET NOT NULL.
• Various updates to improve Postgres compatibility.

For more information, see below.

Downloads

Docker image

$ docker pull cockroachdb/cockroach-unstable:v19.2.0-alpha.20190701

Backward-incompatible changes

• Integer and float columns of less than the max width will now be returned as their own type via the binary protocol. For example, an int4 column will be returned in 32 bits over the pgwire binary protocol instead of 64 bits. #37295

General changes

• Removed kv.range_merge.manual_split_ttl cluster setting. #38004
• Go 1.12.5 is now required to build CockroachDB from source.

SQL language changes

• Added experimental support for using IMPORT INTO to ingest CSV data into an existing table. Usage: IMPORT INTO t(a, b) CSV DATA ('gs://acme-co/employees.csv'). #37451
• Implemented SQL support for timestamp objects to have an optional precision value. #37920
• The logging format for statement execution (as enabled by sql.trace.log_statement_execute) and audit logs (as enabled by ALTER TABLE .. EXPERIMENTAL AUDIT SET ...) is extended with one column that reports internal execution retries performed automatically by the server. The value zero is logged for the first retry attempt and thus corresponds to statements directly received from a client. #38028
• Added WITH EXPIRATION option for SPLIT AT. #38004
• The DISTINCT keyword is now rejected when used in a window function. It was previously accepted, but ignored. #37912
• Added UNSPLIT ALL statement. #38023
• The relam column of the pg_catalog tables pg_class and pg_index is now filled in correctly. #38164
• CockroachDB now supports the special case set timezone = 'utc' as a special alias for set timezone = 'UTC'. The other time zone names are still case-sensitive as previously, pending resolution of issue #36864. #38195
• Added support for ALTER TABLE ... ALTER COLUMN ... SET NOT NULL, which adds a non-null constraint to an existing column. #37554
• pg_get_constraintdef now outputs CHECK constraints exactly the same way that Postgres does. #38318
• The format_type Postgres builtin now properly respects its second argument. #38318
• SHOW CREATE TABLE now prints foreign keys more like Postgres does. #38318
• The pg_type_is_visible builtin from Postgres is now implemented. #38318
• Casting an int2vector to a string now produces a Postgres-compatible result. #38318

Command-line changes

• CockroachDB will now print out an error message and an informative hint if the time zone database is unusable. #38197

Admin UI changes

• The snapshots graph on the Replication page in the Web UI now includes learner snapshots. #38147
• Removed version information from login screen. #38140
• Added Implicit Transaction flag to Statements and Statement Details page. #36341
• Checking the "skip should queue" checkbox in the Manual Enqueue Range advanced debugging page now works for the GC Queue. #38249

Bug fixes

• Fixed help text that erroneously labeled Encryption at Rest as experimental. #38228
• The max_retries column in the statement statistics now properly displays the maximum number of times a statement was retried. #38028
• Previously, due to a bug when estimating result set sizes in the optimizer, queries involving int ranges that were very large could result in poor plans being generated. #38036
• Fixed a bug where MVCC value at future timestamp is returned after a transaction restart. #38085
• The YCSB workload will no longer ignore the --db option. #38236
• Localities in the crdb_internal tables were displayed JSON objects instead of strings. This format stripped them of their order, which is important in localities. This string format now exactly matches what is used in the command line for setting localities. #37944
• Fixed an incorrect type mismatch error when empty array values are used as default values (and potentially in other contexts). #38294
• Fixed a panic that could occur when decoding decimals as query parameters. #38304
• Nulls are now correctly handled by MIN, SUM, and AVG aggregate functions when used as window functions. #38110
• Prevented a possible panic during JSONB binary query parameter parsing. #38346
• Prevented a possible panic during INET binary query parameter parsing. #38346
• Prevented a possible panic during VARBIT binary query parameter parsing. #38346
• Prevented a possible panic during array binary query parameter parsing. #38346
• Fixed possible deadlock when storage engine write fails. #38170
• Fixed potential reappearance of deleted timeseries data, which could trip consistency checker. #38170
• Removed dependency on sync_file_range on Linux platforms on which it returns ENOSYS, such as WSL (Windows Subsystem for Linux). #38170

Build changes

• A recent libedit is now bundled with CockroachDB, which fixes some line editing bugs in the CockroachDB console. On platforms that include libedit as part of the base system, like macOS and FreeBSD, CockroachDB no longer links against the system libedit. #32623

Security

• Only check CN on first certificate in file. #38163

v19.2.0-alpha.20190606

Release Date: June 6, 2019

In addition to various enhancements and bug fixes, with this first alpha release of the v19.2.0 series, we would like to highlight the following changes:

• Parallel Commit: The parallel commit feature is a major optimization that halves the latency of distributed transactions, including those caused by secondary indexes. Previously, transactions would involve two synchronous rounds of replication consensus - one for write operations performed in parallel via transaction pipelining and then another for the commit. Now, replication for the commit runs in parallel with other write operations, avoiding an extra synchronous round trip and dramatically increasing overall transaction performance. This is achieved via a new "implicit" commit status that is distributed yet atomic and durable and that resolves to an "explicit" centralized status quickly for the efficiency of conflicting transactions. Detailed documentation is coming soon; in the meantime, see the Parallel Commit RFC.

• Business Source License (BSL): This release uses the Business Source License instead of the Apache 2.0 License. For more information on why we changed our licensing approach and some practical questions and implications, see our blog post and Licensing FAQs. The full details of the license can be found on Github.

Downloads

Docker image

$ docker pull cockroachdb/cockroach-unstable:v19.2.0-alpha.20190606

Backward-incompatible changes

• Removed the deprecated changefeed.min_high_water metric; use changefeed.max_behind_nanos instead. #36915

General changes

• Added the kv.range_merge.manual_split_ttl cluster setting. #37728
• CockroachDB now uses the Business Source License instead of the Apache License. See our blog post for details. #38007

Enterprise edition changes

• You can now alter the zone configuration for a secondary index partition using the syntax ALTER PARTITION OF INDEX <tablename>@<indexname> CONFIGURE ZONE .... #36868

SQL language changes

• Zigzag joins will now be planned by the optimizer in certain cases. They were previously available via the experimental_enable_zigzag_join session variable and can still be disabled by setting that value to false. #35911
• The LATERAL keyword in a FROM clause is now supported. #36613
• The information_schema.columns table now gives more conformant output for varbit, char, timestamp, and time types. #36598
• Extended the list of advertised OIDs in pg_type to include additional array-related types, such as _int2vector, _regproc, etc. #36598
• CockroachDB now displays correct array type oids for oidvector and int2vector types in the pg_type table, for better PG compatibility. #36598
• CockroachDB now uses the Postgres-compatible name when auto-generating a column alias for a CAST expression. #36598
• Locale names in COLLATE clauses now have dash characters mapped to underscore when printed as part of SQL syntax. #36598
• Parallel statement execution has been removed because the performance improvement it provided was absorbed by transactional pipelining, which doesn't require client buy in. The RETURNING NOTHING syntax will still be accepted but becomes a no-op. #35959
• Only SELECT, INSERT, UPDATE, UPSERT, DELETE, SHOW, EXPLAIN are now supported as data sources using the SELECT ... FROM [ ... ] syntax. #36977
• The atttypmod column of the pg_attribute catalog table is now supported. #37230
• Dates are now fully Postgres-compatible, including support for sentinel values (+/- infinity) and the Postgres date range (4714-11-24 BC to 5874897-12-31). Existing dates outside of this range will be converted to the +/- infinity dates. #36938
• Added an overload for now() that returns a date. #37284
• Added supported for the empty default_tablespace session variable for Postgres compatibility. #37333
• Permitted casting empty arrays to any array type. #37356
• Removed the legacy batch-by-batch index backfiller, as well as the schemachanger.bulk_index_backfill.enabled cluster setting that could be disabled to activate it. #37213
• Removed the transform option from IMPORT. #37338
• Added a new locality_value built-in function that returns the value of the locality key given as its argument. #37369
• CockroachDB now returns NaN instead of an error during invalid decimal operations. #36964
• Foreign keys that are added to an existing table using ALTER TABLE will now be validated for existing rows, with improved performance compared to running ADD CONSTRAINT followed by VALIDATE CONSTRAINT previously. #37433
• SHOW COLUMNS ... WITH COMMENTS now supports printing out column comments using the optional phrase WITH COMMENT, e.g SHOW COLUMNS FROM mytable WITH COMMENT. #37174
• SHOW DATABASES WITH COMMENTS now supports printing out database comments using the optional phrase WITH COMMENT, e.g SHOW DATABASES WITH COMMENT. #37175
• CREATE TABLE statements now always trigger automatic statistics collection on a newly created table. #37611
• Added the split_time column to crdb_internal.ranges and crdb_internal.ranges_no_leases. #37610
• Corrected interval math when multiplying or dividing by floats or ints. #37744
• Added the ALTER TABLE/INDEX ... UNSPLIT AT ... statement. #37603
• Added support for the time value 24:00 and variants. #37898
• CockroachDB now reports a dedicated SQL error code in the particular situation where a multi-statement transaction is committed but one of the schema change (DDL) operations failed. As in previous versions, CockroachDB leaves the transaction partly committed and partly aborted in that case; the new error code aims to facilitate the recognition of this case in client code. #37791
• Added support for a SHOW INDEXES FROM DATABASE command. #37942

Command-line changes

• Included collected goroutine dumps in cockroach debug zip. #36813
• Added the movr workload to the cockroach workload command. This workload simulates a fictional ride sharing company and is based on our open-source MovR GitHub repository. #37709

Admin UI changes

• CockroachDB now includes only successfully executed statements in the statement counters. #37264

Bug fixes

• Fixed a roundtrip typecasting issue where an INET casted to a STRING and back to an INET would fail. #37938
• Fixed a panic when json_build_object is called with BIT/VARBIT values. #37057
• Fixed a panic when planning zigzag joins against unique indexes. #36139
• RESTORE now balances the work of ingesting data more evenly between nodes. #36665
• IMPORT now balances the work of ingesting data more evenly between nodes. #36665
• CockroachDB now computes the result of shifting bit arrays to the right properly and avoids generating invalid bit arrays. #36715
• SHOW ZONE CONFIGURATION no longer emits invalid ALTER syntax in its output when displaying the zone configuration for a table or index partition that is inheriting from the database or the default configuration. #36868
• SHOW ZONE CONFIGURATION FOR TABLE t PARTITION p no longer ignores the clause PARTITION p and now properly displays the zone configuration for that partition instead. #36868
• When a node is restarted with another node's address, CockroachDB now performs fewer attempts to communicate with the wrong node. #34197
• It is now possible to write to columns of type name[]. #37170
• Permitted planning of window functions within mutation statements (and others that cannot be distributed). #37171
• Avoided a source of internal connectivity problems that would resolve after restarting the affected node. #37204
• Permitted arrays and other sequences to be type-checkable in certain contexts when all of their elements are null. #36673
• CockroachDB now correctly escapes non-finite JSON number values when marshaling to text. #37331
• The cockroach init command will now always properly report when a cluster is already initialized, even after the node that it's connecting to is restarted. #37399
• Automated table statistics no longer encounter "batch timestamp must be after replica GC threshold" errors on configurations with low TTL. #36500
• Fixed type inference of columns in subqueries for some expressions of the form scalar IN (subquery). #37578
• Fixed a crash in apply join. #37597
• The age() function is now correctly marked as impure, causing it to be unavailable in certain contexts. #37609
• Fixed a panic when constructing the error message for an invalid partitioning. #37689
• Fixed incorrect multiplication of float and interval types. #37582
• Fixed a case in which cockroach quit would return success even though the server process was still running in a severely degraded state. #37668
• Fixed a potential source of (faux) replica inconsistencies that can be reported while running a mixed v19.1 / v2.1 cluster. This error (in that situation only) is benign and can be resolved by upgrading to the latest v19.1 patch release. Every time this error occurs a "checkpoint" is created which will occupy a large amount of disk space and which needs to be removed manually (see <store directory>/auxiliary/checkpoints). #37668
• Prevented unlimited memory usage during SQL range deletions #36728
• Dividing floats by zero now returns an error instead of -Inf. #37774
• Fixed incorrect results or "incorrectly ordered stream" error in some cases of queries with aggregations. #37713
• A null right operand now causes the sub-operator expression to return null. #37775
• Certain binary encodings of numeric/decimal values no longer result in values that are an order of magnitude off. #35115
• Fixed GROUP BY for empty arrays. #37901
• Fixed a race condition that could cause a panic during query planning. #37972
• The cockroach command line utilities that internally use a RPC connection (e.g., cockroach quit, cockroach init, etc.) once again properly support passing an IPv6 address literal via the --host argument. #37977

Performance improvements

• Improved memory efficiency of bulk ingestion and index backfills buffering. #36743
• Fixed a significant performance regression when building CockroachDB with go1.12. #37967

Security improvement

• Stack memory used by CockroachDB is now marked as non-executable, improving security and compatibility with SELinux. #37939

Was this helpful?