The Migrations page on the CockroachDB Cloud Console features the MOLT Schema Conversion Tool. This tool helps you:
- Convert a schema from a PostgreSQL, MySQL, Oracle, or Microsoft SQL Server database for use with CockroachDB.
- Migrate directly to a CockroachDB Cloud database that uses the converted schema. You specify the target database and database owner when migrating the schema.
Export the converted schema. To migrate to a CockroachDB Self-Hosted database, you can execute the statements in
cockroach sql
, or use a third-party schema migration tool such as Alembic, Flyway, or Liquibase.Note:The Migrations page is used to convert a schema for use with CockroachDB and to create a new database that uses the schema. It does not include moving data to the new database. For details on all steps required to complete a database migration, see the Migration Overview.
To view this page, select a cluster from the Clusters page, and click Migration in the Data section of the left side navigation.
Required privileges
To access the Migrations page, as user must have either of the following roles:
Convert a schema
The steps to convert your schema depend on your source dialect.
- With the Schemas tab open, click the Add Schema button. This opens the Add SQL Schema dialog.
- In step 1 of the Add SQL Schema dialog, select the appropriate Dialect from the pulldown menu.
- Configure the following defaults for schema conversion:
- INT type conversion: On CockroachDB,
INT
is an alias forINT8
, which creates 64-bit signed integers. On PostgreSQL,INT
defaults toINT4
. For details, see Schema design best practices. - Casing of Identifiers: Select Keep case sensitivity to enclose identifiers in double quotes, and Make case insensitive to convert identifiers to lowercase. For details on how CockroachDB handles identifiers, see Identifiers.
- AUTO_INCREMENT Conversion Option: We do not recommend using a sequence to define a primary key column. For details, see Schema design best practices. To understand the differences between the
UUID
andunique_rowid()
options, see the SQL FAQs. - Enum Preferences: On CockroachDB,
ENUMS
are a standalone type. On MySQL, they are part of column definitions. You can select to either deduplicate theENUM
definitions or create a separate type for each column. - GENERATED AS IDENTITY Conversion Option: We do not recommend using a sequence to define a primary key column. For details, see Schema design best practices. To understand the differences between the
UUID
andunique_rowid()
options, see the SQL FAQs. - IDENTITY Conversion Option: We do not recommend using a sequence to define a primary key column. For details, see Schema design best practices. To understand the differences between the
UUID
andunique_rowid()
options, see the SQL FAQs.
- INT type conversion: On CockroachDB,
- Click Next.
- Use either the Upload File or Use Credentials option to add your schema.
Upload File
The Schema Conversion Tool expects to analyze a SQL dump file containing only data definition statements.
To generate an appropriate PostgreSQL schema file, run the pg_dump
utility and specify the -s
or --schema-only
options to extract only the schema of a PostgreSQL database to a .sql
file.
To generate an appropriate MySQL schema file, run the mysqldump
utility and specify the -d
or --no-data
options to extract only the schema of the MySQL database to a .sql
file.
To generate an appropriate Oracle schema file, run the expdp
utility to extract only the schema of the Oracle database to a .sql
file.
To generate an appropriate Microsoft SQL Server schema file, use either SQL Server Management Studio or the equivalent mssql-scripter
utility to extract only the schema of the SQL Server database to a .sql
file.
The dump file must be smaller than 4 MB. INSERT
and COPY
statements will be ignored in schema conversion. To add a schema file:
- In step 2 of the Add SQL Schema dialog, click Upload File. Click the upload box and select a
.sql
file, or drop a.sql
file directly into the box. - Click Convert and wait for the schema to be analyzed. A loading screen is displayed. Depending on the size and complexity of the SQL dump, analyzing the schema can require up to several minutes.
- When analysis is complete, review the Summary Report and edit, add, or remove SQL statements in the Statements list.
Use Credentials
The Schema Conversion Tool can connect directly to a PostgreSQL or MySQL database to obtain the schema. To add a schema using credentials:
- In step 2 of the Add SQL Schema dialog, click Use Credential. Select the credentials to use. If the list is empty, this is because no credentials have been created for the selected database type. You can add credentials directly from the pulldown menu.
Click Convert and wait for the schema to be analyzed. In the background, the Schema Conversion Tool runs the
pg_dump
ormysqldump
utility to obtain the schema.A loading screen is displayed. Depending on the size and complexity of the SQL dump, analyzing the schema can require up to several minutes.
When analysis is complete, review errors and suggestions in the Summary Report. Apply bulk actions in the Summary Report and edit, add, or remove SQL statements in the Statements list.
Add database credentials
Credentials can be added for PostgreSQL and MySQL databases.
- With the Credentials tab open, click the Add Credential button.
- Provide the following information:
- A Credential Name to associate with the credentials.
- The Dialect of the database you are connecting to. Currently, PostgreSQL and MySQL are supported.
- The Host (i.e., hostname or IP address) for accessing the database, excluding the protocol (e.g.,
tcp://
). For example,migrations.cockroachlabs.com
. The following are not allowed:- Local hosts such as
localhost
and127.0.0.1
. - Private IPs, including
127.0.0.0/8
,10.0.0.0/8
,172.16.0.0/12
,192.168.0.0/16
,169.254.0.0/16
,::1/128
,fe80::/10
, andfc00::/7
.
- Local hosts such as
- The Port for accessing the database.
- A valid Username and Password for accessing the database.
- The Database Name to access. The Schema Conversion Tool will obtain the schema for this database.
- The SSL Mode for accessing the database:
None
: Do not force a secure connection.Verify CA
: Force a secure connection and verify that the server certificate is signed by a known CA.Verify Full
: Force a secure connection, verify that the server certificate is signed by a known CA, and verify that the server address matches that specified in the certificate.
If the credentials are valid, they will be added to the Credentials table with a VERIFIED
badge.
The Schema Conversion Tool creates the following internal objects when you convert a schema:
- A database prefixed with
_migration_internal_
is created on the Cloud cluster when you add a schema and each time you retry a schema migration. It does not contain any data apart from the statements in the_migration_internal_statements
table. When you successfully migrate a schema to the Cloud cluster, the final_migration_internal_
database is renamed to your specified database name, and the other_migration_internal_
databases associated with the schema are removed._migration_internal_
databases are also removed when you delete their associated schema from the Schemas table. - A table called
_migration_internal_statements
is created on each_migration_internal_
database. It contains the statements displayed in the Statements list, along with metadata related to the schema conversion. This table is stored indefinitely because it enables you to review and export the converted schema even after migrating it to a Cloud cluster.
Review the schema
Use the Summary Report and Statements list to update the schema and finalize it for migration.
The banner at the top of the page displays:
- The number of Statements Total in the uploaded
.sql
file that were analyzed. - The number of Errors in SQL statements that are blocking schema migration. Errors are further categorized and counted in the Summary Report.
- The number of Incidental Errors in SQL statements that are caused by errors in other SQL statements.
- The number of Incompatible Statements that could not be converted because they have no equivalent syntax on CockroachDB.
- The number of Compatibility Notes on differences in SQL syntax. Although these statements do not block schema migration, you should update them before migrating the schema.
- The number of Suggestions regarding CockroachDB best practices.
Summary Report
This feature is in preview. This feature is subject to change. To share feedback and/or issues, contact Support.
The Summary Report displays the results of the schema analysis and provides bulk actions you can apply to update the schema.
To apply bulk actions to statements, refer to the tables in the Summary Report:
Bulk actions cannot be undone after you retry the migration.
To edit, add, or delete individual statements, click the Statements tab to open the Statements list. Errors and suggestions are displayed for each statement.
After updating the schema, click Retry Migration. If the schema has zero errors, click Migrate Schema to migrate the schema to a new CockroachDB Cloud database.
Required Fixes
Required Fixes indicate errors that must be resolved before you can migrate the schema.
Column | Description |
---|---|
Description | A summary of the error type. |
Category | The category of error:
|
Complexity | The estimated difficulty of addressing the error. |
Remaining Instances | The number of times the error still occurs on the provided schema. This number will change as you update the schema to fix errors. Click the + icon on the row to view up to 20 individual statements where this occurs. |
Actions | The option to Add User to add a missing SQL user, or Delete all statements that contain the error type. This cannot be undone after you retry the migration. |
Compatibility Notes
Compatibility Notes indicate compatibility issues that do not block schema migration. These are only displayed for non-PostgreSQL schemas.
Column | Description |
---|---|
Description | A summary of the SQL compatibility issue. |
Complexity | The estimated difficulty of addressing the suggestion. |
Instances | The number of times the compatibility note occurs on the provided schema. Click the + icon on the row to view up to 20 individual statements where this occurs. |
Actions | The option to Acknowledge all instances of the compatibility note. This is not required for schema migration. |
Suggestions
Suggestions relate to schema design best practices. They do not block schema migration.
Column | Description |
---|---|
Description | One of the following suggestion types:
|
Complexity | The estimated difficulty of addressing the suggestion. |
Instances | The number of times the suggestion occurs on the provided schema. Click the + icon on the row to view up to 20 individual statements where this occurs. |
Actions | The option to Acknowledge all instances of the suggestion. This is not required for schema migration. |
Statements list
The Statements list displays the result of analyzing each statement in the .sql
file that you provided. The numbers from the Summary Report are displayed above the list of statements.
- To migrate the schema and create a new database for migration, click Migrate Schema. The schema must have zero errors.
- If the Migrate Schema button is disabled, use the Statements list to update the schema. Navigate the list by scrolling or by clicking the arrows and Scroll to Top button on the bottom-right.
Statements are displayed as follows:
- A statement that succeeded is displayed without further detail.
- A statement that failed is displayed with
[error]
and a message with error details. If the failure was due to an incidental error, the message also states:This error may automatically resolve once an earlier statement no longer errors
. - A statement that failed due to incompatible syntax is displayed with
[incompat]
, a message with syntax details, and an Acknowledge checkbox. - A statement that has a SQL compatibility issue is displayed with
[compat note]
, a message with syntax details, and an Acknowledge checkbox.Note:Some statements with compatibility issues are automatically removed during conversion. If a statement was removed from the schema, this is stated in the statement's
[incompat]
or[compat note]
message. - A statement that has a suggestion is displayed with
[suggestion]
, a message with suggestion details, and an Acknowledge checkbox.
To edit a statement, click the Edit button or the statement itself and enter your changes. Your changes are saved when you click outside the statement, or when you click the Save button. Click Cancel to discard your changes.
To edit multiple statements, click Find & Replace above the list.
- Enter a search term. Click Prev and Next to display each matching result.
Click > to expand the menu. Enter a replace term and click Replace, Replace & Find Next, or Replace All to replace the matching results.
This will update the corresponding statements. You cannot undo the replace operation.
Note:Replace operations are not currently supported for regex matches.
To remove or add a statement, click the ellipsis above the statement and then click Delete statement, Add statement above, or Add statement below.
Filter the Statements list
To filter the statements that are displayed in the Statements list, click the dropdown menu above the list.
The available filters match the results displayed in the Summary Report:
- Required Fixes
- Suggestions
- Compatibility Notes (non-PostgreSQL schemas)
If a filter is not available, No Options
is displayed.
Click Apply to filter the Statements list using the selected filters.
To remove a filter, click the x
next to the active filter. To remove all active filters, click Clear filters above the Statements list.
Export the schema
To export the current schema, click Download File at the top of the Statements list.
Update the schema
To update the schema:
- Apply bulk actions in the Summary Report.
- Edit, add, or remove statements in the Statements list.
Category | Solution | Bulk Actions | Required for schema migration? |
---|---|---|---|
Unimplemented feature | The feature does not yet exist on CockroachDB. Implement a workaround by editing the statement and adding statements. Otherwise, remove the statement from the schema. If a link to a tracking issue is included, click the link for further context. For more information about unimplemented features, see the Migration Overview. | Delete | Yes |
Uncreated user | Click the Add User button next to the error message. You must be a member of the admin role. This adds the missing user to the cluster. |
Add User, Delete | Yes |
Incidental | Resolve the error in the earlier failed statement that caused the incidental error. | Delete | Yes |
Incompatibility (non-PostgreSQL schemas) | There is no equivalent syntax on CockroachDB. Implement a workaround by replacing the statement. Otherwise, remove the statement from the schema. Then check Acknowledge. | Delete | Yes |
Uncategorized | Edit the statement to fix the error. Otherwise, remove the statement from the schema. | Delete | Yes |
Compatibility note (non-PostgreSQL schemas) | Edit the statement to match the CockroachDB syntax. Then optionally check Acknowledge. | Acknowledge | No |
Suggestion | Review and take any relevant actions indicated by the message. Then optionally check Acknowledge. | Acknowledge | No |
After updating the schema, click Retry Migration. If the schema has zero errors, click Migrate Schema to migrate the schema to a new CockroachDB Cloud database.
Retry the migration
To analyze a schema that you have updated, click Retry Migration at the top of the Summary Report or Statements list.
This will verify that the schema has zero errors and can be migrated.
Migrate the schema
You can migrate the schema directly to a new CockroachDB Cloud database when the number of errors is zero. This value is displayed on the Schemas table, Summary Report, and Statements list.
If you want to migrate to a CockroachDB Self-Hosted database, you can export the schema.
To migrate the schema, click Migrate Schema when viewing the Summary Report or Statements list. A modal will open:
- Name the new database and select a SQL user to own the database.
- Click Migrate.
After migrating the schema and creating the new database, you can load some test data and validate your queries.
Schemas table
If you have added a schema to convert, the following details are displayed when the Schemas tab is open:
Column | Description |
---|---|
Schema Name | The filename of the .sql file that you added. |
Status | The status of the schema migration: READY FOR REVIEW , READY TO MIGRATE , or MIGRATED . You can migrate schemas with READY TO MIGRATE status. |
Date Imported | The timestamp when the SQL dump was uploaded. |
Last Updated | The timestamp when the SQL statements were updated. |
Errors | The number of SQL errors preventing a schema from attaining READY TO MIGRATE status. |
To view the Summary Report or Statements list for a migration, click the migration name.
Credentials table
If you have added any external database credentials (PostgreSQL or MySQL only), the following details are displayed when the Credentials tab is open:
Column | Description |
---|---|
Credential Name | The name associated with the access credentials. A VERIFIED badge will display if the credentials are verified. |
Dialect | The type of database being accessed. |
Host / Port | The host and port used to access the database. |
Database Name | The name of the database being accessed. |
Created At | The timestamp when the credentials were successfully created. |
To delete or verify a set of credentials, select the appropriate option in the Actions column.