Database migration from AWS Aurora Serverless v1

huangapple go评论54阅读模式
英文:

Database migration from AWS Aurora Serverless v1

问题

我们计划创建一个新的应用程序,该应用程序的架构与当前应用程序不同,并且表格结构肯定会有一些差异。我们当前的应用程序数据库运行在AWS Aurora Serverless v1集群上,使用的是MySQL 5.7。我们需要将旧应用程序的数据迁移到新应用程序,并在某些地方进行一些模式更改。

一种解决方案是创建旧表格的转储,以易于阅读的格式,如CSV,然后将数据导入新应用程序,根据需要选择数据。这将需要在两个应用程序上编写定制代码,以从一个应用程序导出数据并导入另一个应用程序。

考虑到我们在AWS中,并且打算继续使用它,选择Aurora作为数据库(我们必须决定是否使用提供的解决方案或选择Serverless v2),是否有任何服务可以从一个Aurora Serverless v1集群导出数据到另一个Aurora集群,并在途中进行更改?

我知道存在数据库快照,但那并不完全符合我们的目标。我们需要将数据从一个数据库迁移到另一个数据库,而不是创建第一个数据库的另一个副本。我看到有一个名为“AWS数据库迁移服务”的服务,用于将数据迁移到RDS,但这项服务是否适合我们的需求,即从一个Aurora Serverless数据库获取数据,进行一些更改,然后将其放入我们选择的另一个RDS数据库中?

英文:

We have a plan to create a new application, which won't have the same architecture of the current one, and will definitely have some differences in the tables' schemas. Our current application's database is running on a AWS Aurora Serverless v1 cluster, with MySQL 5.7. We need to migrate data from the old application to the new one, making some schema changes in some places.

One solution is to create dumps of the old tables, in an easily readable format like csv, then import the data in the new application, picking the data as needed. This will require writing bespoke code on both applications to export data from one and import it into the other.

Considering that we are in AWS, and we intend to keep using it, choosing Aurora as a database (we'll have to decide whether use a provided solution or go with Serverless v2), is there any service to export data from one Aurora Serverless v1 cluster to another Aurora cluster, making changes along the way?

I know that database snapshots exist, but that's not exacly our goal. We need to migrate data from one database to another, not creating another copy of the first one. I've seen that there is a service, called "AWS Database Migration Service", that's used to migrate data into RDS, but does this service suit our needs, to get data from one Aurora Serverless database, do some changes, and then put it into another RDS database of our choice?

答案1

得分: 1

1.- AWS数据库迁移服务,该服务提供了复制数据并在复制过程中执行一些模式转换的功能。您可以创建选择和转换规则,服务将执行这些规则并将更改应用于目标模式。您可以在此链接上查看更多详细信息。

https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Tasks.CustomizingTasks.TableMapping.SelectionTransformation.Transformations.html

2.- AWS Glue Studio,它是一项ETL服务,允许您在源数据库和目标数据库之间提取、转换和传输数据。

什么是AWS Glue Studio?

使用AWS Glue Studio自动化Amazon RDS for SQL Server和Azure托管SQL之间的ETL作业

英文:

You could use following options to accomplish your goal.

1.- AWS Database Migration Service, this service provide you capabilities to replicate your data and also perform some schema transformation during replication process. You can create selection and transformation rules, the service will execute those rules and apply changes to the target schemas. You could review more details on this link.

https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Tasks.CustomizingTasks.TableMapping.SelectionTransformation.Transformations.html

2.- AWS Glue Studio, it is ETL service that allows you extract, convert and transfer data between source and target databases.

What is AWS Glue Studio?

Automate ETL jobs between Amazon RDS for SQL Server and Azure Managed SQL using AWS Glue Studio

答案2

得分: 0

有另一种替代方案可供您探索。它比使用 DMS 更容易。

您可以使用 CLI 将 Serverless v1 修改为 Aurora Provisioned aws rds modify-db-cluster(截至目前尚不支持在控制台上操作),如下所示:

aws rds modify-db-cluster \
 --db-cluster-identifier <ServerlessV1 集群名称> \
 --engine-mode provisioned \
 --allow-engine-mode-change \
 --db-cluster-instance-class <AuroraProvisioned 实例类型> \
 --apply-immediately

完成后,Aurora Serverless V1 将转换为按设计的 Aurora Provisioned 集群。然后,您可以创建 Aurora MySQL 的蓝/绿部署,以便在绿色部署环境中测试应用程序。在进行彻底测试并准备好切换时,您可以从蓝色部署切换到绿色部署。

如果您决定使用 Serverless V2,您可以添加一个读取器(Serverless V2),然后将写入器实例(Provisioned)故障转移到读取器实例(Serverless V2),然后在 Serverless V2 上进行彻底测试并切换蓝/绿。

注意:Aurora 蓝/绿部署 仅适用于 AWS 上的 MySQL。

英文:

There is another alternative which you could explore. It is easier than using DMS.

You could modify Serverless v1 to Aurora Provisioned using CLI aws rds modify-db-cluster (not available on console as of today) with below:

aws rds modify-db-cluster \
 --db-cluster-identifier &lt;ServerlessV1 Cluster-Name&gt; \
 --engine-mode provisioned \
 --allow-engine-mode-change \
 --db-cluster-instance-class &lt;AuroraProvisioned Instance Type&gt; \
 --apply-immediately

Once that's done, the Aurora Serverless V1 will be converted to Aurora Provisioned as cluster as per designed. Then, you could create Aurora MySQL Blue/Green deployment which allow you to test application in green deployment environment. When you have done thorough testing and ready for cut over, you would then switchover from blue to green deployment.

If you decide to go with Serverless V2, you could Add a reader (Serverless V2), then fail over writer instance (Provisioned) to reader instance (Serverless V2) before performing thorough testing on Serverless V2 and switching over blue/green.

NOTE: Aurora blue/green deployment only applicable for MySQL on AWS.

huangapple
  • 本文由 发表于 2023年2月6日 19:31:07
  • 转载请务必保留本文链接:https://go.coder-hub.com/75360753.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定