Migrating Content into Drupal

Drupal offers a migration system named Migrate that allows for one-time or continuous content migration into Drupal. Migrate is built on a set of standard concepts and tooling that help offer a standard to perform content-related migrations. This includes common extract, transform, load (ETL) practices such as source-to-destination mapping and data transformation. Migrate also offers a framework that can be used for more customized needs. It enables various use cases, such as moving content from Drupal 7 to Drupal 10, moving from WordPress, and performing ongoing feed-based content updates.

In this chapter, we’re going to cover the following main topics:

• Migration concepts
• The Migrate system
• Use cases

Migration concepts

Enterprises often have their existing websites and systems, but they may wish to move onto Drupal. To accommodate this, Drupal has a core Migrate system to perform migration. Migration is then made easier through a common set of tools, practices, systems, and frameworks that offer repeatability, predictability, and consistency.

Migrate itself is built from a conventional ETL convention. This is a common design pattern in software engineering and not specific to Drupal. Drupal applies this concept through sources and destinations. Content is extracted from a source. Migrate’s tooling, including features within Drupal, processes that content into a raw data format. Finally, the raw data is loaded into the destination through serialization. This process maintains the foundational, underlying design of the Migrate system.

Migration brings a lot of complexity to both the sources and destinations involved in migrating content. Sources can be any content source...

The Migrate system

Foundationally, the Migrate system is a backend API framework built around ETL concepts. Given the need to support diverse source and destination requirements, it must be highly configurable to work properly. This framework has been used to allow developers to address all of the migration-specific needs with defined plugins, configurations, and so on.

Beyond the API, the Migrate system includes the state of the migrations specific to the Drupal application in which migrations are run. This state maintains useful information, such as which migrations are registered on the application, what content has been successfully or unsuccessfully migrated per migration, what content has not yet been migrated per migration, and execution logs of every migration execution. Given the Migrate system is specific to a Drupal application, developers are empowered to run migrations on their local systems or non-production environments before running a migration on production.

...

Use cases

There are several use cases in which the Migrate system can be used.

Migrations often are designed as one-time or ongoing (continuous migrations). One-time migrations pull from sources only once. It is common to move from one content management system (CMS) to another, where Drupal is the new destination and the old system gets shut down. The migration moves the data only once into the new system. Ongoing migrations are often used when data is pulled from a third-party destination that changes with time and is leveraged by the Drupal application. Ongoing migrations can be executed periodically via drush calls with CRON directly on the server, or through the use of the migrate scheduler or migrate Cron contributed modules.

An example of this could be a daily migration that pulls from public JSON web service feeds into Drupal. The following YAML file, which leverages the Migrate Plus and Migrate Tools modules, defines a migration definition that pulls articles from the...

Summary

Drupal’s Migrate system provides a standard ETL-based framework used for one-time and ongoing migrations. The ETL is implemented through YAML file definitions, source/processing/destination plugins, and common tools to run, roll back, and view the status of migrations. Modules can be developed to extend the out-of-the-box plugins and harness Migrate’s event system for custom processing. The Migrate system helps address use cases where an existing CMS needs to be moved to a Drupal application, or periodic processing of web service feeds.

The next chapter covers multisite management in Drupal, which can be helpful to run multiple sites from the same code base.