Redis Migration With Zero Downtime to Memorystore

Redis Migration With Zero Downtime to Memorystore

In this post we are going to discuss Redis Migration With Zero Downtime from One Server to another server or to Memorystore. There are several ways to migrate from one Redis Server to another Redis Server or MemoryStore but none of them supports zero downtime. Transferring data between systems poses significant risks for businesses. It maybe potential downtime, the risk of data loss, and the possibility of data corruption, all of which are just a few examples of potential complications that can arise.

While the process of migrating data from older or obsolete databases to newer ones carries inherent risks, businesses must embrace innovation to ensure their survival. Modern databases come with advanced features and capabilities that can enhance business operations and maintain competitiveness. For instance, customers transitioning from self-managed databases to fully-managed databases provided by cloud providers can give numerous benefits.

Setup Source Redis Instance in a VM

Installing and Configuring Redis Instance

You can deploy the Redis server by utilizing the APT package manager sourced directly from the official Ubuntu repositories.

Start by updating your local APT package cache:

sudo apt-get update

Install Redis by using the following command

sudo apt install redis-server

The above command will download and install all dependencies for Redis Server, we need to change several options in the Redis Configuration for migration, below is the command to edit the configuration file

sudo nano /etc/redis/redis.conf

Now that we have installed and setup source redis server for migration, before using it let’s check if it is live

Testing Redis Instance

use the following command to check the installed redis server status

sudo systemctl status redis-server

If it is running without issue then below output will come.

Redis Server Migration

This output indicates that Redis is up and running and is ready to boot whenever the server starts. Let’s connect to the Redis Server with redis-cli command


If you enter like below then redis server is running fine.


you will receive the response like below


this confirms that the redis server is running fine just enter exit it will exit from the redis server

Loading Data into Redis Source Server

Load data to the source Redis Server if you have or use the following sample data
Redis Movie sample dataset that available for download from Github repository, Redis Movie Database Sample Dataset.

This sample dataset contains movies stored as Redis Hash. The movie hashes contain the following fields:

movie:id : The key of the hash
title : The title of the movie
plot : A summary of the movie
genre : The genre of the movie, for now a movie will only have one single genre
release_year : The year the movie has been released as a numerical value
rating : The ratings from the public numerical value
votes : Number of votes
poster : Link to the movie poster
imdb_id : id of the movie in the IMDB database

Importing the dataset

Copy and run the following command to import and insert in Redis Source Server


redis-cli -h -p 6379 < ./import_movies.redis

you can check some data using redis-cli with below command:

HMGET "movie:343" title release_year genre


1) "Spider-Man"
2) "2002"
3) "Action"

we are going to use this dataset as the source for our migration to Memorystore Redis.

Note : Here I’m going to use Memorystore as target you can also use any other Redis Server as target just by changing the ip and other details in the migration process.

Setup Memorystore Redis instance

create a Memorystore Redis instance, you can use the Google Cloud console, the gcloud CLI, or the Memorystore API

Using the gcloud CLI:

  1. Open the Cloud Shell from GCP Console
  2. Set the project you want to create your instance in as the default project in gcloud by entering the following command:
    gcloud config set core/project PROJECT_ID
  3. Enter the following command to create a 1 GiB Basic Tier Redis instance in the us-central1 region with Redis version 7 name redis-prod:
    gcloud redis instances create redis-prod --size=1 --region=us-central1 --redis-version=redis_7_0
  4. After the instance is created, enter the describe command to get the IP address and port of the instance:
    gcloud redis instances describe redis-prod --region=us-central1
  5. Check Redis functionality similar to the Source Redis Server setup

Deploy RIOT for Live Replication of Redis Instance
Most of the Redis Migration tools that are available today or offline in nature means need to take backup from the source and restore on the target. But here RIOT implements client-side replication using dump & restore or type-based read & write. Both snapshot and live replication modes are supported.

In live replication mode, RIOT monitors the source database for any changes using keyspace notifications. Whenever a key undergoes modification, RIOT retrieves the associated value and transmits that modification to the target database.

Live replication depends on keyspace notifications. We have to make sure that the source database has keyspace notifications enabled using notify-keyspace-events = KA in redis.conf (or via CONFIG SET). For more details see Redis Keyspace Notifications.

Also note that this live replication mechanism does not guarantee data consistency. It is also possible that RIOT may miss some notifications incase of network failure of source server.

Moreover, depending on factors such as data structure type, size, and the rate of changes in the source database, RIOT may encounter challenges in keeping pace with the stream of changes.

RIOT operates by processing data in batches, where a fixed number of records (referred to as a batch or chunk) are read, processed, and written at a time. This process is repeated until all data from the source has been handled.

Setup RIOT instance

We need a VM to install RIOT tool for the migration, we can also use the source VM for installing the Tool but to avoid any disruption in the prod environment we are going to setup this tool in another VM.

Download and Unzip the RIOT tool

unzip riot-standalone-3.1.5-linux-x86_64

cd riot-standalone-3.1.5-linux-x86_64/bin
Run the Replication

Below is the command to start migration or replication

riot -h <ip-address> -p <port> replicate -h <ip-address> -p <port> –batch <integer> --mode <snapshot|live|compare>
-h, --hostname
Server hostname

-p, --port
Server port

There are many options and tasks that RIOT can handle such as transforms and applying filters. Details of the command and options for the migration can be found in the official documentation here

Let’s start the live migration using the simple following command

./riot -h -p 6379 replicate  -h -p 6379 --mode live
Check the Replication Data

check the replication by inserting new data in our Redis source using below command:

redis-cli -h -p 6379
HSET "movie:1200" title "Guardians of the Galaxy" genre "Action" votes 704613

After the insert, let’s check the new data in our target Redis instance:

redis-cli -h -p 6379
HMGET "movie:1200" title release_year genre


1) “Guardians of the Galaxy”

2) “2014”

3) “Action”

the new data has been replicated successfully to our target VM or Memorystore Redis instance.

Once the replication is completed successfully you can switch the traffic from your old redis instance to new instance by changing the application properties.

Tags : Redis Migration With Zero Downtime, Redis to Memorystore Migration without downtime, Redis live migration ,Redis Server to another Redis Server zero downtime

How to Install and Uninstall Redis Server on Ubuntu

Please leave your feedback or comments below.

Leave a Reply

Your email address will not be published. Required fields are marked *