Migrating Redis, the easy way

I recently migrated our Redis instance from RedisCloud to StackHero. I’d heard good things about StackHero for years, and the fact that we could cut our costs by 90% was the final push.

To be honest, I was dreading it. Switching Redis providers meant I had to:

• Move Sidekiq queues without losing jobs
• Avoid downtime for our Rails app

Surprisingly, it turned out to be straightforward. If your app runs on Ruby on Rails, here’s a step-by-step guide to make the switch (and save 90% in the process).

1. Provision Redis instance

Go to StackHero, create an account and provision a new Redis instance.

Choose the same region as your web servers. This is crucial—when I mistakenly picked the wrong region, latency spiked and Sidekiq queues backed up quickly.

Copy the TSL endpoint of your Redis instance. It looks something like this

rediss://default:<yourPassword>@aiyehk.stackhero-network.com:10828

Replace <yourPassword> with the password you’ll find under the Configure section in the StackHero dashboard.

2. Replace Redis URL in your app

Most Rails apps connect to Redis via the REDIS_URL environment variable.

1. First, make a note of your current REDIS_URL. You’ll need it to migrate existing jobs.
2. Replace it with the TLS endpoint from StackHero.

Once updated, new jobs will automatically be pushed to your StackHero Redis instance.

3. Migrate old jobs to new Redis instance

At this point, any new jobs are safe on StackHero. But what about jobs still sitting in the old Redis instance?

All you need to do now is migrate the jobs from your old Redis instance to your new one with this simple script:

require "redis"
require "json"

old_redis_url = "...." # replace with old REDIS_URL value
old = Redis.new(url: old_redis_url)
new = Redis.new(url: ENV["REDIS_URL"])

# Migrate all Sidekiq queues
old.keys("queue:*").each do |queue|
  jobs = old.lrange(queue, 0, -1)
  jobs.each do |job|
    new.rpush(queue, job)
  end
end

# Also migrate scheduled and retry sets
%w[retry schedule].each do |set|
  jobs = old.zrange("sidekiq:#{set}", 0, -1, withscores: true)
  jobs.each do |job, score|
    new.zadd("sidekiq:#{set}", score, job)
  end
end

This code will migrate every job from all your queues to the new Redis instance, including jobs in the retry and schedule sets.