Migrating Cloudflare Durable Objects Across Accounts
When I needed to move my Cloudflare Durable Objects to another account, I realized there wasn't an easy way to do it. Cloudflare didn't provide a built-in solution for migrating these objects between zones or accounts. After some searching with no luck, I decided to solve this problem the hard way.
The solution involves setting up /dump
and /write
routes in your Durable Object (DO) workers—essentially creating a way to extract data from the source DO and import it into the target DO.
You need to add functionality to both your source and target workers to handle data dumping and writing, respectively. Here's how you do it:
Source Worker: The /dump
Route
In your source worker, you'll set up a /dump
route. This route is responsible for sending all the stored data from your Durable Object. Here's a simplified example from the worker.ts
file:
export { DemoDO } from './do'
export interface Env {
DEMO_DO: DurableObjectNamespace
}
export default {
async fetch(request: Request, env: Env): Promise<Response> {
const url = new URL(request.url);
if (url.pathname.startsWith('/dump')) {
const doId = url.searchParams.get('doId');
if (!doId) return new Response('doID is required', { status: 400 });
const id = env.DEMO_DO.idFromName(doId);
const stub = env.DEMO_DO.get(id);
const dump = await stub.fetch(request);
return new Response(await dump.text(), {
headers: { 'Content-Type': 'application/json' },
status: 200,
});
}
return new Response('Method not allowed', { status: 405 });
},
}
// Source Durable Object
export class DemoDO {
async dumpStorage(startAfter: string = '', limit: number = 100): Promise<{ data: Map<string, any>; lastKey: string | null }> {
const entries: Map<string, any> = await this.state.storage.list({ startAfter, limit })
const lastKey: string | null = Array.from(entries.keys()).pop() || null
return { data: entries, lastKey }
}
async fetch(request: Request): Promise<Response> {
const url = new URL(request.url)
if (url.pathname.startsWith('/dump')) {
const startAfter: string = url.searchParams.get('startAfter') || ''
const limit: number = Number.parseInt(url.searchParams.get('limit')!, 10) || 100
const { data, lastKey } = await this.dumpStorage(startAfter, limit)
return new Response(JSON.stringify({ data: Object.fromEntries(data), lastKey }), {
headers: { 'Content-Type': 'application/json' },
status: 200,
})
}
return new Response('Not found', { status: 404 })
}
}
Target Worker: The /write
Route
On the target worker, the /write
route takes the dumped data and writes it to the Durable Object in the new account. The implementation is similar to the /dump
route, focusing on accepting and storing the incoming data.
// worker.ts
export { DemoDO } from './do'
export interface Env {
DEMO_DO: DurableObjectNamespace
}
export default {
async fetch(request: Request, env: Env): Promise<Response> {
const url = new URL(request.url);
if (url.pathname.startsWith('/write')) {
const doId = url.searchParams.get('doId');
if (!doId) return new Response('doID is required', { status: 400 });
const id = env.DEMO_DO.idFromName(doId);
const stub = env.DEMO_DO.get(id);
const dump = await stub.fetch(request);
return new Response(await dump.text(), {
headers: { 'Content-Type': 'application/json' },
status: 200,
});
}
return new Response('Method not allowed', { status: 405 });
},
}
// Target Durable Object
export class DemoDO {
async bulkWrite(data: KeyValue): Promise<void> {
await this.state.storage.transaction(async (txn) => {
for (const [key, value] of Object.entries(data)) await txn.put(key, value)
})
}
async fetch(request: Request): Promise<Response> {
const url = new URL(request.url)
if (url.pathname.startsWith('/write')) {
const data: KeyValue = await request.json()
if (!data)
return new Response('data not found', { status: 400 })
await this.bulkWrite(data)
return new Response('Data written successfully', { status: 200 })
}
return new Response('Not found', { status: 404 })
}
}
Migrating the Data
To migrate the data between the workers, you'll need a script to automate the process. This script will iterate through all Durable Object IDs, dump the data from the source, and write it to the target. Here's an outline of the Node.js script needed for the migration:
import { promises as fs } from 'node:fs';
import fetch from 'node-fetch';
import type { KeyValue } from './types';
let sourceBaseUrl: string = '<https://source.example.com>'; // Your source worker URL
let targetBaseUrl: string = '<https://target.example.com>'; // Your target worker URL
let batchSize: number = 100; // Adjust based on your preference
let durableObjectIds: string[] = ['your-durable-object-ids']; // List of your DO IDs
// Function to fetch data from source DO
async function fetchData(doId: string): Promise<KeyValue> {
let data: KeyValue = {};
let lastKey: string | undefined = undefined;
do {
const response = await fetch(`${sourceBaseUrl}/dump?doId=${doId}${lastKey ? `&startAfter=${lastKey}` : ''}`);
if (!response.ok) throw new Error(`Failed to fetch data for DO ID ${doId}: ${response.statusText}`);
const { data: fetchedData, lastKey: newLastKey } = await response.json();
data = { ...data, ...fetchedData };
lastKey = newLastKey;
} while (lastKey);
return data;
}
// Function to write data to target DO
async function writeData(doId: string, data: KeyValue): Promise<void> {
const response = await fetch(`${targetBaseUrl}/write?doId=${doId}`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(data),
});
if (!response.ok) throw new Error(`Failed to write data for DO ID ${doId}: ${response.statusText}`);
}
// Main function to migrate data
async function migrateData(): Promise<void> {
for (const doId of durableObjectIds) {
console.log(`Migrating data for DO ID: ${doId}`);
const data = await fetchData(doId);
await writeData(doId, data);
console.log(`Successfully migrated data for DO ID: ${doId}`);
}
}
It would be great if Cloudflare made it easy to back up and move our Durable Objects around with just a couple of clicks. Until they come up with something like that, the method I shared here should do the trick.
To make everyone's life a bit easier, I put together a simple CLI tool. It's designed to simplify the migration processGuide to Migrating Cloudflare Durable Objects Across Accounts. If you're in need of moving some DOs, give mido
a look over on GitHub: yudax42/cf-do-migration.