Handling File Uploads with Cloudflare Workers

This is the first part of a multipart series on how to handle file uploads with Cloudflare Workers as I experiment and learn more about it. The eventual goal is to integrate my file upload service with clerk, nextjs and possibly even cloudflare's queue.

We'll be using Wrangler for this project. Wrangler is a general-purpose cli tool used to help manage workers.

Prerequisites

Before we create our worker, we need to make sure that we

Have a Cloudflare account
Install wrangler
Link wrangler to our Cloudflare account
Create a R2 Bucket to manage uploads

Most of this initial portion is taken from the official cloudflare docs located here

First, you'll need to install wrangler on your local system

yarn global add wrangler

Next, you'll need to login to your Cloudflare account. The following command will open a new browser whereby you'll be able to work on

wrangler login

Once you've succesfully logged in, run the command as seen below

wrangler whoami

This should generate your account details as seen below

schulz wrangler whoami

Now make sure that you've created your bucket. You can create a bucket using the cloudflare dashboard or via the bash command line.

wrangler r2 bucket create <YOUR_BUCKET_NAME>

Verify that your bucket has been created by running the command

➜  file-upload git:(master) ✗ wrangler r2 bucket list
Delegating to locally-installed wrangler@2.17.0 over global wrangler@2.17.0...
Run `npx wrangler r2 bucket list` to use the local version directly.
 
[
  {
    "name": <YOUR_BUCKET_NAME>,
    "creation_date": "2023-04-28T23:23:33.263Z"
  }
]

Creating the worker

We can now create our worker using the command

wranger init <YOUR_WORKER_NAME>

This should give you a similar command as seen below

➜  schulz wrangler init tut

This will create a new directory with the name of your worker. Inside this directory, you'll find a file called index.ts. This is the file that we'll be working with.

In this case, we've selected a few options

We will be using typescript
We have some basic default tests with Vitest set up
We will be using git to manage our worker ( This is optional but recomended. We will use a simple CI/CD to deploy our changes to our workers down the line )

Setting up the worker

We can specify configuration variables using the config.toml file. For this project, we can use the following configuration

config.toml

name = "file-upload-multipart"
main = "src/index.ts" # This is important. Make sure that you have it configured correctly
compatibility_date = "2022-06-30"
 
account_id = "YOUR_ACCOUNT_ID" # ← Replace with your Account ID.
workers_dev = true # Allows for a dev deployment
 
[[r2_buckets]]
binding = 'MY_BUCKET' # <~ valid JavaScript variable name
bucket_name = '<YOUR_BUCKET_NAME>'

For simplicity sake, we are going to use a simple header based authentication that checks a value we'll call AUTH_KEY_SECRET. We can add a new secret using wranger by running the command

wrangler secret put AUTH_KEY_SECRET

You can use the following command to generate a random secret for use.

node -e "console.log(require('crypto').randomBytes(256).toString('base64'));"

Here's the worker file that I'm using for this article

Cloudflare Worker

/**
 * Welcome to Cloudflare Workers! This is your first worker.
 *
 * - Run `wrangler dev src/index.ts` in your terminal to start a development server
 * - Open a browser tab at http://localhost:8787/ to see your worker in action
 * - Run `wrangler publish src/index.ts --name my-worker` to publish your worker
 *
 * Learn more at https://developers.cloudflare.com/workers/
 */
 
export interface Env {
  // Example binding to KV. Learn more at https://developers.cloudflare.com/workers/runtime-apis/kv/
  // MY_KV_NAMESPACE: KVNamespace;
  //
  // Example binding to Durable Object. Learn more at https://developers.cloudflare.com/workers/runtime-apis/durable-objects/
  // MY_DURABLE_OBJECT: DurableObjectNamespace;
  //
  // Example binding to R2. Learn more at https://developers.cloudflare.com/workers/runtime-apis/r2/
  MY_BUCKET: R2Bucket;
  //
  // Example binding to a Service. Learn more at https://developers.cloudflare.com/workers/runtime-apis/service-bindings/
  // MY_SERVICE: Fetcher;
  AUTH_KEY_SECRET: string;
}
 
// Check requests for a pre-shared secret
const hasValidHeader = (request: Request, env: Env) => {
  return request.headers.get("X-Custom-Auth-Key") === env.AUTH_KEY_SECRET;
};
 
function authorizeRequest(request: Request, env: Env, key: string) {
  return hasValidHeader(request, env);
}
 
export default {
  async fetch(
    request: Request,
    env: Env,
    ctx: ExecutionContext
  ): Promise<Response> {
    const url = new URL(request.url);
    const key = url.pathname.slice(1);
 
    if (!authorizeRequest(request, env, key)) {
      return new Response("Forbidden", { status: 403 });
    }
 
    switch (request.method) {
      case "PUT":
        await env.MY_BUCKET.put(key, request.body);
        return new Response(`Put ${key} successfully!`);
      case "GET":
        const object = await env.MY_BUCKET.get(key);
 
        if (object === null) {
          return new Response("Object Not Found", { status: 404 });
        }
 
        const headers = new Headers();
        object.writeHttpMetadata(headers);
        headers.set("etag", object.httpEtag);
 
        return new Response(object.body, {
          headers,
        });
      case "DELETE":
        await env.MY_BUCKET.delete(key);
        return new Response("Deleted!");
 
      default:
        return new Response("Method Not Allowed", {
          status: 405,
          headers: {
            Allow: "PUT, GET, DELETE",
          },
        });
    }
  },
};

Testing the worker

We need to create a fake bucket for testing. This can be done by running the command

wrangler r2 bucket create <fakeBucketName>

and then adding it in our config.toml file as

config.toml

name = '<name of worker>' # ← Replace with your Worker's name.
main = "src/index.ts"
compatibility_date = "2022-06-30"
 
account_id = '<account id which you got from wrangler whoami>'  # ← Replace with your Account ID.
workers_dev = true
 
[[r2_buckets]]
binding = 'MY_BUCKET' # <~ valid JavaScript variable name
bucket_name = '<name of bucket>'
preview_bucket_name = "<name of dev bucket>"

At the same time, we also need to configure a .dev.vars file which contains all the environment secrets that we are using. In our case, since we are using a simple header based authentication, we can add the following to our .dev.vars file

.dev.vars

AUTH_KEY_SECRET = TESTING

Unit Testing

We can now run a local instance of our worker using vitest by running the command npm run test. I've written the following test to check that our header based auth is working

index.test.ts

import { unstable_dev } from "wrangler";
import type { UnstableDevWorker } from "wrangler";
import { describe, expect, it, beforeAll, afterAll } from "vitest";
 
describe("Worker", () => {
  let worker: UnstableDevWorker;
 
  beforeAll(async () => {
    worker = await unstable_dev("src/index.ts", {
      experimental: { disableExperimentalWarning: true },
    });
  });
 
  afterAll(async () => {
    await worker.stop();
  });
 
  it("should throw a forbidden error when header auth is not provided", async () => {
    const resp = await worker.fetch("http://localhost/cat", {
      method: "PUT",
    });
    if (resp) {
      const text = await resp.text();
      expect(text).toMatchInlineSnapshot('"Forbidden"');
    }
  });
 
  it("should allow authenticated results through", async () => {
    const resp = await worker.fetch("http://localhost/cat", {
      method: "PUT",
      headers: {
        "X-Custom-Auth-Key": "TESTING",
      },
    });
    if (resp) {
      const text = await resp.text();
      expect(text).toMatchInlineSnapshot('"Put cat successfully!"');
    }
  });
});

We can now confirm if it works by running the command npm run test. If everything works, we should see the following output

✓ src/index.test.ts (2) 505ms
 
 Test Files  1 passed (1)
      Tests  2 passed (2)
   Start at  12:16:45
   Duration  1.29s (transform 62ms, setup 0ms, collect 387ms, tests 505ms, environment 0ms, prepare 91ms)
 
 
 PASS  Waiting for file changes...
       press h to show help, press q to quit

Integration Testing

Let's now test our endpoint locally. Wrangler allows us to do so by spinning up a local instance connected to a R2 Bucket. We can do so by running the command

wrangler dev --local

Note that we can see here that our worker is connected to the dev R2 Bucket that we created earlier which was listed under the property name of preview_bucket_name.

Make sure to turn off local mode. We can do so by pressing l in the terminal.

Delegating to locally-installed wrangler@2.17.0 over global wrangler@2.17.0...
Run `npx wrangler dev --local` to use the local version directly.

As you can see, my local server can be called by sending requests to http://localhost:8787/.

I recommend using Thunder Client which comes out of the box as a VsCode Extension but you can use any other tool that you prefer....even curl.

I've attached a snapshot of my thunder client configuration so that you can reproduce my api parameters.

Let's try to now run the request and see if we can upload a new file with the key of new-key. If everything works, we should see the following response

Put new-key successfully!

We can also confirm that the file has been uploaded by running a GET command with the key of new-key. This should give us a valid image back that we uploaded previously when we make an api call to http://localhost:8787/new-key

Let's now try to delete the file by running a DELETE command with the key of new-key. If everything works, we should see the following response

Deleted!

If the file has been succesfully deleted, we should get the following 404 response when we try to get an object from our r2-storage with the key of new-key

Object Not Found

Publishing your Worker

Publishing your worker is as simple as simply running the command as

tut git:(master) ✗ npx wrangler publish
 
 
npm WARN config init.author.name Use `--init-author-name` instead.

I hope this helped!