Storage emulator returns 403 Forbidden when trying to access files uploaded with firebaseStorageDownloadTokens set in metadata

### [REQUIRED] Environment info





**firebase-tools:** 9.11.0



**Platform:** Windows 10

### [REQUIRED] Test case



Cloud Function responsible for image optimization (based on [official extension](https://github.com/firebase/extensions/blob/master/storage-resize-images/functions/src/resize-image.ts)). Downloads stored file to tmp directory, performs optimization and overwrites the orignal file, copying over the `firebaseStorageDownloadTokens`


**`functions/optimizeImage.ts`**
```ts
import * as path from 'path';
import * as os from 'os';
import * as functions from 'firebase-functions';
import * as fs from 'fs';
// Admin SDK initialization
import { admin } from '@/lib/firebase/admin';

const storage = admin.storage();
const mkdirp = require('mkdirp');
const sharp = require('sharp');

export const optimizeImage = functions.storage
  .object()
  .onFinalize(async (object) => {
    if (!object.name || !object.contentType?.startsWith('image/')) {
      return null;
    }

    const parts = object.name.split('/');
    const fileName = parts[parts.length - 1];

    const bucket = storage.bucket(object.bucket);
    const originalFile = bucket.file(object.name);
    const tmpFilePath = path.join(os.tmpdir(), object.name);
    const tmpFolderPath = path.dirname(tmpFilePath);
    const optimizedFilePath = path.join(tmpFolderPath, `optimized-${fileName}`);

    try {
      const metaResponse = await originalFile.getMetadata();
      const metadata = metaResponse?.[0]?.metadata;

     // Print the token for debug purposes
     console.log('Firebase Storage download token:', metadata.firebaseStorageDownloadTokens)

      if (metadata.optimized) {
        return;
      }

      await mkdirp(tmpFolderPath);
      await originalFile.download({
        destination: tmpFilePath,
        validation: false,
      });

      await sharp(tmpFilePath)
        .resize({
          width: 1920,
          withoutEnlargement: true,
        })
        .toFormat('jpg')
        .jpeg({
          quality: 70,
          chromaSubsampling: '4:4:4',
          force: true,
        })
        .toFile(optimizedFilePath);

      // Merge original metadata containing firebaseStorageDownloadTokens
      // with custom metadata
      const optimizedFileMetadata = {
        metadata: {
          ...metadata,
          optimized: true,
        },
      };

      await bucket.upload(optimizedFilePath, {
        destination: originalFile,
        metadata: optimizedFileMetadata,
      });

      await fs.unlinkSync(tmpFilePath);
      await fs.unlinkSync(optimizedFilePath);

      return { success: true };
    } catch (error) {
      return Promise.reject(error);
    }
  });

```




### [REQUIRED] Steps to reproduce

1. Run the emulators: `firebase emulators:start`
2. Upload an image to the Storage (for example using the Storage Emulator UI)
3. Open the file from Storage Emulator and copy file URL from 403 error in console (related to #3393) and append
`?alt=media&token=<token>` to the link, replacing token with the token you get from `optimizeImage` function console.log

Alternatively build the URL yourself:
`http://localhost:9199/v0/b/<bucket>/o/<file-path>?alt=media&token=<token>`

Attempting to obtain the download url using `fileRef.getDownloadURL()` results in the following error:
![image](https://user-images.githubusercontent.com/9134970/119239599-75026f80-bb4a-11eb-9d66-0a993bad2dba.png)

The above function works fine and returns the URL when running using production Firebase.



### [REQUIRED] Expected behavior



Setting `firebaseStorageDownloadTokens` should allow you to access the file by providing the `?token=<token>` in the resource url. This is used by the official [image resize extension](https://github.com/firebase/extensions/blob/master/storage-resize-images/functions/src/resize-image.ts#L153) and works fine in production (even though it is not officialy documented):

![image](https://user-images.githubusercontent.com/9134970/119240218-8b122f00-bb4e-11eb-8ce3-94a5e1281b9d.png)
![image](https://user-images.githubusercontent.com/9134970/119240199-746bd800-bb4e-11eb-8888-7b2ed85da847.png)


### [REQUIRED] Actual behavior


After overwriting the file or even creating a new file with `firebaseStorageDownloadTokens` set in the metadata, the file is inaccessible and returns 403 Forbidden

![image](https://user-images.githubusercontent.com/9134970/119240456-e7298300-bb4f-11eb-9cf6-1b0afba89900.png)

```
[2021-05-22T20:44:15.631Z] [runtime-status] [8668] Trigger "optimizeImage" has been found, beginning invocation! {"metadata":{"emulator":{"name":"functions"},"function":{"name":"optimizeImage"},"message":"[runtime-status] [8668] Trigger \"optimizeImage\" has been found, beginning invocation!"}}
i  functions: Beginning execution of "optimizeImage" {"metadata":{"emulator":{"name":"functions"},"function":{"name":"optimizeImage"},"message":"Beginning execution of \"optimizeImage\""}}
[2021-05-22T20:44:15.632Z] [runtime-status] [8668] triggerDefinition {"regions":["europe-central2"],"eventTrigger":{"resource":"projects/_/buckets/<bucket>.appspot.com","eventType":"google.storage.object.finalize","service":"storage.googleapis.com"},"name":"optimizeImage","entryPoint":"optimizeImage"} {"met
adata":{"emulator":{"name":"functions"},"function":{"name":"optimizeImage"},"message":"[runtime-status] [8668] triggerDefinition {\"regions\":[\"europe-central2\"],\"eventTrigger\":{\"resource\":\"projects/_/buckets/<bucket>.appspot.com\",\"eventType\":\"google.storage.object.finalize\",\"service\":\"storag
e.googleapis.com\"},\"name\":\"optimizeImage\",\"entryPoint\":\"optimizeImage\"}"}}
[2021-05-22T20:44:15.632Z] [runtime-status] [8668] Running optimizeImage in mode BACKGROUND {"metadata":{"emulator":{"name":"functions"},"function":{"name":"optimizeImage"},"message":"[runtime-status] [8668] Running optimizeImage in mode BACKGROUND"}}
[2021-05-22T20:44:15.633Z] [runtime-status] [8668] ProcessBackground {"eventId":"1621716254902","timestamp":"2021-05-22T22:44:14.902Z","eventType":"google.storage.object.finalize","resource":{"service":"storage.googleapis.com","name":"projects/_/buckets/<bucket>.appspot.com/objects/danielle-cerullo-CQfNt66t
tZM-unsplash.jpg","type":"storage#object"},"data":{"kind":"#storage#object","name":"danielle-cerullo-CQfNt66ttZM-unsplash.jpg","bucket":"<bucket>.appspot.com","generation":"1621716254898","metageneration":"1","contentType":"image/jpeg","timeCreated":"2021-05-22T22:44:14.897Z","updated":"2021-05-22T22:44:14.
900Z","storageClass":"STANDARD","size":"264694","md5Hash":"HWnJ+ZgmyNHPj9HLQqCXwQ==","etag":"someETag","metadata":{"firebaseStorageDownloadTokens":"c2887239-8b0c-4b47-93e1-974415480b6e","optimized":true},"crc32c":"----6A==","timeStorageClassUpdated":"2021-05-22T22:44:14.897Z","id":"<bucket>.appspot.com/dani
elle-cerullo-CQfNt66ttZM-unsplash.jpg/1621716254898","selfLink":"http://localhost:9199/storage/v1/b/<bucket>.appspot.com/o/danielle-cerullo-CQfNt66ttZM-unsplash.jpg","mediaLink":"http://localhost:9199/download/storage/v1/b/<bucket>.appspot.com/o/danielle-cerullo-CQfNt66ttZM-unsplash.jpg?generation=16
21716254898&alt=media"}} {"metadata":{"emulator":{"name":"functions"},"function":{"name":"optimizeImage"},"message":"[runtime-status] [8668] ProcessBackground {\"eventId\":\"1621716254902\",\"timestamp\":\"2021-05-22T22:44:14.902Z\",\"eventType\":\"google.storage.object.finalize\",\"resource\":{\"service\":\"stora
ge.googleapis.com\",\"name\":\"projects/_/buckets/<bucket>.appspot.com/objects/danielle-cerullo-CQfNt66ttZM-unsplash.jpg\",\"type\":\"storage#object\"},\"data\":{\"kind\":\"#storage#object\",\"name\":\"danielle-cerullo-CQfNt66ttZM-unsplash.jpg\",\"bucket\":\"<bucket>.appspot.com\",\"generation\":\"16
21716254898\",\"metageneration\":\"1\",\"contentType\":\"image/jpeg\",\"timeCreated\":\"2021-05-22T22:44:14.897Z\",\"updated\":\"2021-05-22T22:44:14.900Z\",\"storageClass\":\"STANDARD\",\"size\":\"264694\",\"md5Hash\":\"HWnJ+ZgmyNHPj9HLQqCXwQ==\",\"etag\":\"someETag\",\"metadata\":{\"firebaseStorageDownloadTokens\
":\"c2887239-8b0c-4b47-93e1-974415480b6e\",\"optimized\":true},\"crc32c\":\"----6A==\",\"timeStorageClassUpdated\":\"2021-05-22T22:44:14.897Z\",\"id\":\"<bucket>.appspot.com/danielle-cerullo-CQfNt66ttZM-unsplash.jpg/1621716254898\",\"selfLink\":\"http://localhost:9199/storage/v1/b/<bucket>.appspot.co
m/o/danielle-cerullo-CQfNt66ttZM-unsplash.jpg\",\"mediaLink\":\"http://localhost:9199/download/storage/v1/b/<bucket>.appspot.com/o/danielle-cerullo-CQfNt66ttZM-unsplash.jpg?generation=1621716254898&alt=media\"}}"}}
[2021-05-22T20:44:15.634Z] [runtime-status] [8668] RunBackground {"data":{"kind":"#storage#object","name":"danielle-cerullo-CQfNt66ttZM-unsplash.jpg","bucket":"<bucket>.appspot.com","generation":"1621716254898","metageneration":"1","contentType":"image/jpeg","timeCreated":"2021-05-22T22:44:14.897Z","updated
":"2021-05-22T22:44:14.900Z","storageClass":"STANDARD","size":"264694","md5Hash":"HWnJ+ZgmyNHPj9HLQqCXwQ==","etag":"someETag","metadata":{"firebaseStorageDownloadTokens":"c2887239-8b0c-4b47-93e1-974415480b6e","optimized":true},"crc32c":"----6A==","timeStorageClassUpdated":"2021-05-22T22:44:14.897Z","id":"<bucket>
/danielle-cerullo-CQfNt66ttZM-unsplash.jpg/1621716254898","selfLink":"http://localhost:9199/storage/v1/b/<bucket>.appspot.com/o/danielle-cerullo-CQfNt66ttZM-unsplash.jpg","mediaLink":"http://localhost:9199/download/storage/v1/b/<bucket>.appspot.com/o/danielle-cerullo-CQfNt66ttZM-uns
plash.jpg?generation=1621716254898&alt=media"},"context":{"eventId":"1621716254902","timestamp":"2021-05-22T22:44:14.902Z","eventType":"google.storage.object.finalize","resource":{"service":"storage.googleapis.com","name":"projects/_/buckets/<bucket>.appspot.com/objects/danielle-cerullo-CQfNt66ttZM-unsplash
.jpg","type":"storage#object"}}} {"metadata":{"emulator":{"name":"functions"},"function":{"name":"optimizeImage"},"message":"[runtime-status] [8668] RunBackground {\"data\":{\"kind\":\"#storage#object\",\"name\":\"danielle-cerullo-CQfNt66ttZM-unsplash.jpg\",\"bucket\":\"<bucket>.appspot.com\",\"generation\"
:\"1621716254898\",\"metageneration\":\"1\",\"contentType\":\"image/jpeg\",\"timeCreated\":\"2021-05-22T22:44:14.897Z\",\"updated\":\"2021-05-22T22:44:14.900Z\",\"storageClass\":\"STANDARD\",\"size\":\"264694\",\"md5Hash\":\"HWnJ+ZgmyNHPj9HLQqCXwQ==\",\"etag\":\"someETag\",\"metadata\":{\"firebaseStorageDownloadTo
kens\":\"c2887239-8b0c-4b47-93e1-974415480b6e\",\"optimized\":true},\"crc32c\":\"----6A==\",\"timeStorageClassUpdated\":\"2021-05-22T22:44:14.897Z\",\"id\":\"<bucket>.appspot.com/danielle-cerullo-CQfNt66ttZM-unsplash.jpg/1621716254898\",\"selfLink\":\"http://localhost:9199/storage/v1/b/<bucket>.appsp
ot.com/o/danielle-cerullo-CQfNt66ttZM-unsplash.jpg\",\"mediaLink\":\"http://localhost:9199/download/storage/v1/b/<bucket>.appspot.com/o/danielle-cerullo-CQfNt66ttZM-unsplash.jpg?generation=1621716254898&alt=media\"},\"context\":{\"eventId\":\"1621716254902\",\"timestamp\":\"2021-05-22T22:44:14.902Z\",\"even
tType\":\"google.storage.object.finalize\",\"resource\":{\"service\":\"storage.googleapis.com\",\"name\":\"projects/_/buckets/<bucket>.appspot.com/objects/danielle-cerullo-CQfNt66ttZM-unsplash.jpg\",\"type\":\"storage#object\"}}}"}}
>  Firebase Storage download token: c2887239-8b0c-4b47-93e1-974415480b6e {"user":"Firebase Storage download token: c2887239-8b0c-4b47-93e1-974415480b6e","metadata":{"emulator":{"name":"functions"},"function":{"name":"optimizeImage"},"message":"\u001b[90m> \u001b[39m Firebase Storage download token: c2887239-8b0c-4
b47-93e1-974415480b6e"}}
[2021-05-22T20:44:15.672Z] [runtime-status] [8668] Ephemeral server survived. {"metadata":{"emulator":{"name":"functions"},"function":{"name":"optimizeImage"},"message":"[runtime-status] [8668] Ephemeral server survived."}}
i  functions: Finished "optimizeImage" in ~1s {"metadata":{"emulator":{"name":"functions"},"function":{"name":"optimizeImage"},"message":"Finished \"optimizeImage\" in ~1s"}}
[2021-05-22T20:44:15.675Z] [worker-optimizeImage-70ee1f4d-bb78-4c7e-9017-c7d8108f8742]: IDLE {"metadata":{"emulator":{"name":"functions"},"message":"[worker-optimizeImage-70ee1f4d-bb78-4c7e-9017-c7d8108f8742]: IDLE"}}
[2021-05-22T20:44:15.676Z] [work-queue] {"queueLength":0,"workRunningCount":0}
```

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Storage emulator returns 403 Forbidden when trying to access files uploaded with firebaseStorageDownloadTokens set in metadata #3396

[REQUIRED] Environment info

[REQUIRED] Test case

[REQUIRED] Steps to reproduce

[REQUIRED] Expected behavior

[REQUIRED] Actual behavior

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Uh oh!

Storage emulator returns 403 Forbidden when trying to access files uploaded with firebaseStorageDownloadTokens set in metadata #3396

Description

[REQUIRED] Environment info

[REQUIRED] Test case

[REQUIRED] Steps to reproduce

[REQUIRED] Expected behavior

[REQUIRED] Actual behavior

Metadata

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Issue actions