Skip to content

Commit 9185698

Browse files
Update support for GCS resumable uploads with retries and signed URLs
1 parent dcf587d commit 9185698

File tree

5 files changed

+855
-614
lines changed

5 files changed

+855
-614
lines changed

CHANGELOG.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,15 @@
22

33
All notable changes to this project will be documented in this file.
44

5+
## [1.0.2]
6+
- Implemented support for Google Cloud Storage resumable uploads and chunked client uploads.
7+
- Added retry mechanism with exponential backoff for GCS upload failures based on retryable status codes.
8+
- Enabled support for user-provided signed URLs, allowing resumable uploads to work with externally generated session URIs.
9+
- Updated the API endpoint from https://v1.fastpix.io/on-demand/uploads to https://api.fastpix.io/v1/on-demand/upload for obtaining signed URLs.
10+
11+
## [1.0.1]
12+
- Update readme.md and license
13+
514
## [1.0.0]
615

716
### Features:

README.md

Lines changed: 93 additions & 67 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ This SDK helps you efficiently upload large files from the browser by splitting
44

55
Please note that this SDK is designed to work only with FastPix and is not a general purpose uploads SDK.
66

7-
## Features:
7+
# Features:
88

99
- **Chunking:** Files are automatically split into chunks (configurable, default size is 16MB/chunk).
1010
- **Pause and Resume:** Allows temporarily pausing the upload and resuming after a while.
@@ -13,84 +13,99 @@ Please note that this SDK is designed to work only with FastPix and is not a gen
1313
- **Error Handling and Reporting:** Comprehensive error handling to manage upload failures gracefully and inform users of issues.
1414
- **Customizability:** Developers can customize the chunk size and retry attempts based on their specific needs and network conditions.
1515

16-
## Installation
16+
# Prerequisites:
1717

18-
To install the SDK, you can use npm or your favourite node package manager 😉:
18+
## Getting started with FastPix:
19+
20+
To get started with SDK, you will need a signed URL.
21+
22+
To make API requests, you'll need a valid **Access Token** and **Secret Key**. See the [Basic Authentication Guide](https://docs.fastpix.io/docs/basic-authentication) for details on retrieving these credentials.
23+
24+
Once you have your credentials, use the [Upload media from device](https://docs.fastpix.io/reference/direct-upload-video-media) API to generate a signed URL for uploading media.
25+
26+
## Installation:
27+
28+
To install the SDK, you can use NPM, CDN, or your preferred package manager:
29+
30+
### Using NPM:
1931

2032
```bash
2133
npm i @fastpix/resumable-uploads
2234
```
2335

24-
## Basic Usage
25-
26-
To get started with SDK, you will need a signed URL.
36+
### Using CDN:
2737

28-
To make API requests, you'll need a valid **Access Token** and **Secret Key**. See the [Basic Authentication Guide](https://docs.fastpix.io/docs/basic-authentication) for details on retrieving these credentials.
38+
```bash
39+
<script src="https://cdn.jsdelivr.net/npm/@fastpix/resumable-uploads@latest/dist/uploads.js"></script>
40+
```
2941

30-
Once you have your credentials, use the [Upload media from device](https://docs.fastpix.io/reference/direct-upload-video-media) API to generate a signed URL for uploading media.
42+
## Basic Usage
3143

32-
**Import**
44+
## Import
3345

3446
```javascript
35-
import { Uploader } from "@fastpix/resumable-uploads"
47+
import { Uploader } from "@fastpix/resumable-uploads";
3648
```
3749

38-
**Integration**
50+
## Integration
3951

4052
```javascript
41-
const fileUploader = Uploader.init({
42-
endpoint: 'https://example.com/signed-url', // Replace with the signed URL.
43-
file: mediaFile, // Provide the media file you want to upload.
44-
chunkSize: 5120, // Specify the chunk size in kilobytes (KB). Minimum allowed chunk size is 5120KB (5MB).
53+
try {
54+
const fileUploader = Uploader.init({
55+
endpoint: "https://example.com/signed-url", // Replace with the signed URL.
56+
file: mediaFile, // Provide the media file you want to upload. From <input type="file" />
57+
chunkSize: 5 * 1024, // Minimum allowed chunk size is 5120KB (5MB).
4558

4659
// Additional optional parameters can be specified here as needed
47-
})
60+
});
61+
} catch (error) {
62+
// Handle initialization errors, such as invalid configuration or missing file
63+
console.error("Failed to initialize uploads:", error?.message);
64+
}
4865
```
4966
50-
**Monitor the upload progress through lifecycle events**
67+
## Monitor the upload progress through lifecycle events
5168
5269
```javascript
53-
5470
// Track upload progress
55-
fileUploader.on('progress', event => {
56-
console.log("Upload Progress:", event.detail);
57-
});
71+
fileUploader.on("progress", (event) => {
72+
console.log("Upload Progress:", event.detail.progress);
73+
});
5874

5975
// Handle errors during the upload process
60-
fileUploader.on('error', event => {
61-
console.error("Upload Error:", event.detail.message);
62-
});
76+
fileUploader.on("error", (event) => {
77+
console.error("Upload Error:", event.detail.message);
78+
});
6379

6480
// Trigger actions when the upload completes successfully
65-
fileUploader.on('success', event => {
66-
console.log("Upload Completed");
67-
});
81+
fileUploader.on("success", (event) => {
82+
console.log("Upload Completed");
83+
});
6884

6985
// Track the initiation of each chunk upload
70-
fileUploader.on('attempt', event => {
71-
console.log("Chunk Upload Attempt:", event.detail);
72-
});
86+
fileUploader.on("chunkAttempt", (event) => {
87+
console.log("Chunk Upload Attempt:", event.detail);
88+
});
7389

7490
// Track failures of each chunk upload attempt
75-
fileUploader.on('chunkAttemptFailure', event => {
76-
console.log("Chunk Attempt Failure:", event.detail);
77-
});
91+
fileUploader.on("chunkAttemptFailure", (event) => {
92+
console.log("Chunk Attempt Failure:", event.detail);
93+
});
7894

7995
// Perform an action when a chunk is successfully uploaded
80-
fileUploader.on('chunkSuccess', event => {
81-
console.log("Chunk Successfully Uploaded:", event.detail);
82-
});
96+
fileUploader.on("chunkSuccess", (event) => {
97+
console.log("Chunk Successfully Uploaded:", event.detail);
98+
});
8399

84100
// Triggers when the connection is back online
85-
fileUploader.on('online', event => {
86-
console.log("Connection Online");
87-
});
101+
fileUploader.on("online", (event) => {
102+
console.log("Connection Online");
103+
});
88104

89105
// Triggers when the connection goes offline
90-
fileUploader.on('offline', event => {
91-
console.log("Connection Offline");
106+
fileUploader.on("offline", (event) => {
107+
console.log("Connection Offline");
92108
});
93-
94109
```
95110
96111
## Managing Uploads
@@ -115,35 +130,46 @@ You can control the upload lifecycle with the following methods:
115130
fileUploader.abort(); // Abort the current upload
116131
```
117132
118-
119133
## Parameters Accepted
120134
121-
This SDK supports the following parameters:
122-
123-
124-
- `endpoint` (required):
125-
126-
The URL endpoint where the file will be uploaded.
127-
128-
- `file` (required):
129-
130-
The file object that you want to upload.
131-
132-
- `chunkSize` (optional):
133-
134-
Size of each chunk in kilobytes (KB). Default is 16 MB (16384 KB), with a minimum of 5 MB (5120 KB) and a maximum of 500 MB (512000 KB).
135-
136-
- `maxFileSize` (optional):
137-
138-
The maximum file size allowed for upload, specified in kilobytes (KB). This helps prevent excessively large uploads.
139-
140-
- `retryChunkAttempt` (optional):
135+
The upload function accepts the following parameters:
136+
137+
| Name | Type | Required | Description |
138+
| ------------------- | ----------------------------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
139+
| `endpoint` | `string` or `() => Promise<string>` | Required | The signed URL endpoint where the file will be uploaded. Can be a static string or a function returning a `Promise` that resolves to the upload URL. |
140+
| `file` | `File` or `Object` | Required | The file object to be uploaded. Typically a `File` retrieved from an `<input type="file" />` element, but can also be a generic object representing the file. |
141+
| `chunkSize` | `number` (in KB) | Optional | Size of each chunk in kilobytes. Default is `16384` KB (16 MB).<br>**Minimum:** 5120 KB (5 MB), **Maximum:** 512000 KB (500 MB). |
142+
| `maxFileSize` | `number` (in KB) | Optional | Maximum allowed file size for upload, specified in kilobytes. Files exceeding this limit will be rejected. |
143+
| `retryChunkAttempt` | `number` | Optional | Number of retry attempts per chunk in case of failure. Default is `5`. |
144+
| `delayRetry` | `number` (in seconds) | Optional | Delay between retry attempts after a failed chunk upload. Default is `1` second. |
145+
146+
### Example usage of integrating all parameters with `Uploader.init`
147+
148+
```js
149+
// Get the selected file from the input
150+
const selectedFile = document.querySelector("#fileInput").files[0];
151+
152+
try {
153+
const fileUploader = Uploader.init({
154+
endpoint: "https://example.com/signed-url", // Signed URL for uploading
155+
file: selectedFile, // File or Object to upload
156+
chunkSize: 10 * 1024, // 10 MB per chunk
157+
maxFileSize: 100 * 1024, // 100 MB max file size
158+
retryChunkAttempt: 6, // Retry each failed chunk up to 6 times
159+
delayRetry: 2, // Wait 2 seconds between retry attempts
160+
});
161+
} catch (error) {
162+
// Handle initialization errors
163+
console.error("Failed to initialize upload:", error?.message);
164+
}
165+
```
141166
142-
Number of retries per chunk in case of failure. Default is 5 retries.
167+
# References
143168
144-
- `delayRetry` (optional):
145-
146-
Delay between retry attempts (in milliseconds) after a chunk upload fails. Default is 1000 ms.
169+
[FastPix Homepage](https://www.fastpix.io/)
170+
[FastPix Dashboard](https://dashboard.fastpix.io/)
171+
[Uploads github](https://github.com/FastPix/web-uploads-sdk)
172+
[API Reference](https://docs.fastpix.io/reference/on-demand-overview)
147173
148174
# Detailed Usage:
149175

0 commit comments

Comments
 (0)