-
Notifications
You must be signed in to change notification settings - Fork 399
MSC2246: Asynchronous media uploads #2246
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 3 commits
Commits
Show all changes
30 commits
Select commit
Hold shift + click to select a range
b439277
Proposal for asynchronous media uploads
tulir a83c79c
Add security consideration and mention possible /create request body
tulir 9a395ed
Expand on security consideration
tulir 29e3463
Change error code for existing media PUT
tulir 7cf22be
Add spam prevention error, example response and auth requirements
tulir 658aac8
Add query parameters to control downloading
tulir bbd7d08
Rename max_stall to max_stall_ms
tulir 0bffcb7
Merge branch 'master' into asynchronous_uploads
tulir 4d009a9
Specify /create request body content
tulir 1cbc04e
Update links, versions and prefixes
tulir c65f2bf
Add recommended maximum delay for starting upload
tulir 63cef50
Explicitly deny uploading to other users' media IDs
tulir 8ccf85f
Prevent spam by ratelimiting instead of limiting number of IDs
tulir 12e907b
Remove streaming requirement
tulir d582bb3
Add unstable prefixes for new error codes
tulir 173edf3
Add missing words
tulir f438754
Change /create endpoint to use v1
sumnerevans 725675c
Reorganize /upload spec and integrate feedback
sumnerevans d55f1f9
Rename max_stall_ms -> timeout_ms
sumnerevans 955177b
Mention that maximum value for timeout_ms should be imposed by the se…
sumnerevans 823fcca
Mention that the timeout_ms can be ignored if the media exists already
sumnerevans 3b00026
Change M_NOT_YET_UPLOADED to use 504 status code
sumnerevans 9627af2
Remove retry_after_ms optional field
sumnerevans 045c21e
Make unused_expires_at the deadline for the upload to complete rather…
sumnerevans 011031b
Add notes about suggested rate-limiting techniques
sumnerevans 6cb7e31
Recommend 24 hours instead of 1 minute
sumnerevans fedc697
Merge pull request #3 from beeper/async-uploads-rate-limiting-improve…
tulir 7652f59
Clarify that rate limiting can apply on /create and /upload
sumnerevans 098dd90
Clarify that unused_expires_at is a POSIX millisecond timestamp
sumnerevans 9559ab0
Merge pull request #4 from beeper/asynchronous_uploads
tulir File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,55 @@ | ||
# Asynchronous media uploads | ||
Sending media to Matrix currently requires that clients first upload the media | ||
to the content repository and then send the event. This is a problem for some | ||
use cases, such as bridges that want to preserve message order, as reuploading | ||
a large file would block all messages. | ||
|
||
## Proposal | ||
This proposal proposes a way to send the event containing media before actually | ||
uploading the media, which would make the aformentioned bridge message order | ||
preservation possible without blocking all other messages behind a long upload. | ||
|
||
In the future, this new functionality could be used for streaming file | ||
transfers, as requested in [#1885]. | ||
|
||
### Content repository behavior | ||
The proposal adds two new endpoints to the content repository API and modifies | ||
the behavior of the download endpoint. | ||
|
||
#### `POST /_matrix/media/r0/create` | ||
Create a new MXC URI without content. Like `/upload`, this endpoint returns the | ||
tulir marked this conversation as resolved.
Show resolved
Hide resolved
|
||
`content_uri` that can be used in events. Additionally, the response would | ||
tulir marked this conversation as resolved.
Show resolved
Hide resolved
|
||
contain a `media_id` field with the media ID part of the MXC URI for | ||
convenience (see [MXC format] in the spec). | ||
|
||
This endpoint could have a JSON body that contains metadata, such as the mime | ||
type of the media that's going to be uploaded. In the future, the body could | ||
also contain access control settings (related: [MSC701]). | ||
|
||
TODO: decide if body is needed already and if yes, spec body schema | ||
|
||
#### `PUT /_matrix/media/r0/upload/{mediaId}` | ||
tulir marked this conversation as resolved.
Show resolved
Hide resolved
|
||
Upload content to a MXC URI that was created earlier. If the endpoint is called | ||
with a media ID that already has content, the request should be rejected with | ||
the error code `M_CANNOT_OVERRIDE_MEDIA`. | ||
tulir marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
#### Behavior change in `/download` | ||
When another client tries to download media that has not yet been uploaded, the | ||
content repository should stall the request until it is uploaded. Optionally, | ||
tulir marked this conversation as resolved.
Show resolved
Hide resolved
|
||
content repository implementations may send the data that has already been | ||
uploaded and stream data as it comes in from the sender. | ||
|
||
## Tradeoffs | ||
|
||
## Potential issues | ||
Other clients may time out the download if the sender takes too long to upload | ||
media. | ||
|
||
## Security considerations | ||
turt2live marked this conversation as resolved.
Show resolved
Hide resolved
|
||
There may be DoS risks in creating lots of media IDs. Especially if those media | ||
IDs are then sent to rooms, but not given any content, the proposed stalling | ||
method could cause a lot of open HTTP connections. | ||
turt2live marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
[#1885]: https://github.com/matrix-org/matrix-doc/issues/1885 | ||
[MXC format]: https://matrix.org/docs/spec/client_server/latest#matrix-content-mxc-uris | ||
[MSC701]: https://github.com/matrix-org/matrix-doc/issues/701 |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.