Description
As requested by @fur-q
2016-01-27 13:29:47 daurnimator furq: so the other day I played with multipart form encoding
2016-01-27 13:29:53 daurnimator furq: I'm wondering how you envision the API
2016-01-27 13:31:49 furq er
2016-01-27 13:32:19 furq i guess the same as a normal post but allowing for file handles as values
2016-01-27 13:33:31 furq i take it the api for a form-urlencoded post is request(url, { key = "value" })
2016-01-27 13:33:34 daurnimator furq: so this is where I got up to: https://github.com/daurnimator/lua-http/compare/WIP-multipart-form-data?expand=1
2016-01-27 13:33:34 furq or something similar
2016-01-27 13:34:04 daurnimator furq: the thing about those is that you can provide all sort of fun things in the parts: they all have their own headers section
2016-01-27 13:34:32 daurnimator a section usually has a least content-type. but also often content-disposition
2016-01-27 13:34:40 furq oh yeah
2016-01-27 13:34:58 furq it's been ages since i've touched any of this
2016-01-27 13:35:32 daurnimator furq: so the current encoding function I've got there, you provide an iterator that returnsheaders, bodypairs
2016-01-27 13:36:00 furq it would be nice if you could just do a post with a file handle and have it automatically generate multipart with the file set to application/octet-stream
2016-01-27 13:36:09 daurnimator where body can be a string, a file object, or another iterator...
2016-01-27 13:36:19 furq as far as manual control goes you probably know better than i do
2016-01-27 13:36:58 daurnimator (because multipart/form-data sometimes contain another multipart/form-data as one of their parts.... its often very nested)
2016-01-27 13:37:28 furq i can see why so many http libraries don't bother with this
2016-01-27 13:38:23 furq also isn't the content-disposition always form-data
2016-01-27 13:38:54 daurnimator furq: no. it can be e.g. attachment
2016-01-27 13:38:56 furq oh never mind
2016-01-27 13:39:20 furq for forms it's always form-data but it can also be 'form-data; name="foo"' or whatever
2016-01-27 13:39:38 daurnimator furq: except for file uploads.
2016-01-27 13:40:01 furq that's not what this w3 page says
2016-01-27 13:40:17 daurnimator in which case it can beattachment; name="foo"; filename=somethingascii; filename*=someunicodeencodingmess
2016-01-27 13:40:38 furq maybe the rfc says different
2016-01-27 13:41:06 furq actually the rfc doesn't mention that either
2016-01-27 13:41:31 daurnimator furq: there's a whole IANA registry for it
2016-01-27 13:41:38 furq nice
2016-01-27 13:41:41 daurnimator furq: https://www.iana.org/assignments/cont-disp/cont-disp.xhtml
2016-01-27 13:42:22 furq "attachment" says it's for emails
2016-01-27 13:43:35 daurnimator furq: splitting hairs here; but nothing says that you can't serve an email over HTTP.
2016-01-27 13:43:39 zash You can do attachment in HTTP too, I think browsers ask you were to save it then
2016-01-27 13:43:42 furq "multipart/form-data" contains a series of parts. Each part is expected to contain a content-disposition header [RFC 2183] where the disposition type is "form-data", and where the disposition contains an (additional) parameter of "name", where the value of that parameter is the original field name in the form.
2016-01-27 13:43:43 daurnimator furq: perhaps you're confusing HTML vs HTTP?
2016-01-27 13:43:45 daurnimator zash: yep.
2016-01-27 13:44:21 furq i'm only interested in multipart/form-data
2016-01-27 13:44:33 daurnimator furq: e.g. https://support.microsoft.com/en-us/kb/260519
2016-01-27 13:44:37 zash Uploading files uses some disposition in a form-data thing too
2016-01-27 13:44:40 furq i wasn't aware any other multipart http requests existed, but you can always count on http to have some fucking dismal hole
2016-01-27 13:45:11 furq er
2016-01-27 13:45:18 furq isn't that for responses?
2016-01-27 13:45:26 daurnimator furq: well that link is.
2016-01-27 13:45:31 TheCycoONE a bunch of multiparts
2016-01-27 13:45:43 furq that has nothing to do with multipart though
2016-01-27 13:45:48 daurnimator furq: but why does that matter? this code path is hit for both client and server.
2016-01-27 13:46:02 zash https://en.wikipedia.org/wiki/MIME#Mixed-Replace is fun btw
2016-01-27 13:46:45 zash Server push from 1998 :)
2016-01-27 13:46:49 furq oh christ, multipart responses?
2016-01-27 13:46:55 furq that sounds really awful
2016-01-27 13:47:01 zash furq: No it's awesome
2016-01-27 13:47:17 zash Especially the mixed-replace one, which means you can animate ASCII and stuff
2016-01-27 13:47:36 TheCycoONE saves a lot of tcp sessions
2016-01-27 13:47:51 daurnimator furq: so at this point; I've very tempted to say "get a MIME library; not lua-http's problem"
2016-01-27 13:48:14 furq form-data posts are pretty common though
2016-01-27 13:48:15 zash daurnimator: luasocket includes one...
2016-01-27 13:48:25 furq i don't care if you ignore all the other edge cases that seemed like a good idea in 1998
2016-01-27 13:48:27 daurnimator zash: luasocket also includes an SMTP library.
2016-01-27 13:48:56 zash and base64 encoding
2016-01-27 13:49:08 TheCycoONE ... why not a luacurl that wraps everything curl supports?
2016-01-27 13:49:10 furq apparently browsers don't even support multipart/mixed etc any more
2016-01-27 13:49:18 furq at least firefox dropped it ages ago
2016-01-27 13:49:22 daurnimator infact, IIRC the luasocket MIME routines are locked away in the SMTP section, and can't readily be used by HTTP clients.
2016-01-27 13:49:42 TheCycoONE I think the WAP Push spec requires multipart/mixed
2016-01-27 13:49:43 zash daurnimator: eh, just require"mime"?
2016-01-27 13:49:53 zash WAP? Hahahahaha
2016-01-27 13:50:01 daurnimator TheCycoONE: WAP is something else entirely
2016-01-27 13:50:10 furq oh nvm that's in XHR
2016-01-27 13:51:07 furq even so i doubt it's worth your time supporting that on the server side
2016-01-27 13:51:15 zash Yet I still have no idea how to respond to a form POST with 2xx without the browser re-posting if you refresh the page...
2016-01-27 13:51:20 furq even if you've already written a multipart encoder
2016-01-27 13:51:46 daurnimator zash: the multipart stuff is locked up behind smtp.message()
2016-01-27 13:51:51 daurnimator zash: use a 303.
2016-01-27 13:52:30 TheCycoONE zash, PRG pattern (https://en.wikipedia.org/wiki/Post/Redirect/Get) - what daurnimator said
2016-01-27 13:52:34 zash daurnimator: but that's not very RESTful
2016-01-27 13:52:38 TheCycoONE ah
2016-01-27 13:52:56 rjek What is this, hipster central? Get the fuck out with your REST
2016-01-27 13:53:00 zash also already did that http://hg.prosody.im/issue-tracker/rev/cd49d6c23da3
2016-01-27 13:53:10 TheCycoONE use PUT in rest for things that you don't want duplicated
2016-01-27 13:53:24 TheCycoONE and then handle it with the id
2016-01-27 13:53:54 zash rjek: I'm using RESTful ironically!
2016-01-27 13:54:04 TheCycoONE but why is a web browser interacting with REST directly?
2016-01-27 13:54:49 furq anyway
2016-01-27 13:55:23 furq daurnimator: beyondrequest(url, { file = handle })andrequest(url, { file = { handle = handle, type = "image/png" })i don't care what you do
2016-01-27 13:55:32 furq +}
2016-01-27 13:56:11 furq i'm pretty sure that will cover 99% of use cases anyway
2016-01-27 13:57:09 furq and by 99% i of course mean by number of requests made, not number of potential bullshit rfc-says-SHOULD uses
2016-01-27 13:57:13 daurnimator furq: how about:myrequest:set_multipart_body(function() local h = new_headers(); h:append("content-type", "image/png"); h:append("content-disposition", "form-data; name=file"); coroutine.yield(h, handle) end)
2016-01-27 13:57:41 furq that seems like the kind of thing you could magic up for me when i pass a file handle
2016-01-27 13:58:46 daurnimator furq: I can't inger the file name; and I don't want to infer the mime type :p
2016-01-27 13:58:49 daurnimator s/inger/infer/
2016-01-27 13:59:04 furq the mime type should default to application/octet-stream
2016-01-27 13:59:13 furq and the name should be whatever the key is
2016-01-27 14:00:04 daurnimator furq: the alternative is some sort of multipart/form-data construction class..... local mb = new_multipart_body(); mb:add_form_data("myname", "image/png", handle); request:set_body(mb);
2016-01-27 14:00:41 furq is that how you add form data anyway
2016-01-27 14:00:52 furq minus the new_multipart_body ofc
2016-01-27 14:01:52 furq also if you do it that way then new_multipart_body should take a table of initial form-data values
2016-01-27 14:02:06 furq as well as add_form_data
2016-01-27 14:02:22 daurnimator ehhhhhh
2016-01-27 14:03:05 furq new_multipart_body({ name = { handle, "image/png" } }) is much less hassle
2016-01-27 14:03:34 daurnimator furq: problem is I want to imply order (cause the order large things are sent tends to matter)
2016-01-27 14:03:58 furq if you want a defined order then use add_form_data
2016-01-27 14:04:03 furq and by you i mean the user
2016-01-27 14:05:00 furq http://docs.python-requests.org/en/latest/user/quickstart/#post-a-multipart-encoded-file
2016-01-27 14:05:11 furq it would be nice if it was this simple, although i'm aware this is a high-level api wrapper
2016-01-27 14:05:26 furq maybe you could separate that stuff into a helper libary
2016-01-27 14:05:47 daurnimator furq: helper libraries are hard when you want them to be methods on existing objects....
2016-01-27 14:09:07 furq also you should definitely infer the content-type as "text/plain" for string form-data values
2016-01-27 14:09:22 furq that is the official(tm) default
2016-01-27 14:10:17 furq actually
2016-01-27 14:10:44 furq As with all multipart MIME types, each part has an optional "Content-Type", which defaults to text/plain. If the contents of a file are returned via filling out a form, then the file input is identified as the appropriate media type, if known, or "application/octet-stream".
2016-01-27 14:11:00 furq there are defaults defined for files and strings
2016-01-27 14:11:49 furq also nice grammar "l. masinter"