Version 6 of http::POST

Updated 2020-09-07 15:36:19 by ABU

http::POST is an extension of the standard http::geturl commands, providing an easy way to gather all the POST parameters - including files to be uploaded - and to set them up in a proper multipart data-structure that will be passed to the underlying http::geturl command.

Download

Version 1.0.1 (May 2019)

  • [L1 ] with non-standard required packages included

  Reference manual

http::POST 1.0.1 http POST with multipart/form-data

extension of http::geturl for posting data and files

SYNOPSIS

package require Tcl 8.5

package require http::POST ?1.0.1?

  • ::http::POST url ?options? ?-form formSpec?

DESCRIPTION

The http::POST package provides a command for sending HTTP POST requests with data and (very large) files.

::http::POST url ?options? ?-form formSpec?
http::POST is an extension of the standard http::geturl commans, providing an easy way to gather all the POST parameters - including files to be uploaded - and to set them up in a proper multipart data-structure that will be passed to the underlying http::geturl command.

http::POST takes care to gather all the parameters and files to transmit, in a single, large, virtual, mime-multipart channel. Note that all the assembled parts (including very large files such as images, videos, ...) are NOT assembled in memory, nor in a temporary file; they are assembled in a virtual stream and transmitted one block at a time. The return value of http::POST is a token for the transaction, just the same token returned by the underlying http::geturl command.

Standard parameters and -form specification

http::POST accepts all the valid http::geturl's parameters/options, plus a special option : -form

-form formSpecification
A formSpecification is a list of field-specifictions where a field-specification is a list of three parameters (plus two optional parameters) like the following:
fieldName
The name of the form's field
fieldType
can be -text or -file or -channel
fieldValue
The field value. If fieldType is -file, it should be a filename. If fieldType is -channel, it should be a valid channelID (see command chan).
?mimeType?
(optional) it could be used for specifying the mime-type (eg image/jpeg). It is required if fieldType is -channel.
?mimeExtra?
(optional) is could be used for specifying extra parameters as name=value,name=value .... Usually when file contents are specifified via a -channel, you should provide also a file-name (see example below) (these rules may vary on a service by service basis), Note that a field-specification for a -text field, can be abbreviated as a list with just two parameters

fieldName fieldValue Be aware that if you specify contents through channels (eg. using the fieldType -channel), the http::POST command will take ownership of these channels, and they will be closed at the end of the upload-phase. You should take care of closing these channels only if http::POST fails before starting the upload (e.g. bad-url or network errors..)

Examples

Let's suppose we should send some data to service http://somesite/someService by filling thes following form field name, customerId,... passportPhoto (upload of an picture), videocap.. (upload of a short video..))

  # prepare a form-specification ....
 set myformSpec {
    { "surname" -text "Barrow Smith" }
    { "name" "John" }
    { "customerID" -text "123456" }
    { "attachment1"   -file "/data/document1.pdf" }
    { "passportPhoto" -file "/data/mypic.jpg" }
    { "privacyCheck" -text "yes"}                
 }    
  # then add a last more complex parameter:
  #  in the "videocap" parameter we pass a stream channel..  (eg  $myChannel1 )
  # Note that we should also set the mimeType (since it cannot be inferred from the channelName)
  # 
 lappend myFormSpec [list "videocap" -channel  $myChannel1  video/mp4  "filename=Cap1234567.mp4"]

 set token [http::POST https://xyz.com/registerme -form $myformSpec]

  # get the response from token  ...
  # (usually it's a json-formatted response or an XML-formatted response ... it's your resoponsability to decode it ...)
  # ...
  # ... myDecode [http::data $token]
  # ...

 http:cleanup $token
  #  note that the ownership of myChannel1 has been ''taken' by http::POST.
  #  Now that the transmissin is ended, myChannel1 is closed

NOTES

  • Posting contents through https or trough a proxy, just requires the usual preliminaries used for standard http::geturl command, e.g load packages tls and autoproxy
  • you can also send in a single post two or more large files (photos, videos, ...); http::POST takes care to assembly all the parts on fly, without using temporary files or huge memory buffers; all the parts are trasmitted on fly, block-by block as a single (multipart) stream.

Package Dependencies

http::POST requires some auxiliary standard packages:

  • http 2.8.12 ;# previous versions don't work with the HTTP/1.1 100 Continue
  • tcl::chan::string 1.0.3 ;# part of Tcllib-1.19
  • tls 1.7 ;# previous versions don't support the -autoservername option
  • tcl::chan::join ;# not yet standard ... included in lib subdir

KEYWORDS

http::POST

CATEGORY

Networking

COPYRIGHT

 Copyright (c) 2019 A.Buratti

  Example 1 - getting the car-plate ID from a picture

The following is complete example of how to pass data to a real service. We will use an upload service offered by https://platerecognizer.com allowing to send a picture of a car-plate and then get the plate-id as a string. This service requires an authorization token (APIKEY) that can be obtained for free; see http://docs.platerecognizer.com

package require http::POST
package require tls 1.7

http::register https 443 ::tls::socket
  
set APIKEY 4c33e60562f57aeb04b09978267c412a9878707f   ;# write here your APIKEY
set myCarPlateImg ./carplate1.jpg                         ;# .. 
 # I used a test plate downloaded from 
 #  https://images.summitmedia-digital.com/topgear/images/2018/07/19/license-plate-main.jpg


set URL https://api.platerecognizer.com/v1/plate-reader
set headers [list Authorization [list Token $APIKEY]]
set form {}
lappend form [list upload -file $myCarPlateImg]
http::POST $URL -headers $headers -form $form

set data [http::data $token]
http::cleanup $token

 # result is in $data. It should be parsed ( use package json )
 #  {"processing_time":281.635,"version":1,"results":[{"box":{"xmin":171,"ymin":123,"ymax":283,"xmax":395},"plate":"r08097","score":0.769,"dscore":0.698}],"filename":"17_08_carplate1.jpg"}
 # Note a small error ...  plate should be R0B097 (insted of r09097)

  Example 2 - uploading pictures and comments (other than simple ASCII chars)

For another example, we are going to use a different service: https://vgy.me

This site provides a photo-upload service wiithout requiring a registration, therefore no authorization token. Playing with this service, you could post both a photo and some text.

The most interesting aspect of this service is related to the text we're going to transmit; in general all the API services require text as utf-8 strings. If you just write in English, you could simply ignore all this stuff but, how can you transmit some a greek text like this "Καφές: 1,20 ευρώ" (Coffe: 1,20 euro) ?

Hereis the solution

package require http::POST
package require tls 1.7
 # note: this target server require servername during the https handshake
http::register https 443 {::tls::socket -autoservername true}

set URL https://vgy.me
set form {}
lappend form [list "file" -file "./coffee.jpg"  image/jpeg]
lappend form [list title [encoding convertto utf-8 "Καφές"]]
lappend form [list description  [encoding convertto utf-8 "Καφές: 1,20 ευρώ"]]

set token [http::POST ${URL}/upload -form $form]

set data [http::data $token]
http::cleanup $token

 # result is in $data. It should be parsed ( use package json )
 #{"error":false,"size":151424,"filename":"oiy6Kg","ext":"jpg","url":"https://vgy.me/u/oiy6Kg","image":"https://i.vgy.me/oiy6Kg.jpg","delete":"https://vgy.me/delete/Cuf2MrZAJ4wU"}


# .. please don't abuse of this service; if you just uploaded some images for test, remove them by calling the "delete" URL you found in the response.