Is there a way to read a table from a network url or from a runpipe external command? It seems that DataFrame.readtable only supports reading from file.
For example in R we could do:
df = read.table(url("http://example.com/data.txt"))
x = read.table(pipe("zcat data.txt | sed /^#/d | cut -f '11-13'"), colClasses=c("integer","integer","integer"), fill=TRUE, row.names=NULL)
With the deprecation of Requests in favor of HTTP here is an example on how to use HTTP.request and the body of the resulting call to request.
julia> using CSV, HTTP
julia> res = HTTP.request("GET", "http://users.csc.calpoly.edu/~dekhtyar/365-Winter2015/data/CARS/cars-data.csv")
HTTP.Messages.Response:
"""
HTTP/1.1 200 OK
Date: Wed, 16 May 2018 12:46:39 GMT
Server: Apache/2.4.18 (Ubuntu)
Last-Modified: Mon, 05 Jan 2015 23:29:09 GMT
ETag: "330f-50bf00ea05b40"
Accept-Ranges: bytes
Content-Length: 13071
Content-Type: text/csv
Id,MPG,Cylinders,Edispl,Horsepower,Weight,Accelerate,Year
1,18,8,307,130,3504,12,1970
2,15,8,350,165,3693,11.5,1970
3,18,8,318,150,3436,11,1970
⋮
13071-byte body
"""
julia> res_buffer = IOBuffer(res.body)
IOBuffer(data=UInt8[...], readable=true, writable=false, seekable=true, append=false, size=13071, maxsize=Inf, ptr=1, mark=-1)
julia> using DataFrames, DataStreams
julia> df = CSV.read(res_buffer)
406×8 DataFrames.DataFrame
│ Row │ Id │ MPG │ Cylinders │ Edispl │ Horsepower │ Weight │ Accelerate │ Year │
├─────┼─────┼─────┼───────────┼────────┼────────────┼────────┼────────────┼──────┤
│ 1 │ 1 │ 18 │ 8 │ 307.0 │ 130 │ 3504 │ 12.0 │ 1970 │
│ 2 │ 2 │ 15 │ 8 │ 350.0 │ 165 │ 3693 │ 11.5 │ 1970 │
│ 3 │ 3 │ 18 │ 8 │ 318.0 │ 150 │ 3436 │ 11.0 │ 1970 │
⋮
│ 405 │ 405 │ 28 │ 4 │ 120.0 │ 79 │ 2625 │ 18.6 │ 1982 │
│ 406 │ 406 │ 31 │ 4 │ 119.0 │ 82 │ 2720 │ 19.4 │ 1982 │