I am attempting to using a service I control to proxy a file between two services without loading all of its contents into memory in my serbvice. The basic flow is something like this:
ServiceA -> MyService -> ServiceB
MyService receives a multipart/form-data request from ServiceA that contains a file in it and I need to stream that file to ServiceB. Currently I am attempting to processes the incoming request using multipart.Reader by doing something like this:
formPart, err := formReader.NextPart()
for err == nil {
switch formPart.FormName() {
case "myFileField":
outGoingFile := &os.File{}
_, err = io.Copy(outGoingFile, formPart)
if err != nil {
return nil, "", err
}
....
}
Where, outGoingFile is then included in the outgoing request to ServiceB. The problem is that my call to io.Copy returns an invalid argument error. I'm assuming this is since outGoingFile is just an empty instance that is never really initialized using something like os.Create. I've managed to track the source of the error down to the write function in os/file_posix.go (line 48) which throws an error because file.pfd is nil, which causes a panic. I could create a temporary file on the disc but that seems hacky. Some sort of in-memory equivalent of os.File would probably work too, but I don't think that exists.
How can I stream a file between services as I described above?
Note: I'm also doing several other more trivial things with other parts of the incoming request, it's not like I can forward the request as is. The outgoing request is in a completely different format.