That attempt is definitely not naive, though it is not the usual way of doing things.
Since Reader r a is really just a function r -> a behind the scenes, your definitions of getArguments and produceResponse are essentially:
getArguments :: Config -> Request -> Arguments
produceResponse :: Config -> Header -> Arguments -> Response
Notice that in this case Config is always the first parameter, so things like getArguments req will not work - after all, Request is the second parameter of getArguments, not the first one, so you can't just apply getArguments to req.
What you need to do is to first apply getArguments to Config. Since we're actually working with a Reader Config ... and not a simple function Config -> ..., we do that by binding getArguments:
handle req = do header <- initializeHeader
argumentGetter <- getArguments
-- rest omitted
Since getArguments is a Reader Config (Request -> Arguments), argumentGetter will simply be a function Request -> Arguments. Using such a function is then trivial:
handle :: Request -> Reader Config Response
handle req = do header <- initializeHeader
argumentGetter <- getArguments
let args = argumentGetter req
-- rest omitted
You could then go on and apply the same treatment to produceResponse and things would work.
However, at the start I said that this is not the usual way of doing things. Consider these definitions:
getArguments :: Request -> Reader Config Arguments
produceResponse :: Header -> Arguments -> Reader Config Response
If we unwrap the Reader, we'll see that these are actually just:
getArguments :: Request -> Config -> Arguments
produceResponse :: Header -> Arguments -> Config -> Response
That is, Config always comes as the last parameter. This is in contract to what we had at the start, where Config always came first.
Defining getArguments and produceResponse like this works nicely in your original example:
args <- getArguments req - getArguments applied to req produces a Reader Config Arguments and binding that to args makes args an Arguments
produceResponse header args - produceResponse applied to header and args produces a Reader Config Response, which fits nicely as the last statement in the do block
Note that we could do a transformation like this for Reader since Reader is just a function anyway, but in general for any Monad m, there's a quite a bit of difference between m (x -> y) and x -> m y, with the latter being the most common for "parameterized" monadic operations.
For example, consider readFile :: FilePath -> IO String. You provide a FilePath and it gives you an IO action that produces the contents of the file. If instead it were IO (FilePath -> String), it would be an IO action that produces a function FilePath -> String - that is, a pure function that, when given a FilePath, produces the contents of a file. However, since it's a pure function it can't have any side effects, so it can't actually read the file and thus this would not really work (getFile could do crazy stuff like read the whole filesystem first, but let's not get into that).