Shouldn't fmap (+3) (*3) be equivalent to \x -> ((x+3)*3)?

my intuition tells me that since the functor (*3) is equivalent to \x -> x * 3

Functions form a functor instance

instance Functor ((->) r) where ...

Those are the functions mapping from r.

Given a function g :: r -> a you can form a new function h :: r -> b with the help of f :: a -> b via h = fmap f g. It should now be clear that f :: a -> b cannot be applied first but must be applied second. That is h' = (g :: r -> a) . (f :: a -> b) does not make any sense but h = (f :: a -> b) . (g :: r -> a) does.

fmap has to obey two laws:

1. fmap id == id
2. fmap (f . g) == fmap f . fmap g

Your proposed definition, fmap' f g == g . f, satisfies the first law but violates the second law:

fmap' id f == f . id == f == id f  -- OK

fmap' (f . g) h == h . (f . g)
                == (h . f) . g
                == (fmap' f h) . g
                == fmap' g (fmap' f h)
                == (fmap' g . fmap' f) h  -- violation, since (.) is not commutative

The correct definition, fmap f g = f . h, satisfies both:

fmap id f == id . f == f == id f

fmap (f . g) h == (f . g) . h
               == f . (g . h)
               == fmap f (g . h)
               == fmap f (fmap g h)
               == (fmap f . fmap g) h

A "functor", in Haskell, is a higher order type, F, -- "higher order" meaning, it accepts another type variable, a (denoting any type whatever), -- such that we can have

  F     a                      fa
       (a -> b)                 ab
-----------------
  F          b                 fmap ab fa

which is known as "flip fmap" (flip just means that the order of arguments is flipped, flip fmap fa ab == fmap ab fa). There are also some "common sense" laws it must follow.

For F ~ Maybe, say, it means

flip fmap :: Maybe a ->
                  (a    -> b) ->
             Maybe         b

and for F ~ [],

flip fmap :: []    a ->
                  (a    -> b) ->
             []            b

which is more conventionally written as [a] -> (a -> b) -> [b].

In our case here, F a ~ r -> a, or more formally, F a ~ ((->) r) a, which means F ~ ((->) r),

flip fmap :: ((->) r) a ->
                     (a    -> b) ->
             ((->) r)         b

which is more conventionally written as (r -> a) -> (a -> b) -> (r -> b),

  r -> a
       a -> b
 -------------
  r ->      b

which is the same as (r -> a) -> (a -> b) -> r -> b since with types, the arrows associate on the right, corresponding to the fact that applications associate on the left: f a b c is actually ((f a) b) c and

f    a    b    c =  d          --     f  a  b  c
f    a    b = \c -> d          --    (f  a  b) c
f    a = \b    c -> d          --   ((f  a) b) c
f = \a    b    c -> d          --  (((f) a) b) c

are all different ways to write down the same definition, and different ways to write down the same function call.

This then means we need to implement

fmap :: (a -> b) -> (r -> a) -> (r -> b)
fmap    ab           ra          r =  b
   where
   b =

So what could the definition be? Is it up to us to decode what goes where? Well, we must produce a b type value. The only thing we have that can do it for us, is ab :: a -> b.

Can we produce a b without it? Out of the blue? Except for erroring out, no, we can't -- we know nothing about that b type. It can be anything. So we're left with

   b = ab a
   a = 

and now we must get an a somewhere, to use it as an argument to ab. Fortunately, ra can give it to us:

   a = ra r

and r, we already got! So the types did write this implementation for us:

fmap :: (a -> b) -> (r -> a) -> (r -> b)
fmap    ab           ra          r =  b
   where
   b = ab a
   a = ra r

or, simplifying and renaming, we get

fmap f g r = f  ( g  r)
           = (f . g) r

by definition of the function composition, ., as

(.) :: (a -> b) -> (r -> a) -> (r -> b)
(f . g) r = f (g r)

which is a valid syntax definition, otherwise written as

(.) :: (a -> b) -> (r -> a) -> (r -> b)
(.) f g r = f (g r)

or

(.) :: (a -> b) -> (r -> a) -> (r -> b)
(.) f g = \ r -> f (g r)

or

(.) :: (a -> b) -> (r -> a) -> (r -> b)
(.) = \f g r -> f (g r)

All these are equivalent. And its type diagram

         a -> b
    r -> a
   --------------
    r ->      b

As for the intuition, a functorial type value of type F A is, loosely, an F-type "something" than can somehow produce an A-type something, in some F-type sense.

The functor laws mean that F does so in some purely "structural", mechanical way, without regard to what that A value that it produces actually is. In other words, the A values do not influence how they are produced, only the F type itself determines that.

For example, Maybe Int could Maybe produce an Int. Or [Int] could produce several Ints. (*3) can also produce an Int, if we supply it with an Int argument.

What then this fmap is? What does it do? It transforms that would-be produced value. Every functor type must define its fmap, that is what makes it be a functorial type, that it defines the

instance Functor Maybe where
  fmap ab (Just a) = (Just (ab a))

etc. So then, with functions r -> a, which produce that a type value promised by their type, after being applied to an argument, we transform that value by applying the transformation function to it:

 fmap transf mult3 arg = tansf (mult3 arg)

which is just the definition of the functional composition itself, with arguments renamed.

So that's why, in this case,

 fmap (+3) (*3) r = (+3) ((*3) r)
                  = (+3) (r*3)
                  =      (r*3) + 3

we (+3) transform the value produced by (*3) in the ((->) r) sense, which is application to some user-supplied argument, r. So (*3) must be applied first, to get (it to produce) that value.