Symfony 3.3 and Swiftmailer - mail created and sent by controller deferred by server

I'm trying to use Swiftmailer to send emails out from a website. The emails keep getting deferred because Swiftmailer is attempting to use my server's IP address rather than localhost as the relay:

Aug  2 14:18:28 picus sm-mta[21171]: v72IIS0I021171: from=<Test@test.com>, size=347, class=0, nrcpts=1, msgid=<91d4a1a70de9fed0a2c04e682e435405@swift.generated>, proto=ESMTP, daemon=MTA-v4, relay=localhost [127.0.0.1]
Aug  2 14:18:28 picus sm-mta[21173]: v72IIS0I021171: to=<person@gmail.com>, delay=00:00:00, xdelay=00:00:00, mailer=esmtp, pri=120347, relay=example.com. [my.servers.ip.address], dsn=4.0.0, stat=Deferred: Connection refused by example.com.

My Symfony controller code, config, and parameters -

Relevant controller code:

if ($form->isSubmitted() && $form->isValid()) {
    $data = $form->getData();

    $this->addFlash('success', 'Message sent successfully');

    $data['message'] = str_replace("\n.", "\n..", $data['message']);

    $mail = (new \Swift_Message())
        ->setSubject("[From My Website] - {$data['subject']}")
        ->setFrom($data['email'])
        ->setTo('person@gmail.com')
        ->setBody("{$data['name']} wrote the following message:\n\n{$data['message']}");

    $this->get('mailer')->send($mail);

    return $this->redirect($this->generateUrl('_home'));
}

config.yml:

# Swiftmailer Configuration
swiftmailer:
    transport: '%mailer_transport%'
    host: '%mailer_host%'
    username: '%mailer_user%'
    password: '%mailer_password%'
    port: '%mailer_port%'
    spool:
        type: file
        path: '%kernel.cache_dir%/swiftmailer/spool'

parameters.yml:

parameters:
    mailer_transport: sendmail
    mailer_host: 127.0.0.1
    mailer_user: null
    mailer_password: null
    mailer_port: null

What's really frustrating is that if I create a message using bin/console swiftmailer:email:send, and then flush the spool (bin/console swiftmailer:spool:send) it is sent properly. It's only when I create and send a message through my controller that there's an issue.

What am I doing wrong?