serverless.yml Output exports does not work as expected

All of my exports in the serverless files are not setting the export values in cloudformation.

As an example, I have the following section in one of my serverless.yml files:

Outputs:
  UserPoolName:
    Value: ${self:custom.userPoolName}
    Export:
      Name:  ${self:custom.stage}-userPoolName

But when I try to access it with

pool: 
  'Fn::ImportValue': ${self:custom.stage}-userPoolName

it says:

'user-service-dev - No export named dev-userPoolName found.'

And I checked the exports section in the cloud formation console and its empty. This is same for the other serverless files as well.

Another export I have is

Outputs:
  UserTableArn:
    Value:
        Fn::GetAtt:
        - UserTable
        - Arn
    Export:
      Name: ${self:custom.stage}-userTableArn

This value is also not available. What have I done incorrectly here?