How to avoid fleet option while generating spot instances?

Question

I am using this cloudformation template to create spot instances and it is working as expected.

https://github.com/shantanuo/cloudformation/blob/master/updated/linux_training.tpl.txt

The only change I would like is to remove "Fleet" part from the type here...

Type: 'AWS::EC2::SpotFleet

The reason: When I try to stop an instance, I get a message that says "Fleet instances can not be stopped".

How do I buy spot instances using "normal" or default option?

The (AWS User interface) wizard allows me to generate spot instances those can be stopped and started as and when required. Looking for the same functionality from cloudformation template.

As suggested in the comments, I have modified the template to look like this. It works as expected. But it will create only a "template". I will still need to login to console to initiate a new instance using the template. I need to automate end-to-end process.

Parameters:
  LatestAmiId:
    Type: 'AWS::SSM::Parameter::Value'
    Default: /aws/service/ami-amazon-linux-latest/amzn2-ami-minimal-hvm-arm64-ebs
  OksoftSG:
    Type: 'AWS::EC2::SecurityGroup::Id'
  KeyName:
    Type: 'AWS::EC2::KeyPair::KeyName'
  MyInstanceType:
    Type: String
    Default: r6g.medium
    AllowedValues:
      - r6g.medium
      - r6g.4xlarge
      - r6g.12xlarge
  MyIpAddress:
    Type: String
 
Resources:
  MySpotInstance:
    Type: 'AWS::EC2::LaunchTemplate'
    Properties:
      LaunchTemplateData:
        ImageId: !Ref LatestAmiId
        KeyName: !Ref KeyName
        SecurityGroupIds: 
          - !Ref OksoftSG
        InstanceType: !Ref MyInstanceType

        BlockDeviceMappings:
          - DeviceName: /dev/xvda
            Ebs:
              DeleteOnTermination: true
              VolumeType: standard
              VolumeSize: 400

        InstanceMarketOptions: 
          MarketType: spot
          SpotOptions:
            InstanceInterruptionBehavior: stop
            SpotInstanceType: persistent

  MyInstance:
    Type: 'AWS::EC2::Instance'
    Properties:
      "LaunchTemplate" : 
        LaunchTemplateId: !Ref MySpotInstance
        Version: !GetAtt MySpotInstance.LatestVersionNumber

When I delete the cloudformation template, it does not remove the spot request that it has created.

user_174777 · Answer

A Spot Fleet is a way of requesting a certain amount of compute resources (RAM, CPU) that will "keep running" even if some Spot Instances are removed. This is why you cannot request to stop an instance -- because it will be automatically replaced.

Normally, a request for Spot Instance can be specified in a RunInstances() command. However, I was unable to find this option in AWS::EC2::Instance.

The closest I could find was using AWS::EC2::Instance and specifying a AWS::EC2::LaunchTemplate, which contains SpotOptions for specifying Spot Instances.

Frankly, I think that it is quite unusual to use a CloudFormation template to launch Spot Instances. This is because CloudFormation is normally used to launch resources in a reliable, consistent manner. In contrast, Spot Instances can disappear with very little notice. Therefore, Spot Instances are normally launched via specific API requests rather than via CloudFormation templates.

user_248823 · Answer

Here is working example of creating LaunchTemplate with persistent spot request which creates an instance based on the template. The instances running on persistent spot request can be stopped, but not terminated (a replacement will be launch as soon as there is a spot place):

Parameters:
  LatestAmiId:
    Type: 'AWS::SSM::Parameter::Value'
    Default: /aws/service/ami-amazon-linux-latest/amzn-ami-hvm-x86_64-ebs
  OksoftSG:
    Type: 'AWS::EC2::SecurityGroup::Id'
  KeyName:
    Type: 'AWS::EC2::KeyPair::KeyName'
  MyInstanceType:
    Type: String
    Default: m3.medium
    AllowedValues:
      - t2.small
      - m3.medium
      - m3.xlarge
      - m3.2xlarge
      - i3.xlarge
  MyIpAddress:
    Type: String    
 
Resources:

  SpotPersistantLaunchTemplate:
    Type: AWS::EC2::LaunchTemplate
    Properties:  
      LaunchTemplateData:
        InstanceType: !Ref MyInstanceType
        ImageId: !Ref LatestAmiId
        SecurityGroupIds:
          - !Ref OksoftSG
        KeyName: !Ref KeyName
        InstanceMarketOptions: 
          MarketType: spot
          SpotOptions:           
            InstanceInterruptionBehavior: stop
            SpotInstanceType: persistent
        BlockDeviceMappings:
          - DeviceName: /dev/xvda
            Ebs:
              DeleteOnTermination: true
              VolumeType: standard
              VolumeSize: 400                  
        UserData: !Base64 
          'Fn::Join':
            - ''
            - - |
                #!/bin/bash -xe
              - >
                yum install -y docker mysql git python-pip >
                /tmp/line1_succ.txt 2> /tmp/line1_err.txt
              - >
                service docker start > /tmp/line2_succ.txt 2>
                /tmp/line2_err.txt 
              - >
                echo 'india' | passwd ec2-user --stdin >
                /tmp/line4a_succ.txt 2> /tmp/line4a_err.txt 
              - >
                sed -i 's|[#]*PasswordAuthentication
                no|PasswordAuthentication yes|g' /etc/ssh/sshd_config >
                /tmp/line4b_succ.txt 2> /tmp/line4b_err.txt 
              - >
                /etc/init.d/sshd restart > /tmp/line4c_succ.txt 2>
                /tmp/line4c_err.txt 
              - >
                docker run -d -p 8887:8888 -v /tmp:/tmp shantanuo/notebook >
                /tmp/line3_succ.txt 2> /tmp/line3_err.txt 
              - >-
                pip install aws-ec2-assign-elastic-ip > /tmp/line4_succ.txt
                2> /tmp/line4_err.txt 
                 
              - 'sudo sh -c "echo '
              - !ImportValue secretKey
              - |2
                 >> /home/ec2-user/mysecret.txt" 
              - 'sudo sh -c "echo '
              - !ImportValue accessKey
              - |2
                 >> /home/ec2-user/myaccesskey.txt" 
              - >-
                /usr/local/bin/aws-ec2-assign-elastic-ip --access-key ''`cat
                /home/ec2-user/myaccesskey.txt`''  --secret-key ''`cat
                /home/ec2-user/mysecret.txt`'' --valid-ips '
              - !Ref MyIpAddress
              - |
                '                

  MyPersistantSpotInstance:
    Type: AWS::EC2::Instance    
    Properties:
      LaunchTemplate:
        LaunchTemplateId: !Ref SpotPersistantLaunchTemplate
        Version: 1

How to avoid fleet option while generating spot instances?

2 Answers