YAML - Appending Content before comments/New lines

Question

I have the following YAML file

template.yaml

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
    sam-app

    Sample SAM Template for sam-app

# More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst
Globals:
    Function:
        Timeout: 20
        MemorySize: 512
        Environment: # More info about Env Vars: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#environment-object
          Variables:
            TABLE_NAME: !Ref OrdersTable
            ENDPOINT_OVERRIDE: ""

Resources:
  GetOrderFunction:
    Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
    Properties:
      CodeUri: target/aws-sam-java-rest-1.0.0.jar
      Handler: com.amazonaws.handler.GetOrderHandler::handleRequest
      Runtime: java8
      Policies:
        - DynamoDBReadPolicy:
            TableName: !Ref OrdersTable
      Events:
        GetOrder:
          Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
          Properties:
            Path: /orders/{order_id}
            Method: get
  OrdersTable:
    Type: AWS::DynamoDB::Table
    Properties:
      TableName: orders_table
      AttributeDefinitions:
        - AttributeName: orderId
          AttributeType: S
      KeySchema:
        - AttributeName: orderId
          KeyType: HASH
      BillingMode: PAY_PER_REQUEST


Outputs:
    HelloWorldApi:
      Description: "API Gateway endpoint URL for Prod stage for Hello World function"
      Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/hello/"

    GetOrderFunction:
      Description: "GetOrder Lambda Function ARN"
      Value: !GetAtt GetOrderFunction.Arn

    GetOrdersFunction:
      Description: "GetOrders Lambda Function ARN"
      Value: !GetAtt GetOrdersFunction.Arn

    UpdateOrderFunction:
      Description: "UpdateOrder Lambda Function ARN"
      Value: !GetAtt UpdateOrderFunction.Arn

    DeleteOrderFunction:
      Description: "DeleteOrder Lambda Function ARN"
      Value: !GetAtt DeleteOrderFunction.Arn

    CreateOrderFunction:
      Description: "CreateOrder Lambda Function ARN"
      Value: !GetAtt CreateOrderFunction.Arn

and I am trying to add another Property to Resources:OrdersTable:Properties the following way

main.py

from ruamel.yaml import YAML
yaml = YAML()
with open('template.yaml', 'r') as f:
    dataMap = yaml.load(f)
    dataMap['Resources']['OrdersTable']['Properties']['Tags'] = [
        {
            'Key': 'Name1',
            'Value': 'Value1'
        },
        {
            'Key': 'Name2',
            'Value': 'Value2'
        },
        {
            'Key': 'Name3',
            'Value': 'Value3'
        }]
        yaml.dump(dataMap, open('new.yaml', 'w'))

the problem is that the output file keeps the new lines that was between BillingMode and Outputs

here is the output file (only the effected section)

new.yaml

Resources:
  OrdersTable:
    Type: AWS::DynamoDB::Table
    Properties:
      TableName: orders_table
      AttributeDefinitions:
      - AttributeName: orderId
        AttributeType: S
      KeySchema:
      - AttributeName: orderId
        KeyType: HASH
      BillingMode: PAY_PER_REQUEST


      Tags:
      - Key: Name1
        Value: Value1
      - Key: Name2
        Value: Value2
      - Key: Name3
        Value: Value3
Outputs:
  HelloWorldApi:
    Description: API Gateway endpoint URL for Prod stage for Hello World function
    Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/hello/"

and this is how I want the new file to be

Resources:
  OrdersTable:
    Type: AWS::DynamoDB::Table
    Properties:
      TableName: orders_table
      AttributeDefinitions:
      - AttributeName: orderId
        AttributeType: S
      KeySchema:
      - AttributeName: orderId
        KeyType: HASH
      BillingMode: PAY_PER_REQUEST
      Tags:
      - Key: Name1
        Value: Value1
      - Key: Name2
        Value: Value2
      - Key: Name3
        Value: Value3


Outputs:
  HelloWorldApi:
    Description: API Gateway endpoint URL for Prod stage for Hello World function
    Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/hello/"

I've found that the BillingMode map looks like this {'BillingMode': [None, None, CommentToken(' ', line: 104, col: 19), None]}, but I dont know what to do with that knowledge. Any help will be appreciated.

user_1307905 · Answer

Comments (and blank lines are treated as comments), in ruamel.yaml are currently (0.17) associated with a node that comes before it. In this case of a mapping comment occuring in a mapping is associated with the key.

So what you need to do is reassociate the comment with the new key. Doing

 mapvar =  dataMap['Resources']['OrdersTable']['Properties']
 mapvar.ca.items['Tags'] = mapvar.ca.items.pop('BillingMode')

should do the trick. The line: 104 is not used on output and doesn't have to be changed. You can this more generic and search for the last key in the mapping, pop its comment and then assign that to the newly inserted key (if the new key is already in the mapping, you don't want to do that).

YAML - Appending Content before comments/New lines

1 Answers