AWS CodeBuild S3 cache for swift lambda

Question

So I have some aws swift lambdas that I deployed via sam deploy. That works fine. The swift lambda looks like this: aws-samples

I am currently in the middle of building a CI/CD pipeline using Codepipeline and Codebuild. My Codebuild project executes the following buildspec.yml and is configured to cache to S3:

version: 0.2
phases:
  build:
    commands:
      - sam build
      - sam package -t template.yml --s3-bucket bucketName --output-template-file packaged.yaml
artifacts:
  files:
    - packaged.yaml
cache:
  paths:
    - ".aws-sam/**/*"

To build the swift lambdas sam buildexecuted the following makefile as the function in template.yml is set to: BuildMethod: makefile

Makefile:

### Add functions here and link them to builder-bot format MUST BE "build-FunctionResourceName in template.yaml"

build-ExpiredMediaItemProcessorLambda: builder-bot

builder-bot:
    $(eval $@PRODUCT = $(subst build-,,$(MAKECMDGOALS)))
    $(eval $@BUILD_DIR = $(PWD)/.aws-sam/build-$($@PRODUCT))
    $(eval $@STAGE = $($@BUILD_DIR)/lambda)
    $(eval $@ARTIFACTS_DIR = $(PWD)/.aws-sam/build/$($@PRODUCT))
    
    # prep directories
    mkdir -p $($@BUILD_DIR)/lambda $($@ARTIFACTS_DIR)

    # Compile application
    swift build --product $($@PRODUCT) -c release --build-path $($@BUILD_DIR)

    # copy deps
    ldd '/$($@BUILD_DIR)/release/$($@PRODUCT)' | grep swift | cut -d' ' -f3 | xargs cp -Lv -t /$($@BUILD_DIR)/lambda

    # copy binary to stage
    cp $($@BUILD_DIR)/release/$($@PRODUCT) $($@BUILD_DIR)/lambda/bootstrap

    # copy app from stage to artifacts dir
    cp $($@STAGE)/* $($@ARTIFACTS_DIR)

I got the makefile from the linked aws-samples project above which I modified slightly to work on codebuild.

Now to my question: How do I get codebuild S3 cache to work with swift lambdas? when I cache the .aws-sam/**/* folder swift cannot compile because the build paths are different due to the different build machines. The error looks like so:

swift build --product LambdaName -c release --build-path /codebuild/output/src409372700/src/.swift-build/build-LambdaName
33  [1/863] Compiling CSotoExpat xmltok_impl.c
34  [2/863] Compiling CSotoExpat xmltok_ns.c
35  [3/865] Compiling INIParser INIParser.swift
36  :0: error: PCH was compiled with module cache path '.aws-sam/build-LambdaName/x86_64-unknown-linux-gnu/release/ModuleCache/1LD7OVICEM9JB', but the path is currently '/codebuild/output/src409372700/src/.aws-sam/build-LambdaName/x86_64-unknown-linux-gnu/release/ModuleCache/1LD7OVICEM9JB'
37  :0: error: missing required module 'SwiftShims'
38  [4/865] Compiling Logging Locks.swift
39  :0: error: PCH was compiled with module cache path '.aws-sam/build-LambdaName/x86_64-unknown-linux-gnu/release/ModuleCache/1LD7OVICEM9JB', but the path is currently '/codebuild/output/src409372700/src/.aws-sam/build-LambdaName/x86_64-unknown-linux-gnu/release/ModuleCache/1LD7OVICEM9JB'
40  :0: error: missing required module 'SwiftShims'

As far as I understand this is due to the fact that swift build cannot cope with a changing or relative path.

After some digging I found this stack overflow post which has a similar problem. I tried there solution and the swift compile error is gone but after a rebuild swift compiles the entire project again even if no code was changed.

Any help is much appreciated! Thanks!

AWS CodeBuild S3 cache for swift lambda

0 Answers