By default, AWS accounts have a Lambda storage limit of 75 GB. Uploading new packages of code to the lambda service counts towards this limit. Additionally, when using Lambda managed versions, the storage size of each version of your lambdas will also count towards this limit. Without pruning those older versions, you may find yourself greeted with the Code storage limit exceeded error.

1
2
3

 An error occurred: [LambdaName] - Code storage limit exceeded. \
    (Service: AWSLambdaInternal; Status Code: 400; \
    Error Code: CodeStorageExceededException; Proxy: null).

You’ve typically got two options here. Option one, switch off version management of your lambda functions. If you are not using lambda versions for something like canary rollouts or rollbacks, maybe you don’t require them in the first place. YAGNI doesn’t just apply to code! Option two, prune the versions so you only hold on to the latest n versions. Either way, if you’re seeing the error above, you’ve got some versions to clean up before you can start deploying again.

There are some great tools out there already that can integrate into existing workflows.

• If you’re using the Serverless Framework, there’s the Prune Plugin.
• Lumigo’s Lambda Janitor will provision a lambda that cleans up old versions for you with the use of a periodic CloudWatch event schedule.

Alternatively, for something a little more ad-hoc, the script below uses the AWS CLI and JQ to prune your lambda versions, with the option to hold on to a number of the latest ones.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55

#!/usr/bin/env bash

# Exit immediately if a command exits with a non-zero status.
set -e

# Check we have valid AWS credentials to hand
# This will exit and stop the script if we don't.
SESSION_VALID=$(aws sts get-caller-identity --output json)

# Check arguments are present
USAGE="Usage: $0 <function-name> <versions-to-keep>"
[ $# -eq 0 ] && echo "Function name required. $USAGE" \
  && exit 1
[ $# -eq 1 ] && echo "Number of versions to keep required. $USAGE" \
  && exit 1

# Dry run first before actually deleting the versions
DRY_RUN=${DRY_RUN:-true}
# Take the number of versions to keep and add one as tail \
# is not zero indexed
KEEP_COUNT=$(($2 + 1))

if [ "$DRY_RUN" = "true" ]; then
  echo -ne "Planning to delete the following versions for $1:\n\t"
else
  echo -ne "Deleting the following versions for $1:\n\t"
fi

aws lambda list-versions-by-function --function-name $1 --output json \
  | jq -r '.Versions | sort_by(.LastModified) | reverse[] | .Version' \
  | grep -wv '$LATEST' \
  | tail -n +"$KEEP_COUNT" \
  | tail -r \
  | while read -r versionQualifier; do
    if [ "$DRY_RUN" != "true" ]; then
      aws lambda delete-function \
        --function-name $1 \
        --qualifier $versionQualifier
    fi
    echo -ne "$versionQualifier "
  done

if [ "$DRY_RUN" = "true" ]; then
  echo ""
  read -r -p "Would you like to delete these versions? [y/N] " response
  if [[ "$response" =~ ^([yY][eE][sS]|[yY])$ ]]
  then
    # Rerun with DRY_RUN to false to actually delete the versions
    DRY_RUN=false ./$0 $1 $2
  else
      echo "Cancelled - No deletes performed!"
  fi
else
  echo -n "\nVersions for $1 successfully pruned."
fi

[Download prune-lambda-versions.sh]

At the core of the script is the aws lambda list-versions-by-function and subsequent piped commands. To delete all the lambda versions except the number you want to keep the script performs the following steps.

1. Grab a list of all the versions available for a given lambda and return the JSON object.
2. Using JQ to select the Versions array, we sort by .LastModified, reverse the list (putting newest at the top) and return an array of version qualifiers.
3. Remove the $LATEST version from the list using grep and the invert match flag (-v). We certainly don’t want to remove that.
4. Use tail to return the list of versions minus the latest n versions that we want to keep.
5. Reverse the list again using the tail reverse flag (-r) so we delete from the oldest version to newest. Just in case a panicked CTRL+C is in order.
6. If we’re not performing a dry run, delete the function version for each qualifier.

There is something reassuring about a shell script. To shell script is to capture the unexpected tasks of the day. The unforeseen tasks, the laborious tasks, the kind of tasks you didn’t expect to be fighting with. It’s a good feeling, cherry-picking your terminal history and committing it for relative eternity. To take a “step-by-step” guide and turn it into a simple dot slash command. If only to help the next engineer who finds themselves as you did, an unforeseen task in hand, with a newly open terminal on screen.