Measuring Lambda cold starts across three runtimes

Published

Updated

Cold start numbers quoted in blog posts are almost never reproducible, because the two variables that dominate init duration are package size and what the module scope does before the handler is called. So instead of trusting a table, I built a small harness that deploys the same handler shape to three runtimes and reads initDuration straight out of the platform report records.

The handler shape

Every runtime gets the same three phases: import the SDK, construct one client, then answer. Nothing else runs at module scope.

import { DynamoDBClient, GetItemCommand } from "@aws-sdk/client-dynamodb";

const client = new DynamoDBClient({});
const table = process.env.TABLE_NAME!;

export const handler = async (event: { id: string }) => {
  const result = await client.send(
    new GetItemCommand({
      TableName: table,
      Key: { pk: { S: event.id } },
    }),
  );

  return { found: result.Item !== undefined };
};

The Python version is deliberately boring, so the only difference between the two is the runtime itself:

import os

import boto3

client = boto3.client("dynamodb")
TABLE = os.environ["TABLE_NAME"]


def handler(event, _context):
    result = client.get_item(TableName=TABLE, Key={"pk": {"S": event["id"]}})
    return {"found": "Item" in result}

Forcing a cold start

Publishing a new version is the only way I trust to guarantee a cold container. Updating the environment variables works too, but version publishing gives you an alias to invoke, which keeps the measurement honest.

set -euo pipefail

version=$(aws lambda publish-version \
  --function-name cold-start-node \
  --query Version --output text)

aws lambda invoke \
  --function-name "cold-start-node:${version}" \
  --payload '{"id":"probe"}' \
  --cli-binary-format raw-in-base64-out \
  /dev/null

Reading the init duration

The platform report record carries the number, so there is no need to instrument the handler at all. One CloudWatch Logs Insights query covers every function in the experiment:

{
  "queryString": "filter @type = 'REPORT' | stats count(*) as invocations, avg(@initDuration) as avgInit, pct(@initDuration, 95) as p95Init by @log",
  "logGroupNames": [
    "/aws/lambda/cold-start-node",
    "/aws/lambda/cold-start-python",
    "/aws/lambda/cold-start-rust"
  ],
  "startTime": 1770768000,
  "endTime": 1770854400
}

What actually moved the needle

Across 200 forced cold starts per runtime, the ranking was stable but the gaps were smaller than folklore suggests. What mattered, in order:

  1. Bundling. Shipping the tree-shaken client instead of the whole SDK cut Node.js init by roughly half.
  2. Module-scope work. One extra SSMClient fetch at import time cost more than the runtime choice.
  3. Package size beyond about 10 MB, where the download phase starts to dominate.

The runtime is the variable everyone argues about and the one I would tune last. Bundle first, move configuration out of module scope second, and only then go shopping for a faster runtime.