Dev: Learn About AWS Lambda (Spike)

[Rayna-Yu]

Motivation

• Throughout the project, we’ll be using AWS Lambda for hosting and our Davis API data ingestion pipeline. For this “spike” ticket, you’ll be learning about AWS Lambda and gain an understanding of the tool so we can use it later on.

Context

• Lambda is AWS’s serverless compute service. You upload a function, and AWS runs it on demand in response to a trigger (a schedule, an event from another AWS service, etc.) without you managing a server. You are billed per invocation and execution time, rather than for a server sitting idle.

Learning Goals

• How do you actually write, package, and deploy a Lambda function (console vs. CLI vs. IaC)?
• How does a Lambda get triggered. Specifically, how does EventBridge Scheduler hook up to a Lambda on a fixed interval?
• What are the practical limits worth knowing (execution timeout, memory/CPU tradeoffs, payload size, concurrency limits)?
• What does “cold start” actually mean, when does it happen, and how noticeable is it?
• How do environment variables, secrets, and IAM permissions work for a Lambda (e.g., how would it hold Davis API credentials safely)?
• How do you view logs and debug a Lambda after it runs (CloudWatch)?
• What does local testing/development look like before deploying?

Acceptance Criteria

• A Google Doc is sent in Slack channel before standup that covers everything in the task list

Tasks

• Task 1: Understand what Lambda is and how it fits our stack
  Read AWS's own Lambda overview/docs to understand the core concept (function-as-a-service, event-driven, pay-per-invocation)
  Identify how Lambda is meant to connect to EventBridge Scheduler (for the Davis ingestion pipeline)
• Task 2: Research deployment & packaging
  Research how a Lambda function is typically written, packaged, and deployed (console, CLI, IaC like SAM/CDK/Terraform )
  Research what a Node.js Lambda handler looks like (function signature, event object, context object, what gets returned)
  Look at a few example handler snippets in docs/tutorials to get a feel for the code shape
• Task 3: Research triggers relevant to BHO
  Research how EventBridge Scheduler → Lambda works at a 1-minute cadence (docs, articles, forum posts on typical setups)
  Research how API Gateway → Lambda works for a typical REST endpoint (request/response shape, what's automatically handled vs. what we'd write)
• Task 4: Research limits and constraints
  Look up default and maximum execution timeout, memory limits, and payload size limits
  Look up concurrency limits and what happens if EventBridge fires while a previous invocation is still running (relevant for a 1-minute schedule)
  Research cold starts — what causes them, typical impact for Node.js, whether it's a real concern at our expected invocation frequency
• Task 5: Research config, secrets, and permissions
  Research how environment variables work in Lambda
  Research whether Secrets Manager or Parameter Store is the more standard practice for credentials instead of plain environment variables
  Research IAM execution roles at a conceptual level — what a Lambda needs permission to do, and how that's typically scoped
• Task 6: Research observability
  Research how CloudWatch Logs work for Lambda by default (what gets logged automatically, how to find a given invocation's logs)
  Research common patterns for surfacing Lambda failures somewhere other than raw logs (e.g., writing status to a database, alerting)
• Task 7: Write it up
  Draft a findings/reference doc answering the "what we're trying to learn" questions from the ticket and any other notes based on your research
  Summarize the deployment/trigger model in plain English for teammates who haven't read the docs themselves
  List open questions for the TL