How to Design a Petabyte Scale Agentic Lakehouse Using Only AWS Services

You can build a petabyte scale lakehouse that AI agents can safely query using nothing but AWS services. This session covers how we did it at UC Irvine, what AWS gave us out of the box, and the gaps we had to fill ourselves. The environment made the constraint interesting — student records, research data, and operational systems land in the same lakehouse, owned by departments that do not report to each other, under rules that differ per dataset. We walk through cataloging and classification in AWS Glue, Apache Iceberg on S3, fine-grained row and column permissions in Lake Formation, and how partitioning and compaction had to be reworked around how agents query rather than how dashboards were built. We close with the honest part: where AWS native services ended, what we built to cover the gap, and the readiness checks we run before any agent touches sensitive data.

Building production-ready AI Agents with Spring AI and Amazon Bedrock AgentCore

Amazon Bedrock AgentCore enables deploying and operating highly capable AI agents securely at scale, with infrastructure purpose-built for dynamic agent workloads. In this talk, we dive deep into how to implement a production-ready AI Agent in Java using Spring AI, including using the MCP Streamable HTTP Client to communicate with MCP Servers. We host our agent on Amazon Bedrock AgentCore Runtime and explore AgentCore features including Observability in OpenTelemetry-compatible format, Gateway for securely connecting to MCP-compatible tools, short and long-term Memory, and Identity. We close with the Spring AI AgentCore project and its integrations between Spring AI and Bedrock AgentCore.

Self-Hosting an LLM on ECS: Serving, Distribution, Disk, and Autoscaling

Not so long ago I had a customer who needed to serve an open model inside their own AWS account. They already ran ECS for everything, so the question was whether that was enough, or whether they had to bring in Kubernetes or a managed ML platform first. It was enough. In this talk I will show how we put Qwen2.5-7B behind an ALB on ECS GPU instances with vLLM, and what broke on the way to production. The build was the easy part. Three things surprised us, and in each case the measurement disagreed with the obvious answer. Distribution. A scale-out has to get 15 GB of weights onto every node. A peer-to-peer swarm looked like the clever answer, so I built a test rig and ran it at 4, 8, 16 and 32 nodes. Plain S3 in the same region beat it by 11.7x at 32 nodes. Disk. Downloads plateaued at 1.05 Gbit/s and we blamed the network. The network was fine at 4.4 Gbit/s. A default gp3 root volume caps at 125 MB/s, and that number decides your cold start. Autoscaling. The GPU scaling policy sat in INSUFFICIENT_DATA and never fired. On Amazon Linux 2023 a CloudWatch agent running as a sidecar cannot see the GPU, so nvidia-smi never runs and the metric never publishes. I will show the numbers behind each one and the fixes we shipped in our open source Terraform module. I will close with a checklist you can run against your own service: how to confirm your GPU metric is real, how to tell a disk-bound load from a network-bound one, and when local NVMe storage is worth the trouble. This talk is for anyone running model inference on AWS, or about to.