Large scale inference for the enterprise – architecture, security, and best practices

This keynote presentation will explore large-scale inference infrastructure for enterprises, focusing on architecture, security, and best practices. As an AI CTO at Dell, I had a privilege to be involved in our largest projects and I will share insights into both hardware and software solutions, highlighting deployed architectures and key learnings from customer implementations. I will show you how to build a SW stack that is able to deliver 30 billion recommendations daily across 4 billion web pages (which is live with one of our customers). Learnings will cover GPU utilization optimization, K8s scheduling mechanisms, monitoring distributed systems using open telemetry approaches balancing security best practices with innovation coming from open source world.