Marko Radosavljevic

Leading initiatives to optimize Large Language Model (LLM) inference performance on GPUs, targeting significant improvements in throughput and latency for advanced AI applications.

Focusing on maximizing hardware utilization and efficiency across GPU architectures to enhance the scalability and cost-effectiveness of LLM deployments.

Designed and tuned high-performance ML inference pipelines for custom Tensor Processing hardware, utilizing PyTorch models as reference, with a strong focus on low-latency and high-throughput execution.

Implemented advanced parallel algorithms to leverage hardware concurrency, scaling the inference pipeline to a multi-chip distributed system and achieving a remarkable 45% end-to-end latency reduction.

Developed and optimized an internal tool for High Availability and Disaster Recovery (HADR) within Azure SQL Managed Instance, leveraging C# and PowerShell for robust solutions.

Improved the cloud migration experience for customers transitioning from on-premises solutions, streamlining the process and enhancing user satisfaction.

Engineered a client-side ML inference system utilizing cached (local) models instead of repeated network calls, achieving approximately 60% speedup in inference time.

Developed and implemented system components using C# and TypeScript, ensuring robust functionality and efficient data handling.