Meta's Monster: Unveiling XFaaS, the Hyperscale Serverless Powerhouse

At Meta, handling trillions of daily function calls necessitates groundbreaking solutions. In a recent paper titled "XFaaS: A Serverless Framework for Hyperscale Stateless Functions" [1], researches delve into their internal serverless champion. This article explores the inner workings of XFaaS, the challenges Meta faced with traditional serverless approaches, and the key takeaways for companies building large-scale serverless systems.

The Hyperscale Serverless Struggle

Serverless functions offer a developer-friendly approach: write code, define triggers, and let the platform manage the infrastructure. But at Meta's scale, traditional serverless limitations emerge. As detailed in the XFaaS paper, Meta grappled with two key challenges:

• Cold Starts: When a function is invoked for the first time, it experiences a longer startup time due to the lack of a pre-running instance. This can significantly impact performance, especially for user-facing functions.

• Resource Management: Efficiently managing resource utilization across a vast server network is crucial for cost control. Traditional serverless approaches often struggle to handle the high variance in load experienced by Meta's services, leading to either wasted resources or overloaded downstream services.

XFaaS: The Cold-Start Slayer

Meta's answer is XFaaS, a serverless platform architected for handling massive workloads and addressing the challenges outlined in the paper. Here's a glimpse into its core functionalities:

• High-Density Execution: Unlike traditional serverless where each function runs in its own container, XFaaS packs multiple functions into a single Linux process. This approach, feasible due to the high trust environment of Meta's private cloud, maximizes resource utilization.

• Cooperative JIT Compilation: To combat cold starts, XFaaS utilizes a clever technique: when a new function version is deployed, one server gathers profiling data and transmits it to others. This enables pre-compilation of the function on other servers, significantly reducing subsequent startup times.

• Time-Shifted Execution: Not all functions are created equal. XFaaS prioritizes essential tasks by leveraging "time-shifted computing.". Critical functions that directly impact user experience are executed immediately. Meanwhile, non-critical tasks like background jobs are placed in a queue and executed during periods of lower load. This ensures responsiveness for users while maximizing resource utilization.

• Global Resource Management: Efficient resource allocation is vital for cost-effectiveness. XFaaS employs a global Resource Isolation and Management (RIM) system. RIM gathers resource usage data across the entire platform, allowing XFaaS to dynamically adjust the number of concurrent functions running and prevent overloading downstream services. Additionally, downstream services can send feedback to XFaaS, further optimizing resource allocation.

Lessons for the Serverless Masses

While the specifics of XFaaS may not be directly applicable to everyone, the underlying principles offer valuable insights:

• Prioritize Cold Start Reduction: Even minor improvements in cold start times can significantly enhance performance (as demonstrated by XFaaS's cooperative JIT compilation).

• Resource Management is Paramount: A holistic view of resource utilization across the entire system is essential for cost-effective scaling.

• Embrace Time-Based Flexibility: Consider delaying non-critical tasks to optimize resource allocation during peak usage periods.

Conclusion

Meta's XFaaS is a testament to the ingenuity required for managing serverless functions at massive scale. By tackling cold starts, optimizing resource management, and leveraging a unique execution architecture, they've achieved a cost-effective and performant platform. The core takeaways from their work offer valuable guidance on how to leverage serverless functions at scale.

To stay ahead of the curve and make the best decisions for yourself and your team, subscribe to the Manager's Tech Edge newsletter! Weekly actionable insights in decision-making, AI, and software engineering.

References

1. Alireza Sahraei, Soteris Demetriou, Amirali Sobhgol, Haoran Zhang, Abhigna Nagaraja, Neeraj Pathak, Girish Joshi, Carla Souza, Bo Huang, Wyatt Cook, Andrii Golovei, Pradeep Venkat, Andrew McFague, Dimitrios Skarlatos, Vipul Patel, Ravinder Thind, Ernesto Gonzalez, Yun Jin and Chunqiang Tang. (2023). XFaaS: A Serverless Framework for Hyperscale Stateless Functions.