Sep. 10, August 2024 – Phala Network, a pioneering blockchain platform focused on Trusted Execution Environments (TEEs), has announced a significant milestone in its roadmap to decentralized AI. The company, which has been at the forefront of verifiable computing since its founding in 2019, has just released the first-ever benchmark for TEE-enabled GPUs, marking a new era in secure, high-performance decentralized AI.
Phala Network’s vision for decentralized AI
Phala Network has consistently pushed the boundaries of what blockchain can achieve, particularly in secure computation.
Since launching its mainnet in 2021, Phala has grown to support over 40,000 TEE CPU nodes, providing robust infrastructure for decentralized applications. These nodes enable Web3 developers to offload complex computations from smart contracts to Phala's off-chain network, ensuring data privacy and security while generating verifiable proofs and oracles.
Phala’s technology has been instrumental in powering various Web3 use cases, including social apps, games, and AI-driven agents.
With the release of their latest GPU-TEE benchmark, Phala is taking a decisive step toward realizing its vision of decentralized AI. The new benchmark focuses on the performance of Nvidia’s H100 and H200 GPUs when integrated into Phala’s TEE technology. This integration is crucial for scaling decentralized AI, offering the high-performance compute power necessary for training and running large language models (LLMs) like LLaMA 3 and Microsoft Phi while maintaining stringent security and privacy standards.
The role of TEE in decentralized AI
The integration of Trusted Execution Environments (TEEs) within decentralized AI systems is crucial for ensuring the protection of sensitive AI workloads, particularly as AI becomes increasingly critical in various industries. By extending trusted computing principles beyond CPUs to AI accelerators like GPUs, TEEs can provide a new layer of defense, safeguarding model weights and inference data through cryptographic measures. This evolution in infrastructure security enables the secure deployment of decentralized AI systems, ensuring that data remains protected even in the event of a compromise.
In 2024, Nvidia introduced its H100 Tensor Core GPUs, equipped with advanced Confidential Computing (CC) solutions that integrate TEE functionality directly into the GPU architecture. These solutions include features like encrypted memory, secure boot, and hardware firewalls, all designed to protect sensitive data during processing.
Phala’s adoption of Nvidia’s TEE-enabled GPUs represents a significant advancement in decentralized AI, providing a foundation for secure, transparent AI systems that are not controlled by any single entity. This move is aligned with the broader goals of decentralized AI, which aims to democratize access to AI technologies, making them more accessible and equitable.
Benchmarking GPU-TEE performance
Phala Network’s latest benchmark for TEE-enabled Nvidia H100 GPUs provides critical insights into secure AI performance. Key findings include:
- Minimal overhead: The introduction of TEE incurs a performance overhead of less than 7% on typical LLM queries, with almost zero impact on larger models like LLaMA-3.1-70B. For smaller models, the overhead is primarily linked to CPU-GPU data transfers via PCIe rather than GPU computation itself.
- Scalability: Phala’s report shows that the overhead becomes negligible for larger AI models, highlighting that the GPU’s compute-heavy tasks are not hindered by TEE mode.
These results validate the viability of TEE-enabled GPUs for developers looking to implement secure, decentralized AI applications without compromising performance.
The future of Decentralized AI
As the demand for decentralized AI grows, the need for robust and secure infrastructure becomes paramount. The future of decentralized AI hinges on advancements in technologies like confidential computing, which offers the promise of enhanced security by encrypting data at the hardware level. Moreover, the integration of network and tenant isolation guarantees, along with innovations in operational and physical security, will be critical in building resilient AI systems. These measures not only protect against external threats but also ensure that decentralized AI can scale securely, providing equitable access to advanced AI capabilities.
Phala Network’s work in decentralized AI is a critical step toward addressing these challenges. By integrating TEE technology into GPUs and providing the first comprehensive benchmark, Phala is not only advancing the technical capabilities of decentralized AI but also setting new standards for security and transparency in AI systems.
The release of this benchmark is just the beginning. As Phala continues to innovate, the decentralized AI ecosystem is poised to grow, offering new opportunities for developers, businesses, and communities to harness the power of AI in a way that is secure, transparent, and equitable for all.