A new 5G-based test platform quantifies generative AI performance under varying wireless conditions, helping engineers optimize AI applications, define future testing metrics, and accelerate 6G-ready intelligent network development.
As generative AI applications expand across smartphones, AI glasses, robots, and other connected devices, accurately measuring user experience under real-world network conditions has become increasingly challenging. Addressing this gap, Rohde & Schwarz and the China Mobile Research Institute have demonstrated a testing framework that quantitatively evaluates generative AI performance over wireless networks using the CMX500 5G one-box signalling tester.
The testbed focuses on measuring how network conditions affect the responsiveness of cloud-based large language models (LLMs). In the demonstrated architecture, the LLM runs on a cloud or server, while a client application on a smart terminal communicates through the CMX500, which emulates a broad range of wireless environments. Engineers can configure network latency, jitter, fading models, RF impairments, and signal strength to recreate conditions ranging from ideal coverage to cell-edge connectivity.
The platform measures key AI performance indicators including Time to First Token (TTFT)—the delay before an AI model begins generating a response—and Tokens Per Second (TPS), which reflects inference throughput. Test results are collected and analyzed through host PC software, enabling engineers to compare AI responsiveness under different network scenarios using graphical performance reports.
The approach provides a quantitative framework for evaluating the impact of wireless networks on generative AI applications, an area where standardized testing methodologies are still emerging. With organizations such as 3GPP and the GTI alliance exploring AI service metrics, such measurement platforms could contribute to future benchmarking and standardization efforts. The collected data can also guide optimization of AI application design while helping network operators fine-tune infrastructure for latency-sensitive AI services.
The testing solution is built around the CMX500, an integrated 5G signaling tester supporting standalone (SA) and non-standalone (NSA) networks across FR1, FR2, and FR3 frequency ranges up to 18 GHz. It combines RF conformance testing, protocol validation, VoNR acoustic testing, carrier aggregation, and an integrated IMS server within a single platform, making it suitable for both commercial 5G validation and advanced 5G-Advanced and 6G research.
By linking wireless network performance directly with AI inference quality, the demonstration highlights a practical step toward developing measurable user experience standards for next-generation AI-enabled mobile networks and future 6G services.
