Silent Comet project: AI-Powered Network Troubleshooting Assistant

Network troubleshooting has always been a challenging aspect of IT operations, often requiring years of experience and deep technical knowledge. To address this challenge, Amartus has developed and showcased an innovative AIOps solution at MEF’s flagship Global NaaS Event. This groundbreaking solution bridges the expertise gap in Network Troubleshooting by leveraging advanced AI technology. It was created as part of the MEF NaaS Accelerator Program, an initiative launched by MEF to foster collaboration among MEF member service providers and technology suppliers, with the goal of defining the next generation of Network-as-a-Service (NaaS).

Our solution offers a significant improvement in network automation. It combines Large Language Models (LLMs) with Retrieval-Augmented Generation (RAG) technology to revolutionize network troubleshooting processes and accelerate Root Cause Analysis (RCA) resolution. By making vast standardization knowledge easily accessible and eliminating traditional barriers to network expertise. As part of the MEF Accelerator project Silent Comet, we demonstrated how AI can be effectively integrated into practical network operations while maintaining high-performance standards.

The business value of the solution lies in its ability to reduce downtime and operational inefficiencies caused by network issues. By streamlining troubleshooting processes, organizations can minimize their reliance on highly specialized personnel, resulting in reduced costs and improved response times.

Here are more details about how our AI-powered solution is transforming network troubleshooting.

The Challenge

Network engineers, especially those early in their careers, face several critical challenges:

• Complex network topologies requiring extensive knowledge
• Diverse equipment from multiple vendors
• Time-consuming diagnostic processes
• Gather real-time data from the network
• Possess organization-specific network knowledge

Our goal was to create an intelligent assistant that could support engineers while minimizing operational costs and maintaining high accuracy.

Our Solution

We developed a sophisticated multi-agent RAG application that combines the power of Large Language Models (LLMs) with practical network diagnostics tools.

1. Intelligent Communication Layer

• Natural language processing for intuitive interaction
• Context-aware responses based on network topology
• Real-time diagnostic capabilities

Figure 1 – A screenshot of our application after uploading the network topology diagram.

2. Network Integration

• Seamless integration with existing network infrastructure
• Support for multiple vendor environments
• Real-time data collection through ping, traceroute, and direct device communication
• Draw.io XML-based network topology diagram parsing
• NetworkX library for internal network representation

3. User-Friendly Interface

• Chat-based interaction for quick problem-solving
• Interactive network topology visualization
• Real-time feedback on diagnostic processes
• Tool activity visualization
• Text chunk streaming for immediate response delivery
• Contextual network path visualization

Figure 2 – Our real-time AI tool visualization feature shows users in real time which processes are active and how their inputs are being processed.

Figure 3 – Our contextual view feature visualizes paths between network devices, allowing users to instantly grasp device relationships and dependencies for more effective troubleshooting.

Technical Implementation

The solution leverages several cutting-edge technologies:

Core Architecture

• Multi-Agent System:
• • Topology Agent: Handles network diagram data collection
  • Devices Agent: Manages gathering devices’ data
  • Insights Agent: Analyzes collected data
  • Router Agent: Orchestrates query distribution

• LangChain/LangGraph for multi-agent RAG implementation
• Custom-built network topology parser
• Advanced prompt engineering for accurate responses

Figure 4 – Solution architecture.

User Interface

• Streamlit-based frontend for smooth user experience
• Real-time visualization of diagnostic processes
• Intuitive chat interface

Network Environment

• ContainerLab for simulating multi-vendor networks (Cisco, Juniper, Nokia)
• Automation scripts for provisioning and deprovisioning the environment

Evaluation Framework

• Comprehensive testing framework
• Continuous evaluation system
• Integration with DeepEval and PyTest
• LangSmith for debugging and response analysis

Benefits

Our solution demonstrates the potential of Silent Comet project in the following areas:

Enhanced Efficiency

• Reduced troubleshooting time
• Lower operational costs
• Improved first-time resolution rates

Knowledge Integration

• Successfully combined public and private network data
• Implemented sophisticated reasoning capabilities
• Created adaptive learning mechanisms

User Experience

• Minimal learning curve
• Real-time feedback
• Transparent decision-making process

Lessons Learned

Throughout the development process, we gained valuable insights:

1. Knowledge Integration

• Mastering prompt engineering is crucial for accurate responses
• Data preparation significantly impacts AI performance
• Small adjustments can lead to substantial improvements

2. LLM Capabilities

• Network operational knowledge varies across models and vendors
• Custom tools must be tailored to specific LLM capabilities
• Continuous evaluation and adjustment are essential

3. System Evaluation

• Comprehensive testing is crucial for reliability
• Domain expertise remains vital for system development
• Regular updates to test sets ensure continued accuracy

Industry Recognition

Project Silent Comet was recently showcased at MEF GNE NaaS Accelerator Live 2024 in Dallas. Together with TATA Communications and NTT Communications, we presented pioneering AI-driven solutions leveraging Large Language Models and Retrieval-Augmented Generation to streamline network provisioning and troubleshooting. We demonstrated the system’s ability to effectively correlate information from multiple sources and provide actionable insights.

Project Silent Comet was honored with the “Best of Show MEF Accelerator Live” award and received highly positive feedback from the audience and industry experts.

Looking Forward

The integration of AI in network engineering is still in its early stages, with new developments emerging regularly. Our experience with Silent Comet has shown that combining AI capabilities with domain expertise can create powerful tools for network management.

We are committed to exploring these possibilities further, focusing on our core goal: making network troubleshooting automated, efficient and accessible.

Want to learn more about Silent Comet project or see it in action? Contact our team for a demonstration.