MFLib uses Kibana to visualize metrics and logs gathered by ELK.
The Measurement Framework (MFLib) uses Kibana to access and visualize Elastic Search (ELK) metrics and logs. Visit Elastic Search for more information about ELK.
The Role of Kibana in FABRIC Measurement Framework
Data Collection: Lightweight agents called Beats (like Packetbeat or Metricbeat) are deployed across the FABRIC nodes to collect network traffic data, system metrics, and logs.
Data Processing: The collected data can optionally be processed and enriched using Logstash before being sent to Elasticsearch. This might include parsing logs, adding geographical information, or performing DNS lookups.
Data Storage: Elasticsearch is the distributed search and analytics engine where all the measurement data is indexed and stored. It’s highly scalable and allows for fast, complex queries.
Data Visualization: Kibana connects to Elasticsearch, providing an intuitive, web-based dashboard where users can search, analyze, and visualize the data in real time.
Kibana’s Key Features for Network Measurement
- Dashboards: Users can create customizable, real-time dashboards to monitor key performance indicators (KPIs) like network throughput, latency, packet loss, and link utilization. These dashboards can combine various visualizations (charts, graphs, maps) into a single, comprehensive view.
- Discover: The “Discover” feature allows for interactive data exploration. Users can search, filter, and drill down into raw network data to find specific events, troubleshoot issues, or investigate anomalies.
- Visualizations: Kibana offers a wide array of visualization types, including:
- Time series charts to track performance over time.
- Geospatial maps to visualize data from different geographical locations within the FABRIC testbed. (this will require geographical location information to be used)
- Bar charts and pie charts to show data distributions and top contributors (e.g., top talkers, most used protocols).
- Machine Learning: For licensed versions, Kibana’s machine learning capabilities can automatically detect anomalies in network traffic. This is crucial for identifying unusual behavior, such as a sudden spike in latency or a denial-of-service attack, that might be missed by manual monitoring. Note that this feature may require licensed subscriptions.
- Alerting: Users can set up alerts based on specific thresholds or detected anomalies. This enables automated notifications to be sent when a critical event occurs, allowing for a rapid response to network issues. Note that this feature may require licensed subscriptions.
- Search: Kibana uses Elasticsearch’s powerful search capabilities, allowing users to perform complex queries on the indexed data using the query language. This is more powerful than simple filters and enables detailed analysis.
Accessing Kibana in MFLib’s measurement node (_meas_node)
Kibana runs on the Measurement Node(_meas_node) and is accessed via a web browser. Since the Measurement Node is behind the FABRIC Bastion Host, you must create an SSH tunnel. ssh -L 10020:localhost:80 -F <fabric-ssh-config-file> -i <your portal_slice_id_rsa-file> <slice-username>@<meas_node-ip> Then browse to https://localhost:10020.
For detailed examples for using Kibana in MFLib is shown in this Jupyterhub notebook examples.