1. Charles Carpenter

Charles Carpenter

Forum Replies Created

Viewing 7 posts - 16 through 22 (of 22 total)
  • Author
    Posts
  • in reply to: Mflib – Prometheus instrumentize error #7091
    Charles Carpenter
    Participant

      The ELK mirror problem with Centos/Rocky 8 has been fixed.

       

      in reply to: Mflib – Prometheus instrumentize error #7042
      Charles Carpenter
      Participant

        The MeasurementFramework has been updated to fix the docker conflict.
        Prometheus system is now working. There is an error in a script due to a mirror problem, but this does not affect the Prometheus install.

        The ELK install has a fatal mirror problem that remains to be fixed. I will post here when that is completed.

        in reply to: Mflib – Prometheus instrumentize error #7040
        Charles Carpenter
        Participant

          Thanks for bringing this to our attention.

          In the above result, setting up Prometheus returned ‘success’: False, ‘msg’: ‘Prometheus playbook install failed’ This means that Grafana, which is part of the Promtheus install was most likely not installed.
          The ssh tunnel to trying to connect to Grafana’s port is unable to connect since there is no Grafana running, thus the channel 3: open failed: connect failed: Connection refused error.
          The error was caused by an ansible related update which broke the installation process. We have found the problem and will be pushing out a fix today.

           

          Charles Carpenter
          Participant
            There are a couple of methods to export metrics from your running python code.
            For a general overview on writing prometheus exporters see https://prometheus.io/docs/instrumenting/writing_exporters/
            Option 1) Create your own exporter in python. This will run a small http server and will allow Prometheus to query your code every x seconds (30 seconds is usually the default). This is best for a consistently running process. Python has a promtheus_client module, pip install prometheus-client that handles most of the work for you. You will need to add a function that will be called whenever a prometheus instance makes the request. See https://prometheus.github.io/client_python/getting-started/three-step-demo/  and  https://pypi.org/project/prometheus-client/  and  https://github.com/prometheus/client_python
            You can do a test of the running exporter using curl or wget with the address of the exporter and the path /metrics.
            Next configure the prometheus on the meas_node to scrape your newly created exporter. The config file is /opt/fabric_prometheus/prometheus/prometheus_config.yml . ssh to the meas_node and sudo vim /opt/fabric_prometheus/prometheus/prometheus_config.yml Add the new scrape section at the end of the file.
            Something like

             

            # My Exporter
            - job_name: 'my_exporter_name'
              static_configs:
              - targets: ['my-exporter-address:port']
            The scrape will default to the /metrics path.

            Save the file and

            docker restart fabric_prometheus_prometheus
            Use the Explore tab in Grafana with the PromQl {job=”my_exporter_name”} to see the metrics.

             

            Option 2) Use the node_exporters textfile collector. This is best for sporatic metrics, perhaps a cron job that runs hourly. The collector reads text files found in the /var/lib/node_exporter directory. The files need to be in the format found at https://prometheus.io/docs/instrumenting/exposition_formats/#text-based-format There is a python module for writing out text files See https://prometheus.github.io/client_python/exporting/textfile/ https://github.com/prometheus/node_exporter#textfile-collector

            That should get you started. Let me know if you have more questions.
            -Charles

             

            in reply to: metrics used in Grafana #6629
            Charles Carpenter
            Participant

              Those use the SI prefixes see https://en.wikipedia.org/wiki/Metric_prefix

              So lower case m would be milli or 0.001 or 1/1000 of a packet.

              Capital M would be mega or 1,000,000 or a million packet

              in reply to: How to discover Router/Switch load in Fabnet Ipv4 #6550
              Charles Carpenter
              Participant

                Some very basic metrics will soon be available for public use at KNIT8. These will include the up status of the dataplane switches and their packet counts, but not the other data you are seeking.

                in reply to: Error instrumentizing a slice #4605
                Charles Carpenter
                Participant

                  Hello Acheme,
                  The MFLib Instrumentize errors are now resolved.  They have been tested on the KNIT6 workshop notebooks found in the default JupyterHub login with jupyter-examples-rel1.5.1

                  -Charles

                Viewing 7 posts - 16 through 22 (of 22 total)