[Acheme Acheme]

Adding the error

on installing the driver, sudo apt-get install mlnx-en-vma

See attached…

• This reply was modified 1 year, 5 months ago by Acheme Acheme.

[Acheme Acheme]

Thank you very much.

[Acheme Acheme]

Thank you, I will try that and see.

[Acheme Acheme]

Thank you for your response. See the output from nfdc face list on the three nodes below:

ubuntu@Node1:~$ nfdc face list
faceid=1 remote=internal:// local=internal:// congestion={base-marking-interval=100ms default-threshold=65536B} mtu=8800 counters={in={0i 47d 0n 30928B} out={117i 0d 0n 9206B}} flags={local permanent point-to-point local-fields}
faceid=254 remote=contentstore:// local=contentstore:// mtu=8800 counters={in={0i 0d 0n 0B} out={0i 0d 0n 0B}} flags={local permanent point-to-point}
faceid=255 remote=null:// local=null:// mtu=8800 counters={in={0i 0d 0n 0B} out={0i 0d 0n 0B}} flags={local permanent point-to-point}
faceid=256 remote=ether://[01:00:5e:00:17:aa] local=dev://ens3 congestion={base-marking-interval=100ms default-threshold=65536B} mtu=8800 counters={in={0i 0d 0n 0B} out={0i 0d 0n 0B}} flags={non-local permanent multi-access}
faceid=257 remote=ether://[01:00:5e:00:17:aa] local=dev://ens7 congestion={base-marking-interval=100ms default-threshold=65536B} mtu=1500 counters={in={0i 0d 0n 0B} out={0i 0d 0n 0B}} flags={non-local permanent multi-access}
faceid=258 remote=fd://30 local=unix:///run/nfd.sock congestion={base-marking-interval=100ms default-threshold=65536B} mtu=8800 counters={in={114i 4d 0n 8834B} out={4i 38d 0n 29283B}} flags={local on-demand point-to-point local-fields congestion-marking}
faceid=259 remote=udp4://224.0.23.170:56363 local=udp4://10.30.6.78:45175 congestion={base-marking-interval=100ms default-threshold=65536B} mtu=8800 counters={in={0i 0d 0n 0B} out={0i 0d 0n 0B}} flags={non-local permanent multi-access congestion-marking}
faceid=260 remote=udp6://[ff02::1234%ens3]:56363 local=udp6://[2610:148:1f00:9f01:f816:3eff:fea9:58b3]:42060 congestion={base-marking-interval=100ms default-threshold=65536B} mtu=8800 counters={in={0i 0d 0n 0B} out={0i 0d 0n 0B}} flags={non-local permanent multi-access congestion-marking}
faceid=261 remote=udp6://[ff02::1234%ens7]:56363 local=udp6://[fe80::85:bfff:fed3:2e09%ens7]:57430 congestion={base-marking-interval=100ms default-threshold=65536B} mtu=8800 counters={in={0i 0d 0n 0B} out={0i 0d 0n 0B}} flags={non-local permanent multi-access congestion-marking}
faceid=262 remote=fd://37 local=unix:///run/nfd.sock congestion={base-marking-interval=100ms default-threshold=65536B} mtu=8800 counters={in={1i 0d 0n 286B} out={0i 1d 0n 321B}} flags={local on-demand point-to-point congestion-marking}
faceid=264 remote=ether://[e8:eb:d3:81:b7:fe] local=dev://ens7 congestion={base-marking-interval=100ms default-threshold=65536B} mtu=1500 counters={in={0i 0d 0n 0B} out={0i 0d 0n 0B}} flags={non-local persistent point-to-point}
faceid=267 remote=udp4://224.0.23.170:56363 local=udp4://192.168.1.1:50111 congestion={base-marking-interval=100ms default-threshold=65536B} mtu=8800 counters={in={0i 0d 0n 0B} out={0i 0d 0n 0B}} flags={non-local permanent multi-access congestion-marking}
faceid=268 remote=fd://38 local=unix:///run/nfd.sock congestion={base-marking-interval=100ms default-threshold=65536B} mtu=8800 counters={in={1i 0d 0n 43B} out={0i 0d 0n 0B}} flags={local on-demand point-to-point congestion-marking}

 

ubuntu@Node2:~$ nfdc face list
faceid=1 remote=internal:// local=internal:// congestion={base-marking-interval=100ms default-threshold=65536B} mtu=8800 counters={in={0i 43d 0n 28758B} out={114i 0d 0n 8138B}} flags={local permanent point-to-point local-fields}
faceid=254 remote=contentstore:// local=contentstore:// mtu=8800 counters={in={0i 0d 0n 0B} out={0i 0d 0n 0B}} flags={local permanent point-to-point}
faceid=255 remote=null:// local=null:// mtu=8800 counters={in={0i 0d 0n 0B} out={0i 0d 0n 0B}} flags={local permanent point-to-point}
faceid=256 remote=ether://[01:00:5e:00:17:aa] local=dev://ens3 congestion={base-marking-interval=100ms default-threshold=65536B} mtu=8800 counters={in={0i 0d 0n 0B} out={0i 0d 0n 0B}} flags={non-local permanent multi-access}
faceid=257 remote=ether://[01:00:5e:00:17:aa] local=dev://ens7 congestion={base-marking-interval=100ms default-threshold=65536B} mtu=1500 counters={in={0i 0d 0n 0B} out={0i 0d 0n 0B}} flags={non-local permanent multi-access}
faceid=258 remote=fd://30 local=unix:///run/nfd.sock congestion={base-marking-interval=100ms default-threshold=65536B} mtu=8800 counters={in={111i 2d 0n 7261B} out={2i 34d 0n 26559B}} flags={local on-demand point-to-point local-fields congestion-marking}
faceid=259 remote=udp4://224.0.23.170:56363 local=udp4://10.30.6.187:36951 congestion={base-marking-interval=100ms default-threshold=65536B} mtu=8800 counters={in={0i 0d 0n 0B} out={0i 0d 0n 0B}} flags={non-local permanent multi-access congestion-marking}
faceid=260 remote=udp6://[ff02::1234%ens3]:56363 local=udp6://[2610:148:1f00:9f01:f816:3eff:fe80:5c5b]:47746 congestion={base-marking-interval=100ms default-threshold=65536B} mtu=8800 counters={in={0i 0d 0n 0B} out={0i 0d 0n 0B}} flags={non-local permanent multi-access congestion-marking}
faceid=261 remote=udp6://[ff02::1234%ens7]:56363 local=udp6://[fe80::436:eff:feb8:aee6%ens7]:41187 congestion={base-marking-interval=100ms default-threshold=65536B} mtu=8800 counters={in={0i 0d 0n 0B} out={0i 0d 0n 0B}} flags={non-local permanent multi-access congestion-marking}
faceid=262 remote=fd://37 local=unix:///run/nfd.sock congestion={base-marking-interval=100ms default-threshold=65536B} mtu=8800 counters={in={1i 0d 0n 285B} out={0i 1d 0n 322B}} flags={local on-demand point-to-point congestion-marking}
faceid=264 remote=ether://[e8:eb:d3:81:b7:ff] local=dev://ens7 congestion={base-marking-interval=100ms default-threshold=65536B} mtu=1500 counters={in={0i 0d 0n 0B} out={10i 0d 0n 450B}} flags={non-local persistent point-to-point}
faceid=267 remote=udp4://224.0.23.170:56363 local=udp4://192.168.1.2:42008 congestion={base-marking-interval=100ms default-threshold=65536B} mtu=8800 counters={in={0i 0d 0n 0B} out={0i 0d 0n 0B}} flags={non-local permanent multi-access congestion-marking}
faceid=268 remote=fd://42 local=unix:///run/nfd.sock congestion={base-marking-interval=100ms default-threshold=65536B} mtu=8800 counters={in={1i 0d 0n 43B} out={0i 0d 0n 0B}} flags={local on-demand point-to-point congestion-marking}

 

ubuntu@Switch:~$ nfdc face list
faceid=1 remote=internal:// local=internal:// congestion={base-marking-interval=100ms default-threshold=65536B} mtu=8800 counters={in={0i 65d 0n 49003B} out={130i 0d 0n 10004B}} flags={local permanent point-to-point local-fields}
faceid=254 remote=contentstore:// local=contentstore:// mtu=8800 counters={in={0i 0d 0n 0B} out={0i 0d 0n 0B}} flags={local permanent point-to-point}
faceid=255 remote=null:// local=null:// mtu=8800 counters={in={0i 0d 0n 0B} out={0i 0d 0n 0B}} flags={local permanent point-to-point}
faceid=256 remote=ether://[01:00:5e:00:17:aa] local=dev://ens3 congestion={base-marking-interval=100ms default-threshold=65536B} mtu=8800 counters={in={0i 0d 0n 0B} out={0i 0d 0n 0B}} flags={non-local permanent multi-access}
faceid=257 remote=ether://[01:00:5e:00:17:aa] local=dev://ens7 congestion={base-marking-interval=100ms default-threshold=65536B} mtu=1500 counters={in={0i 0d 0n 0B} out={0i 0d 0n 0B}} flags={non-local permanent multi-access}
faceid=258 remote=ether://[01:00:5e:00:17:aa] local=dev://ens8 congestion={base-marking-interval=100ms default-threshold=65536B} mtu=1500 counters={in={0i 0d 0n 0B} out={0i 0d 0n 0B}} flags={non-local permanent multi-access}
faceid=259 remote=ether://[01:00:5e:00:17:aa] local=dev://ens9 congestion={base-marking-interval=100ms default-threshold=65536B} mtu=1500 counters={in={0i 0d 0n 0B} out={0i 0d 0n 0B}} flags={non-local permanent multi-access}
faceid=260 remote=ether://[01:00:5e:00:17:aa] local=dev://ens10 congestion={base-marking-interval=100ms default-threshold=65536B} mtu=1500 counters={in={0i 0d 0n 0B} out={0i 0d 0n 0B}} flags={non-local permanent multi-access}
faceid=261 remote=fd://42 local=unix:///run/nfd.sock congestion={base-marking-interval=100ms default-threshold=65536B} mtu=8800 counters={in={125i 3d 0n 8874B} out={3i 48d 0n 44617B}} flags={local on-demand point-to-point local-fields congestion-marking}
faceid=262 remote=udp4://224.0.23.170:56363 local=udp4://10.30.6.190:37885 congestion={base-marking-interval=100ms default-threshold=65536B} mtu=8800 counters={in={0i 0d 0n 0B} out={0i 0d 0n 0B}} flags={non-local permanent multi-access congestion-marking}
faceid=263 remote=udp6://[ff02::1234%ens3]:56363 local=udp6://[2610:148:1f00:9f01:f816:3eff:fe3a:e5d6]:60036 congestion={base-marking-interval=100ms default-threshold=65536B} mtu=8800 counters={in={0i 0d 0n 0B} out={0i 0d 0n 0B}} flags={non-local permanent multi-access congestion-marking}
faceid=264 remote=udp6://[ff02::1234%ens7]:56363 local=udp6://[fe80::eaeb:d3ff:fe81:b7fe%ens7]:38222 congestion={base-marking-interval=100ms default-threshold=65536B} mtu=8800 counters={in={0i 0d 0n 0B} out={0i 0d 0n 0B}} flags={non-local permanent multi-access congestion-marking}
faceid=265 remote=udp6://[ff02::1234%ens8]:56363 local=udp6://[fe80::eaeb:d3ff:fe81:b7ff%ens8]:36929 congestion={base-marking-interval=100ms default-threshold=65536B} mtu=8800 counters={in={0i 0d 0n 0B} out={0i 0d 0n 0B}} flags={non-local permanent multi-access congestion-marking}
faceid=266 remote=udp6://[ff02::1234%ens9]:56363 local=udp6://[fe80::eaeb:d3ff:fe81:c5ba%ens9]:38572 congestion={base-marking-interval=100ms default-threshold=65536B} mtu=8800 counters={in={0i 0d 0n 0B} out={0i 0d 0n 0B}} flags={non-local permanent multi-access congestion-marking}
faceid=267 remote=udp6://[ff02::1234%ens10]:56363 local=udp6://[fe80::eaeb:d3ff:fe81:c5bb%ens10]:47330 congestion={base-marking-interval=100ms default-threshold=65536B} mtu=8800 counters={in={0i 0d 0n 0B} out={0i 0d 0n 0B}} flags={non-local permanent multi-access congestion-marking}
faceid=268 remote=fd://55 local=unix:///run/nfd.sock congestion={base-marking-interval=100ms default-threshold=65536B} mtu=8800 counters={in={1i 0d 0n 286B} out={0i 1d 0n 321B}} flags={local on-demand point-to-point congestion-marking}
faceid=270 remote=ether://[02:85:bf:d3:2e:09] local=dev://ens7 congestion={base-marking-interval=100ms default-threshold=65536B} mtu=1500 counters={in={0i 0d 0n 0B} out={0i 0d 0n 0B}} flags={non-local persistent point-to-point}
faceid=273 remote=ether://[06:36:0e:b8:ae:e6] local=dev://ens9 congestion={base-marking-interval=100ms default-threshold=65536B} mtu=1500 counters={in={0i 0d 0n 0B} out={0i 0d 0n 0B}} flags={non-local persistent point-to-point}
faceid=276 remote=ether://[01:00:5e:00:17:aa] local=dev://br0 congestion={base-marking-interval=100ms default-threshold=65536B} mtu=1500 counters={in={0i 0d 0n 0B} out={0i 0d 0n 0B}} flags={non-local permanent multi-access}
faceid=277 remote=udp6://[ff02::1234%br0]:56363 local=udp6://[fe80::eaeb:d3ff:fe81:b7fe%br0]:36593 congestion={base-marking-interval=100ms default-threshold=65536B} mtu=8800 counters={in={0i 0d 0n 0B} out={0i 0d 0n 0B}} flags={non-local permanent multi-access congestion-marking}
faceid=278 remote=fd://56 local=unix:///run/nfd.sock congestion={base-marking-interval=100ms default-threshold=65536B} mtu=8800 counters={in={1i 0d 0n 43B} out={0i 0d 0n 0B}} flags={local on-demand point-to-point congestion-marking}

What can I do differently, please. Does this have to do with creating the OpenVSwitch after the faces and routes were created?

 

Regards,

Acheme

[Acheme Acheme]

Hi yoursunny,

Kindly find the notebook here

See also the terminal commands to setup OVSwitch on the “switch” below:

sudo apt-get -y update && sudo apt-get -y upgrade

sudo apt-get -y install openvswitch-switch openvswitch-common

sudo ovs-vsctl add-br br0

sudo ovs-vsctl add-port br0 ens7

sudo ovs-vsctl add-port br0 ens9

sudo ifconfig ens7 0

sudo ifconfig ens9 0

sudo ifconfig br0 up

 

Regards,

Acheme

[Acheme Acheme]

Hi Hussam,

It can wait for a full resolution.

Thank you very much.

 

Regards,

Acheme

[Acheme Acheme]

Hi Hussam,

Did you get it this time?

[Acheme Acheme]

Sorry about that, please see below:

Setting up Prometheus…
{‘grafana_admin_pass’: ‘wrRwrFZl’, ‘success’: False, ‘msg’: ‘Prometheus playbook install failed..’, ‘play_recap’: ‘PLAY RECAP *********************************************************************\nmeas-node : ok=0 changed=0 unreachable=1 failed=0 skipped=0 rescued=0 ignored=0 \n\nPlaybook run took 0 days, 0 hours, 0 minutes, 52 seconds\nMonday 26 June 2023 13:41:21 +0000 (0:00:52.271) 0:00:52.287 *********** \n=============================================================================== \nGathering Facts ——————————————————– 52.27s\nMonday 26 June 2023 13:41:21 +0000 (0:00:52.271) 0:00:52.286 *********** \n=============================================================================== \ngather_facts ———————————————————– 52.27s\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ \ntotal —————————————————————— 52.27s\n’}

[Acheme Acheme]

Works fine now.

Thank you Komal

 

Regards,

Acheme

[Acheme Acheme]

Please how do I restart the container?

[Acheme Acheme]

Hi Komal,

I am using 1.5.1

(base) fabric@jupyter-aacheme-40clemson-2eedu:~/work$ pip list | grep fabric
fabric 3.1.0
fabric-credmgr-client 1.3.2
fabric-fim 1.4.3
fabric_fss_utils 1.5.0
fabric-orchestrator-client 1.4.4
fabrictestbed 1.4.4
fabrictestbed-extensions 1.4.1

 

(base) fabric@jupyter-aacheme-40clemson-2eedu:~/work$ cat ~/work/fabric_config/requirements.txt
fabrictestbed-extensions==1.4.1

Thank you.

Regards,

Acheme

[Acheme Acheme]

Thank you Ilya

[Author]

Posts