MFLIB error in instrumentazion

[Bjoern Sagstad]

Hi,

Recently I get an error intermittently. The error is added as a screenshot below. It happens in the mf.instumentize() method. Since it’s intermittent it’s not entirely clear to me how to give good steps for someone to reproduce, but if I figure that out I will update this post.

It only happens sometimes, and I can solve it by deleting the slice and starting over, but it still takes a bit extra time when it happens. Because of that I am looking to see if there is something, a command or anything that can potentially fix this error without making me start from scratch.

Sites used: Unsure
Slice name: MyMonitoredSliceTwo
Slice ID: b8e31bd6-3c85-4262-9b64-7d3cf349b282
Fablib version: 1.6.5
MFLib version: 1.0.40

• This topic was modified 6 months, 1 week ago by Bjoern Sagstad. Reason: adding topic tags

[Hussam Nasir]

Is this slice still active? If so, could you please let us know which rack it is on ?. Also it would be good if you can also send us  your MFlib log file which should be in /tmp/mflib/* on your jupyterhub session.

[Hussam Nasir]

Hello,

I looked at your nodes, but as I mentioned earlier, the relevant log files are on your jupyterhub session in /tmp/mflib/*. We would require those to look into the cause. I tried the same slice using my account and have not encountered the error you mentioned.

[Bjoern Sagstad]

Hi,
I just encountered the error again. I think it’s might be related to when I restart the notebook. So, not when I first create the slice but when I reload the slice to use it without having to delete the slice and go through all the setup again.

Sites used:BRIST, MICH, RUTG
Slice name: MyMonitoredSliceTwo
Fablib version: 1.6.5
MFLib version: 1.0.40

I have also attached the log to this.

Update: I wasn’t able to upload the log file so here are the logs in plain text:

06/18/2024 15:33:41 PM mflib.core INFO Using core_class_version 1.0.38
06/18/2024 15:33:41 PM mflib.core INFO -----Set slice name MyMonitoredSliceTwo.-----
06/18/2024 15:33:43 PM mflib.mflib INFO Inititializing slice "MyMonitoredSliceTwo" for MeasurementFramework.
06/18/2024 15:33:43 PM mflib.mflib INFO Found meas node as meas-node at 2620:0:c80:1003:f816:3eff:fe84:7927
06/18/2024 15:35:56 PM mflib.mflib DEBUG Instrumentizing slice "MyMonitoredSliceTwo"
06/18/2024 15:35:56 PM mflib.mflib DEBUG Setting up Prometheus...
06/18/2024 15:35:56 PM mflib.core INFO Run create for prometheus
06/18/2024 15:35:56 PM mflib.core DEBUG Data is None.
06/18/2024 16:37:38 PM mflib.core INFO Creating mflib object.
06/18/2024 16:37:38 PM mflib.core INFO Using core_class_version 1.0.38
06/18/2024 16:37:38 PM mflib.core INFO Using core_class_version 1.0.38
06/18/2024 16:37:38 PM mflib.core INFO -----Set slice name MyMonitoredSliceTwo.-----
06/18/2024 16:37:38 PM mflib.core INFO -----Set slice name MyMonitoredSliceTwo.-----
06/18/2024 16:37:41 PM mflib.mflib INFO Inititializing slice "MyMonitoredSliceTwo" for MeasurementFramework.
06/18/2024 16:37:41 PM mflib.mflib INFO Inititializing slice "MyMonitoredSliceTwo" for MeasurementFramework.
06/18/2024 16:37:41 PM mflib.mflib INFO Found meas node as meas-node at 2620:0:c80:1003:f816:3eff:fe6e:b221
06/18/2024 16:37:41 PM mflib.mflib INFO Found meas node as meas-node at 2620:0:c80:1003:f816:3eff:fe6e:b221
06/18/2024 16:39:54 PM mflib.mflib DEBUG Instrumentizing slice "MyMonitoredSliceTwo"
06/18/2024 16:39:54 PM mflib.mflib DEBUG Instrumentizing slice "MyMonitoredSliceTwo"
06/18/2024 16:39:54 PM mflib.mflib DEBUG Setting up Prometheus...
06/18/2024 16:39:54 PM mflib.mflib DEBUG Setting up Prometheus...
06/18/2024 16:39:54 PM mflib.core INFO Run create for prometheus
06/18/2024 16:39:54 PM mflib.core INFO Run create for prometheus
06/18/2024 16:39:54 PM mflib.core DEBUG Data is None.
06/18/2024 16:39:54 PM mflib.core DEBUG Data is None.
06/18/2024 16:42:19 PM mflib.core DEBUG STDOUT: ""
06/18/2024 16:42:19 PM mflib.core DEBUG STDOUT: ""
06/18/2024 16:42:19 PM mflib.core ERROR Unable to convert returned comand json.
Traceback (most recent call last):
File "/opt/conda/lib/python3.11/site-packages/mflib/core.py", line 815, in _run_service_command
ret_data = json.loads(jsonstr)
^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
06/18/2024 16:42:19 PM mflib.core ERROR Unable to convert returned comand json.
Traceback (most recent call last):
File "/opt/conda/lib/python3.11/site-packages/mflib/core.py", line 815, in _run_service_command
ret_data = json.loads(jsonstr)
^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
06/18/2024 16:42:19 PM mflib.core DEBUG STDERR: sudo: unknown user: mfuser
sudo: unable to initialize policy plugin

06/18/2024 16:42:19 PM mflib.core DEBUG STDERR: sudo: unknown user: mfuser
sudo: unable to initialize policy plugin

 

• This reply was modified 6 months ago by Bjoern Sagstad.

[Hussam Nasir]

Do you mean that firs t instrumentized the slice, then logged out and came back and tried to re-instrumentize the same slice ?

[Bjoern Sagstad]

Yes. The last time I got the error that was how I got it.

[Author]

Posts