Operations on Slices taking time

[Nirmala Shenoy]

Hello

I am working on  site “TACC” and any commands I give to the remote nodes are taking time /just hanging there. I worked on this slice yesterday and did not have problems.

I am in the process of reserving resources in “HAWI”. The reservation has now been going for more than 1 hour – it is stuck at running post_boot_config.

thanks

Nirmala

[Komal Thareja]

Hi Nirmala,

Could you please check if you see any errors in /tmp/fablib/fablib.log ?

Another thing for the Post Boot config delays could be expired bastion keys. Please run the notebook jupyter-examples-*/configure_and_validate/configure_and_validate.ipynb This shall renew your bastion keys if they are expired.

Please remember to update the Project ID in this notebook.

Please let me know in case you continue to run into issues.

Best,

Komal

[Nirmala Shenoy]

the bastion key has not expired Komal. I am able to ssh into some nodes in the TACC site, but looks like I lost access to some nodes.

I am not seeing a tmp/fablib directory

thanks

 

[Komal Thareja]

Hi Nirmala,

We’re currently investigating what appears to be an unplanned outage on our Kafka service, which may be causing the slowness you’re experiencing. We are actively working on recovery, and I’ll keep you updated on our progress.

Best, Komal

[Nirmala Shenoy]

thanks Komal

[Komal Thareja]

Thank you for reporting this Nirmala! System has been recovered. Please let us know if you continue to run into issues.

Best,

Komal

[Nirmala Shenoy]

thanks Komal. I will be testing them tomorrow.

[Author]

Posts