- This topic has 4 replies, 3 voices, and was last updated 1 year, 5 months ago by
.
-
Posts
-
Dear experimenters,
We appear to be experiencing multiple simultaneous problems with the testbed:
– There is a network provider hardware failure at WASH site (site has been placed into maintenance until new equipment is installed)(RESOLVED)– SRI management is not working reliably (site has been placed into maintenance)
– GATech worker 3 and
STAR worker 6(resolved) have malfunctioned and have also been placed into maintenance– In addition, we are experiencing problems with Kafka message delivery across all sites (we are investigating)(resolved)For the moment we will place the testbed into maintenance mode until we know more. Existing slices should not be affected, unless there is a management network problem at the site. We will post updates on this thread as we know more.
Dear experimenters,
We believe we have gotten to the bottom of the Kafka issues and the testbed is reopening. SRI will remain in maintenance, WASH will also remain in maintenance. Workers at GATech (#3) and STAR (#6) will be in maintenance and note that we have a number of planned outages for workers across multiple sites to install FPGAs (these were mentioned on previous announcements) in the past few days. As the work gets completed, those will be taken off maintenance in due course.
WASH is back online
star-w6 issue was resolved
SRI MTU issue is being worked on but this rack is not in production yet.
As an update on this:
– SRI did not complete acceptance tests (site has been placed into maintenance) and should not be used for experiments.
– GATech worker-2 (GPU worker) has a hardware fault and it’s placed into maintenance. Other GATECH resources are available for experiments.
- The topic ‘Multiple problems in FABRIC [Partially Resolved]’ is closed to new replies.