slice.submit(wait=True) fails

[Timothy Middelkoop]

If I set “wait=True” in slice.submit I get a

+++ start slice
Running post boot config ... Traceback (most recent call last):
  File "/Users/tmiddelkoop/projects/zero/two/k8/fabric/k8.py", line 132, in 
    sys.exit(main(sys.argv[1]))
  File "/Users/tmiddelkoop/projects/zero/two/k8/fabric/k8.py", line 117, in main
    slice.create()
  File "/Users/tmiddelkoop/projects/zero/two/k8/fabric/k8.py", line 47, in create
    slice.submit(wait=True)
  File "/Users/tmiddelkoop/projects/zero/two/k8/fabric/.venv/lib/python3.9/site-packages/fabrictestbed_extensions/fablib/slice.py", line 1231, in submit
    self.post_boot_config()
  File "/Users/tmiddelkoop/projects/zero/two/k8/fabric/.venv/lib/python3.9/site-packages/fabrictestbed_extensions/fablib/slice.py", line 1086, in post_boot_config
    node_thread.result()
  File "/usr/local/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/concurrent/futures/_base.py", line 439, in result
    return self.__get_result()
  File "/usr/local/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/concurrent/futures/_base.py", line 391, in __get_result
    raise self._exception
  File "/usr/local/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/tmiddelkoop/projects/zero/two/k8/fabric/.venv/lib/python3.9/site-packages/fabrictestbed_extensions/fablib/node.py", line 1127, in network_manager_stop
    raise e
  File "/Users/tmiddelkoop/projects/zero/two/k8/fabric/.venv/lib/python3.9/site-packages/fabrictestbed_extensions/fablib/node.py", line 1109, in network_manager_stop
    stdout, stderr = self.execute(f"sudo systemctl stop NetworkManager")
  File "/Users/tmiddelkoop/projects/zero/two/k8/fabric/.venv/lib/python3.9/site-packages/fabrictestbed_extensions/fablib/node.py", line 671, in execute
    raise Exception(f"node.execute: Management IP Invalid: {management_ip}")
Exception: node.execute: Management IP Invalid: None

error. If I set “wait=False” it will run.

Here is the code to reproduce it.

        print("+++ start slice")
        slice.submit(wait=False)
        print(slice)

        slice.wait()
        slice.wait_ssh()
        slice.test_ssh()

[Paul Ruth]

Are you able to repeat this consistently?  I can’t seem to reproduce it.

The error you are seeing is that a control framework does not have a management IP for the VM (or does not yet have a management IP for the VM) at a time when the library is trying to ssh to it.

I think this might be a transient issue that has to do with a VM that was not issues a management  IP.

Let us know if you continue to be able to repeat it.

[Timothy Middelkoop]

I am still able to replicate it. Here is the code.

## Dump Config
for i in ['FABRIC_PROJECT_ID']:
    print(f'{i}={os.environ}')

## Ping auth test
print(fablib.get_site_names())

## Create slice
slice = fablib.new_slice(name="oneslice")
node = slice.add_node(name="node1", image="default_ubuntu_22")

slice.submit(wait=True) # Change to False to run correctly.

print(slice)
slice.wait_ssh()
slice.test_ssh()

print(node)

The environment is built via the following:

python3 -m venv .venv
. .venv/bin/activate
python3 -m pip install --upgrade pip
python3 -m pip install git+https://github.com/fabric-testbed/fabric-cli.git
python3 -m pip install git+https://github.com/fabric-testbed/fabrictestbed-extensions.git

All config-values are set via env. I have a complete example w/ docs if you want.

It’s run on MACOS, tools installed via brew.

(.venv) tmiddelkoop@tmiddelkoopsMBP fabric % python3 --version
Python 3.9.13

[Paul Ruth]

I think the issue is that we have moved to a more formal release mechanism for fabrictestbed-extensions.  I think the main branch is a bit ahead of what you should be using (and apparently broken).  Try pulling release 1.2 instead of the main branch (https://github.com/fabric-testbed/fabrictestbed-extensions/releases)

Alternatively, you can pull from pypi directly with:

pip install fabrictestbed-extensions

I think the instructions need to be updated to point to the pypi release.

[Author]

Posts