How to ensure IP addresses are free?

[Sunjay Cauligi]

I’m using Network.get_available_ips(), .make_ip_publicly_routable(), and .allocate_ip() (although that last one doesn’t seem to do anything) to allocate IP addresses for my nodes.

I recently spun up a second slice, which happened to have nodes at one of the same sites (PRIN) as my first slice. None of the above method calls prevent me from attempting to reserve the same IP addresses as the nodes of my first slice; in fact, if I follow the FABRIC official examples, I would end up trying to use the exact same IP addresses for nodes on both slices.

Looking at the code, .get_available_ips() simply enumerates addresses based on the gateway IP address, and the other two methods only update FIM labels without checking anything about existing allocations.

Is there some way to check and prevent IP collisions? Especially since these need to be publicly routable IP addresses.

[Komal Thareja]

Hi Sunjay,

Thank you for reporting this. The collision shouldn’t occur, I will look at this and will share updates on this soon.

Thanks,

Komal

[Sunjay Cauligi]

Possibly related, I can’t seem to modify my slice with new public IP addresses:

>>> slc = fablib.get_slice('qbss-tester')
>>> [(net.get_name(), net.get_public_ips()) for net in slc.get_networks()]
[('netPRIN', None), ('netGPN', None), ('netSALT', None)]
>>> for net in slc.get_networks():
...     net.make_ip_publicly_routable(ipv4=[str(iface.get_ip_addr()) for iface in net.get_interfaces()])
...
>>> [(net.get_name(), net.get_public_ips()) for net in slc.get_networks()]
[
    ('netPRIN', [IPv4Address('23.134.233.117'), IPv4Address('23.134.233.118'), IPv4Address('23.134.233.119')]),
    ('netGPN', [IPv4Address('23.134.232.226'), IPv4Address('23.134.232.227'), IPv4Address('23.134.232.228')]),
    ('netSALT', [IPv4Address('23.134.232.196'), IPv4Address('23.134.232.194'), IPv4Address('23.134.232.195')])
]
>>> slc.modify()
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ in :1                                                                                    │
│                                                                                                  │
│ /Users/scauligi/research/icsi/fabric/ei-fablib/venv/lib/python3.10/site-packages/fabrictestbed_e │
│ xtensions/fablib/slice.py:2745 in modify                                                         │
│                                                                                                  │
│   2742 │   │   │   slice_id=self.slice_id, slice_graph=slice_graph                               │
│   2743 │   │   )                                                                                 │
│   2744 │   │   if return_status != Status.OK:                                                    │
│ ❱ 2745 │   │   │   raise Exception(                                                              │
│   2746 │   │   │   │   "Failed to submit modify slice: {}, {}".format(                           │
│   2747 │   │   │   │   │   return_status, slice_reservations                                     │
│   2748 │   │   │   │   )                                                                         │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
Exception: Failed to submit modify slice: Status.FAILURE, (500)
Reason: INTERNAL SERVER ERROR
HTTP response headers: HTTPHeaderDict({'Server': 'nginx/1.21.6', 'Date': 'Tue, 06 Aug 2024 21:20:26 GMT', 'Content-Type': 'text/html;
charset=utf-8', 'Content-Length': '219', 'Connection': 'keep-alive', 'Access-Control-Allow-Credentials': 'true',
'Access-Control-Allow-Headers': 'DNT, User-Agent, X-Requested-With, If-Modified-Since, Cache-Control, Content-Type, Range, Authorization',
'Access-Control-Allow-Methods': 'GET, POST, PUT, PATCH, DELETE, OPTIONS', 'Access-Control-Allow-Origin': '*',
'Access-Control-Expose-Headers': 'Content-Length, Content-Range, X-Error', 'X-Error': "'NoneType' object has no attribute 'get_type'"})
HTTP response body: b'{\n    "errors": [\n        {\n            "details": "\'NoneType\' object has no attribute \'get_type\'",\n
"message": "Internal Server Error"\n        }\n    ],\n    "size": 1,\n    "status": 500,\n    "type": "error"\n}'

• This reply was modified 3 months, 2 weeks ago by Sunjay Cauligi.

[Komal Thareja]

Hi Sunjay,

That Modify error was a bug and a patch has been deployed for that. There is also an issue in the notebook. For FabNetv4Ext service, IP allocations can be updated by the Control Framework as the FabNetv4Ext subnet is shared across the slices on a site. Collisions are handled for FabNetv4Ext by ControlFramework instead of the Fablib.

Getting the latest slice topology and retrieving the networks would ensure you have the actual IPs allocated. This should ensure correct IPs are configured on the VMs. The following steps are missing on the example notebook. Will work to update that.


slice = fablib.get_slice(slice_name)
network1 = slice.get_network(network1_name)
network2 = slice.get_network(network2_name)
print(network1.get_public_ips())
print(network2.get_public_ips())


Please try your slice again with above suggestions and let me know if you still see the same issue.

Thanks,

Komal

[Sunjay Cauligi]

Hi Komal, I’m not entirely sure how the code you posted is supposed to help?
The .get_public_ips() call only checks the addresses listed in the NetworkService’s FIM user data, and doesn’t seem to check at all what the state of the shared site subnet is.

(I updated to fabrictestbed_extensions 1.7.3 before running the following)

 

>>> print(fablib.get_slice('devscript').get_network('netPRIN').get_public_ips())
[IPv4Address('23.134.233.114'), IPv4Address('23.134.233.115'), IPv4Address('23.134.233.116')]
>>> print(fablib.get_slice('qbss-tester').get_network('netPRIN').get_public_ips())
None

Additionally, I still don’t think I understand how the IP allocation functions are supposed to be used.
For example, after a slice modify request, the list of allocated IPs seems to be reset?

>>> [(net.get_name(), net.get_allocated_ips()) for net in slc.get_networks()]
[
    (
        'netINDI',
        [IPv4Address('23.134.233.129'), IPv4Address('23.134.233.130'), IPv4Address('23.134.233.131'), IPv4Address('23.134.233.132')]
    ),
    (
        'netPRIN',
        [IPv4Address('23.134.233.113'), IPv4Address('23.134.233.114'), IPv4Address('23.134.233.115'), IPv4Address('23.134.233.116')]
    ),
    (
        'netCERN',
        [IPv4Address('23.134.233.209'), IPv4Address('23.134.233.210'), IPv4Address('23.134.233.211'), IPv4Address('23.134.233.212')]
    )
]
>>> slc.modify()
Waiting for slice . Slice state: StableOK
Waiting for ssh in slice . ssh successful
Running post boot config ... Running post boot config threads ...
Post boot config node3d2, Done! (8 sec)
Post boot config node3d3, Done! (8 sec)
Post boot config node3d1, Done! (8 sec)
Post boot config node1d1, Done! (9 sec)
Post boot config node1d3, Done! (9 sec)
Post boot config node1d2, Done! (9 sec)
Post boot config node2d2, Done! (13 sec)
Post boot config node2d1, Done! (13 sec)
Post boot config node2d3, Done! (13 sec)
Saving fablib data...  Done!
Done!
'0e651854-b2fa-4fa2-a4e6-073f76c77fdd'
>>> [(net.get_name(), net.get_allocated_ips()) for net in slc.get_networks()]
[('netINDI', [IPv4Address('23.134.233.129')]), ('netPRIN', [IPv4Address('23.134.233.113')]), ('netCERN', [IPv4Address('23.134.233.209')])]

[Komal Thareja]

Hi Sunjay,

I apologize for not clarifying this earlier. Please note that when using FabNetv4Ext, IP allocation is managed by the Control Framework, not the user. While users can request a public IP starting with the first IP from the allocated subnet, the Control Framework assigns an available FabNetv4Ext IP based on the available options. This assigned IP is what gets returned to the user and can be accessed through net.get_public_ips(). Since the Control Framework updates this information, it’s recommended to retrieve the latest slice topology after the slice reaches a stable state.

So for FabNetv4Ext, please use get_public_ips() as opposed to get_allocated_ips().

NOTE: The example notebook uses get_public_ips() after requesting Publicly Routable IP.

Thanks,

Komal

[Sunjay Cauligi]

Hi Komal, is there no way to check which IP addresses are already in use at a given site(‘s network)? Otherwise if I understand correctly, it seems like a user would have to call make_ip_publicly_routable() -> modify() -> get_public_ips() in a loop until they finally receive public IP addresses, which may take a not-insignificant amount of time due to modify().

[Komal Thareja]

Hi Sunjay,

Currently, that’s not possible. Even if you could query the allocated public IPs, you would still need to invoke the make_ip_publicly_routable followed by modify or submit. Since FabNetv4Ext has a limited number of IPs available, we only enable it when explicitly requested via the modifyor submit operation.

Without this step, the service exists but won’t pass traffic. I hope this clarifies things! I’ll note this as a potential enhancement to allow querying the allocated IPs.

Please note this should be done only at the slice setup time.

Thanks,

Komal

• This reply was modified 3 months, 2 weeks ago by Komal Thareja.

[Author]

Posts