User is not a member of project:

This topic has 8 replies, 4 voices, and was last updated 2 years, 3 months ago by ADAM GRIFFITH.

Viewing 9 posts - 1 through 9 (of 9 total)

Author

Posts
July 20, 2022 at 11:36 am #2388
Xander Maddox Weintraut
Participant
Hi,
I’ve recently encountered a bug that is preventing me from making any progress. Beginning this past Monday evening, I have been unable to create a slice using any notebook in Jupyter.
When I first start my Jupyter server, the first time I try to create a slice I receive the following error (this was while executing the hello_fabric notebook):
```
Exception: error: User is not a member of project: <Update Project Id>, refresh_token: NB2HI4DTHIXS6Y3JNRXWO33OFZXXEZZPN5QXK5DIGIXTENBUGMZGMMLCGI2TAM3GGIYDKYJVGU4TGNZUHAYWGZTGHBSTQP3UPFYGKPLSMVTHEZLTNBKG623FNYTHI4Z5GE3DKOBTGM2DGNJUGE2DOJTWMVZHG2LPNY6XMMROGATGY2LGMV2GS3LFHU4DMNBQGAYDAMA
```
From then on, all subsequent requests generate the following Exception:
```
Exception: (invalid_grant) invalid refresh token
```
The issue is not resolved by creating a new FABRIC API token and placing it at ~/.tokens.json
The issue is also not resolved by restarting the Kernel for the notebook, and if I restart the Jupyter server, I get the first error again, followed by the second on subsequent requests.

Can someone please help me with this? I don’t know what to try next, and I am unable to make any progress until this issue is resolved.
Thank you
July 20, 2022 at 11:52 am #2389
Chengyi Qu
Participant
Facing the same problem…. Seems like Fabric is updating the format of environment configuration…. with a fabric_config directory…
July 20, 2022 at 12:54 pm #2397
Paul Ruth
Keymaster
We have simplified the environment configuration process and this is a side effect. There are a couple ways to remedy this.
- Get the updated jupyter-examples notebooks and edit/run the configure environment notebook. This will create the config files you need and put them in fabric_config.
- Alternatively, you could delete the existing fabric_config folder and stop/start you jupyter container. This will reset the fabric_config files to sane defaults that allow existing notebooks to work with the new fablib.
- This reply was modified 2 years, 4 months ago by Paul Ruth.
July 20, 2022 at 1:03 pm #2398
Xander Maddox Weintraut
Participant
That worked! Thank you
August 19, 2022 at 3:18 pm #2659
ADAM GRIFFITH
Participant
Is there a work-around for this error outside of Jupyter Lab? I download a fresh token from the Portal, set FABRIC_TOKEN_LOCATION and received:
```
Traceback (most recent call last):
  File "~/repos/fabric-testbed-api/cluster/main.py", line 75, in <module>
    raise ex
  File "~/repos/fabric-testbed-api/cluster/main.py", line 72, in <module>
    slice_id = fabric_slice.submit()
  File "~/.conda/envs/slate-fabrictestbed/lib/python3.9/site-packages/fabrictestbed_extensions/fablib/slice.py", line 1254, in submit
    return_status, slice_reservations = self.fablib_manager.get_slice_manager().create(slice_name=self.slice_name,
  File "~/.conda/envs/slate-fabrictestbed/lib/python3.9/site-packages/fabrictestbed/slice_manager/slice_manager.py", line 200, in create
    self.refresh_tokens()
  File "~/.conda/envs/slate-fabrictestbed/lib/python3.9/site-packages/fabrictestbed/slice_manager/slice_manager.py", line 160, in refresh_tokens
    raise SliceManagerException(tokens.get(CredmgrProxy.ERROR))
fabrictestbed.slice_manager.slice_manager.SliceManagerException: error: User is not a member of project: 'XXXX', refresh_token: abcdefg1234
```
August 20, 2022 at 6:21 am #2660
Paul Ruth
Keymaster
@Adam I think you need to set the FABRIC_PROJECT_ID env var too. You can get the value from the projects page in the FABRIC portal.

This was new in a recent FABRIC update so it might not be in the docs you are working from. Which docs are you following? I would like to update them.
August 20, 2022 at 11:05 am #2661
ADAM GRIFFITH
Participant
@Paul I am following your Configure your Jupyter Environment docs.

I believe all the variables are present below.
```
cluster_logger.info('Setting up OS environmental variables for FABlib...')
os.environ['FABRIC_CREDMGR_HOST'] = 'cm.fabric-testbed.net'
os.environ['FABRIC_ORCHESTRATOR_HOST'] = 'orchestrator.fabric-testbed.net'
os.environ['FABRIC_BASTION_HOST'] = 'bastion-1.fabric-testbed.net'
os.environ['FABRIC_PROJECT_ID'] = config['secrets']['fabric_project_id']
os.environ['FABRIC_BASTION_USERNAME'] = config['secrets']['fabric_bastion_username']
os.environ['FABRIC_BASTION_KEY_LOCATION'] = f"{fabric_config_dir}/fabric-bastion-key"
os.environ['FABRIC_SLICE_PRIVATE_KEY_FILE'] = f"{fabric_config_dir}/slice_key"
os.environ['FABRIC_SLICE_PUBLIC_KEY_FILE'] = f"{fabric_config_dir}/slice_key.pub"
os.environ['FABRIC_LOG_LEVEL'] = 'INFO'
os.environ['FABRIC_LOG_FILE'] = f"{fabric_config_dir}/fablib.log"
os.environ['FABRIC_TOKEN_LOCATION'] = f"{fabric_config_dir}/token.json"
```
That membership error does shift to an “invalid refresh token” that won’t go away despite regenerating new token JSON files (submitted as FIP-91).
August 23, 2022 at 10:01 am #2678
Paul Ruth
Keymaster
You might just need to restart the notebook kernel to get this working. Restarting the notebook kernel clears out any old var from the python application. Are you doing this?
August 23, 2022 at 10:19 am #2679
ADAM GRIFFITH
Participant
While testing this in Jupyter I am yes. cat .tokens.json does show an up-to-date created_at but the oscillation between the group membership or invalid refresh token errors remain. Komal has been assigned my ticket and hopefully he can do some behind the scenes magic to clear this up.
Author

Posts

Viewing 9 posts - 1 through 9 (of 9 total)

You must be logged in to reply to this topic.