fablib

This module exports a FablibManager class and a fablib class available, which allows you to, among other things:

• Query FABRIC testbed resources.

• Create, modify, and delete slices.

• Manage the SSH keys you use with FABRIC.

• etc.

In most cases you would need to create a FablibManager instance to interact with FABRIC testbed:

from fabrictestbed_extensions.fablib.fablib import FablibManager

fablib = FablibManager()

slice = fablib.new_slice(name="MySlice")
node = slice.add_node(name="node1")
slice.submit();

See FABRIC project’s Jupyter notebook examples for more complete code samples.

Note

Some configuration in the form of a configuration file, environment variables, or FablibManager constructor parameters is required for the library to work. Please see the FABRIC project’s documentation on getting started.

class fabrictestbed_extensions.fablib.fablib.FablibManager(fabric_rc=None, credmgr_host=None, orchestrator_host=None, core_api_host=None, am_host=None, ceph_mgr_host=None, token_location=None, id_token=None, project_id=None, bastion_username=None, bastion_key_location=None, log_level='INFO', log_file='/tmp/fablib/fablib.log', log_propagate=False, data_dir='/tmp/fablib', execute_thread_pool_size=64, offline=False, auto_token_refresh=True, validate_config=True, no_ssh=False, raise_on_not_found=False, **kwargs)

FablibManager is the main interface to FABRIC services.

A FablibManager object is used to query FABRIC testbed for available resources, create and configure slices, manage SSH keys in nodes in slices and FABRIC’s bastion host, etc. This requires some configuration, which is gathered from:

• constructor parameters (highest priority)

• a configuration file (medium priority, optional)

• environment variables (low priority)

• defaults (lowest priority, if needed and when possible)

The configuration file is optional. You can provide configuration entirely through constructor parameters and/or environment variables. If using a config file, it typically would be located at "${HOME}/work/fabric_config/fabric_rc".

Parameters:

• fabric_rc (str) – Path to fablib configuration file. Optional. Defaults to "${HOME}/work/fabric_config/fabric_rc", but the file doesn’t need to exist.

• credmgr_host (str) – Name of credential manager host.

• orchestrator_host (str) – Name of FABRIC orchestrator host.

• core_api_host (str) – Name of Core API host.

• am_host (str) – Name of Aggregate Manager host.

• ceph_mgr_host (str) – Name of ceph manager host.

• token_location (str) – Path to the file that contains your FABRIC auth token.

• id_token (str) – ID token string to use directly (optional). If provided, token_location file check is skipped and auto_token_refresh is automatically disabled.

• project_id (str) – Your FABRIC project ID, obtained from https://cm.fabric-testbed.net/, usually via FABRIC portal.

• bastion_username (str) – Your username on FABRIC bastion host, obtained from FABRIC portal.

• bastion_key_location (str) – Path to your bastion SSH key.

• log_file (str) – Path where fablib logs are written; defaults to "/tmp/fablib/fablib.log".

• log_level (str) – Level of detail in the logs written. Defaults to "DEBUG"; other possible log levels are "INFO", "WARNING", "ERROR", and "CRITICAL", in reducing order of verbosity.

• log_propagate (bool) – Whether fablib logs propagate to the root logger. Defaults to False to avoid duplicate logging in notebooks.

• data_dir (str) – directory for fablib to store temporary data.

• output – Format of fablib output; can be either "pandas" or "text". Defaults to "pandas" in a Jupyter notebook environment; "text" otherwise.

• execute_thread_pool_size (int) – Number of worker threads in the thread pool fablib uses to execute commands in nodes. Defaults to 64.

• offline (bool) – Avoid using FABRIC services when initializing. This is False by default, and set to True only in some unit tests.

• auto_token_refresh (bool) – Auto refresh tokens (automatically disabled if id_token is provided)

• validate_config (bool) – Whether to verify and persist configuration during initialization.

• no_ssh (bool) – Disable SSH operations. When True, skips SSH-related initialization (thread pool, bastion probing, SSH key validation) and prevents SSH operations on nodes. Useful for API-only mode (e.g., MCP server). Can also be set via the FABRIC_NO_SSH environment variable.

• raise_on_not_found (bool) – Global default for singular getter methods (get_network, get_interface, etc.). When True, these methods raise ResourceNotFoundError if the resource is not found; when False (default), they return None. Individual calls can override this via their raise_exception parameter.

cache_slice(slice_object)

Caches a Slice object by its name and ID.

Adds the given slice object to both the __slices_by_name and __slices_by_id dictionaries for quick retrieval based on either its name or its ID.

If a slice with the same name already exists in cache, it will be replaced. This ensures that the cache always contains the most recent slice object.

Parameters:

slice_object (Slice) – The Slice object to be cached.

close()

Clean up resources including the SSH thread pool executor.

Call this when done using the FablibManager instance, or use the context manager protocol instead:

with FablibManager() as fablib:
    # use fablib
# resources automatically cleaned up

create_artifact(artifact_title, description_short, description_long, authors, tags, visibility=Author, update_existing=True)

Create a new artifact or update an existing one.

Parameters:

• artifact_title (str) – Title of the artifact

• description_short (str) – Short description of the artifact

• description_long (str) – Long description of the artifact

• authors (List[str]) – List of authors associated with the artifact

• tags (List[str]) – List of tags associated with the artifact

• visibility (Visibility) – Visibility level of the artifact

• update_existing (bool) – Flag indicating whether to update an existing artifact

Return type:

Artifact

Returns:

Dictionary containing the artifact details

Raises:

FabricManagerV2Exception – If there is an error in creating or updating the artifact.

create_bastion_keys(*, bastion_key_location=None, store_pubkey=True, overwrite=False)

Create Bastion Keys

Parameters:

• bastion_key_location (str) – bastion key location

• store_pubkey (bool) – flag indicating if the public key should be saved

• overwrite (bool) – overwrite the bastion key file if it exists already

create_sliver_keys(*, sliver_priv_key_location=None, store_pubkey=True, overwrite=False)

Create Sliver Keys

Parameters:

• sliver_priv_key_location (str) – sliver key location

• store_pubkey (bool) – flag indicating if the public key should be saved

• overwrite (bool) – overwrite the bastion key file if it exists already

create_ssh_config(overwrite=False)

Create SSH config file

Parameters:

overwrite (bool) – overwrite the configuration if True, return otherwise

create_ssh_tunnel_config(overwrite=False)

Create zip file containing SSH keys and SSH config file to use for SSH tunnels

Parameters:

overwrite (bool) – overwrite the configuration if True, return otherwise

delete_all(progress=True)

Deletes all slices on the slice manager.

Parameters:

progress (Bool) – optional progress printing to stdout

delete_artifact(artifact_id=None, artifact_title=None)

Delete an artifact by its ID or title.

This method deletes an artifact from the system. Either the artifact_id or artifact_title must be provided to identify the artifact to be deleted. If artifact_id is not provided, the method will search for the artifact using artifact_title and then delete it.

Parameters:

• artifact_id (str) – The unique identifier of the artifact to be deleted.

• artifact_title (str) – The title of the artifact to be deleted.

Raises:

• ValueError – If neither artifact_id nor artifact_title is provided.

• FabricManagerV2Exception – If an error occurs during the deletion process.

delete_slice(slice_name=None)

Deletes a slice by name.

Parameters:

slice_name (String) – the name of the slice to delete

determine_bastion_username()

Determine Bastion Username. Query User Information from Core API and updates the bastion username

discover_ceph_clusters(verify=True)

Discover Ceph clusters via the Ceph Manager API.

Calls CephFsUtils.list_clusters_from_api() using this object’s Ceph Manager base URL and token file.

Parameters:

verify (bool) – Verify TLS certificates when calling the API. Defaults to True.

Returns:

List of cluster name

Return type:

list

Raises:

RuntimeError – If the API call fails.

discover_user_ceph_clusters(verify=True)

Discover Ceph clusters where the current user has a CephX keyring.

Queries all clusters and returns only those where the user’s entity has an active keyring.

Parameters:

verify (bool) – Verify TLS certificates when calling the API.

Returns:

List of cluster names where the user has credentials.

Return type:

list

download_artifact(download_dir, artifact_id=None, artifact_title=None, version=None, version_urn=None)

Download an artifact to a specified directory.

This method downloads an artifact identified by either its artifact_id or artifact_title to the specified download_dir. If artifact_id is not provided, the method will search for the artifact using artifact_title.

Parameters:

• download_dir (str) – The directory where the artifact will be downloaded.

• artifact_id (str) – The unique identifier of the artifact to download.

• artifact_title (str) – The title of the artifact to download.

• version (str) – The specific version of the artifact to download (optional).

• version_urn (str) – Version urn for the artifact

Return type:

str

Returns:

The path to the downloaded artifact.

Raises:

• ValueError – If neither artifact_id nor artifact_title is provided.

• FabricManagerV2Exception – If an error occurs during the download process.

find_resource_slot(start, end, duration, slice=None, resources=None, max_results=1, use_live_data=False)

Find time windows where requested resources are simultaneously available.

Resources can be specified either by passing an unsubmitted Slice object (built with add_node(), add_l2network(), etc.) or by providing a raw list of resource requirement dicts.

When providing resources directly, component keys in the "components" dict can use either the fablib model name (e.g. "NIC_ConnectX_5") or the calendar/DB format (e.g. "SmartNIC-ConnectX-5"); they will be normalized automatically.

Parameters:

• start (datetime) – start of the search window (UTC)

• end (datetime) – end of the search window (UTC)

• duration (int) – required slot length in hours

• slice (Optional[Slice]) – an unsubmitted Slice whose topology will be converted to resource requirements automatically

• resources (Optional[List[Dict[str, Any]]]) – list of resource requirement dicts (advanced usage)

• max_results (int) – maximum number of slots to return

• use_live_data (bool) – when True, use live orchestrator data instead of Reports API for more accurate, real-time availability

Return type:

Dict[str, Any]

Returns:

dict with slot results

Raises:

ValueError – if both or neither of slice and resources are provided

generate_ceph_bundle(cluster, out_base='~/.ceph', mount_root='/mnt/cephfs', verify=True)

Generate a local bundle for mounting CephFS paths for the current user on a specific cluster.

This:

1. Fetches the minimal ceph.conf for region.

2. Exports the user’s keyring from that cluster.

3. Writes files under out_base/<region>/: ceph.conf, ceph.client.<user>.secret, ceph.client.<user>.keyring, and a mount script mount_<user>.sh that mounts every path found in the MDS caps.

Parameters:

• cluster (str) – Target cluster name (e.g., "europe").

• out_base (str) – Output root directory for artifacts (default: "~/.ceph").

• mount_root (str) – Mount prefix used by the generated script (default: "/mnt/cephfs").

• verify (bool) – Verify TLS certificates when calling the API. Defaults to True.

Returns:

Details of the generated bundle, for example:

{

”cluster_dir”: “~/.ceph/europe”, “ceph_conf”: “~/.ceph/europe/ceph.conf”, “secret_file”: “~/.ceph/europe/ceph.client.alice.secret”, “keyring_file”: “~/.ceph/europe/ceph.client.alice.keyring”, “mount_script”: “~/.ceph/europe/mount_alice.sh”, “entity”: “client.alice”, “user”: “alice”, “mounts”: [

{

“fsname”: “CEPH-FS-01”, “path”: “/volumes/_nogroup/alice/….”, “mount_point”: “/mnt/cephfs/europe/alice/volumes__nogroup_alice____”

]

}

Return type:

dict

Raises:

• ValueError – If cluster is unknown or the user name is empty.

• RuntimeError – If no clusters are discovered or keyring is unavailable.

get_artifacts(artifact_title=None, artifact_id=None, tag=None)

Gets a list of artifacts either based on artifact id, artifact title or tag. :type artifact_title: str :param artifact_title: :type artifact_id: str :param artifact_id: :type tag: str :param tag:

Returns:

a list of Artifacts

Return type:

List[Artifact]

get_available_resources(update=False, force_refresh=False, start=None, end=None, avoid=None, includes=None)

Get the available resources.

Optionally update the available resources by querying the FABRIC services. Otherwise, this method returns the existing information.

Parameters:

• update (bool)

• force_refresh (bool)

• start (Optional[<module ‘datetime’ from ‘/home/docs/.asdf/installs/python/3.11.14/lib/python3.11/datetime.py’>]) – start time in UTC format: %Y-%m-%d %H:%M:%S %z

• end (Optional[<module ‘datetime’ from ‘/home/docs/.asdf/installs/python/3.11.14/lib/python3.11/datetime.py’>]) – end time in UTC format: %Y-%m-%d %H:%M:%S %z

• avoid (Optional[List[str]]) – list of sites to avoid

• includes (Optional[List[str]]) – list of sites to include

Type:

datetime

Type:

datetime

Type:

list of string

Type:

list of string

Return type:

ResourcesV2

Returns:

Available ResourcesV2Wrapper object

get_crinkle_slice(name=None, slice_id=None, user_only=True, show_un_submitted=False, pcaps_dir='.query_analysis_pcaps', name_prefix='C')

Gets a Crinkle slice by name or slice_id. Dead and Closing slices may have non-unique names and must be queried by slice_id. Slices in all other states are guaranteed to have unique names and can be queried by name.

If both a name and slice_id are provided, the slice matching the slice_id will be returned.

Parameters:

• name (String) – The name of the desired slice

• slice_id (String) – The ID of the desired slice

• user_only (bool) – True indicates return own slices; False indicates return project slices

• show_un_submitted (bool) – Show unsubmitted slices

• pcaps_dir (String) – The working directory for .pcap file transfers

• name_prefix (String) – The name prefix to use for autogenerated Crinkle objects. Must match what was used in slice creation

Raises:

Exception: if slice name or slice id are not inputted

Returns:

the slice, if found

Return type:

Slice

get_crinkle_slices(excludes=[Dead, Closing], slice_name=None, slice_id=None, user_only=True, show_un_submitted=False, pcaps_dir='.query_analysis_pcaps', name_prefix='C')

Gets a list of Crinkle slices from the slice manager.

By default this method ignores Dead and Closing slices. Optional, parameter allows excluding a different list of slice states. Pass an empty list (i.e. excludes=[]) to get a list of all slices.

Parameters:

• excludes (List[SliceState]) – A list of slice states to exclude from the output list

• slice_name (str)

• slice_id (str)

• user_only (bool) – True indicates return own slices; False indicates return project slices

• show_un_submitted (bool) – Show unsubmitted slices;

• pcaps_dir (String) – The working directory for .pcap file transfers

• name_prefix (String) – The name prefix to use for autogenerated Crinkle objects. Must match what was used in slice creation

Returns:

a list of slices

Return type:

List[Slice]

get_default_slice_key()

Gets the current default_slice_keys as a dictionary containing the public and private slice keys.

Important! Slice key management is under development and this functionality will likely change going forward.

Returns:

default_slice_key dictionary

Return type:

dict[str, str]

get_facility_ports(update=False, start=None, end=None)

Get the resources object (which includes facility ports).

Parameters:

• update (Optional[bool]) – whether to refresh resources

• start (Optional[<module ‘datetime’ from ‘/home/docs/.asdf/installs/python/3.11.14/lib/python3.11/datetime.py’>]) – start time in UTC format: %Y-%m-%d %H:%M:%S %z

• end (Optional[<module ‘datetime’ from ‘/home/docs/.asdf/installs/python/3.11.14/lib/python3.11/datetime.py’>]) – end time in UTC format: %Y-%m-%d %H:%M:%S %z

Return type:

ResourcesV2

Returns:

ResourcesV2Wrapper

get_image_names()

Gets a list of available image names.

Returns:

Dictionary of images with default user and description

Return type:

dict[str, dict]

get_links(update=True)

Get the resources object (which includes links).

Parameters:

update (bool) – whether to refresh resources

Return type:

ResourcesV2

Returns:

ResourcesV2Wrapper

get_manager()

Gets the manager of this fablib object.

Lazily initializes the FabricManagerV2 on first access, avoiding network calls during FablibManager construction.

Returns:

the manager on this fablib object

Return type:

FabricManagerV2

get_project_tags()

Return the set of permission tags for the current project from the token.

The result is cached for the lifetime of this manager instance. Returns an empty frozenset in offline mode or if the token cannot be decoded.

Returns:

frozenset of tag strings (e.g. "Component.GPU")

Return type:

frozenset

get_random_site(avoid=None, filter_function=None, update=True)

Get a random site.

Parameters:

• avoid (List[String]) – list of site names to avoid choosing

• filter_function – filter_function

• update (bool) – flag indicating if fetch latest availability information

Returns:

one site name

Return type:

String

get_random_sites(count=1, avoid=None, filter_function=None, update=True)

Get a list of random sites names. Each site will be included at most once.

Parameters:

• count (int) – number of sites to return.

• avoid (List[String]) – list of site names to avoid choosing

• filter_function – filter_function

• update (bool) – flag indicating if fetch latest availability information

Returns:

list of random site names.

Return type:

List[String]

get_resources(update=False, force_refresh=False, start=None, end=None, avoid=None, includes=None)

Get a reference to the resources object. The resources object is used to query for available resources and capacities.

Parameters:

• update (bool)

• force_refresh (bool)

• start (Optional[<module ‘datetime’ from ‘/home/docs/.asdf/installs/python/3.11.14/lib/python3.11/datetime.py’>]) – start time in UTC format: %Y-%m-%d %H:%M:%S %z

• end (Optional[<module ‘datetime’ from ‘/home/docs/.asdf/installs/python/3.11.14/lib/python3.11/datetime.py’>]) – end time in UTC format: %Y-%m-%d %H:%M:%S %z

• avoid (Optional[List[str]]) – list of sites to avoid

• includes (Optional[List[str]]) – list of sites to include

Type:

datetime

Type:

datetime

Type:

list of string

Type:

list of string

Returns:

the resources object

Return type:

ResourcesV2

get_site_names()

Gets a list of all available site names.

Returns:

list of site names as strings

Return type:

list[str]

get_slice(name=None, slice_id=None, user_only=True, show_un_submitted=False)

Gets a slice by name or slice_id. Dead and Closing slices may have non-unique names and must be queried by slice_id. Slices in all other states are guaranteed to have unique names and can be queried by name.

If both a name and slice_id are provided, the slice matching the slice_id will be returned.

Parameters:

• name (String) – The name of the desired slice

• slice_id (String) – The ID of the desired slice

• user_only (bool) – True indicates return own slices; False indicates return project slices

• show_un_submitted (bool) – Show unsubmitted slices

Raises:

Exception: if slice name or slice id are not inputted

Returns:

the slice, if found

Return type:

Slice

get_slices(excludes=None, slice_name=None, slice_id=None, user_only=True, show_un_submitted=False)

Gets a list of slices from the slice manager.

By default this method ignores Dead and Closing slices. Optional, parameter allows excluding a different list of slice states. Pass an empty list (i.e. excludes=[]) to get a list of all slices.

Parameters:

• excludes (List[SliceState]) – A list of slice states to exclude from the output list. Defaults to [SliceState.Dead, SliceState.Closing].

• slice_name (str) – Filter by slice name

• slice_id (str) – Filter by slice ID

• user_only (bool) – True indicates return own slices; False indicates return project slices

• show_un_submitted (bool) – Show unsubmitted slices

Returns:

a list of slices

Return type:

List[Slice]

get_ssh_thread_pool_executor()

Get ThreadPoolExecutor that runs SSH commands.

Return type:

ThreadPoolExecutor

get_storage(uuid)

Get a specific storage volume by UUID.

Parameters:

uuid (str) – Storage volume UUID.

Returns:

Storage volume details.

Return type:

list

Raises:

FabricManagerV2Exception – If there is an error retrieving the storage volume.

get_tags()

Retrieve all tags associated with artifacts.

This method returns a list of all tags that are associated with artifacts in the system. Tags are useful for categorizing and searching for artifacts.

Returns:

A list of tags.

Raises:

FabricManagerV2Exception – If an error occurs while retrieving the tags.

get_user_info()

Get User information

Returns:

returns a dictionary containing User’s Information

Return type:

dict

list_artifacts(output=None, fields=None, quiet=False, filter_function=None, pretty_names=True)

List artifacts based on a search query.

Parameters:

• search – Search query for filtering artifacts

• output – Output format - ‘text’, ‘pandas’, ‘json’

• fields – List of fields (table columns) to show

• quiet – True to suppress printing/display

• filter_function – Lambda function to filter data by field values

• pretty_names – Whether to use pretty names for fields

Return type:

object

Returns:

Table in format specified by output parameter

Raises:

FabricManagerV2Exception – If there is an error in listing the artifacts.

list_facility_ports(output=None, fields=None, quiet=False, filter_function=None, update=False, pretty_names=True, start=None, end=None)

Lists all the facility ports and their attributes.

There are several output options: “text”, “pandas”, and “json” that determine the format of the output that is returned and (optionally) displayed/printed.

output: ‘text’: string formatted with tabular

‘pandas’: pandas dataframe ‘json’: string in json format

fields: json output will include all available fields/columns.

Example: TODO

filter_function: A lambda function to filter data by field values.

Example: filter_function=lambda s: s[‘ConnectX-5 Available’] > 3 and s[‘NVMe Available’] <= 10

Parameters:

• output (str) – output format

• fields (List[str]) – list of fields (table columns) to show

• quiet (bool) – True to specify printing/display

• filter_function (lambda) – lambda function

• update (bool)

• pretty_names (bool)

• start (Optional[<module ‘datetime’ from ‘/home/docs/.asdf/installs/python/3.11.14/lib/python3.11/datetime.py’>]) – start time in UTC format: %Y-%m-%d %H:%M:%S %z

• end (Optional[<module ‘datetime’ from ‘/home/docs/.asdf/installs/python/3.11.14/lib/python3.11/datetime.py’>]) – end time in UTC format: %Y-%m-%d %H:%M:%S %z

Type:

datetime

Type:

datetime

Returns:

table in format specified by output parameter

Return type:

Object

list_hosts(output=None, fields=None, quiet=False, filter_function=None, update=False, pretty_names=True, force_refresh=False, start=None, end=None, avoid=None, includes=None)

Lists all the hosts and their attributes.

There are several output options: “text”, “pandas”, and “json” that determine the format of the output that is returned and (optionally) displayed/printed.

output: ‘text’: string formatted with tabular

‘pandas’: pandas dataframe ‘json’: string in json format

fields: json output will include all available fields/columns.

Example: fields=[‘Name’,’ConnectX-5 Available’, ‘NVMe Total’]

filter_function: A lambda function to filter data by field values.

Example: filter_function=lambda s: s[‘ConnectX-5 Available’] > 3 and s[‘NVMe Available’] <= 10

Parameters:

• output (str) – output format

• fields (List[str]) – list of fields (table columns) to show

• quiet (bool) – True to specify printing/display

• filter_function (lambda) – lambda function

• update (bool)

• pretty_names (bool)

• force_refresh (bool)

• start (Optional[<module ‘datetime’ from ‘/home/docs/.asdf/installs/python/3.11.14/lib/python3.11/datetime.py’>]) – start time in UTC format: %Y-%m-%d %H:%M:%S %z

• end (Optional[<module ‘datetime’ from ‘/home/docs/.asdf/installs/python/3.11.14/lib/python3.11/datetime.py’>]) – end time in UTC format: %Y-%m-%d %H:%M:%S %z

• avoid (Optional[List[str]]) – list of sites to avoid

• includes (Optional[List[str]]) – list of sites to include

Return type:

object

Returns:

table in format specified by output parameter

Type:

datetime

Type:

datetime

Type:

list of string

Type:

list of string

list_links(output=None, fields=None, quiet=False, filter_function=None, update=True, pretty_names=True)

Lists all the links and their attributes.

There are several output options: “text”, “pandas”, and “json” that determine the format of the output that is returned and (optionally) displayed/printed.

output: ‘text’: string formatted with tabular

‘pandas’: pandas dataframe ‘json’: string in json format

fields: json output will include all available fields/columns.

Example: TODO

filter_function: A lambda function to filter data by field values.

Example: filter_function=lambda s: s[‘ConnectX-5 Available’] > 3 and s[‘NVMe Available’] <= 10

Parameters:

• output (str) – output format

• fields (List[str]) – list of fields (table columns) to show

• quiet (bool) – True to specify printing/display

• filter_function (lambda) – lambda function

• update (bool)

• pretty_names (bool)

Returns:

table in format specified by output parameter

Return type:

Object

list_sites(output=None, fields=None, quiet=False, filter_function=None, update=False, pretty_names=True, force_refresh=False, start=None, end=None, avoid=None, includes=None)

Lists all the sites and their attributes.

There are several output options: “text”, “pandas”, and “json” that determine the format of the output that is returned and (optionally) displayed/printed.

output: ‘text’: string formatted with tabular

‘pandas’: pandas dataframe ‘json’: string in json format

fields: json output will include all available fields/columns.

Example: fields=[‘Name’,’ConnectX-5 Available’, ‘NVMe Total’]

filter_function: A lambda function to filter data by field values.

Example: filter_function=lambda s: s[‘ConnectX-5 Available’] > 3 and s[‘NVMe Available’] <= 10

Parameters:

• output (str) – output format

• fields (List[str]) – list of fields (table columns) to show

• quiet (bool) – True to specify printing/display

• filter_function (lambda) – lambda function

• update (bool)

• pretty_names (bool)

• force_refresh (bool)

• latlon – convert address to latlon, makes online call to openstreetmaps.org

• start (Optional[<module ‘datetime’ from ‘/home/docs/.asdf/installs/python/3.11.14/lib/python3.11/datetime.py’>]) – start time in UTC format: %Y-%m-%d %H:%M:%S %z

• end (Optional[<module ‘datetime’ from ‘/home/docs/.asdf/installs/python/3.11.14/lib/python3.11/datetime.py’>]) – end time in UTC format: %Y-%m-%d %H:%M:%S %z

• avoid (Optional[List[str]]) – list of sites to avoid

• includes (Optional[List[str]]) – list of sites to include

Return type:

object

Returns:

table in format specified by output parameter

Type:

Object

Type:

datetime

Type:

datetime

Type:

list of string

Type:

list of string

list_slices(excludes=None, output=None, fields=None, quiet=False, filter_function=None, pretty_names=True, user_only=True, show_un_submitted=False)

Lists all the slices created by a user.

There are several output options: “text”, “pandas”, and “json” that determine the format of the output that is returned and (optionally) displayed/printed.

output: ‘text’: string formatted with tabular

‘pandas’: pandas dataframe ‘json’: string in json format

fields: json output will include all available fields/columns.

Example: fields=[‘Name’,’State’]

filter_function: A lambda function to filter data by field values.

Example: filter_function=lambda s: s[‘State’] == ‘Configuring’

Parameters:

• excludes (list[SliceState]) – slice states to exclude. Defaults to [SliceState.Dead, SliceState.Closing].

• output (str) – output format

• fields (List[str]) – list of fields (table columns) to show

• quiet (bool) – True to specify printing/display

• filter_function (lambda) – lambda function

• pretty_names (bool) – pretty_names

• user_only (bool) – True indicates return own slices; False indicates return project slices

• show_un_submitted (bool) – True indicates to also show unsubmitted slices

Returns:

table in format specified by output parameter

Return type:

Object

list_storage(offset=0, limit=200, fetch_all=False, output=None, fields=None, quiet=False, filter_function=None)

List storage volumes available for the project.

Parameters:

• offset (int) – Pagination offset (default: 0).

• limit (int) – Maximum number of records to fetch per page (default: 200).

• fetch_all (bool) – If True, automatically fetch all storage volumes across all pages (default: False).

• output – Output format - ‘text’, ‘pandas’, ‘json’

• fields – List of fields (table columns) to show

• quiet – True to suppress printing/display

• filter_function – Lambda function to filter data by field values

Return type:

object

Returns:

Table in format specified by output parameter

Raises:

FabricManagerV2Exception – If there is an error in listing storage volumes.

new_crinkle_slice(name, pcaps_dir='.query_analysis_pcaps', name_prefix='C')

Creates a new Crinkle slice with the given name and with an Analyzer node.

Parameters:

• name (String) – the name to give the slice

• analyzer_name (String) – Name for the

• cores (int)

• ram (int)

• disk (int)

• site (String)

• pcaps_dir (String)

Returns:

A new Crinkle slice

Return type:

CrinkleSlice

new_slice(name, storage=False, storage_cluster=None)

Creates a new slice with the given name.

Parameters:

• name (String) – the name to give the slice

• storage (bool) – Enable CephFS storage on every node added to this slice. Default: False

• storage_cluster (str) – Ceph cluster name. When None, the first available cluster is auto-discovered at post-boot time.

Returns:

a new slice

Return type:

Slice

probe_bastion_host()

See if bastion will admit us with our configuration.

Bastion hosts are configured to block hosts that attempts to use it with too many repeated authentication failures. We want to avoid that.

Returns True if connection attempt succeeds. Raises an error in the event of failure.

Return type:

Optional[bool]

remove_slice_from_cache(slice_object)

Removes a Slice object from the cache by its name and ID.

Removes the slice from both __slices_by_name and __slices_by_id if present.

Parameters:

slice_object (Slice) – The Slice object to be removed from the cache.

resources_calendar(start, end, interval='day', site=None, host=None, exclude_site=None, exclude_host=None, show='all', output=None, fields=None, quiet=False, filter_function=None)

Fetch and display resource availability calendar over time slots.

There are several output options: "text", "pandas", and "json" that determine the format of the output that is returned and (optionally) displayed/printed.

output: 'text': string formatted with tabulate

'pandas': pandas dataframe (default in Jupyter) 'json': string in json format 'list': raw list of dicts

fields: list of columns to include in the output table.

Example: fields=['Date', 'Name', 'Cores (avail/cap)', 'RAM GB (avail/cap)']

filter_function: A lambda to filter the flattened rows.

Example: filter_function=lambda r: r['Name'] == 'RENC'

Parameters:

• start (datetime.datetime) – start of the calendar window (UTC)

• end (datetime.datetime) – end of the calendar window (UTC)

• interval (str) – time slot granularity: hour, day, or week

• site (list[str]) – list of site names to include

• host (list[str]) – list of host names to include

• exclude_site (list[str]) – list of site names to exclude

• exclude_host (list[str]) – list of host names to exclude

• show (str) – which resource types to display: "sites", "hosts", or "all" (default)

• output (str) – output format

• fields (list[str]) – list of fields (table columns) to show

• quiet (bool) – True to suppress printing/display

• filter_function (callable) – lambda function to filter rows

Returns:

table in format specified by output parameter

show_config(output=None, fields=None, quiet=False, pretty_names=True)

Show a table containing the current FABlib configuration parameters.

There are several output options: “text”, “pandas”, and “json” that determine the format of the output that is returned and (optionally) displayed/printed.

output: ‘text’: string formatted with tabular

‘pandas’: pandas dataframe ‘json’: string in json format

fields: json output will include all available fields.

Example: fields=[‘credmgr_host’,’project_id’, ‘fablib_log_file’]

Parameters:

• output (str) – output format

• fields (List[str]) – list of fields to show

• quiet (bool) – True to specify printing/display

• pretty_names (bool)

Returns:

table in format specified by output parameter

Return type:

Object

show_site(site_name, output=None, fields=None, quiet=False, pretty_names=True)

Show a table with all the properties of a specific site

There are several output options: “text”, “pandas”, and “json” that determine the format of the output that is returned and (optionally) displayed/printed.

output: ‘text’: string formatted with tabular

‘pandas’: pandas dataframe ‘json’: string in json format

fields: json output will include all available fields.

Example: fields=[‘credmgr_host’,’project_id’, ‘fablib_log_file’]

Parameters:

• site_name (str) – the name of a site

• output (str) – output format

• fields (List[str]) – list of fields to show

• quiet (bool) – True to specify printing/display

• pretty_names (bool)

• latlon (bool) – convert address to lat/lon

Returns:

table in format specified by output parameter

Return type:

Object

show_slice(name=None, id=None, output=None, fields=None, quiet=False, pretty_names=True, user_only=True, show_un_submitted=False)

Show a table with all the properties of a specific site

There are several output options: “text”, “pandas”, and “json” that determine the format of the output that is returned and (optionally) displayed/printed.

output: ‘text’: string formatted with tabular

‘pandas’: pandas dataframe ‘json’: string in json format

fields: json output will include all available fields.

Example: fields=[‘Name’,’State’]

Parameters:

• name (str) – the name of a slice

• id (Optional[str]) – the slice id

• output (str) – output format

• fields (List[str]) – list of fields to show

• quiet (bool) – True to specify printing/display

• pretty_names (bool) – pretty_names

• user_only (bool) – True indicates return own slices; False indicates return project slices

• show_un_submitted (bool) – True indicates to also show unsubmitted slices

Returns:

table in format specified by output parameter

Return type:

Object

update_slice_cache_id(slice_object)

Updates the cache when a slice gets a new ID (e.g., after submit).

This should be called after a slice is submitted and receives its slice_id.

Parameters:

slice_object (Slice) – The Slice object whose ID was updated.

upload_file_to_artifact(file_to_upload, artifact_id=None, artifact_title=None)

Upload a file to an existing artifact.

This method uploads a file to an artifact identified by either its artifact_id or artifact_title. If artifact_id is not provided, the method will search for the artifact using artifact_title before uploading the file.

Parameters:

• file_to_upload (str) – The path to the file that should be uploaded.

• artifact_id (str) – The unique identifier of the artifact to which the file will be uploaded.

• artifact_title (str) – The title of the artifact to which the file will be uploaded.

Return type:

dict

Returns:

A dictionary containing the details of the uploaded file.

Raises:

• ValueError – If neither artifact_id nor artifact_title is provided.

• FabricManagerV2Exception – If an error occurs during the upload process.

validate_and_update_bastion_keys(validate_only=False)

Validate Bastion Key; if key does not exist or is expired, it creates bastion keys

Parameters:

validate_only (bool) – flag to specify to only do config validation

validate_config()

Validate and create Fablib config - checks if all the required configuration exists for slice provisioning to work successfully

• Checks Credential Manager Host is configured properly

• Checks Orchestrator Host is configured properly

• Checks Core API Host is configured properly

• Checks Bastion Host is configured properly

• Check Sliver keys exist; create sliver keys if they do not exist

• Check Bastion keys exist and are not expired; update/create bastion keys if expired or do not exist

• Check Bastion Username is configured

• Check Project Id is configured

Deprecated since version 1.6.5: Use verify_and_configure() instead.

@raises Exception if the configuration is invalid

validate_node(node, allocated=None)

Validate a node w.r.t available resources on a site before submission.

Delegates to NodeValidatorV2 with pre-fetched resources. This method must remain on FablibManager because Node.add_component() calls self.get_fablib_manager().validate_node().

Returns:

Tuple indicating status for validation and error message in case of failure

Return type:

Tuple[bool, str]

verify_and_configure(validate_only=False)

Validate and create Fablib config - checks if all the required configuration exists for slice provisioning to work successfully

• Checks Credential Manager Host is configured properly

• Checks Orchestrator Host is configured properly

• Checks Core API Host is configured properly

• Checks Bastion Host is configured properly

• Check Sliver keys exist; create sliver keys if they do not exist

• Check Bastion keys exist and are not expired; update/create bastion keys if expired or do not exist

• Check Bastion Username is configured

• Check Project Id is configured

Parameters:

validate_only (bool) – flag to specify to only do config validation

:raises Exception if the configuration is invalid

class fabrictestbed_extensions.fablib.fablib.FablibManager(fabric_rc=None, credmgr_host=None, orchestrator_host=None, core_api_host=None, am_host=None, ceph_mgr_host=None, token_location=None, id_token=None, project_id=None, bastion_username=None, bastion_key_location=None, log_level='INFO', log_file='/tmp/fablib/fablib.log', log_propagate=False, data_dir='/tmp/fablib', execute_thread_pool_size=64, offline=False, auto_token_refresh=True, validate_config=True, no_ssh=False, raise_on_not_found=False, **kwargs)

FablibManager is the main interface to FABRIC services.

A FablibManager object is used to query FABRIC testbed for available resources, create and configure slices, manage SSH keys in nodes in slices and FABRIC’s bastion host, etc. This requires some configuration, which is gathered from:

• constructor parameters (highest priority)

• a configuration file (medium priority, optional)

• environment variables (low priority)

• defaults (lowest priority, if needed and when possible)

The configuration file is optional. You can provide configuration entirely through constructor parameters and/or environment variables. If using a config file, it typically would be located at "${HOME}/work/fabric_config/fabric_rc".

Parameters:

• fabric_rc (str) – Path to fablib configuration file. Optional. Defaults to "${HOME}/work/fabric_config/fabric_rc", but the file doesn’t need to exist.

• credmgr_host (str) – Name of credential manager host.

• orchestrator_host (str) – Name of FABRIC orchestrator host.

• core_api_host (str) – Name of Core API host.

• am_host (str) – Name of Aggregate Manager host.

• ceph_mgr_host (str) – Name of ceph manager host.

• token_location (str) – Path to the file that contains your FABRIC auth token.

• id_token (str) – ID token string to use directly (optional). If provided, token_location file check is skipped and auto_token_refresh is automatically disabled.

• project_id (str) – Your FABRIC project ID, obtained from https://cm.fabric-testbed.net/, usually via FABRIC portal.

• bastion_username (str) – Your username on FABRIC bastion host, obtained from FABRIC portal.

• bastion_key_location (str) – Path to your bastion SSH key.

• log_file (str) – Path where fablib logs are written; defaults to "/tmp/fablib/fablib.log".

• log_level (str) – Level of detail in the logs written. Defaults to "DEBUG"; other possible log levels are "INFO", "WARNING", "ERROR", and "CRITICAL", in reducing order of verbosity.

• log_propagate (bool) – Whether fablib logs propagate to the root logger. Defaults to False to avoid duplicate logging in notebooks.

• data_dir (str) – directory for fablib to store temporary data.

• output – Format of fablib output; can be either "pandas" or "text". Defaults to "pandas" in a Jupyter notebook environment; "text" otherwise.

• execute_thread_pool_size (int) – Number of worker threads in the thread pool fablib uses to execute commands in nodes. Defaults to 64.

• offline (bool) – Avoid using FABRIC services when initializing. This is False by default, and set to True only in some unit tests.

• auto_token_refresh (bool) – Auto refresh tokens (automatically disabled if id_token is provided)

• validate_config (bool) – Whether to verify and persist configuration during initialization.

• no_ssh (bool) – Disable SSH operations. When True, skips SSH-related initialization (thread pool, bastion probing, SSH key validation) and prevents SSH operations on nodes. Useful for API-only mode (e.g., MCP server). Can also be set via the FABRIC_NO_SSH environment variable.

• raise_on_not_found (bool) – Global default for singular getter methods (get_network, get_interface, etc.). When True, these methods raise ResourceNotFoundError if the resource is not found; when False (default), they return None. Individual calls can override this via their raise_exception parameter.

cache_slice(slice_object)

Caches a Slice object by its name and ID.

Adds the given slice object to both the __slices_by_name and __slices_by_id dictionaries for quick retrieval based on either its name or its ID.

If a slice with the same name already exists in cache, it will be replaced. This ensures that the cache always contains the most recent slice object.

Parameters:

slice_object (Slice) – The Slice object to be cached.

close()

Clean up resources including the SSH thread pool executor.

Call this when done using the FablibManager instance, or use the context manager protocol instead:

with FablibManager() as fablib:
    # use fablib
# resources automatically cleaned up

create_artifact(artifact_title, description_short, description_long, authors, tags, visibility=Author, update_existing=True)

Create a new artifact or update an existing one.

Parameters:

• artifact_title (str) – Title of the artifact

• description_short (str) – Short description of the artifact

• description_long (str) – Long description of the artifact

• authors (List[str]) – List of authors associated with the artifact

• tags (List[str]) – List of tags associated with the artifact

• visibility (Visibility) – Visibility level of the artifact

• update_existing (bool) – Flag indicating whether to update an existing artifact

Return type:

Artifact

Returns:

Dictionary containing the artifact details

Raises:

FabricManagerV2Exception – If there is an error in creating or updating the artifact.

create_bastion_keys(*, bastion_key_location=None, store_pubkey=True, overwrite=False)

Create Bastion Keys

Parameters:

• bastion_key_location (str) – bastion key location

• store_pubkey (bool) – flag indicating if the public key should be saved

• overwrite (bool) – overwrite the bastion key file if it exists already

create_sliver_keys(*, sliver_priv_key_location=None, store_pubkey=True, overwrite=False)

Create Sliver Keys

Parameters:

• sliver_priv_key_location (str) – sliver key location

• store_pubkey (bool) – flag indicating if the public key should be saved

• overwrite (bool) – overwrite the bastion key file if it exists already

create_ssh_config(overwrite=False)

Create SSH config file

Parameters:

overwrite (bool) – overwrite the configuration if True, return otherwise

create_ssh_tunnel_config(overwrite=False)

Create zip file containing SSH keys and SSH config file to use for SSH tunnels

Parameters:

overwrite (bool) – overwrite the configuration if True, return otherwise

delete_all(progress=True)

Deletes all slices on the slice manager.

Parameters:

progress (Bool) – optional progress printing to stdout

delete_artifact(artifact_id=None, artifact_title=None)

Delete an artifact by its ID or title.

This method deletes an artifact from the system. Either the artifact_id or artifact_title must be provided to identify the artifact to be deleted. If artifact_id is not provided, the method will search for the artifact using artifact_title and then delete it.

Parameters:

• artifact_id (str) – The unique identifier of the artifact to be deleted.

• artifact_title (str) – The title of the artifact to be deleted.

Raises:

• ValueError – If neither artifact_id nor artifact_title is provided.

• FabricManagerV2Exception – If an error occurs during the deletion process.

delete_slice(slice_name=None)

Deletes a slice by name.

Parameters:

slice_name (String) – the name of the slice to delete

determine_bastion_username()

Determine Bastion Username. Query User Information from Core API and updates the bastion username

discover_ceph_clusters(verify=True)

Discover Ceph clusters via the Ceph Manager API.

Calls CephFsUtils.list_clusters_from_api() using this object’s Ceph Manager base URL and token file.

Parameters:

verify (bool) – Verify TLS certificates when calling the API. Defaults to True.

Returns:

List of cluster name

Return type:

list

Raises:

RuntimeError – If the API call fails.

discover_user_ceph_clusters(verify=True)

Discover Ceph clusters where the current user has a CephX keyring.

Queries all clusters and returns only those where the user’s entity has an active keyring.

Parameters:

verify (bool) – Verify TLS certificates when calling the API.

Returns:

List of cluster names where the user has credentials.

Return type:

list

download_artifact(download_dir, artifact_id=None, artifact_title=None, version=None, version_urn=None)

Download an artifact to a specified directory.

This method downloads an artifact identified by either its artifact_id or artifact_title to the specified download_dir. If artifact_id is not provided, the method will search for the artifact using artifact_title.

Parameters:

• download_dir (str) – The directory where the artifact will be downloaded.

• artifact_id (str) – The unique identifier of the artifact to download.

• artifact_title (str) – The title of the artifact to download.

• version (str) – The specific version of the artifact to download (optional).

• version_urn (str) – Version urn for the artifact

Return type:

str

Returns:

The path to the downloaded artifact.

Raises:

• ValueError – If neither artifact_id nor artifact_title is provided.

• FabricManagerV2Exception – If an error occurs during the download process.

find_resource_slot(start, end, duration, slice=None, resources=None, max_results=1, use_live_data=False)

Find time windows where requested resources are simultaneously available.

Resources can be specified either by passing an unsubmitted Slice object (built with add_node(), add_l2network(), etc.) or by providing a raw list of resource requirement dicts.

When providing resources directly, component keys in the "components" dict can use either the fablib model name (e.g. "NIC_ConnectX_5") or the calendar/DB format (e.g. "SmartNIC-ConnectX-5"); they will be normalized automatically.

Parameters:

• start (datetime) – start of the search window (UTC)

• end (datetime) – end of the search window (UTC)

• duration (int) – required slot length in hours

• slice (Optional[Slice]) – an unsubmitted Slice whose topology will be converted to resource requirements automatically

• resources (Optional[List[Dict[str, Any]]]) – list of resource requirement dicts (advanced usage)

• max_results (int) – maximum number of slots to return

• use_live_data (bool) – when True, use live orchestrator data instead of Reports API for more accurate, real-time availability

Return type:

Dict[str, Any]

Returns:

dict with slot results

Raises:

ValueError – if both or neither of slice and resources are provided

generate_ceph_bundle(cluster, out_base='~/.ceph', mount_root='/mnt/cephfs', verify=True)

Generate a local bundle for mounting CephFS paths for the current user on a specific cluster.

This:

1. Fetches the minimal ceph.conf for region.

2. Exports the user’s keyring from that cluster.

3. Writes files under out_base/<region>/: ceph.conf, ceph.client.<user>.secret, ceph.client.<user>.keyring, and a mount script mount_<user>.sh that mounts every path found in the MDS caps.

Parameters:

• cluster (str) – Target cluster name (e.g., "europe").

• out_base (str) – Output root directory for artifacts (default: "~/.ceph").

• mount_root (str) – Mount prefix used by the generated script (default: "/mnt/cephfs").

• verify (bool) – Verify TLS certificates when calling the API. Defaults to True.

Returns:

Details of the generated bundle, for example:

{

”cluster_dir”: “~/.ceph/europe”, “ceph_conf”: “~/.ceph/europe/ceph.conf”, “secret_file”: “~/.ceph/europe/ceph.client.alice.secret”, “keyring_file”: “~/.ceph/europe/ceph.client.alice.keyring”, “mount_script”: “~/.ceph/europe/mount_alice.sh”, “entity”: “client.alice”, “user”: “alice”, “mounts”: [

{

“fsname”: “CEPH-FS-01”, “path”: “/volumes/_nogroup/alice/….”, “mount_point”: “/mnt/cephfs/europe/alice/volumes__nogroup_alice____”

]

}

Return type:

dict

Raises:

• ValueError – If cluster is unknown or the user name is empty.

• RuntimeError – If no clusters are discovered or keyring is unavailable.

get_artifacts(artifact_title=None, artifact_id=None, tag=None)

Gets a list of artifacts either based on artifact id, artifact title or tag. :type artifact_title: str :param artifact_title: :type artifact_id: str :param artifact_id: :type tag: str :param tag:

Returns:

a list of Artifacts

Return type:

List[Artifact]

get_available_resources(update=False, force_refresh=False, start=None, end=None, avoid=None, includes=None)

Get the available resources.

Optionally update the available resources by querying the FABRIC services. Otherwise, this method returns the existing information.

Parameters:

• update (bool)

• force_refresh (bool)

• start (Optional[<module ‘datetime’ from ‘/home/docs/.asdf/installs/python/3.11.14/lib/python3.11/datetime.py’>]) – start time in UTC format: %Y-%m-%d %H:%M:%S %z

• end (Optional[<module ‘datetime’ from ‘/home/docs/.asdf/installs/python/3.11.14/lib/python3.11/datetime.py’>]) – end time in UTC format: %Y-%m-%d %H:%M:%S %z

• avoid (Optional[List[str]]) – list of sites to avoid

• includes (Optional[List[str]]) – list of sites to include

Type:

datetime

Type:

datetime

Type:

list of string

Type:

list of string

Return type:

ResourcesV2

Returns:

Available ResourcesV2Wrapper object

get_crinkle_slice(name=None, slice_id=None, user_only=True, show_un_submitted=False, pcaps_dir='.query_analysis_pcaps', name_prefix='C')

Gets a Crinkle slice by name or slice_id. Dead and Closing slices may have non-unique names and must be queried by slice_id. Slices in all other states are guaranteed to have unique names and can be queried by name.

If both a name and slice_id are provided, the slice matching the slice_id will be returned.

Parameters:

• name (String) – The name of the desired slice

• slice_id (String) – The ID of the desired slice

• user_only (bool) – True indicates return own slices; False indicates return project slices

• show_un_submitted (bool) – Show unsubmitted slices

• pcaps_dir (String) – The working directory for .pcap file transfers

• name_prefix (String) – The name prefix to use for autogenerated Crinkle objects. Must match what was used in slice creation

Raises:

Exception: if slice name or slice id are not inputted

Returns:

the slice, if found

Return type:

Slice

get_crinkle_slices(excludes=[Dead, Closing], slice_name=None, slice_id=None, user_only=True, show_un_submitted=False, pcaps_dir='.query_analysis_pcaps', name_prefix='C')

Gets a list of Crinkle slices from the slice manager.

By default this method ignores Dead and Closing slices. Optional, parameter allows excluding a different list of slice states. Pass an empty list (i.e. excludes=[]) to get a list of all slices.

Parameters:

• excludes (List[SliceState]) – A list of slice states to exclude from the output list

• slice_name (str)

• slice_id (str)

• user_only (bool) – True indicates return own slices; False indicates return project slices

• show_un_submitted (bool) – Show unsubmitted slices;

• pcaps_dir (String) – The working directory for .pcap file transfers

• name_prefix (String) – The name prefix to use for autogenerated Crinkle objects. Must match what was used in slice creation

Returns:

a list of slices

Return type:

List[Slice]

get_default_slice_key()

Gets the current default_slice_keys as a dictionary containing the public and private slice keys.

Important! Slice key management is under development and this functionality will likely change going forward.

Returns:

default_slice_key dictionary

Return type:

dict[str, str]

get_facility_ports(update=False, start=None, end=None)

Get the resources object (which includes facility ports).

Parameters:

• update (Optional[bool]) – whether to refresh resources

• start (Optional[<module ‘datetime’ from ‘/home/docs/.asdf/installs/python/3.11.14/lib/python3.11/datetime.py’>]) – start time in UTC format: %Y-%m-%d %H:%M:%S %z

• end (Optional[<module ‘datetime’ from ‘/home/docs/.asdf/installs/python/3.11.14/lib/python3.11/datetime.py’>]) – end time in UTC format: %Y-%m-%d %H:%M:%S %z

Return type:

ResourcesV2

Returns:

ResourcesV2Wrapper

get_image_names()

Gets a list of available image names.

Returns:

Dictionary of images with default user and description

Return type:

dict[str, dict]

get_links(update=True)

Get the resources object (which includes links).

Parameters:

update (bool) – whether to refresh resources

Return type:

ResourcesV2

Returns:

ResourcesV2Wrapper

get_manager()

Gets the manager of this fablib object.

Lazily initializes the FabricManagerV2 on first access, avoiding network calls during FablibManager construction.

Returns:

the manager on this fablib object

Return type:

FabricManagerV2

get_project_tags()

Return the set of permission tags for the current project from the token.

The result is cached for the lifetime of this manager instance. Returns an empty frozenset in offline mode or if the token cannot be decoded.

Returns:

frozenset of tag strings (e.g. "Component.GPU")

Return type:

frozenset

get_random_site(avoid=None, filter_function=None, update=True)

Get a random site.

Parameters:

• avoid (List[String]) – list of site names to avoid choosing

• filter_function – filter_function

• update (bool) – flag indicating if fetch latest availability information

Returns:

one site name

Return type:

String

get_random_sites(count=1, avoid=None, filter_function=None, update=True)

Get a list of random sites names. Each site will be included at most once.

Parameters:

• count (int) – number of sites to return.

• avoid (List[String]) – list of site names to avoid choosing

• filter_function – filter_function

• update (bool) – flag indicating if fetch latest availability information

Returns:

list of random site names.

Return type:

List[String]

get_resources(update=False, force_refresh=False, start=None, end=None, avoid=None, includes=None)

Get a reference to the resources object. The resources object is used to query for available resources and capacities.

Parameters:

• update (bool)

• force_refresh (bool)

• start (Optional[<module ‘datetime’ from ‘/home/docs/.asdf/installs/python/3.11.14/lib/python3.11/datetime.py’>]) – start time in UTC format: %Y-%m-%d %H:%M:%S %z

• end (Optional[<module ‘datetime’ from ‘/home/docs/.asdf/installs/python/3.11.14/lib/python3.11/datetime.py’>]) – end time in UTC format: %Y-%m-%d %H:%M:%S %z

• avoid (Optional[List[str]]) – list of sites to avoid

• includes (Optional[List[str]]) – list of sites to include

Type:

datetime

Type:

datetime

Type:

list of string

Type:

list of string

Returns:

the resources object

Return type:

ResourcesV2

get_site_names()

Gets a list of all available site names.

Returns:

list of site names as strings

Return type:

list[str]

get_slice(name=None, slice_id=None, user_only=True, show_un_submitted=False)

Gets a slice by name or slice_id. Dead and Closing slices may have non-unique names and must be queried by slice_id. Slices in all other states are guaranteed to have unique names and can be queried by name.

If both a name and slice_id are provided, the slice matching the slice_id will be returned.

Parameters:

• name (String) – The name of the desired slice

• slice_id (String) – The ID of the desired slice

• user_only (bool) – True indicates return own slices; False indicates return project slices

• show_un_submitted (bool) – Show unsubmitted slices

Raises:

Exception: if slice name or slice id are not inputted

Returns:

the slice, if found

Return type:

Slice

get_slices(excludes=None, slice_name=None, slice_id=None, user_only=True, show_un_submitted=False)

Gets a list of slices from the slice manager.

By default this method ignores Dead and Closing slices. Optional, parameter allows excluding a different list of slice states. Pass an empty list (i.e. excludes=[]) to get a list of all slices.

Parameters:

• excludes (List[SliceState]) – A list of slice states to exclude from the output list. Defaults to [SliceState.Dead, SliceState.Closing].

• slice_name (str) – Filter by slice name

• slice_id (str) – Filter by slice ID

• user_only (bool) – True indicates return own slices; False indicates return project slices

• show_un_submitted (bool) – Show unsubmitted slices

Returns:

a list of slices

Return type:

List[Slice]

get_ssh_thread_pool_executor()

Get ThreadPoolExecutor that runs SSH commands.

Return type:

ThreadPoolExecutor

get_storage(uuid)

Get a specific storage volume by UUID.

Parameters:

uuid (str) – Storage volume UUID.

Returns:

Storage volume details.

Return type:

list

Raises:

FabricManagerV2Exception – If there is an error retrieving the storage volume.

get_tags()

Retrieve all tags associated with artifacts.

This method returns a list of all tags that are associated with artifacts in the system. Tags are useful for categorizing and searching for artifacts.

Returns:

A list of tags.

Raises:

FabricManagerV2Exception – If an error occurs while retrieving the tags.

get_user_info()

Get User information

Returns:

returns a dictionary containing User’s Information

Return type:

dict

list_artifacts(output=None, fields=None, quiet=False, filter_function=None, pretty_names=True)

List artifacts based on a search query.

Parameters:

• search – Search query for filtering artifacts

• output – Output format - ‘text’, ‘pandas’, ‘json’

• fields – List of fields (table columns) to show

• quiet – True to suppress printing/display

• filter_function – Lambda function to filter data by field values

• pretty_names – Whether to use pretty names for fields

Return type:

object

Returns:

Table in format specified by output parameter

Raises:

FabricManagerV2Exception – If there is an error in listing the artifacts.

list_facility_ports(output=None, fields=None, quiet=False, filter_function=None, update=False, pretty_names=True, start=None, end=None)

Lists all the facility ports and their attributes.

There are several output options: “text”, “pandas”, and “json” that determine the format of the output that is returned and (optionally) displayed/printed.

output: ‘text’: string formatted with tabular

‘pandas’: pandas dataframe ‘json’: string in json format

fields: json output will include all available fields/columns.

Example: TODO

filter_function: A lambda function to filter data by field values.

Example: filter_function=lambda s: s[‘ConnectX-5 Available’] > 3 and s[‘NVMe Available’] <= 10

Parameters:

• output (str) – output format

• fields (List[str]) – list of fields (table columns) to show

• quiet (bool) – True to specify printing/display

• filter_function (lambda) – lambda function

• update (bool)

• pretty_names (bool)

• start (Optional[<module ‘datetime’ from ‘/home/docs/.asdf/installs/python/3.11.14/lib/python3.11/datetime.py’>]) – start time in UTC format: %Y-%m-%d %H:%M:%S %z

• end (Optional[<module ‘datetime’ from ‘/home/docs/.asdf/installs/python/3.11.14/lib/python3.11/datetime.py’>]) – end time in UTC format: %Y-%m-%d %H:%M:%S %z

Type:

datetime

Type:

datetime

Returns:

table in format specified by output parameter

Return type:

Object

list_hosts(output=None, fields=None, quiet=False, filter_function=None, update=False, pretty_names=True, force_refresh=False, start=None, end=None, avoid=None, includes=None)

Lists all the hosts and their attributes.

There are several output options: “text”, “pandas”, and “json” that determine the format of the output that is returned and (optionally) displayed/printed.

output: ‘text’: string formatted with tabular

‘pandas’: pandas dataframe ‘json’: string in json format

fields: json output will include all available fields/columns.

Example: fields=[‘Name’,’ConnectX-5 Available’, ‘NVMe Total’]

filter_function: A lambda function to filter data by field values.

Example: filter_function=lambda s: s[‘ConnectX-5 Available’] > 3 and s[‘NVMe Available’] <= 10

Parameters:

• output (str) – output format

• fields (List[str]) – list of fields (table columns) to show

• quiet (bool) – True to specify printing/display

• filter_function (lambda) – lambda function

• update (bool)

• pretty_names (bool)

• force_refresh (bool)

• start (Optional[<module ‘datetime’ from ‘/home/docs/.asdf/installs/python/3.11.14/lib/python3.11/datetime.py’>]) – start time in UTC format: %Y-%m-%d %H:%M:%S %z

• end (Optional[<module ‘datetime’ from ‘/home/docs/.asdf/installs/python/3.11.14/lib/python3.11/datetime.py’>]) – end time in UTC format: %Y-%m-%d %H:%M:%S %z

• avoid (Optional[List[str]]) – list of sites to avoid

• includes (Optional[List[str]]) – list of sites to include

Return type:

object

Returns:

table in format specified by output parameter

Type:

datetime

Type:

datetime

Type:

list of string

Type:

list of string

list_links(output=None, fields=None, quiet=False, filter_function=None, update=True, pretty_names=True)

Lists all the links and their attributes.

There are several output options: “text”, “pandas”, and “json” that determine the format of the output that is returned and (optionally) displayed/printed.

output: ‘text’: string formatted with tabular

‘pandas’: pandas dataframe ‘json’: string in json format

fields: json output will include all available fields/columns.

Example: TODO

filter_function: A lambda function to filter data by field values.

Example: filter_function=lambda s: s[‘ConnectX-5 Available’] > 3 and s[‘NVMe Available’] <= 10

Parameters:

• output (str) – output format

• fields (List[str]) – list of fields (table columns) to show

• quiet (bool) – True to specify printing/display

• filter_function (lambda) – lambda function

• update (bool)

• pretty_names (bool)

Returns:

table in format specified by output parameter

Return type:

Object

list_sites(output=None, fields=None, quiet=False, filter_function=None, update=False, pretty_names=True, force_refresh=False, start=None, end=None, avoid=None, includes=None)

Lists all the sites and their attributes.

There are several output options: “text”, “pandas”, and “json” that determine the format of the output that is returned and (optionally) displayed/printed.

output: ‘text’: string formatted with tabular

‘pandas’: pandas dataframe ‘json’: string in json format

fields: json output will include all available fields/columns.

Example: fields=[‘Name’,’ConnectX-5 Available’, ‘NVMe Total’]

filter_function: A lambda function to filter data by field values.

Example: filter_function=lambda s: s[‘ConnectX-5 Available’] > 3 and s[‘NVMe Available’] <= 10

Parameters:

• output (str) – output format

• fields (List[str]) – list of fields (table columns) to show

• quiet (bool) – True to specify printing/display

• filter_function (lambda) – lambda function

• update (bool)

• pretty_names (bool)

• force_refresh (bool)

• latlon – convert address to latlon, makes online call to openstreetmaps.org

• start (Optional[<module ‘datetime’ from ‘/home/docs/.asdf/installs/python/3.11.14/lib/python3.11/datetime.py’>]) – start time in UTC format: %Y-%m-%d %H:%M:%S %z

• end (Optional[<module ‘datetime’ from ‘/home/docs/.asdf/installs/python/3.11.14/lib/python3.11/datetime.py’>]) – end time in UTC format: %Y-%m-%d %H:%M:%S %z

• avoid (Optional[List[str]]) – list of sites to avoid

• includes (Optional[List[str]]) – list of sites to include

Return type:

object

Returns:

table in format specified by output parameter

Type:

Object

Type:

datetime

Type:

datetime

Type:

list of string

Type:

list of string

list_slices(excludes=None, output=None, fields=None, quiet=False, filter_function=None, pretty_names=True, user_only=True, show_un_submitted=False)

Lists all the slices created by a user.

There are several output options: “text”, “pandas”, and “json” that determine the format of the output that is returned and (optionally) displayed/printed.

output: ‘text’: string formatted with tabular

‘pandas’: pandas dataframe ‘json’: string in json format

fields: json output will include all available fields/columns.

Example: fields=[‘Name’,’State’]

filter_function: A lambda function to filter data by field values.

Example: filter_function=lambda s: s[‘State’] == ‘Configuring’

Parameters:

• excludes (list[SliceState]) – slice states to exclude. Defaults to [SliceState.Dead, SliceState.Closing].

• output (str) – output format

• fields (List[str]) – list of fields (table columns) to show

• quiet (bool) – True to specify printing/display

• filter_function (lambda) – lambda function

• pretty_names (bool) – pretty_names

• user_only (bool) – True indicates return own slices; False indicates return project slices

• show_un_submitted (bool) – True indicates to also show unsubmitted slices

Returns:

table in format specified by output parameter

Return type:

Object

list_storage(offset=0, limit=200, fetch_all=False, output=None, fields=None, quiet=False, filter_function=None)

List storage volumes available for the project.

Parameters:

• offset (int) – Pagination offset (default: 0).

• limit (int) – Maximum number of records to fetch per page (default: 200).

• fetch_all (bool) – If True, automatically fetch all storage volumes across all pages (default: False).

• output – Output format - ‘text’, ‘pandas’, ‘json’

• fields – List of fields (table columns) to show

• quiet – True to suppress printing/display

• filter_function – Lambda function to filter data by field values

Return type:

object

Returns:

Table in format specified by output parameter

Raises:

FabricManagerV2Exception – If there is an error in listing storage volumes.

new_crinkle_slice(name, pcaps_dir='.query_analysis_pcaps', name_prefix='C')

Creates a new Crinkle slice with the given name and with an Analyzer node.

Parameters:

• name (String) – the name to give the slice

• analyzer_name (String) – Name for the

• cores (int)

• ram (int)

• disk (int)

• site (String)

• pcaps_dir (String)

Returns:

A new Crinkle slice

Return type:

CrinkleSlice

new_slice(name, storage=False, storage_cluster=None)

Creates a new slice with the given name.

Parameters:

• name (String) – the name to give the slice

• storage (bool) – Enable CephFS storage on every node added to this slice. Default: False

• storage_cluster (str) – Ceph cluster name. When None, the first available cluster is auto-discovered at post-boot time.

Returns:

a new slice

Return type:

Slice

probe_bastion_host()

See if bastion will admit us with our configuration.

Bastion hosts are configured to block hosts that attempts to use it with too many repeated authentication failures. We want to avoid that.

Returns True if connection attempt succeeds. Raises an error in the event of failure.

Return type:

Optional[bool]

remove_slice_from_cache(slice_object)

Removes a Slice object from the cache by its name and ID.

Removes the slice from both __slices_by_name and __slices_by_id if present.

Parameters:

slice_object (Slice) – The Slice object to be removed from the cache.

resources_calendar(start, end, interval='day', site=None, host=None, exclude_site=None, exclude_host=None, show='all', output=None, fields=None, quiet=False, filter_function=None)

Fetch and display resource availability calendar over time slots.

There are several output options: "text", "pandas", and "json" that determine the format of the output that is returned and (optionally) displayed/printed.

output: 'text': string formatted with tabulate

'pandas': pandas dataframe (default in Jupyter) 'json': string in json format 'list': raw list of dicts

fields: list of columns to include in the output table.

Example: fields=['Date', 'Name', 'Cores (avail/cap)', 'RAM GB (avail/cap)']

filter_function: A lambda to filter the flattened rows.

Example: filter_function=lambda r: r['Name'] == 'RENC'

Parameters:

• start (datetime.datetime) – start of the calendar window (UTC)

• end (datetime.datetime) – end of the calendar window (UTC)

• interval (str) – time slot granularity: hour, day, or week

• site (list[str]) – list of site names to include

• host (list[str]) – list of host names to include

• exclude_site (list[str]) – list of site names to exclude

• exclude_host (list[str]) – list of host names to exclude

• show (str) – which resource types to display: "sites", "hosts", or "all" (default)

• output (str) – output format

• fields (list[str]) – list of fields (table columns) to show

• quiet (bool) – True to suppress printing/display

• filter_function (callable) – lambda function to filter rows

Returns:

table in format specified by output parameter

show_config(output=None, fields=None, quiet=False, pretty_names=True)

Show a table containing the current FABlib configuration parameters.

There are several output options: “text”, “pandas”, and “json” that determine the format of the output that is returned and (optionally) displayed/printed.

output: ‘text’: string formatted with tabular

‘pandas’: pandas dataframe ‘json’: string in json format

fields: json output will include all available fields.

Example: fields=[‘credmgr_host’,’project_id’, ‘fablib_log_file’]

Parameters:

• output (str) – output format

• fields (List[str]) – list of fields to show

• quiet (bool) – True to specify printing/display

• pretty_names (bool)

Returns:

table in format specified by output parameter

Return type:

Object

show_site(site_name, output=None, fields=None, quiet=False, pretty_names=True)

Show a table with all the properties of a specific site

There are several output options: “text”, “pandas”, and “json” that determine the format of the output that is returned and (optionally) displayed/printed.

output: ‘text’: string formatted with tabular

‘pandas’: pandas dataframe ‘json’: string in json format

fields: json output will include all available fields.

Example: fields=[‘credmgr_host’,’project_id’, ‘fablib_log_file’]

Parameters:

• site_name (str) – the name of a site

• output (str) – output format

• fields (List[str]) – list of fields to show

• quiet (bool) – True to specify printing/display

• pretty_names (bool)

• latlon (bool) – convert address to lat/lon

Returns:

table in format specified by output parameter

Return type:

Object

show_slice(name=None, id=None, output=None, fields=None, quiet=False, pretty_names=True, user_only=True, show_un_submitted=False)

Show a table with all the properties of a specific site

There are several output options: “text”, “pandas”, and “json” that determine the format of the output that is returned and (optionally) displayed/printed.

output: ‘text’: string formatted with tabular

‘pandas’: pandas dataframe ‘json’: string in json format

fields: json output will include all available fields.

Example: fields=[‘Name’,’State’]

Parameters:

• name (str) – the name of a slice

• id (Optional[str]) – the slice id

• output (str) – output format

• fields (List[str]) – list of fields to show

• quiet (bool) – True to specify printing/display

• pretty_names (bool) – pretty_names

• user_only (bool) – True indicates return own slices; False indicates return project slices

• show_un_submitted (bool) – True indicates to also show unsubmitted slices

Returns:

table in format specified by output parameter

Return type:

Object

update_slice_cache_id(slice_object)

Updates the cache when a slice gets a new ID (e.g., after submit).

This should be called after a slice is submitted and receives its slice_id.

Parameters:

slice_object (Slice) – The Slice object whose ID was updated.

upload_file_to_artifact(file_to_upload, artifact_id=None, artifact_title=None)

Upload a file to an existing artifact.

This method uploads a file to an artifact identified by either its artifact_id or artifact_title. If artifact_id is not provided, the method will search for the artifact using artifact_title before uploading the file.

Parameters:

• file_to_upload (str) – The path to the file that should be uploaded.

• artifact_id (str) – The unique identifier of the artifact to which the file will be uploaded.

• artifact_title (str) – The title of the artifact to which the file will be uploaded.

Return type:

dict

Returns:

A dictionary containing the details of the uploaded file.

Raises:

• ValueError – If neither artifact_id nor artifact_title is provided.

• FabricManagerV2Exception – If an error occurs during the upload process.

validate_and_update_bastion_keys(validate_only=False)

Validate Bastion Key; if key does not exist or is expired, it creates bastion keys

Parameters:

validate_only (bool) – flag to specify to only do config validation

validate_config()

Validate and create Fablib config - checks if all the required configuration exists for slice provisioning to work successfully

• Checks Credential Manager Host is configured properly

• Checks Orchestrator Host is configured properly

• Checks Core API Host is configured properly

• Checks Bastion Host is configured properly

• Check Sliver keys exist; create sliver keys if they do not exist

• Check Bastion keys exist and are not expired; update/create bastion keys if expired or do not exist

• Check Bastion Username is configured

• Check Project Id is configured

Deprecated since version 1.6.5: Use verify_and_configure() instead.

@raises Exception if the configuration is invalid

validate_node(node, allocated=None)

Validate a node w.r.t available resources on a site before submission.

Delegates to NodeValidatorV2 with pre-fetched resources. This method must remain on FablibManager because Node.add_component() calls self.get_fablib_manager().validate_node().

Returns:

Tuple indicating status for validation and error message in case of failure

Return type:

Tuple[bool, str]

verify_and_configure(validate_only=False)

Validate and create Fablib config - checks if all the required configuration exists for slice provisioning to work successfully

• Checks Credential Manager Host is configured properly

• Checks Orchestrator Host is configured properly

• Checks Core API Host is configured properly

• Checks Bastion Host is configured properly

• Check Sliver keys exist; create sliver keys if they do not exist

• Check Bastion keys exist and are not expired; update/create bastion keys if expired or do not exist

• Check Bastion Username is configured

• Check Project Id is configured

Parameters:

validate_only (bool) – flag to specify to only do config validation

:raises Exception if the configuration is invalid