fablib¶

This module exports a FablibManager class and a fablib class available, which allows you to, among other things:

Query FABRIC testbed resources.

Create, modify, and delete slices.

Manage the SSH keys you use with FABRIC.

etc.

In most cases you would need to create a FablibManager instance to interact with FABRIC testbed:

from fabrictestbed_extensions.fablib.fablib import FablibManager

fablib = FablibManager()

slice = fablib.new_slice(name="MySlice")
node = slice.add_node(name="node1")
slice.submit();

See FABRIC project’s Jupyter notebook examples for more complete code samples.

Note

Some configuration in the form of a configuration file, environment variables, or FablibManager constructor parameters is required for the library to work. Please see the FABRIC project’s documentation on getting started.

class fabrictestbed_extensions.fablib.fablib.fablib¶

Convenience static methods to work with FABRIC testbed.

static delete_all(progress: bool = True)¶

Deletes all slices on the slice manager.

Parameters:: progress (Bool) – optional progress printing to stdout

static delete_slice(slice_name: str | None = None)¶

Deletes a slice by name.

Parameters:: slice_name (String) – the name of the slice to delete

static get_available_resources(update: bool = False) → Resources¶

Get the available resources.

Optionally update the available resources by querying the FABRIC services. Otherwise, this method returns the existing information.

Parameters:: update (Bool) – update
Returns:: Available Resources object
Return type:: Resources

static get_bastion_host() → str¶

Gets the FABRIC Bastion host address.

Returns:: Bastion host public address
Return type:: String

static get_bastion_key_filename() → str¶

Gets the FABRIC Bastion key filename.

Returns:: FABRIC Bastion key filename
Return type:: String

static get_bastion_username() → str¶

Gets the FABRIC Bastion username.

Returns:: FABRIC Bastion username
Return type:: String

static get_config() → Dict[str, str]¶

Gets a dictionary mapping keywords to configured FABRIC environment variable values values.

Returns:: dictionary mapping keywords to FABRIC values
Return type:: Dict[String, String]

static get_credmgr_host() → str¶

Gets the credential manager host site value.

Returns:: the credential manager host site
Return type:: String

static get_default_slice_key() → Dict[str, str]¶

Gets the current default_slice_keys as a dictionary containing the public and private slice keys.

Important! Slice key management is underdevelopment and this functionality will likely change going forward.

Returns:: default_slice_key dictionary from superclass
Return type:: Dict[String, String]

static get_default_slice_private_key_file() → str¶

Gets the path to the default slice private key file.

Important! Slices key management is underdevelopment and this functionality will likely change going forward.

Returns:: the path to the slice private key on this fablib object
Return type:: String

static get_default_slice_private_key_passphrase() → str¶

Gets the passphrase to the default slice private key.

Important! Slices key management is underdevelopment and this functionality will likely change going forward.

Returns:: the passphrase to the slice private key on this fablib object
Return type:: String

static get_default_slice_public_key() → str¶

Gets the default slice public key.

Important! Slice key management is underdevelopment and this functionality will likely change going forward.

Returns:: the slice public key on this fablib object
Return type:: String

static get_default_slice_public_key_file() → str¶

Gets the path to the default slice public key file.

Important! Slice key management is underdevelopment and this functionality will likely change going forward.

Returns:: the path to the slice public key on this fablib object
Return type:: String

static get_fabric_token() → str¶

Gets the FABRIC token location.

Returns:: FABRIC token location
Return type:: String

static get_facility_ports() → FacilityPorts¶

Get a string used to print a tabular list of facility ports

Returns:: tabulated string of facility ports
Return type:: str

static get_fim_slice(excludes: List[SliceState] = [Dead, Closing], user_only: bool = True) → List[OrchestratorSlice]¶

Not intended for API use.

Gets a list of fim slices from the slice manager.

By default this method ignores Dead and Closing slices. Optional, parameter allows excluding a different list of slice states. Pass an empty list (i.e. excludes=[]) to get a list of all slices.

Parameters:

excludes (List[SliceState]) – A list of slice states to exclude from the output list
user_only (bool) – True indicates return own slices; False indicates return project slices

Returns:

a list of slices

Return type:

List[Slice]

static get_image_names() → List[str]¶

Gets a list of available image names.

Returns:: list of image names as strings
Return type:: list[str]

static get_links() → Links¶

Get a string used to print a tabular list of links

Returns:: tabulated string of links
Return type:: str

static get_log_level()¶: Gets the current log level for logging

static get_orchestrator_host() → str¶

Gets the orchestrator host site value.

Returns:: the orchestrator host site
Return type:: String

static get_random_site(avoid: List[str] = []) → str¶

Get a random site.

Parameters:: avoid – list of site names to avoid choosing
Returns:: one site name
Return type:: String

static get_random_sites(count: int = 1, avoid: List[str] = []) → List[str]¶

Get a list of random sites names. Each site will be included at most once.

Parameters:

count (int) – number of sites to return.
avoid (List[String]) – list of site names to avoid choosing

Returns:

list of random site names.

Return type:

List[Sting]

static get_resources() → Resources¶

Get a reference to the resources object. The resources object is used to query for available resources and capacities.

Returns:: the resources object
Return type:: Resources

static get_site_advertisement(site: str) → Node¶

Not intended for API use.

Given a site name, gets fim topology object for this site.

Parameters:: site (String) – a site name
Returns:: fim object for this site
Return type:: Node

static get_site_names() → List[str]¶

Gets a list of all available site names.

Returns:: list of site names as strings
Return type:: list[str]

static get_slice(name: str | None = None, slice_id: str | None = None, user_only: bool = True) → Slice¶

Gets a slice by name or slice_id. Dead and Closing slices may have non-unique names and must be queried by slice_id. Slices in all other states are guaranteed to have unique names and can be queried by name.

If both a name and slice_id are provided, the slice matching the slice_id will be returned.

Parameters:

name (String) – The name of the desired slice
slice_id (String) – The ID of the desired slice
user_only (bool) – True indicates return own slices; False indicates return project slices

Raises:

Exception: if slice name or slice id are not inputted

Returns:

the slice, if found

Return type:

Slice

static get_slice_manager() → SliceManager¶

Not intended as API call

Gets the slice manager of this fablib object.

Returns:: the slice manager on this fablib object
Return type:: SliceManager

static get_slices(excludes: List[SliceState] = [Dead, Closing], user_only: bool = True) → List[Slice]¶

Gets a list of slices from the slice manager.

By default this method ignores Dead and Closing slices. Optional, parameter allows excluding a different list of slice states. Pass an empty list (i.e. excludes=[]) to get a list of all slices.

Parameters:

excludes (List[SliceState]) – A list of slice states to exclude from the output list
user_only (bool) – True indicates return own slices; False indicates return project slices

Returns:

a list of slices

Return type:

List[Slice]

static init_fablib()¶

Not intended to be called by the user.

Static initializer for the fablib object.

static list_facility_ports() → object¶

Print the facility ports in pretty format

Returns:: Formatted list of facility ports
Return type:: object

static list_links() → object¶

Print the links in pretty format

Returns:: Formatted list of links
Return type:: object

static list_sites(latlon: bool = True) → object¶

Get a string used to print a tabular list of sites with state

Returns:: tabulated string of site state
Return type:: str

static new_slice(name: str) → Slice¶

Creates a new slice with the given name.

Parameters:: name (String) – the name to give the slice
Returns:: a new slice
Return type:: Slice

static set_log_level(log_level: str)¶

Sets the current log level for logging

Options: logging.DEBUG: logging.INFO logging.WARNING logging.ERROR logging.CRITICAL

Parameters:: log_level (Level) – new log level

static show_site(site_name: str)¶

Get a string used to print tabular info about a site

Parameters:: site_name (String) – the name of a site
Returns:: tabulated string of site state
Return type:: String

class fabrictestbed_extensions.fablib.fablib.FablibManager(fabric_rc: str | None = None, credmgr_host: str | None = None, orchestrator_host: str | None = None, core_api_host: str | None = None, token_location: str | None = None, project_id: str | None = None, bastion_username: str | None = None, bastion_key_location: str | None = None, log_level: str = 'INFO', log_file: str = '/tmp/fablib/fablib.log', data_dir: str = '/tmp/fablib', output: str | None = None, execute_thread_pool_size: int = 64, offline: bool = False, auto_token_refresh: bool = True, **kwargs)¶

FablibManager is the main interface to FABRIC services.

A FablibManager object is used to query FABRIC testbed for available resources, create and configure slices, manage SSH keys in nodes in slices and FABRIC’s bastion host, etc. This requires some configuration, which is gathered from:

constructor parameters (high priority)

a configuration file (medium priority)

environment variables (low priority)

defaults (if needed, and when possible)

Typically you would use the configuration file located at "${HOME}/work/fabric_config/fabric_rc", and/or environment variables.

Parameters:

fabric_rc – Path to fablib configuration file. Defaults to "${HOME}/work/fabric_config/fabric_rc".
credmgr_host – Name of credential manager host.
orchestrator_host – Name of FABRIC orchestrator host.
fabric_token – Path to the file that contains your FABRIC auth token.
project_id – Your FABRIC project ID, obtained from https://cm.fabric-testbed.net/, usually via FABRIC portal.
bastion_username – Your username on FABRIC bastion host, obtained from FABRIC portal.
bastion_key_filename – Path to your bastion SSH key.
log_file – Path where fablib logs are written; defaults to "/tmp/fablib/fablib.log".
log_level – Level of detail in the logs written. Defaults to "DEBUG"; other possible log levels are "INFO", "WARNING", "ERROR", and "CRITICAL", in reducing order of verbosity.
data_dir – directory for fablib to store temporary data.
output – Format of fablib output; can be either "pandas" or "text". Defaults to "pandas" in a Jupyter notebook environment; "text" otherwise.
execute_thread_pool_size – Number of worker threads in the thread pool fablib uses to execute commands in nodes. Defaults to 64.
offline – Avoid using FABRIC services when initializing. This is False by default, and set to True only in some unit tests.
auto_token_refresh – Auto refresh tokens

create_bastion_keys(*, bastion_key_location: str | None = None, store_pubkey: bool = True, overwrite: bool = False)¶

Create Bastion Keys

Parameters:

bastion_key_location (str) – bastion key location
store_pubkey (bool) – flag indicating if the public key should be saved
overwrite (bool) – overwrite the bastion key file if it exists already

create_sliver_keys(*, sliver_priv_key_location: str | None = None, store_pubkey: bool = True, overwrite: bool = False)¶

Create Sliver Keys

Parameters:

sliver_priv_key_location (str) – sliver key location
store_pubkey (bool) – flag indicating if the public key should be saved
overwrite (bool) – overwrite the bastion key file if it exists already

create_ssh_config(overwrite: bool = False)¶

Create SSH config file

Parameters:: overwrite (bool) – overwrite the configuration if True, return otherwise

delete_all(progress: bool = True)¶

Deletes all slices on the slice manager.

Parameters:: progress (Bool) – optional progress printing to stdout

delete_slice(slice_name: str | None = None)¶

Deletes a slice by name.

Parameters:: slice_name (String) – the name of the slice to delete

determine_bastion_username()¶: Determine Bastion Username. Query User Information from Core API and updates the bastion username

get_available_resources(update: bool = False, force_refresh: bool = False) → Resources¶

Get the available resources.

Optionally update the available resources by querying the FABRIC services. Otherwise, this method returns the existing information.

Parameters:

update
force_refresh

Returns:

Available Resources object

get_facility_ports(update: bool = True) → FacilityPorts¶

Get the facility ports.

Optionally update the available resources by querying the FABRIC services. Otherwise, this method returns the existing information.

Parameters:: update
Returns:: Links

get_fim_slices(excludes: List[SliceState] = [Dead, Closing], user_only: bool = True) → List[OrchestratorSlice]¶

Gets a list of fim slices from the slice manager.

This is not recommended for most users and should only be used to bypass fablib inorder to create custom low-level functionality.

By default this method ignores Dead and Closing slices. Optional, parameter allows excluding a different list of slice states. Pass an empty list (i.e. excludes=[]) to get a list of all slices.

Parameters:

excludes (List[SliceState]) – A list of slice states to exclude from the output list
user_only (bool) – True indicates return own slices; False indicates return project slices

Returns:

a list of fim models of slices

Return type:

List[Slice]

get_links(update: bool = True) → Links¶

Get the links.

Optionally update the available resources by querying the FABRIC services. Otherwise, this method returns the existing information.

Parameters:: update
Returns:: Links

get_random_site(avoid: List[str] = [], filter_function=None, update: bool = True) → str¶

Get a random site.

Parameters:

avoid (List[String]) – list of site names to avoid choosing
filter_function – filter_function
update (bool) – flag indicating if fetch latest availability information

Returns:

one site name

Return type:

String

get_random_sites(count: int = 1, avoid: List[str] = [], filter_function=None, update: bool = True, unique: bool = True) → List[str]¶

Get a list of random sites names. Each site will be included at most once.

Parameters:

count (int) – number of sites to return.
avoid (List[String]) – list of site names to avoid chosing
filter_function – filter_function
update (bool) – flag indicating if fetch latest availability information
unique

Returns:

one site name

Returns:

list of random site names.

Return type:

List[Sting]

get_resources(update: bool = True, force_refresh: bool = False) → Resources¶

Get a reference to the resources object. The resources object is used to query for available resources and capacities.

Returns:: the resources object
Return type:: Resources

get_site_advertisement(site: str) → Node¶

Not intended for API use.

Given a site name, gets fim topology object for this site.

Parameters:: site (String) – a site name
Returns:: fim object for this site
Return type:: Node

get_site_names() → List[str]¶

Gets a list of all available site names.

Returns:: list of site names as strings
Return type:: list[str]

get_slice(name: str | None = None, slice_id: str | None = None, user_only: bool = True) → Slice¶

If both a name and slice_id are provided, the slice matching the slice_id will be returned.

Parameters:

name (String) – The name of the desired slice
slice_id (String) – The ID of the desired slice
user_only (bool) – True indicates return own slices; False indicates return project slices

Raises:

Exception: if slice name or slice id are not inputted

Returns:

the slice, if found

Return type:

Slice

get_slice_manager() → SliceManager¶

Not intended as API call

Gets the slice manager of this fablib object.

Returns:: the slice manager on this fablib object
Return type:: SliceManager

get_slices(excludes: List[SliceState] = [Dead, Closing], slice_name: str | None = None, slice_id: str | None = None, user_only: bool = True) → List[Slice]¶

Gets a list of slices from the slice manager.

By default this method ignores Dead and Closing slices. Optional, parameter allows excluding a different list of slice states. Pass an empty list (i.e. excludes=[]) to get a list of all slices.

Parameters:

excludes (List[SliceState]) – A list of slice states to exclude from the output list
slice_name
slice_id
user_only (bool) – True indicates return own slices; False indicates return project slices

Returns:

a list of slices

Return type:

List[Slice]

get_user_info() → dict¶

Get User information

:return returns a dictionary containing User’s Information :rtype: dict

static is_jupyter_notebook() → bool¶

Test for running inside a jupyter notebook

Returns:: bool, True if in jupyter notebook
Return type:: bool

list_facility_ports(output: str | None = None, fields: str | None = None, quiet: bool = False, filter_function=None, update: bool = True, pretty_names=True) → object¶

Lists all the facility ports and their attributes.

There are several output options: “text”, “pandas”, and “json” that determine the format of the output that is returned and (optionally) displayed/printed.

output: ‘text’: string formatted with tabular: ‘pandas’: pandas dataframe ‘json’: string in json format

fields: json output will include all available fields/columns.

Example: TODO

filter_function: A lambda function to filter data by field values.

Example: filter_function=lambda s: s[‘ConnectX-5 Available’] > 3 and s[‘NVMe Available’] <= 10

Parameters:

output (str) – output format
fields (List[str]) – list of fields (table columns) to show
quiet (bool) – True to specify printing/display
filter_function (lambda) – lambda function
update (bool)
pretty_names (bool)

Returns:

table in format specified by output parameter

Return type:

Object

list_links(output: str | None = None, fields: str | None = None, quiet: bool = False, filter_function=None, update: bool = True, pretty_names=True) → object¶

Lists all the links and their attributes.

There are several output options: “text”, “pandas”, and “json” that determine the format of the output that is returned and (optionally) displayed/printed.

output: ‘text’: string formatted with tabular: ‘pandas’: pandas dataframe ‘json’: string in json format

fields: json output will include all available fields/columns.

Example: TODO

filter_function: A lambda function to filter data by field values.

Example: filter_function=lambda s: s[‘ConnectX-5 Available’] > 3 and s[‘NVMe Available’] <= 10

Parameters:

output (str) – output format
fields (List[str]) – list of fields (table columns) to show
quiet (bool) – True to specify printing/display
filter_function (lambda) – lambda function
update (bool)
pretty_names (bool)

Returns:

table in format specified by output parameter

Return type:

Object

list_sites(output: str | None = None, fields: str | None = None, quiet: bool = False, filter_function=None, update: bool = True, pretty_names: bool = True, force_refresh: bool = False, latlon: bool = True) → object¶

Lists all the sites and their attributes.

There are several output options: “text”, “pandas”, and “json” that determine the format of the output that is returned and (optionally) displayed/printed.

output: ‘text’: string formatted with tabular: ‘pandas’: pandas dataframe ‘json’: string in json format

fields: json output will include all available fields/columns.

Example: fields=[‘Name’,’ConnectX-5 Available’, ‘NVMe Total’]

filter_function: A lambda function to filter data by field values.

Example: filter_function=lambda s: s[‘ConnectX-5 Available’] > 3 and s[‘NVMe Available’] <= 10

Parameters:

output (str) – output format
fields (List[str]) – list of fields (table columns) to show
quiet (bool) – True to specify printing/display
filter_function (lambda) – lambda function
update (bool)
pretty_names (bool)
force_refresh (bool)
latlon – convert address to latlon, makes online call to openstreetmaps.org

Returns:

table in format specified by output parameter

Return type:

Object

list_slices(excludes=[Dead, Closing], output=None, fields=None, quiet=False, filter_function=None, pretty_names=True, user_only: bool = True)¶

Lists all the slices created by a user.

There are several output options: “text”, “pandas”, and “json” that determine the format of the output that is returned and (optionally) displayed/printed.

output: ‘text’: string formatted with tabular: ‘pandas’: pandas dataframe ‘json’: string in json format

fields: json output will include all available fields/columns.

Example: fields=[‘Name’,’State’]

filter_function: A lambda function to filter data by field values.

Example: filter_function=lambda s: s[‘State’] == ‘Configuring’

Parameters:

excludes (list[slice.state]) – slice status to exclude
output (str) – output format
fields (List[str]) – list of fields (table columns) to show
quiet (bool) – True to specify printing/display
filter_function (lambda) – lambda function
pretty_names (bool) – pretty_names
user_only (bool) – True indicates return own slices; False indicates return project slices

Returns:

table in format specified by output parameter

Return type:

Object

new_slice(name: str) → Slice¶

Creates a new slice with the given name.

Parameters:: name (String) – the name to give the slice
Returns:: a new slice
Return type:: Slice

probe_bastion_host() → bool¶

See if bastion will admit us with our configuration.

Bastion hosts are configured to block hosts that attempts to use it with too many repeated authentication failures. We want to avoid that.

Returns True if connection attempt succeeds. Raises an error in the event of failure.

set_slice_manager(slice_manager: SliceManager)¶

Not intended as API call

Sets the slice manager of this fablib object.

Parameters:: slice_manager (SliceManager) – the slice manager to set

show_config(output: str | None = None, fields: list[str] | None = None, quiet: bool = False, pretty_names=True)¶

Show a table containing the current FABlib configuration parameters.

There are several output options: “text”, “pandas”, and “json” that determine the format of the output that is returned and (optionally) displayed/printed.

output: ‘text’: string formatted with tabular: ‘pandas’: pandas dataframe ‘json’: string in json format

fields: json output will include all available fields.

Example: fields=[‘credmgr_host’,’project_id’, ‘fablib_log_file’]

Parameters:

output (str) – output format
fields (List[str]) – list of fields to show
quiet (bool) – True to specify printing/display
pretty_names (bool)

Returns:

table in format specified by output parameter

Return type:

Object

show_site(site_name: str, output: str | None = None, fields: list[str] | None = None, quiet: bool = False, pretty_names=True, latlon=True)¶

Show a table with all the properties of a specific site

There are several output options: “text”, “pandas”, and “json” that determine the format of the output that is returned and (optionally) displayed/printed.

output: ‘text’: string formatted with tabular: ‘pandas’: pandas dataframe ‘json’: string in json format

fields: json output will include all available fields.

Example: fields=[‘credmgr_host’,’project_id’, ‘fablib_log_file’]

Parameters:

site_name (str) – the name of a site
output (str) – output format
fields (List[str]) – list of fields to show
quiet (bool) – True to specify printing/display
pretty_names (bool)
latlon (bool) – convert address to lat/lon

Returns:

table in format specified by output parameter

Return type:

Object

show_slice(name: str | None = None, id: str | None = None, output=None, fields=None, quiet=False, pretty_names=True, user_only: bool = True)¶

Show a table with all the properties of a specific site

There are several output options: “text”, “pandas”, and “json” that determine the format of the output that is returned and (optionally) displayed/printed.

output: ‘text’: string formatted with tabular: ‘pandas’: pandas dataframe ‘json’: string in json format

fields: json output will include all available fields.

Example: fields=[‘Name’,’State’]

Parameters:

name (str) – the name of a slice
id – the slice id
output (str) – output format
fields (List[str]) – list of fields to show
quiet (bool) – True to specify printing/display
pretty_names (bool) – pretty_names
user_only (bool) – True indicates return own slices; False indicates return project slices

Returns:

table in format specified by output parameter

Return type:

Object

validate_and_update_bastion_keys()¶: Validate Bastion Key; if key does not exist or is expired, it create bastion keys

validate_config()¶

Validate and create Fablib config - checks if all the required configuration exists for slice provisioning to work successfully

Checks Credential Manager Host is configured properly
Checks Orchestrator Host is configured properly
Checks Core API Host is configured properly
Checks Bastion Host is configured properly
Check Sliver keys exist; create sliver keys if they do not exist
Check Bastion keys exist and are not expired; update/create bastion keys if expired or do not exist
Check Bastion Username is configured
Check Project Id is configured

Deprecated since version 1.6.5: Use verify_and_configure() instead.

@raises Exception if the configuration is invalid

verify_and_configure()¶

Validate and create Fablib config - checks if all the required configuration exists for slice provisioning to work successfully

Checks Credential Manager Host is configured properly
Checks Orchestrator Host is configured properly
Checks Core API Host is configured properly
Checks Bastion Host is configured properly
Check Sliver keys exist; create sliver keys if they do not exist
Check Bastion keys exist and are not expired; update/create bastion keys if expired or do not exist
Check Bastion Username is configured
Check Project Id is configured

@raises Exception if the configuration is invalid