sebs.faas package

Submodules

sebs.faas.config module

Configuration management for Function-as-a-Service (FaaS) systems.

This module provides abstract base classes for managing configurations across different FaaS platforms (AWS Lambda, Azure Functions, Google Cloud Functions, OpenWhisk, etc.). It defines the core interfaces for:

• Credentials management and authentication

• Resource allocation and management

• Platform-specific configuration settings

• Configuration serialization and caching

The module follows a hierarchical structure where each platform implements these abstract classes with their specific authentication methods, resource types, and configuration parameters. All configurations support caching to avoid repeated initialization and provide persistence across benchmark runs.

Classes:

Credentials: Abstract base for platform authentication credentials Resources: Abstract base for cloud resource management Config: Abstract base for complete platform configuration

The credentials initialization follows this precedence order: 1. Load credentials with values provided in config 2. Fall back to environment variables 3. Report failure if no credentials are available

class sebs.faas.config.Config(name: str)

Bases: ABC, LoggingBase

Abstract base class for complete FaaS platform configuration.

This class combines credentials and resources into a complete platform configuration, along with platform-specific settings like region selection. It provides the top-level configuration interface used throughout the benchmarking framework.

The Config class coordinates: - Platform credentials for authentication - Resource allocation and management - Regional deployment settings - Configuration persistence and caching - Platform-specific parameter handling

abstract property credentials: Credentials

Get the platform credentials.

Returns:

Platform-specific credentials instance

Return type:

Credentials

abstractmethod static deserialize(config: dict, cache: Cache, handlers: LoggingHandlers) → Config

Create configuration instance from user config and cached values.

This method serves as a factory for platform-specific configurations, dynamically loading the appropriate implementation based on the platform name specified in the configuration. To do that, it calls the appropriate subclass’s deserialize method.

Parameters:

• config – User-provided configuration dictionary

• cache – Cache instance for loading stored configuration

• handlers – Logging handlers for error reporting

Returns:

Platform-specific configuration instance

Return type:

Config

Raises:

AssertionError – If the platform type is unknown or unsupported

abstractmethod static initialize(cfg: Config, dct: dict)

Initialize a Config instance from configuration dictionary.

Parameters:

• cfg – Config instance to initialize

• dct – Configuration dictionary

property region: str

Get the cloud region for deployment.

Returns:

Cloud region identifier

Return type:

str

abstract property resources: Resources

Get the platform resources.

Returns:

Platform-specific resources instance

Return type:

Resources

abstractmethod serialize() → dict

Serialize configuration to dictionary for cache storage.

Subclasses should call super().serialize() and extend the dictionary. This base implementation serializes name and region.

Returns:

Serialized configuration including platform name and region

Return type:

dict

abstractmethod update_cache(cache: Cache)

Update the cache with current configuration settings.

Parameters:

cache – Cache instance to update

class sebs.faas.config.Credentials

Bases: ABC, LoggingBase

Abstract base class for FaaS platform authentication credentials.

This class defines the interface for managing authentication credentials across different FaaS platforms. Each platform implementation provides specific credential types (API keys, service account files, connection strings, etc.) while following the common serialization and caching patterns defined here.

abstractmethod static deserialize(config: dict, cache: Cache, handlers: LoggingHandlers) → Credentials

Create credentials instance from user config and cached values.

This method implements the credential loading hierarchy: 1. Use new config values if provided 2. Load from environment variables 3. Fail if no credentials available

Credentials are NOT cached.

Parameters:

• config – User-provided configuration dictionary

• cache – Cache instance for loading stored credentials

• handlers – Logging handlers for error reporting

Returns:

Platform-specific credentials instance

Return type:

Credentials

Raises:

RuntimeError – If no valid credentials can be loaded

abstractmethod serialize() → dict

Serialize credentials to dictionary for cache storage.

Returns:

Serialized credential data suitable for JSON storage

Return type:

dict

Note

Implementations should be careful about storing sensitive information and may choose to exclude certain fields.

class sebs.faas.config.Resources(name: str)

Bases: ABC, LoggingBase

Abstract base class for FaaS platform resource management.

This class manages cloud resources allocated for function execution and deployment across different FaaS platforms. Resources include infrastructure components like IAM roles, API gateways, networking components, and storage buckets needed to support serverless function deployment and execution.

Storage resources (object storage, NoSQL databases) are handled separately through dedicated storage classes, while this class focuses on compute and deployment infrastructure.

Key responsibilities: - Resource ID management and generation - Storage bucket lifecycle management - Platform-specific resource provisioning - Resource serialization and caching - Resource cleanup and deallocation

class StorageBucketType(*values)

Bases: str, Enum

Enumeration of storage bucket types used by SeBS.

Different bucket types serve different purposes in the benchmarking workflow: - DEPLOYMENT: Stores function deployment packages (ZIP files, containers) - BENCHMARKS: Stores benchmark input data and test files - EXPERIMENTS: Stores experiment results and output data

BENCHMARKS = 'benchmarks'

DEPLOYMENT = 'deployment'

EXPERIMENTS = 'experiments'

static deserialize(val: str) → StorageBucketType

Deserialize a string value to a StorageBucketType enum.

Parameters:

val – String value to convert to enum

Returns:

Corresponding enum value

Return type:

StorageBucketType

Raises:

Exception – If the value doesn’t match any enum member

cleanup_deleted_buckets(cache_client: Cache)

Clean local and cached entries for already deleted storage buckets.

Parameters:

cache_client – SeBS cache client.

abstractmethod static deserialize(config: dict, cache: Cache, handlers: LoggingHandlers) → Resources

Create resources instance from user config and cached values.

Parameters:

• config – User-provided configuration dictionary

• cache – Cache instance for loading stored resources

• handlers – Logging handlers for error reporting

Returns:

Platform-specific resources instance

Return type:

Resources

get_buckets() → List[str]

Produces a list of all buckets.

Returns:

list of bucket names

get_storage_bucket(bucket_type: StorageBucketType) → str | None

Get the bucket name for a specific bucket type.

Parameters:

bucket_type – Type of bucket to retrieve

Returns:

Bucket name if set, None otherwise

Return type:

Optional[str]

get_storage_bucket_name(bucket_type: StorageBucketType) → str

Generate a standardized bucket name for a bucket type.

Creates bucket names following the pattern: sebs-{type}-{resource_id}

Parameters:

bucket_type – Type of bucket to name

Returns:

Generated bucket name

Return type:

str

property has_resources_id: bool

Check if a resource ID has been assigned.

Returns:

True if resource ID is set, False otherwise

Return type:

bool

abstractmethod static initialize(res: Resources, dct: dict)

Initialize a Resources instance from configuration dictionary.

This base implementation handles common resource initialization including resource ID and storage bucket configuration. Platform-specific implementations should call this method and add their own initialization.

Parameters:

• res – Resources instance to initialize

• dct – Configuration dictionary from cache or user config

property region: str

Get the cloud region for resource deployment.

Returns:

Cloud region identifier

Return type:

str

property resources_id: str

Get the unique resource ID for this deployment.

Returns:

Unique resource identifier

Return type:

str

Raises:

AssertionError – If no resource ID has been set

abstractmethod serialize() → dict

Serialize resources to dictionary for cache storage.

Subclasses should call super().serialize() and extend the dictionary. This base implementation serializes resources_id and storage_buckets.

Returns:

Serialized resource data including resource ID and bucket mappings

Return type:

dict

set_storage_bucket(bucket_type: StorageBucketType, bucket_name: str)

Set the bucket name for a specific bucket type.

Parameters:

• bucket_type – Type of bucket to set

• bucket_name – Name of the bucket

update_cache(cache: Cache)

Update the cache with current resource configuration.

Stores the resource ID and storage bucket mappings in the cache for future retrieval.

Parameters:

cache – Cache instance to update

sebs.faas.container module

Docker container management for serverless function deployments.

This module provides the DockerContainer class for building and managing Docker containers for serverless function deployments. It handles:

• Building benchmark Docker images for different platforms

• Cross-architecture container compilation with emulation

• Container registry operations (push/pull)

• Progress tracking for container operations

• Platform-specific container naming and tagging

The module supports container-based deployments across different serverless platforms, with automatic detection of the host architecture and appropriate configuration for cross-compilation when needed.

class sebs.faas.container.DockerContainer(system_config: SeBSConfig, docker_client: DockerClient, experimental_manifest: bool = False)

Bases: LoggingBase

Abstract base class for Docker container management in serverless deployments.

This class provides common functionality for building, pushing, and managing Docker containers for serverless function deployments. Each platform implementation (AWS, Azure, GCP, etc.) extends this class to provide platform-specific container handling.

Key features: - Container image building with cross-architecture support - Container registry operations (push/pull/inspect) - Progress tracking for long-running operations - Platform-specific image naming and tagging - Caching and optimization for repeated builds

docker_client

Docker client for container operations

experimental_manifest

Whether to use experimental manifest inspection

system_config

SeBS configuration for image management

_disable_rich_output

Flag to disable rich progress output

build_base_image(directory: str, language: Language, language_version: str, architecture: str, benchmark: str, is_cached: bool, builder_image: str) → Tuple[bool, str, float]

Build benchmark Docker image. When building function for the first time (according to SeBS cache), check if Docker image is available in the registry. If yes, then skip building. If no, then continue building.

For every subsequent build, we rebuild image and push it to the registry. These are triggered by users modifying code and enforcing a build.

Parameters:

• directory – build directory

• language – benchmark language

• language_version – benchmark language version

• architecture – CPU architecture

• benchmark – benchmark name

• is_cached – true if the image is currently cached

• builder_image – Base image for containers

Returns:

True if image was rebuilt, and image URI and size in MB

Return type:

Tuple[bool, str, float]

property disable_rich_output: bool

Get whether rich output is disabled.

Returns:

True if rich output is disabled, False otherwise

Return type:

bool

find_image(repository_name: str, image_tag: str) → bool

Check if a Docker image exists in the registry.

Attempts to find an image in the registry using either experimental manifest inspection (if enabled) or by attempting to pull the image.

Parameters:

• repository_name – Name of the repository (e.g., ‘my-repo/my-image’)

• image_tag – Tag of the image to find

Returns:

True if the image exists, False otherwise

Return type:

bool

abstractmethod static name() → str

Get the platform name for this container implementation.

Returns:

Platform name (e.g., ‘aws’, ‘azure’, ‘gcp’)

Return type:

str

push_image(repository_uri: str, image_tag: str)

Push a Docker image to a container registry.

Delegates to the static method in DockerImageBuilder for consistent image pushing with progress tracking across SeBS.

Parameters:

• repository_uri – URI of the container registry repository

• image_tag – Tag of the image to push

Raises:

• docker.errors.APIError – If the push operation fails

• RuntimeError – If an error occurs during the push stream

push_to_registry(benchmark: str, language_name: str, language_version: str, architecture: str) → str

Push an existing local Docker image to the registry and return its URI.

Used when the container is cached locally but its registry URI is unknown (e.g., after previously cleaning resources). Computes the registry name, pushes the image, and returns the full image URI.

Parameters:

• benchmark – Benchmark name (e.g., ‘110.dynamic-html’)

• language_name – Programming language name (e.g., ‘python’)

• language_version – Language version (e.g., ‘3.8’)

• architecture – Target architecture (e.g., ‘x64’)

Returns:

Full URI of the pushed image.

Return type:

str

abstractmethod registry_name(benchmark: str, language_name: str, language_version: str, architecture: str) → Tuple[str, str, str, str]

Generate registry name and image URI for a benchmark.

Creates platform-specific naming for container images including registry URL, repository name, image tag, and complete image URI.

Parameters:

• benchmark – Name of the benchmark (e.g., ‘110.dynamic-html’)

• language_name – Programming language (e.g., ‘python’, ‘nodejs’)

• language_version – Language version (e.g., ‘3.8’, ‘14’)

• architecture – Target architecture (e.g., ‘x64’, ‘arm64’)

Returns:

Registry name, repository name, image tag, full image URI

Return type:

Tuple[str, str, str, str]

sebs.faas.function module

Function and execution model for the serverless benchmarking framework.

This module defines the core abstractions for serverless functions, including: - Function class: Represents a deployed serverless function - Trigger class: Represents invocation mechanisms for functions - Runtime and FunctionConfig: Configuration parameters for functions - ExecutionResult and related classes: Data model for capturing measurements

These abstractions provide a unified interface for handling functions across different FaaS platforms, allowing for consistent deployment, invocation, and measurement collection.

class sebs.faas.function.ExecutionBilling

Bases: object

Billing information for function execution.

Tracks billing-related metrics such as allocated memory, billed execution time, and GB-seconds consumed.

memory

Allocated memory in MB

billed_time

Billed execution time in milliseconds

gb_seconds

GB-seconds consumed (memory/1024 * billed_time/1000)

property billed_time: int | None

Get the billed execution time in milliseconds.

Returns:

Billed time in milliseconds, or None if not available

Return type:

int

static deserialize(cached_obj: dict) → ExecutionBilling

Create an ExecutionBilling instance from a dictionary.

Parameters:

cached_obj – Dictionary containing serialized billing data

Returns:

New instance with the deserialized data

Return type:

ExecutionBilling

property gb_seconds: float

Get the GB-seconds consumed.

Returns:

GB-seconds consumed

Return type:

float

property memory: int | None

Get the allocated memory in MB.

Returns:

Memory allocation in MB, or None if not available

Return type:

int

class sebs.faas.function.ExecutionResult

Bases: object

Comprehensive result of a function execution.

This class captures all timing information, provider metrics, and function output from a single function invocation. It provides methods for parsing benchmark output and calculating metrics.

output

Dictionary containing function output

Type:

dict

request_id

Unique identifier for the request

Type:

str

times

ExecutionTimes containing client-side timing measurements

Type:

sebs.faas.function.ExecutionTimes

provider_times

ProviderTimes containing platform-reported timings

Type:

sebs.faas.function.ProviderTimes

stats

ExecutionStats containing resource usage statistics

Type:

sebs.faas.function.ExecutionStats

billing

ExecutionBilling containing cost-related information

Type:

sebs.faas.function.ExecutionBilling

billing: ExecutionBilling

static deserialize(cached_config: dict) → ExecutionResult

Create an ExecutionResult instance from a cached configuration.

Parameters:

cached_config – Dictionary containing serialized execution result

Returns:

New instance with the deserialized data

Return type:

ExecutionResult

static from_times(client_time_begin: datetime, client_time_end: datetime) → ExecutionResult

Create an ExecutionResult with client-side timing information.

Parameters:

• client_time_begin – Timestamp when the request was initiated

• client_time_end – Timestamp when the response was received

Returns:

New instance with calculated client-side timing

Return type:

ExecutionResult

output: dict

parse_benchmark_output(output: dict)

Parse the output from a benchmark execution.

Extracts timing information and cold start status from the benchmark output.

Parameters:

output – Dictionary containing benchmark output

Raises:

RuntimeError – If the invocation failed (missing required fields)

provider_times: ProviderTimes

request_id: str

stats: ExecutionStats

times: ExecutionTimes

class sebs.faas.function.ExecutionStats

Bases: object

Statistics for function execution.

Tracks execution statistics such as memory usage, cold start status, and execution failure.

memory_used

Amount of memory used in MB (if available)

Type:

float | None

cold_start

Whether this was a cold start execution

Type:

bool

failure

Whether the execution failed

Type:

bool

cold_start: bool

static deserialize(cached_obj: dict) → ExecutionStats

Create an ExecutionStats instance from a dictionary.

Parameters:

cached_obj – Dictionary containing serialized statistics

Returns:

New instance with the deserialized data

Return type:

ExecutionStats

failure: bool

memory_used: float | None

class sebs.faas.function.ExecutionTimes

Bases: object

Client-side timing measurements for function execution.

Stores various timing measurements from the client’s perspective, including total execution time, HTTP connection times, and benchmark runtime. All times are reported in microseconds unless otherwise specified.

client

Total client-side execution time in microseconds

Type:

int

client_begin

Timestamp when the request was initiated

Type:

datetime.datetime

client_end

Timestamp when the response was received

Type:

datetime.datetime

benchmark

Benchmark execution time in microseconds

Type:

int

initialization

Function initialization time in microseconds

Type:

int

http_startup

Time to establish HTTP connection in seconds

Type:

int

http_first_byte_return

Time to first byte in seconds

Type:

int

benchmark: int

client: int

client_begin: datetime

client_end: datetime

static deserialize(cached_obj: dict) → ExecutionTimes

Create an ExecutionTimes instance from a dictionary.

Parameters:

cached_obj – Dictionary containing serialized timing data

Returns:

New instance with the deserialized data

Return type:

ExecutionTimes

http_first_byte_return: int

http_startup: int

initialization: int

class sebs.faas.function.Function(benchmark: str, name: str, code_hash: str, cfg: FunctionConfig)

Bases: LoggingBase

Abstract base class for serverless functions.

This class represents a deployed serverless function with its configuration and contains a list of associated triggers. Each cloud provider (AWS, Azure, GCP, etc.) implements a subclass with platform-specific functionality.

Represents a deployable unit of code on a FaaS platform. Contains details about the benchmark it belongs to, its name, code hash, configuration, and associated triggers. Subclasses implement provider-specific details.

config

Function configuration

name

Name of the deployed function

benchmark

Name of the benchmark implemented by this function

code_package_hash

Hash of the deployed code package

updated_code

Whether the code has been updated since deployment

add_trigger(trigger: Trigger)

Add a trigger to this function.

Parameters:

trigger – Trigger to add

property benchmark: str

Get the name of the benchmark.

Returns:

Name of the benchmark

Return type:

str

property code_package_hash: str

Get the hash of the code package.

Returns:

Hash of the code package

Return type:

str

property config: FunctionConfig

Get the function configuration.

Returns:

Configuration of the function

Return type:

FunctionConfig

abstractmethod static deserialize(cached_config: dict) → Function

Create a Function instance from a cached configuration.

Parameters:

cached_config – Dictionary containing serialized function

Returns:

New instance with the deserialized data

Return type:

Function

property name: str

Get the name of the function.

Returns:

Name of the function

Return type:

str

serialize() → dict

Serialize the function to a dictionary.

Returns:

Dictionary representation of the function

Return type:

dict

triggers(trigger_type: TriggerType) → List[Trigger]

Get triggers of a specific type associated with this function.

Parameters:

trigger_type – Type of triggers to get

Returns:

List of triggers of the specified type

Return type:

List[Trigger]

triggers_all() → List[Trigger]

Get all triggers associated with this function.

Returns:

List of all triggers

Return type:

List[Trigger]

property updated_code: bool

Check if the code has been updated since deployment.

Returns:

True if the code has been updated, False otherwise

Return type:

bool

class sebs.faas.function.FunctionConfig(timeout: int, memory: int, runtime: Runtime, architecture: Architecture = Architecture.X86)

Bases: object

Configuration for a serverless function.

Defines the resources, runtime, and architecture for a function deployment.

timeout

Maximum execution time in seconds

Type:

int

memory

Memory allocation in MB

Type:

int

runtime

Runtime environment configuration

Type:

sebs.faas.function.Runtime

architecture

CPU architecture for deployment

Type:

sebs.types.Architecture

architecture: Architecture = 'x64'

static deserialize(data: dict) → FunctionConfig

Create a FunctionConfig instance from a dictionary.

Parameters:

data – Dictionary containing serialized function configuration

Returns:

New instance with the deserialized data

Return type:

FunctionConfig

static from_benchmark(benchmark: Benchmark) → FunctionConfig

Create a FunctionConfig instance from a benchmark.

Parameters:

benchmark – Benchmark to extract configuration from

Returns:

New instance with the benchmark’s configuration

Return type:

FunctionConfig

memory: int

runtime: Runtime

serialize() → dict

Serialize the function configuration to a dictionary.

Returns:

Dictionary representation of the function configuration

Return type:

dict

timeout: int

class sebs.faas.function.ProviderTimes

Bases: object

Provider-reported timing measurements for function execution.

Stores timing measurements reported by the cloud provider, including initialization time and execution time.

initialization

Function initialization time in microseconds

Type:

int

execution

Function execution time in microseconds

Type:

int

static deserialize(cached_obj: dict) → ProviderTimes

Create a ProviderTimes instance from a dictionary.

Parameters:

cached_obj – Dictionary containing serialized timing data

Returns:

New instance with the deserialized data

Return type:

ProviderTimes

execution: int

initialization: int

class sebs.faas.function.Runtime(language: Language, version: str)

Bases: object

Runtime configuration for a serverless function.

Defines the language and version for a function’s runtime environment.

language

Programming language (Python, Node.js)

Type:

sebs.types.Language

version

Version string of the language runtime

Type:

str

static deserialize(config: dict) → Runtime

Create a Runtime instance from a dictionary.

Parameters:

config – Dictionary containing serialized runtime

Returns:

New instance with the deserialized data

Return type:

Runtime

language: Language

serialize() → dict

Serialize the runtime to a dictionary.

Returns:

Dictionary representation of the runtime

Return type:

dict

version: str

class sebs.faas.function.Trigger

Bases: ABC, LoggingBase

Abstract base class for function triggers.

A trigger represents a mechanism for invoking a serverless function, such as HTTP requests, direct SDK invocations, or event-based triggers. Each trigger type implements synchronous and asynchronous invocation methods.

Includes a helper method for HTTP invocations using pycurl.

class TriggerType(*values)

Bases: Enum

Enumeration of supported trigger types.

Defines the different mechanisms for invoking serverless functions: - HTTP: Invocation via HTTP requests - LIBRARY: Invocation via cloud provider SDK - STORAGE: Invocation via storage events

HTTP = 'http'

LIBRARY = 'library'

STORAGE = 'storage'

static get(name: str) → TriggerType

Get a TriggerType by name (case-insensitive).

Parameters:

name – Name of the trigger type

Returns:

The matching trigger type

Return type:

TriggerType

Raises:

Exception – If no matching trigger type is found

abstractmethod async_invoke(payload: dict) → Future

Asynchronously invoke a function with the given payload.

Parameters:

payload – Dictionary containing the function input

Returns:

Future object representing the pending execution

Return type:

Future

abstractmethod static deserialize(cached_config: dict) → Trigger

Create a Trigger instance from a cached configuration.

Parameters:

cached_config – Dictionary containing serialized trigger

Returns:

New instance with the deserialized data

Return type:

Trigger

abstractmethod serialize() → dict

Serialize the trigger to a dictionary.

Returns:

Dictionary representation of the trigger

Return type:

dict

abstractmethod sync_invoke(payload: dict) → ExecutionResult

Synchronously invoke a function with the given payload.

Parameters:

payload – Dictionary containing the function input

Returns:

Result of the function execution

Return type:

ExecutionResult

abstractmethod static trigger_type() → TriggerType

Get the type of this trigger.

Returns:

The type of this trigger

Return type:

TriggerType

sebs.faas.nosql module

Module for NoSQL database storage abstraction in the Serverless Benchmarking Suite.

This module provides an abstract base class for NoSQL database implementations across different cloud platforms (AWS DynamoDB, Azure CosmosDB, Google Cloud Datastore) and local development environments. It handles table creation, data writing, and cache management for benchmark data stored in NoSQL databases.

class sebs.faas.nosql.NoSQLStorage(region: str, cache_client: Cache, resources: Resources)

Bases: ABC, LoggingBase

Abstract base class for NoSQL database storage implementations.

This class defines the interface for NoSQL database operations across different cloud platforms and local environments. Concrete implementations handle the platform-specific details of creating tables, writing data, and managing resources.

cache_client

Client for caching database information

region

Cloud region where the database is deployed

property cache_client: Cache

Get the cache client.

Returns:

The cache client for database information

Return type:

Cache

cleanup_tables(dry_run: bool = False) → List[str]

Remove all allocated NoSQL tables.

Parameters:

dry_run – when true, skips actual deletion

Returns:

list of deleted table names

abstractmethod clear_table(name: str) → str

Clear all items from a table/container. Currently not implemented for any of the providers.

Provider-specific implementation details: - AWS DynamoDB: Removing & recreating table looks like the cheapest & fastest option. - Azure CosmosDB: Recreate container or use specific API to delete items. - Google Cloud: Likely recreate collection or use specific API.

Parameters:

name – Name of the table to clear

Returns:

Result message or status

Return type:

str

create_benchmark_tables(benchmark: str, name: str, primary_key: str, secondary_key: str | None = None)

Checks if the table already exists in the cache. If not, creates a new table with the specified keys.

Each table name follows this pattern: sebs-benchmarks-{resource_id}-{benchmark-name}-{table-name}

Each implementation should do the following: 1. Retrieve cached data 2. Create missing tables that do not exist 3. Update cached data if anything new was created (done separately in benchmark.py once the data is uploaded by the benchmark)

Parameters:

• benchmark – Name of the benchmark

• name – Logical name of the table

• primary_key – Primary key field name

• secondary_key – Optional secondary key field name

abstractmethod create_table(benchmark: str, name: str, primary_key: str, secondary_key: str | None = None) → str

Create a new table for a benchmark.

Provider-specific implementation details: - AWS: DynamoDB Table - Azure: CosmosDB Container - Google Cloud: Firestore in Datastore Mode, Database/Collection

Parameters:

• benchmark – Name of the benchmark

• name – Logical name of the table

• primary_key – Primary key field name

• secondary_key – Optional secondary key field name

Returns:

Physical name of the created table

Return type:

str

abstractmethod static deployment_name() → str

Get the name of the deployment platform.

Returns:

Name of the deployment platform (e.g., ‘aws’, ‘azure’, ‘gcp’)

Return type:

str

envs() → dict

Return a dictionary of environment variables that are required by functions to access this NoSQL storage (e.g., connection strings, table names). Default implementation returns an empty dictionary. Subclasses should override if they need to expose environment variables.

Returns:

Dictionary of environment variables

Return type:

dict

abstractmethod get_tables(benchmark: str) → Dict[str, str]

Get a mapping of benchmark-defined table names to actual cloud provider table names.

Parameters:

benchmark – Name of the benchmark

Returns:

Dictionary mapping table logical names to physical table names

Return type:

Dict[str, str]

property region: str

Get the cloud region.

Returns:

The cloud region where the database is deployed

Return type:

str

abstractmethod remove_table(name: str) → str

Remove a table completely.

Parameters:

name – Name of the table to remove

Returns:

Result message or status

Return type:

str

abstractmethod retrieve_cache(benchmark: str) → bool

Retrieve cached table information for a benchmark. Implementations should populate internal structures with cached table names/details.

Parameters:

benchmark – Name of the benchmark

Returns:

True if cache was successfully retrieved, False otherwise

Return type:

bool

abstractmethod update_cache(benchmark: str)

Update the cache with the latest table information for a benchmark.

Parameters:

benchmark – Name of the benchmark

abstractmethod write_to_table(benchmark: str, table: str, data: dict, primary_key: Tuple[str, str], secondary_key: Tuple[str, str] | None = None)

Write an item/document to the specified table/container. This is used by benchmarks to populate tables with test data.

Args: Write data to a table.

benchmark: Name of the benchmark table: Logical name of the table data: Dictionary of data to write primary_key: Tuple of (key_name, key_value) for the primary key secondary_key: Optional tuple of (key_name, key_value) for the secondary key

sebs.faas.resources module

System resource management for FaaS platforms.

This module provides the abstract base class for managing system-level resources across different serverless platforms. It coordinates access to storage services, NoSQL databases, and other cloud resources needed for benchmark execution.

Each platform implementation (AWS, Azure, GCP, Local, etc.) provides concrete implementations that handle platform-specific resource management while following the common interface defined here.

class sebs.faas.resources.SystemResources(config: Config, cache_client: Cache, docker_client: DockerClient)

Bases: ABC, LoggingBase

Abstract base class for system-level resource management.

This class provides a common interface for managing cloud resources needed by benchmark functions across different serverless platforms. It handles the provisioning and access to storage services, NoSQL databases, and other platform-specific resources.

_config

Platform configuration containing credentials and settings

_cache_client

Cache client for storing resource configurations

_docker_client

Docker client for container-based resource management

abstractmethod get_nosql_storage() → NoSQLStorage

Get or create a NoSQL database storage instance.

Provides access to NoSQL database services (DynamoDB, CosmosDB, Datastore, ScyllaDB) for benchmarks that require structured data storage with key-value or document-based operations. The storage instance may be a cloud service or a locally deployed container.

Returns:

Configured NoSQL storage instance ready for use

Return type:

NoSQLStorage

Raises:

RuntimeError – If NoSQL service cannot be provisioned or accessed

abstractmethod get_storage(replace_existing: bool | None = None) → PersistentStorage

Get or create a persistent storage instance.

Provides access to object storage services (S3, Azure Blob, GCS, MinIO) for storing benchmark input data, function packages, and results. The storage instance may be a cloud service or a locally deployed container.

Parameters:

replace_existing – Whether to replace existing benchmark data. If None, uses the default behavior for the platform.

Returns:

Configured storage instance ready for use

Return type:

PersistentStorage

Raises:

RuntimeError – If storage service cannot be provisioned or accessed

sebs.faas.storage module

Object storage abstraction for serverless benchmarks.

This module provides the PersistentStorage abstract base class for managing object storage across different cloud platforms and local deployments. It handles bucket management, file operations, and benchmark data organization.

The storage abstraction supports: - Cross-platform object storage (S3, Azure Blob, GCS, MinIO) - Benchmark data organization with input/output separation - Bucket lifecycle management and naming conventions - Benchmark files upload/download operations with caching - Deployment discovery and resource management

Each platform provides concrete implementations that handle platform-specific API calls while following the common interface defined here.

class sebs.faas.storage.PersistentStorage(region: str, cache_client: Cache, resources: Resources, replace_existing: bool)

Bases: ABC, LoggingBase

Abstract base class for persistent object storage implementations.

This class defines the interface for object storage services across different cloud platforms. It manages buckets, files, and benchmark data organization while providing a consistent API regardless of the underlying storage service.

cached

Whether bucket configuration is cached

_cache_client

Cache client for storing configuration

_input_prefixes

List of input data prefixes for benchmarks

_output_prefixes

List of output data prefixes for benchmarks

input_prefixes_files

Files associated with input prefixes

_replace_existing

Whether to replace existing files during uploads

_region

Cloud region for storage operations

_cloud_resources

Resource configuration for the platform

benchmark_data(benchmark: str, requested_buckets: Tuple[int, int]) → Tuple[List[str], List[str]]

Allocate storage prefixes for benchmark input and output data.

Creates logical prefixes within the benchmarks bucket for organizing benchmark input and output data. Checks cache first to avoid redundant allocation and validates existing prefix configuration.

Prefix naming format: - Input: “benchmark-{idx}-input” - Output: “benchmark-{idx}-output”

Parameters:

• benchmark – Name of the benchmark

• requested_buckets – Tuple of (input_prefix_count, output_prefix_count)

Returns:

Lists of (input_prefixes, output_prefixes)

Return type:

Tuple[List[str], List[str]]

property cache_client: Cache

Get the cache client for configuration storage.

Returns:

Cache client instance

Return type:

Cache

abstractmethod clean_bucket(bucket_name: str) → None

Remove all objects from a storage bucket.

Parameters:

bucket_name – Name of the bucket to clean

Raises:

Platform-specific exceptions for deletion failures –

cleanup_buckets(dry_run: bool = False) → List[str]

Remove all allocated object storage buckets.

Parameters:

dry_run – when true, skips actual deletion

Returns:

list of deleted buckets

abstractmethod correct_name(name: str) → str

Correct a bucket name to comply with platform naming requirements.

Different platforms have different naming restrictions (character sets, length limits, etc.). This method applies platform-specific corrections.

Parameters:

name – Original bucket name

Returns:

Corrected bucket name that complies with platform requirements

Return type:

str

abstractmethod static deployment_name() → str

Return the name of the FaaS deployment this storage belongs to (e.g., “aws”, “azure”).

Returns:

Platform name (e.g., ‘aws’, ‘azure’, ‘gcp’, ‘minio’)

Return type:

str

abstractmethod download(bucket_name: str, key: str, filepath: str) → None

Download a file from a storage bucket.

Parameters:

• bucket_name – Name of the source bucket

• key – Storage source filepath (object key)

• filepath – Local destination filepath

Raises:

Platform-specific exceptions for download failures –

download_bucket(bucket_name: str, output_dir: str) → None

Download all files from a storage bucket to a local directory.

Downloads every file from the specified bucket to a local output directory. Only downloads files that don’t already exist locally.

Warning

Assumes flat directory structure in bucket. Does not handle object keys with directory separators (e.g., ‘dir1/dir2/file’).

Parameters:

• bucket_name – Name of the bucket to download from

• output_dir – Local directory to download files to

Raises:

Platform-specific exceptions for download failures –

abstractmethod exists_bucket(bucket_name: str) → bool

Check if a storage bucket/container exists.

Parameters:

bucket_name – Name of the bucket to check

Returns:

True if bucket exists, False otherwise

Return type:

bool

Raises:

Platform-specific exceptions for access failures –

find_deployments() → List[str]

Find existing SeBS deployments by scanning bucket names.

Scans all buckets in the storage service and extracts deployment IDs from bucket names that follow the SeBS naming convention. This helps identify existing deployments that can be reused.

Looks for buckets named “sebs-benchmarks-*”.

Returns:

List of deployment resource IDs found in bucket names

Return type:

List[str]

get_bucket(bucket_type: StorageBucketType) → str

Get or create a storage bucket for the specified type.

Checks if the bucket is already known in _cloud_resources. If not, generates a bucket name following the standard naming convention, checks if it exists in the cloud, creates it if necessary, and then stores it in _cloud_resources.

Parameters:

bucket_type – Type of bucket to retrieve (BENCHMARKS, EXPERIMENTS, DEPLOYMENT)

Returns:

Name of the bucket for the specified type

Return type:

str

Raises:

Platform-specific exceptions for bucket operations –

property input_prefixes: List[str]

Get the list of input data prefixes for benchmarks. These are paths within the benchmark data bucket.

Returns:

List of input prefix names

Return type:

List[str]

abstractmethod list_bucket(bucket_name: str, prefix: str = '') → List[str]

Retrieve list of files in a storage bucket.

Parameters:

• bucket_name – Name of the bucket to list

• prefix – Optional prefix to filter objects

Returns:

List of file keys in the bucket matching the prefix

Return type:

List[str]

Raises:

Platform-specific exceptions for listing failures –

abstractmethod list_buckets(bucket_name: str | None = None) → List[str]

List all storage buckets/containers, optionally filtering them with a prefix.

Parameters:

bucket_name – Optional specific bucket prefix name to check for

Returns:

List of bucket names. If bucket_name is provided,

returns [bucket_name] if it exists, empty list otherwise.

Return type:

List[str]

Raises:

Platform-specific exceptions for listing failures –

property output_prefixes: List[str]

Get the list of output data prefixes for benchmarks. These are paths within the benchmark data bucket.

Returns:

List of output prefix names

Return type:

List[str]

property region: str

Get the cloud region for storage operations.

Returns:

Cloud region identifier

Return type:

str

abstractmethod remove_bucket(bucket: str) → None

Delete a storage bucket completely. The bucket must often be emptied afterwards.

Parameters:

bucket – Name of the bucket to remove

Raises:

Platform-specific exceptions for deletion failures –

property replace_existing: bool

Flag indicating whether to replace existing files during operations.

Returns:

True if existing files should be replaced, False otherwise

Return type:

bool

abstractmethod upload(bucket_name: str, filepath: str, key: str) → None

Upload a file to a storage bucket.

Bypasses caching and directly uploads the file. Useful for uploading code packages to storage when required by the deployment platform.

Parameters:

• bucket_name – Name of the destination bucket

• filepath – Local source filepath

• key – Storage destination filepath (object key)

Raises:

Platform-specific exceptions for upload failures –

abstractmethod uploader_func(bucket_idx: int, file: str, filepath: str) → None

Upload benchmark input data to storage with smart caching.

Implements a utility function for uploading benchmark input data that respects caching preferences. Skips uploading existing files unless the storage client has been configured to override existing data.

This is used by each benchmark to prepare input benchmark files.

Parameters:

• bucket_idx – Index of the input prefix/bucket

• file – Name of the file to upload

• filepath – Storage destination filepath (object key)

Raises:

Platform-specific exceptions for upload failures –

sebs.faas.system module

Module providing the core abstraction for Function-as-a-Service (FaaS) systems.

This module defines the base System class that provides consistent interfaces for working with different serverless platforms (AWS Lambda, Azure Functions, Google Cloud Functions, OpenWhisk, etc.). It handles function lifecycle management, code packaging, deployment, triggering, and metrics collection while abstracting away platform-specific details.

class sebs.faas.system.System(system_config: SeBSConfig, cache_client: Cache, docker_client: DockerClient, system_resources: SystemResources)

Bases: ABC, LoggingBase

Abstract base class for FaaS system implementations.

This class provides basic abstractions for all supported FaaS platforms. It defines the interface for system initialization, resource management, function deployment, code packaging, function invocation, and metrics collection. Each cloud provider implements a concrete subclass of this abstract base.

The class handles: - System and storage service initialization - Creation and updating of serverless functions - Function code packaging and deployment - Trigger creation and management - Metrics collection and error handling - Caching of functions to avoid redundant deployments - Cold start management

system_config

Global SeBS configuration

docker_client

Docker client for building code packages and containers

cache_client

Cache client for storing function and deployment information

cold_start_counter

Counter for generating unique function names to force cold starts

system_resources

Resources manager for the specific cloud platform

build_function(code_package: Benchmark, func_name: str | None = None)

Create a build deployment of the selected function.

Parameters:

• code_package – seleted benchmark

• func_name – user-specified function name

Raises:

RuntimeError – unsupported language version for the platform

property cache_client: Cache

Get the cache client.

Returns:

The cache client

Return type:

Cache

abstractmethod cached_function(function: Function)

Perform any necessary operations for a cached function.

This method is called when a function is found in the cache. It may perform platform-specific operations such as checking if the function still exists in the cloud, updating permissions, re-initializing transient client objects, or ensuring associated resources (like triggers) are correctly configured.

Parameters:

function – The cached function instance

cleanup_functions(dry_run: bool) → List[str]

Remove all created cloud functions.

Parameters:

dry_run – when true, skips actual deletion

Returns:

list of deleted function names

cleanup_resources(dry_run: bool = False) → dict

Discover and delete all SeBS resources of the deployment.

Parameters:

dry_run – when true, it does not delete anything.

Returns:

Dict mapping resource type names to lists of deleted resource identifiers.

property cold_start_counter: int

Get the cold start counter.

A counter used in attempts to enforce cold starts. Its value might be incorporated into function environment variables.

Returns:

The current cold start counter value

Return type:

int

abstract property config: Config

Get the platform-specific configuration.

Returns:

The platform-specific configuration

Return type:

Config

property container_client: DockerContainer | None

Get the platform-specific container manager. For example, on OpenWhisk we push to DockerHub, while on AWS we push images to ECR.

Returns:

Container manager instance.

abstractmethod create_function(code_package: Benchmark, func_name: str, container_deployment: bool, container_uri: str | None) → Function

Create a new function in the FaaS platform. The implementation is responsible for creating all necessary cloud resources.

Parameters:

• code_package – Benchmark containing the function code

• func_name – Name of the function

• container_deployment – Whether to deploy as a container

• container_uri – URI of the container image

Returns:

Created function instance

Return type:

Function

Raises:

NotImplementedError – If container deployment is requested but not supported

abstractmethod create_trigger(function: Function, trigger_type: TriggerType) → Trigger

Create a trigger for a function.

Parameters:

• function – The function to create a trigger for

• trigger_type – Type of trigger to create

Returns:

The created trigger

Return type:

Trigger

abstractmethod default_function_name(code_package: Benchmark, resources: Resources | None = None) → str

Generate a default function name for a benchmark.

Parameters:

• code_package – The benchmark to generate a name for

• resources – Optional resources configuration

Returns:

Generated function name

Return type:

str

delete_function(func_name: str) → None

Delete cloud deployment of a function.

Parameters:

func_name – function name in the cloud.

disable_rich_output()

Disable rich output for platforms that support it, e.g, progress of pushing Docker images.

This is mostly used in testing environments or CI pipelines.

property docker_client: DockerClient

Get the Docker client.

Returns:

The Docker client

Return type:

docker.client.DockerClient

abstractmethod download_metrics(function_name: str, start_time: int, end_time: int, requests: Dict[str, ExecutionResult], metrics: dict)

Download provider-specific performance metrics from the cloud platform.

This typically involves querying a logging or monitoring service (e.g., CloudWatch, Application Insights) for details like actual execution duration, memory usage, etc., and populating the requests (ExecutionResult objects) and metrics dictionaries.

Parameters:

• function_name – Name of the function to get metrics for

• start_time – Start timestamp for metrics collection

• end_time – End timestamp for metrics collection

• requests – Dictionary of execution results

• metrics – Dictionary to store the downloaded metrics

abstractmethod enforce_cold_start(functions: List[Function], code_package: Benchmark)

Force cold starts for the specified functions.

This method implements platform-specific techniques to ensure that subsequent invocations of the functions will be cold starts. In practice, this usually uses an update of environment variables with new values.

Parameters:

• functions – List of functions to enforce cold starts for

• code_package – The benchmark associated with the functions

finalize_container_build() → Callable[[str, Language, str, str, str, bool], Tuple[str, int]] | None

Default behavior of container deployment is that no code package is needed. Thus, we return None to signal that.

One exception is OpenWhisk: we deploy code package + container.

Returns:

Null, as no code package.

find_deployments() → List[str]

Find existing deployments in the cloud platform. Default implementation uses storage buckets to identify deployments. This can be overridden by platform-specific implementations, e.g., Azure that looks for unique storage accounts.

Returns:

List of existing deployment resource IDs

Return type:

List[str]

abstractmethod static function_type() → Type[Function]

Get the platform-specific Function class type.

Returns:

The Function class for this platform

Return type:

Type[Function]

get_function(code_package: Benchmark, func_name: str | None = None) → Function

Get or create a function for a benchmark.

This method handles the complete function creation/update workflow:

1. If a cached function with the given name exists and code has not changed, returns the cached function (after potential configuration checks/updates)

2. If a cached function exists but the code hash differs or rebuild is foreced, update the function code in the cloud.

3. If no cached function exists, creates a new function

Benchmark code is built (via code_package.build) before these steps. The build might be skipped if source code hasn’t changed and no update is forced.

Parameters:

• code_package – The benchmark containing the function code

• func_name – Optional name for the function (will be generated if not provided)

Returns:

The function instance

Return type:

Function

Raises:

Exception – If the language version is not supported by this platform

initialize(config: Dict[str, str] = {}, resource_prefix: str | None = None)

Initialize the system.

After this call completes, the local or remote FaaS system should be ready to allocate functions, manage storage resources, and invoke functions. Subclasses should override this to perform provider-specific initialization.

Parameters:

• config – System-specific parameters

• resource_prefix – Optional prefix for resource naming

initialize_resources(select_prefix: str | None)

Initialize cloud resources for the deployment.

This method either: 1. Uses an existing resource ID from configuration 2. Finds existing deployment in the cloud and reuses it, matching the optional prefix 3. If no suitable existing deployment is found or specified,

a new unique resource ID is generated.

Parameters:

select_prefix – Optional prefix to match when looking for existing deployments

is_configuration_changed(cached_function: Function, benchmark: Benchmark) → bool

Check if a function’s configuration needs to be updated.

This function checks for common function parameters to verify if their values are still up to date with the benchmark configuration.

Parameters:

• cached_function – The existing function

• benchmark – The benchmark with potential new configuration

Returns:

True if configuration has changed, False otherwise

Return type:

bool

abstractmethod static name() → str

Get the name of the platform.

Returns:

Platform name (e.g., ‘aws’, ‘azure’, ‘gcp’)

Return type:

str

abstractmethod package_code(directory: str, language: Language, language_version: str, architecture: str, benchmark: str, is_cached: bool) → Tuple[str, int]

Apply system-specific code packaging to prepare a deployment package.

The benchmark creates a code directory with the following structure: - [benchmark sources] - [benchmark resources], e.g., HTML template or ffmpeg binary - [dependence specification], e.g. requirements.txt or package.json - [language-specific wrappers implementation for the specific system]

This step transforms that structure to fit platform-specific deployment requirements, such as creating a zip file for AWS or container image.

Parameters:

• directory – Path to the code directory

• language – Programming language name

• language_version – Programming language version

• architecture – Target architecture (e.g., ‘x64’, ‘arm64’)

• benchmark – Benchmark name

• is_cached – Whether the code is cached

Returns:

• Path to packaged code

• Size of the package in bytes

Return type:

Tuple containing

abstractmethod shutdown() → None

Shutdown the FaaS system.

This should release any acquired resources, stop any running local services (like Docker containers started by SeBS for CLI interactions), and update the cache with the final system configuration. This should be called when the system is no longer needed.

property system_config: SeBSConfig

Get the global SeBS configuration.

Returns:

The system configuration

Return type:

SeBSConfig

property system_resources: SystemResources

Get the platform-specific resources manager.

Returns:

The resources manager

Return type:

SystemResources

abstractmethod update_function(function: Function, code_package: Benchmark, container_deployment: bool, container_uri: str | None)

Update an existing function in the FaaS platform with new code and/or configuration.

Parameters:

• function – Existing function instance to update

• code_package – New benchmark containing the function code

• container_deployment – Whether to deploy as a container

• container_uri – URI of the container image

Raises:

NotImplementedError – If container deployment is requested but not supported

abstractmethod update_function_configuration(cached_function: Function, benchmark: Benchmark)

Update the configuration of an existing function on the FaaS plaform.

This method is called when a function’s code is up-to-date but its configuration (memory, timeout, environment variable, etc.) needs to be updated.

Parameters:

• cached_function – The function to update

• benchmark – The benchmark containing the new configuration

Module contents

Main interfaces for FaaS system implementation.

For each cloud platform, we implement system abstractions characteristics creates and manages functions; resource management; object storage; NoSQL storage; and function abstraction.