Infrastructure Composition Language (ICL)

Spheron Network uses a declarative system for resource allocation. Users specify their deployment requirements, services, datacenters and pricing parameters through a "manifest" file called deploy.yaml, written in Infrastructure Composition Language (ICL). ICL is designed to be user-friendly and follows the YAML standard, making it similar to Docker Compose files.

The deploy.yaml file, which can also use the .yml extension, serves as a request form for network resources.

It's structured into several key sections:

1. Network Configuration
2. Version
3. Services
4. Profiles
5. Deployment
6. GPU Support
7. Private Container Registry Integration
8. Port Range
9. Pull Policy
10. Shared Memory (SHM) Support

1. Network Configuration

The Infrastructure Composition Language (ICL) file allows you to define networking settings for your deployment. This determines how workloads can connect to each other and be accessed externally. By default, workloads within a deployment group are isolated, meaning no external connections are permitted. However, these restrictions can be adjusted as needed.

2. Version Configuration

The Spheron configuration file requires a version specification. At present, the only accepted value is "1.0".

3. Services Configuration

The services section at the top level of the configuration file defines the workloads for your Spheron deployment. It's structured as a map where each key represents a unique service name, and the corresponding value is another map with the following possible fields:

Environment Variables

The env field allows you to specify a list of environment variables that will be made available to the running container. These variables are defined in a key-value format. For example:

env:
  - WALLET_ADDR=0xabcdedghijke
  - VERSION=1.0

Port Exposure

When configuring port exposure for your services, keep these points in mind:

• HTTPS Support: Spheron deployments can use HTTPS, but only with self-signed certificates.
• Signed Certificates: To implement properly signed certificates, you'll need to use a third-party solution like Cloudflare as a front-end.
• Flexible Port Mapping: You're not limited to just port 80 for HTTP/HTTPS ingress. You can expose other ports and map them to port 80 using the as: 80 directive, provided your application understands HTTP/HTTPS protocols. This is particularly useful for applications like React web apps.
• Simplified Port Exposure: In the ICL, it's only required to expose port 80 for web applications. However, this setup specifies that both ports 80 and 443 are exposed.

- port: 3000
  as: 80
  to:
    - global: true
  accept:
    - www.yoursite.com

Port Range

When deploying to fizz nodes, you can specify a port range using the port_range and port_range_as fields. Here's an example:

- port_range: 8443-8445
  port_range_as: 8443-8445
  to:
    - global: true

You can specify either port, port_range, or both, but make sure to specify at least one of them.

The expose parameter is a list that defines the connections allowed to the service. Each entry in this list is a map that can include one or more of the following fields:

Keep in mind, The as parameter determines the default proto value.

Configuring expose.to

The expose.to field defines a list of clients allowed to connect to a service. Each item in this list is a map that can contain one or both of these entries:

If no service is specified and global is set to true, any client can connect from any location (this is commonly desired for web servers).

If a service name is specified and global is set to false, only services within the current datacenter can connect. If a service name is specified and global is set to true, services from other datacenters within the deployment can connect.

When global is set to false, a service name must be provided.

Pull Policy

The pull_policy field allows you to specify how the container runtime should handle pulling the image for your service. This can be particularly useful when working with frequently updated images or when you want to ensure you're always using the latest version.

There are three possible values for pull_policy:

• Always: The image is pulled whenever the pod is started or restarted.
• IfNotPresent: The image is pulled only if it's not already present on the node.
• Never: The image is never pulled, and the deployment will fail if the image isn't already present on the node.

Example usage:

services:
  myapp:
    image: myregistry.com/myuser/myapp:latest
    pull_policy: Always

4. Profiles

The profiles section is used to define named compute and placement profiles that can be referenced in the deployment section. It's also define the name of the deployment, duration of the lease created & the provider tiers. Below is the table describing the parameters:

Deployment Mode

Spheron offers two deployment modes:

1. Provider Mode: Deploys directly to data center-grade providers. This mode offers:

   • Higher stability
   • Larger compute resources
   • Better bandwidth connections
   • Suitable for production-grade applications
   • To deploy in this mode, just write provider.

2. Fizz Mode: Deploys to a network of smaller, consumer-grade devices. This mode offers:

   • Lower costs
   • Distributed deployment across many nodes
   • Less stability compared to Provider Mode
   • Suitable for testing or less resource-intensive applications
   • To deploy in this mode, just write fizz.

Choose the mode that best fits your application's requirements and budget.

Lease Duration

During the deployment you also pass duration to run the deployment which will be in 1s, 1min, 1h, 1d, 1mon, & 1y. This will specify how much time the lease need to run and accordingly lock funds to continue to run the deployment. When you close the deployment prematurely, then you will unlock the amount that has not been spent.

Deployment Tier

During deployment, you have the option to specify the tiers on which you want your deployment to be placed, whether on a specific or generalized provider tier. This feature is beneficial for developers who need high reliability and are willing to pay a premium for high-tier providers. Conversely, users with less critical requirements can choose lower-tier providers at a reduced cost. During the testnet phase, there is no premium margin on deployment as we are still finalizing the idea.

We have two general tiers: Secured and Community.

• Secured Tier: This tier consists of high-tier providers who have consistently demonstrated high uptime in the network. It includes Provider Tiers 1 to 3.
• Community Tier: This tier consists of lower-tier providers who have recently joined the network or have less reliable hardware. It includes Provider Tiers 4 to 7.

To deploy your services on any tier, use the following values:

Compute Profiles

Within profiles.compute, you can create a map of named compute profiles. Each profile specifies the compute resources that will be allocated to service instances using that profile.

Example

This defines a profile named api with resource requirements of 2 vCPUs, 4 gigabytes of memory, and 20 gigabytes of storage.

api:
  cpu: 2
  memory: "4Gi"
  storage: "20Gi"

cpu units indicate a vCPU share, which can be fractional. Without a suffix, the value denotes a fraction of a whole CPU share. If an m suffix is used, the value specifies the number of milli-CPU shares, which equals 1/1000 of a CPU share.

Example

memory and storage units are expressed in terms of bytes, with specific suffixes utilized to simplify their representation as follows:

Placement Profiles

profiles.placement is a map of named datacenter profiles. Each profile outlines the necessary datacenter attributes and the pricing configuration for each compute profile that will be utilized within the datacenter.

Example

useast:
  attributes:
    region: us-east
  pricing:
    web:
      token: USDT
      amount: 80

This defines a profile named useast with the required attributes {region="us-east"} and a maximum price of 80 USDT per hour for the web compute profiles.

Providers can only assign themselves to a predefined set of regions. If you specify an unlisted region, no provider will bid on your deployment, causing it to fail automatically. Please refer to the list mentioned in the Supported Regions section.

5. Deployment Configuration

The deployment section is where you define the specific deployment strategy for your services. It's structured as a map, with each service name corresponding to its deployment configuration.

For each service you want to deploy, you create an entry in the deployment section. This entry combines datacenter profiles with compute profiles to specify the final resource configuration for the service.

Example

api:
  useast:
    profile: api
    count: 30

This specifies that 30 instances of the api service should be deployed to a datacenter that matches the useast datacenter profile. Each instance will have access to the resources defined in the api compute profile.

6. GPU Integration Support in Compute Profiles

You can add GPUs to your workload by including them in the compute profile section. The placement of the GPU stanza is illustrated in the full compute profile example below.

profiles:
  compute:
    gpudep:
      resources:
        cpu:
          units: 8.0
        memory:
          size: 12Gi
        storage:
          size: 10Gi
        gpu:
          units: 1
          attributes:
            vendor:
              nvidia:
                - model: a100

Additional GPU Use Notes

Complete GPU ICL Example

For a comprehensive example of a GPU-enabled ICL, refer to this example (opens in a new tab) which includes the declaration of several GPU models.

Optional Model Specification

Specifying a GPU model in the ICL is optional. If your deployment does not require a specific GPU model, you can leave the model declaration blank, as demonstrated in the following example.

gpu:
  units: 1
  attributes:
    vendor:
      nvidia:

Declaring Multiple Models

If your deployment is optimized to run on multiple GPU models, include the relevant list of models as shown in the example below. In this setup, any Spheron provider with a listed model will bid on the deployment.

gpu:
  units: 1
  attributes:
    vendor:
      nvidia:
        - model: rtx4090
        - model: t4

Specifying GPU RAM

Optionally, the ICL can include a GPU RAM/VRAM requirement, as illustrated in the example below.

gpu:
  units: 1
  attributes:
    vendor:
      nvidia:
        - model: a100
          ram: 80Gi

Specifying GPU Interface

Optionally, the ICL can include a GPU interface requirement, as shown in the example below.

gpu:
  units: 1
  attributes:
    vendor:
      nvidia:
        - model: h100
          interface: sxm

Specifying GPU with RAM and Interface

Here is an example of specifying both RAM and interface in the ICL GPU section.

gpu:
  units: 1
  attributes:
    vendor:
      nvidia:
        - model: h100
          interface: pcie
          ram: 90Gi

7. Private Container Registry Integration

Spheron Network now supports private container registries, allowing you to use images from your private repositories securely in your deployments. This feature enhances security and flexibility for users who need to work with proprietary or sensitive container images.

Configuring Private Registry Access

To use images from a private registry, you'll need to provide authentication details in your deploy.yaml file. Here's an example of how to structure this information:

services:
  myapp:
    image: myregistry.com/myuser/myapp:latest
    credentials:
      registry: myregistry.com
      username: myusername
      password: "mysecretpassword"
    expose:
      - port: 3000
        as: 80
        to:
          - global: true

Important Notes:

1. Registry Specification:

   • For Docker Hub, use docker.io
   • For GitHub Container Registry, use ghcr.io
   • For Gitlab Container Registry, use registry.gitlab.com
   • For AWS ECR, use public.ecr.aws
   • For Azure Container Registry, use myregistry.azurecr.io

2. Authentication:

   • Docker Hub: Use your account password in the password field
   • GitHub Container Registry: Use a Personal Access Token with appropriate permissions in the password field

3. Compatibility: This feature has been tested with Docker Hub and GitHub Container Registry. Other registries may work but are not officially supported.

By leveraging private registry support, you can maintain tighter control over your container images while benefiting from Spheron's robust deployment infrastructure.

8. Port Range

When deploying to fizz nodes, you can specify a port range using the port_range and port_range_as fields. This feature allows you to expose multiple consecutive ports for your service. Here's an example:

expose:
  - port_range: 8443-8445
    port_range_as: 8443-8445
    to:
      - global: true

Key points about using port range:

1. You can specify either port or port_range or both, but make sure to specify at least one of them.
2. The port_range field defines the range of ports to be exposed from the container.
3. The port_range_as field defines how these ports should be mapped externally.
4. The range is inclusive, so ports 8443, 8444, and 8445 would be exposed in the example above.

9. Pull Policy

The pull_policy field allows you to specify how the container runtime should handle pulling the image for your service. This can be particularly useful when working with frequently updated images or when you want to ensure you're always using the latest version.

There are three possible values for pull_policy:

• Always: The image is pulled every time the pod is started or restarted.
• IfNotPresent: The image is pulled only if it's not already present on the node.
• Never: The image is never pulled, and the deployment will fail if the image isn't already present on the node.

Example usage:

services:
  myapp:
    image: myregistry.com/myuser/myapp:latest
    pull_policy: Always

Using the pull policy feature give you more control over how and when your container images are updated, which can be crucial for maintaining consistency across deployments or ensuring you're always running the latest version of your application.

10. Shared Memory (SHM) Support

Spheron's Infrastructure Composition Language (ICL) now supports the configuration of Shared Memory (SHM) for services that require inter-process communication or temporary file storage with high-speed access. This feature is particularly useful for applications that need to share data quickly between multiple processes within the same container.

Configuring SHM in Compute Profiles

To enable SHM, you can add a new storage class named ram in your compute profile definition. Here's an example of how to include SHM in your ICL:

profiles:
  compute:
    myapp:
      resources:
        cpu:
          units: 2
        memory:
          size: 2Gi
        storage:
          - size: 10Gi
          - name: sharedmem
            size: 2Gi
            attributes:
              persistent: false
              class: ram

In this example, we've defined a compute profile named datavis that includes:

• Standard storage of 10Gi
• A shared memory storage named sharedmem of 2Gi using the ram class

Important Notes:

1. SHM must be non-persistent. The ICL validation will raise an error if SHM is defined as persistent.
2. The class: ram attribute is used to specify that this storage should be treated as shared memory.
3. The name of the shared memory storage (e.g., sharedmem) can only contain alphanumeric characters and hyphens (-). Underscores (_) are not allowed.

Using SHM in Services

Once you've defined the SHM in your compute profile, you can use it in your service definition:

services:
  visualization:
    image: myregistry.com/datavis:latest
    expose:
      - port: 8080
        as: 80
        to:
          - global: true
    params:
      storage:
        sharedmem:
          mount: /dev/shm

In this service definition:

• We're using the visualization service with a specified Docker image.
• The params.storage.shared_mem.mount field is used to mount the shared memory to /dev/shm in the container.

Benefits of Using SHM

1. High-speed Inter-process Communication: SHM allows rapid data sharing between processes in the same container.
2. Temporary File Storage: It provides a fast storage option for temporary files that don't need to persist beyond the container's lifecycle.
3. Resource Efficiency: By using memory for storage, you can reduce I/O operations and improve overall application performance.

By leveraging Shared Memory in your Spheron deployments, you can optimize performance for applications that require fast, temporary storage or efficient inter-process communication.