Version: Next

Configuration files

There are two files generated after running opctl init --provider <provider>:

  • params.yaml - contains the fields that need to be updated to configure your deployment.
  • config.yaml - contains components that are going to be included in the deployment. This file is generated automatically and should not be updated.

For further information on populating the params.yaml file, refer to the sections below. This information is also available inside the generated params.yaml template.

tip

It is highly recommended that you commit params.yaml file into a private repository and encrypt it with BlackBox or use a secret management service like Azure Key Vault, AWS Secret Manager, GCP Secret Manager or HashiCorp Vault.

Content of configuration file params.yaml

important

The template below is automatically generated when your run opctl init for your provider.

This particular one was generated by running

opctl init --database --provider aks --artifact-repository-provider s3 --gpu-device-plugins nvidia --enable-https --enable-cert-manager --dns-provider route53
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
# Generated with Onepanel CLI
# CLI version: v1.0.0
# Command: opctl init --database --provider aks --artifact-repository-provider s3 --gpu-device-plugins nvidia --enable-https --enable-cert-manager --dns-provider route53
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
# Component: Onepanel
# Description: Onepanel application information
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
application:
# First namespace that will be created in Onepanel, more can be added later
# Note that you cannot use any of the following reserved namespaces:
# onepanel, default, application-system, cert-manager, istio-system, knative-serving, kube-public, kube-system
defaultNamespace: <namespace>
# Domain where Onepanel is hosted
# Use a first-level or multi-level subdomain like example.com or sub.example.com
domain: <domain>
# The Fully Qualified Domain (FQDN) where Onepanel will be hosted.
# If `domain` above is set to example.com or sub.example.com, then your FQDN could be: app.example.com or app.sub.example.com respectively
fqdn: <fully-qualified-domain-name>
# HTTP or HTTPS - Do not change, determined by `opctl init --enable-https`
# CLI flag: --enable-https
insecure: false
# Node pool key and values used for AutoScaling
nodePool:
# Cloud providers will automatically set label key as "node.kubernetes.io/instance-type" on all nodes
# For all Kubernetes 1.16.x versions, use "beta.kubernetes.io/instance-type"
# You would also need to use "beta.kubernetes.io/instance-type" for all versions of AKS and GKE 1.17.x
label: node.kubernetes.io/instance-type
# These are the machine type options that will be available in Onepanel
# `name` can be any user friendly name
# `value` should be the instance type in your cloud provider
# `resources.limits` should only be set if the node pool has GPUs
# The first option will be used as default.
options:
- name: 'CPU: X, RAM: Z GB'
value: <instance-type-name-1>
- name: 'CPU: X, GPU: Y, RAM: Z GB'
value: <instance-type-name-2>
resources:
limits:
nvidia.com/gpu: 1
# The kubernetes cluster where Onepanel will be deployed.
# Valid values: aks, eks, gke, microk8s
provider: aks
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
# Component: Artifact repository
# Description: S3 compatible object storage for storing files across Onepanel
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
artifactRepository:
s3:
# S3 access key
accessKey: <access-key>
# Name of bucket, example: my-bucket
bucket: <bucket-name>
# Endpoint for S3 compatible storage
# Supported provider endpoints:
# AWS: s3.amazonaws.com
# GCS: storage.googleapis.com
# Minio: my-minio-endpoint.default:9000
endpoint: s3.amazonaws.com
# Change to true if endpoint does NOT support HTTPS
insecure: false
# Key Format for objects stored by Workflows. This can reference Workflow variables
keyFormat: artifacts/{{workflow.namespace}}/{{workflow.name}}/{{pod.name}}
# Only used if using a local minio setup
publicEndpoint: s3.amazonaws.com
# Bucket region
region: us-west-2
# S3 secret key
secretKey: <secret-key>
certManager:
# Enter certificate admin email
# Example: [email protected]
email: <cert-admin-email>
# DNS Provider: Amazon Route53
# Docs: https://onepanelio.github.io/core-docs/docs/deployment/configuration/tls#route53
# CLI flag: --dns-provider=route53
route53:
access_key: <aws-access-key>
region: <aws-region>
secret_key: <aws-secret-key>
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
# Component: Database
# Description: Database connection information
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
database:
# Name of database
# For in-cluster database, leave as `onepanel`.
# If using an external managed database, use the name of that database.
databaseName: onepanel
# Do not change, only `postgres` driver is supported at this time.
driverName: postgres
# Database host
# For in-cluster database, leave as `postgres`.
# If using an external managed databases, use the host for that database.
host: postgres
# Database password
# For in-cluster database, leave as is.
# If using an external database, use the password for that database.
password: <password>
# Database port
port: 5432
# Database username
# For in-cluster database, leave as `onepanel`.
# If using an external managed database, use the username for that database.
username: onepanel

Sections

What follows is a more detailed description of each section of the params.yaml file.

application

This is where you set the basic application configuration.

defaultNamespace

This is the first Namespace you want created. This could be a project name or a team name. It is set to default by default but we recommend you use something more meaningful.

domain

This is the domain for your Onepanel resources. Some resources like Workspaces create subdomains of this domain so they can be accessed by a browser. This can be a top level domain like example.com or a subdomain sub.example.com.

fqdn

This is where Onepanel UI and API will be deployed. This should be a subdomain of the domain field mentioned above. For example: app.example.com or app.sub.example.com.

insecure

The insecure field is automatically set to true by default and will be set to false if you add the --enable-https when running opctl init. Do not update this field manually.

nodePool

Depending on your provider, these are either called node pools or node groups. They are labels on Kubernetes nodes that Onepanel uses for auto scaling nodes on demand.

A common label to identify these is node.kubernetes.io/instance-type which most cloud providers automatically set. The value of this label is usually set to the instance type of the cloud provider.

important

Kubernetes 1.16.x you will need to use beta.kubernetes.io/instance-type instead.

AKS (all versions) and GKE 1.17.x: you would need to use beta.kubernetes.io/instance-type as well.

EKS 1.16.x: make sure to use beta.kubernetes.io/instance-type label in tags when adding node groups.

You can see all labels on your nodes by running:

kubectl get nodes --show-labels

Note that this lists many different labels, so you can pick and choose any label key/value that is unique to that node.

For example after running the kubectl command above, you may get the following list of labels:

agentpool=nodepool1,
kubernetes.io/arch=amd64,
node.kubernetes.io/instance-type=Standard_D2s_v3,
kubernetes.io/os=linux,

You can then use the label key/value pairs as follows. Note that setting resources.limits field is required for GPUs.

nodePool:
label: node.kubernetes.io/instance-type # node label key
options:
- name: 'CPU: 4, RAM: 16GB' # friendly name for instance
value: m5.xlarge # node label value
- name: 'GPU: 1xV100, CPU: 8, RAM: 61GB'
value: p3.2xlarge
resources:
limits:
# Type and number of GPUs, possible values are:
# amd.com/gpu: <number-of-gpus>
# nvidia.com/gpu: <number-of-gpus>
nvidia.com/gpu: 1

See adding more nodes for more information on adding additional CPU or GPU nodes to your cluster.

artifactRepository

In this section, you can set up the default object storage to store your Workflow and Workspace artifacts. The default object storage is set per namespace and is used to automatically save snapshots of logs, models and any other output from you Workflows or easily sync data between Workspaces and Workflows.

Currently Amazon S3, MinIO S3, Google Cloud Storage (GCS) and Azure Blob Storage are supported.

Amazon S3

Example Amazon S3 configuration:

artifactRepository:
s3:
accessKey: AKIAJSIE27KKMHXI3BJQ
bucket: my-data-bucket
endpoint: s3.amazonaws.com
region: us-west-2
secretKey: 5bEYu26084qjSFyclM/f2pz4gviSfoOg+mFwBH39

MinIO S3

MinIO configurations would be similar to Amazon S3 with a different endpoint:

artifactRepository:
s3:
accessKey: AKIAJSIE27KKMHXI3BJQ
bucket: my-data-bucket
endpoint: my-minio-endpoint.default:9000
region: us-west-2 # The label for the Minio server location
secretKey: 5bEYu26084qjSFyclM/f2pz4gviSfoOg+mFwBH39
insecure: true # Set this to true if Minio is deployed internally into the Cluster

Google Cloud Storage (GCS)

Example Google Cloud Storage (GCS) configuration:

artifactRepository:
gcs:
# Name of bucket, example: my-bucket
bucket: my-data-bucket
# Key Format for objects stored by Workflows. This can reference Workflow variables
keyFormat: artifacts/{{workflow.namespace}}/{{workflow.name}}/{{pod.name}}
projectId: my-project-id
region: us-west1
serviceAccountKey: |
{
"type": "service_account",
"project_id": "my-project-id",
"private_key_id": "private_key_id",
"private_key": "private_key",
"client_email": "client_email",
"client_id": "client_id",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://oauth2.googleapis.com/token",
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
"client_x509_cert_url": "cert_url"
}
tip

You can get the serviceAccount JSON with the following command:

gcloud iam service-accounts keys create key.json \
--iam-account ACCOUNT_NAME@$PROJECT_ID.iam.gserviceaccount.com

Azure Blob Storage

Example Azure Blob Storage configuration:

artifactRepository:
abs:
# Name of Azure block storage container, example: my-container
container: my-data-container
# Azure storage account key
storageAccountKey: VsMyhYqFiFK1rNj5cNhMd6h9m01pCIQaMTDXmUIEOeMDtjspm12HvKxLs1dxvag2RJ1sYTuPWMQBDKvmIGPWIZ==
# Azure storage account name
storageAccountName: my-storage-account-name

certManager

If you have run opctl init with --enable-https, --enable-cert-manager and --dns-provider flags set, you need to configure your respective DNS provider here so that Onepanel can create and renew your TLS certificates for you.

See TLS certificates for more information about configuring this section.

database

This is the database settings section.

Production database

For a production deployment, use a managed PostgresSQL database like Amazon RDS, Azure Database or Google Cloud SQL.

Note that you can update your settings as many times as you like and simply run opctl apply again to update your your database settings in the cluster.

Example production database settings:

database:
databaseName: onepanel
driverName: postgres
host: my-onepanel-db.postgres.database.azure.com
password: verystrongproductionpassword
port: 5432
username: onepanel

Test database

For a test cluster, you can set the database host to postgres and use any username or password. This database will be automatically created in the cluster with the information you entered.

Note that you cannot change the username/password for the test database once it's created, so if you make a mistake, you will have to delete the test database and run opctl apply again:

# delete test database and its related volume
kubectl delete statefulset postgres -n onepanel
kubectl delete pvc postgres-pv-claim-postgres-0 -n onepanel
# Update your database settings
# Run opctl apply again to recreate the test database
opctl apply

Example test database settings:

database:
databaseName: onepanel
driverName: postgres
host: postgres
password: mypassword
port: 5432
username: onepanel

workflowEngine

containerRuntimeExecutor

The executor workflow engine uses to perform certain actions like monitoring pod logs, collecting artifacts, managing container lifecycles, etc.

The possible values are docker and pns:

  • docker is more reliable, however it mounts the docker.sock of the host makes it less secure.
  • pns is more secure, however in some versions of Kubernetes, it tends to fail on tasks that take less than 15 seconds.
Edit this page