Install SigOpt Orchestrate
SigOpt Orchestrate is a command-line tool for managing training clusters and running optimization experiments.
There are five steps to getting SigOpt Orchestrate installed.
- Install Docker
- Install SigOpt Orchestrate
- Configure SigOpt Orchestrate
- Configure AWS
- Create a Kubernetes cluster on AWS
- Download Docker Images
- Test Cluster Connected
1. Install Docker Back to Top
If you don't already have Docker installed, the Docker documentation recommends installing the Docker Community Edition (Docker CE).
SigOpt Orchestrate containerizes your model for you using Docker, industry-leading containerization technology. You do not need to know how to use Docker to use SigOpt Orchestrate. However, because Docker requires greater system privileges, you will need to install Docker separately from SigOpt Orchestrate.
Orchestrate requires that you can access Docker without root privileges. If you can run
docker ps without issue, then you should be good to go. Otherwise you probably need to add your user to the
On Linux, run the following commands to create the “docker” group and then add your user to that group.
sudo groupadd docker sudo usermod -aG docker $USER
Log out and in again then try running
docker ps again. For more information, please refer to the official Docker documentation.
Test your docker installation:
docker run hello-world
2. Install SigOpt Orchestrate Back to Top
SigOpt Orchestrate is a Python package you can install from:
pip install https://public.sigopt.com/orchestrate/releases/v0.9.3/orchestrate-0.9.3-py2.py3-none-any.whl
3. Configure SigOpt Orchestrate Back to Top
SigOpt Orchestrate is built to integrate with your SigOpt account. You will need to locally configure your SigOpt credentials.
Grab your API token below, or from the API Tokens page.
SigOpt Orchestrate will prompt you with:
> SigOpt API Token? null > Enable log collection? yes
Great! Now SigOpt Orchestrate is installed and running.
Alternatively, you can set the
SIGOPT_API_TOKEN environment variable, preferably somewhere such as a
.bash_profile or a
Learn more about Authentication with SigOpt from our docs, including information on the dev token.
4. Configure AWS Back to Top
SigOpt Orchestrate helps you manage clusters. This guide uses AWS EKS to create a cluster. If you already have a cluster on AWS or if you are using another cluster service, see our guide on Bring Your Own Cluster and skip to the test step below.
If your local development environment is not already configured to use AWS, the easiest way to get started is to configure your AWS Access Key and AWS Secret Key via the aws command line interface:
See the AWS docs for more about configuring your AWS credentials.
5. Create a Kubernetes cluster on AWS Back to Top
SigOpt Orchestrate references a cluster configuration YAML file in the creation of a Kubernetes cluster on AWS.
cluster.yml file. The example YAML file below defines a CPU cluster named
tiny-cluster with two
t2.small AWS instances.
# cluster.yml provider: aws cluster_name: tiny-cluster cpu: instance_type: t2.small max_nodes: 2 min_nodes: 2 kubernetes_version: '1.18'
To create the cluster, from the same directory as your cluster configuration file, run:
orchestrate cluster create -f cluster.yml
Cluster creation can take between 15-30 minutes. If you notice an error, please try re-running the same command. SigOpt Orchestrate will reuse the same EKS cluster so the second run will be much faster.
6. Download Docker Images Back to Top
Next, you'll need to download and load the Docker images of SigOpt Orchestrate to run on your cluster.
You'll need the image that matches your Python version - supported versions are Python 3.6 - 3.9. Replace the 3.9 version number below with the correct version for your Python environment:
curl -O https://public.sigopt.com/orchestrate/releases/v0.9.3/python-3.9.tar docker load -i python-3.9.tar
You will need to download and push the SigOpt Orchestrate controller image to your container registry.
In order to push an image to your container registry you must be logged in to the registry. To log in to AWS ECR with
awscli Version 1, run the following command:
aws ecr get-login --no-include-email | sh
If you're using
awscli version 2:
aws ecr get-login-password --region $REGION | docker login --username AWS --password-stdin https://$REGISTRY
Next, create the
orchestrate/controller repository on ECR.
aws ecr create-repository --repository-name=orchestrate/controller
Take note of the
repositoryUri in the output.
Now, download the controller image:
docker tag orchestrate/controller:0.9.3 [yourRepositoryUri]:0.9.3 docker push [yourRepositoryUri]:0.9.3
docker pushsteps above, where
Remaining logged in to Docker may interfere with cluster creation later, so it is best to remove the Docker login credentials from
~/.docker/config.json at this point.
7. Test Cluster Connected Back to Top
When you have completed the above steps, run our test command to verify that your install is working:
Now that you've got SigOpt Orchestrate connected to a cluster, we recommend the tutorials on Orchestrate a tracked Training Run or Orchestrate an HPO Experiment to run a training run or a set of training runs on your cluster.