Projects give you the option to group related experiments as you run experiments with SigOpt. You may need to run multiple optimization experiments per model, and grouping related experiments into projects allows you to more easily locate your work in the SigOpt Experiment Insights dashboard. This makes it possible to view related experiments next to each other and analyze them more effectively.
For example, if you were working on a neural network to perform image classification, you might change the architecture, run an optimization experiment, then change the parameter space, then run another experiment. You might continue this experimentation process until your model meets certain criteria. All these experiments tune the same underlying model and metric, and grouping them into a project allows you to view them all in one place. In general, you can think of a project as mapping to all the experiments that are performed on one set of code.
Starting a New Project
Before you start tuning the parameters of your model or system, navigate to the SigOpt experiments page and click “Create Project.” You can configure the project name and ID. The project ID is used in code when you are creating your experiment.
While it’s easiest to create a Project from the SigOpt web dashboard, you can also create a project via the API.
Creating Experiments in Projects
A project is a group of experiments with the same project ID. When you create an experiment, you configure what project it is placed in by setting the
project field with your project ID.
In general, we recommend one project be associated with one set of model code. In practice, this means you usually only need to use the project ID once when you initially define the Experiment Create call. Each subsequent time you run your optimization job, the
project field will have already been set. And if you modify your Experiment Create definition e.g. change observation budget or add a parameter, you can leave the
project field as is.
from sigopt import Connection conn = Connection(client_token="SIGOPT_API_TOKEN") conn.set_api_url("https://api.staging.sigopt.com") experiment = conn.experiments().create( name="RNN Version 2", parameters=[ dict( name="gamma", bounds=dict( min=0.001, max=1 ), type="double" ) ], observation_budget=50, project="sentiment-analysis" ) print("Created experiment: https://app.staging.sigopt.com/experiment/" + experiment.id)
While putting most experiments in a project can be beneficial for organization, setting the
project field is not required. If it is not set, then the experiment is not placed in a project.
Projects in the Dashboard
In addition to viewing your own and your team’s projects, you can view all the experiments that are contained in a project for easy access and comparison.
From the main experiments page, you can also select multiple experiments and move them into a project all at once.