Multisolution Experiments
In the standard optimization problem, SigOpt will attempt to maximize a metric of interest, returning a single set of parameters. For experiments with multiple solutions, SigOpt will attempt to find multiple solutions that are sufficiently diverse, and return multiple sets of parameters.
Example with Multiple Solutions
The contour plots below depict a function where two solutions are sufficiently far apart, and both are close to the absolute maximum of the function.
Defining an Experiment with Multiple Solutions in SigOpt
A SigOpt Experiment with multiple solutions can be conducted to return multiple sets of parameters that are sufficiently diverse. In the following code block, add the key num_solutions
and value as the number of solutions desired. An observation budget is required for experiments with multiple solutions, and cannot be updated for a given experiment.
def evaluate_function(assignments):
x1 = assignments['x1']
x2 = assignments['x2']
f1_val = -((x1 - 7) ** 2 + (x2 - 8) ** 2)
return [{'name': 'f1', 'value': f1_val}]
experiment_meta = {
'name': '2D Quadratic Polynomials',
'project': 'sigopt-examples',
'parameters': [
{'bounds': {'max': 10, 'min': 0}, 'name': 'x1', 'type': 'double'},
{'bounds': {'max': 10, 'min': 0}, 'name': 'x2', 'type': 'double'},
],
'observation_budget': 100,
'metrics': [{'name': 'f1'}],
'num_solutions': 2,
'parallel_bandwidth': 2,
}
Best Assignments for Multiple Solutions
Calling Best Assignments will return a list of solutions ordered in decreasing function value. Note that SigOpt will not always return the number of solutions desired. Prior to completing the observation budget, SigOpt may not feel that it has sufficiently searched the space and may return less than num_solutions
when calling Best Assignments. After completing the observation budget, there will always be num_solutions
entries in the best assignments call.
Limitations
observation_budget
must be set when a multisolution experiment is created- Multisolution supports finding up to 3 solutions i.e.
num_solutions <= 3
- Categorical parameters are not permitted
- Multimetric experiments are not permitted
- Constraints are not permitted
While this document discusses metric maximization, you can also minimize your metric.