Multisolution Experiments

In the standard optimization problem, SigOpt will attempt to maximize a metric of interest, returning a single set of parameters. For experiments with multiple solutions, SigOpt will attempt to find multiple solutions that are sufficiently diverse, and return multiple sets of parameters.

Example with Multiple Solutions

The contour plots below depict a function where two solutions are sufficiently far apart, and both are close to the absolute maximum of the function.

Defining an Experiment with Multiple Solutions in SigOpt

A SigOpt Experiment with multiple solutions can be conducted to return multiple sets of parameters that are sufficiently diverse. In the following code block, add the key num_solutions and value as the number of solutions desired. An observation budget is required for experiments with multiple solutions, and cannot be updated for a given experiment.

def evaluate_function(assignments):
  x1 = assignments['x1']
  x2 = assignments['x2']
  f1_val = -((x1 - 7) ** 2 + (x2 - 8) ** 2)
  return [{'name': 'f1', 'value': f1_val}]

experiment_meta = {
  'name': '2D Quadratic Polynomials',
  'project': 'sigopt-examples',
  'parameters': [
    {'bounds': {'max': 10, 'min': 0}, 'name': 'x1', 'type': 'double'},
    {'bounds': {'max': 10, 'min': 0}, 'name': 'x2', 'type': 'double'},
  ],
  'observation_budget': 100,
  'metrics': [{'name': 'f1'}],
  'num_solutions': 2,
  'parallel_bandwidth': 2,
}

Best Assignments for Multiple Solutions

Calling Best Assignments will return a list of solutions ordered in decreasing function value. Note that SigOpt will not always return the number of solutions desired. Prior to completing the observation budget, SigOpt may not feel that it has sufficiently searched the space and may return less than num_solutions when calling Best Assignments. After completing the observation budget, there will always be num_solutions entries in the best assignments call.

Limitations

• observation_budget must be set when a multisolution experiment is created
• Multisolution supports finding up to 3 solutions i.e. num_solutions <= 3
• Categorical parameters are not permitted
• Multimetric experiments are not permitted
• Constraints are not permitted

While this document discusses metric maximization, you can also minimize your metric.