Response Optimization

Hybrid Search (Grid over Categorical × Nelder–Mead over Continuous)

Optimises models with mixed factors: enumerates the Cartesian product of categorical level combinations and runs a continuous multi-start Nelder–Mead inside each cell.

Statusvalidated

Version1.0.0

Minitab equivalentStat > DOE > Response Optimizer (handles mixed factors via per-level enumeration)

Description

Classical derivative-free optimisers like Nelder–Mead operate on a continuous space. Categorical factors (e.g. tool A vs. B vs. C) are discrete and need different treatment. The hybrid search sidesteps this structurally: for every possible categorical-level combination, the categorical factors are fixed, and only the continuous factors are optimised. With n_cat combinations, n_cat independent Nelder–Mead runs are performed and the global best is returned. The categorical search is thus exhaustive (no local-minimum risk between categories), the continuous part is derivative-free and multi-start.

Formulas

Outer enumeration

C = \bigtimes_{j \in cat} L_{j}, ∣ C ∣ = j \in cat \prod ∣ L_{j} ∣

Cartesian product of all categorical level sets — exhaustive

Inner optimisation

\forall c \in C : x_{c}^{*} = ar g x_{cont} min - D ([x_{cont}; c])

Per categorical cell, one multi-start Nelder–Mead over the continuous slots

Global best

x^{*} = ar g c \in C max D ([x_{c}^{*}; c])

Global maximiser across all cells

Assumptions

Model carries a ModelSpec describing continuous/categorical predictors
Each categorical factor has a finite level set

Limitations

Scales exponentially with the number of categorical factors — 5 ternary factors = 243 NM runs
Inefficient for very large level sets (> 10) — consider a mixed-integer solver

References

Myers, R.H., Montgomery, D.C., Anderson-Cook, C.M. (2016). Response Surface Methodology, 4th Ed., Wiley — Chapter 7 (Mixed Factors in Optimisation)
Nelder, J.A. & Mead, R. (1965). A simplex method for function minimization. The Computer Journal 7(4): 308-313