The following are the test problems currently making up the database, together with the performances obtained by the different tested algorithms. To make it easier to pair up problems with algorithms and to analyze the results, the problems are broken up into three classes, with Class 1 problems denoting those with only bound constraints, Class 2 problems as those with general numerical constraints but no experimental constraints, and Class 3 problems being those that possess experimental constraints.

Class 1: P1, P7, P9

Class 2: P8, P10

Class 3: P2, P3, P4, P5, P6, P11

In reporting the metric values, both the average (left) and the standard error (right) are given, with the plus-minus (±) sign used to separate the two. For Metrics 8-10, which attempt to gauge convergence speed, the percentage of the trials that achieved the specified convergence is also given. In the case that convergence was not achieved for any of the trials, the letters "NA" (not available) are reported. Currently, testing is done automatically by dedicated computers, with a maximum of 100 trials done for each testing combination (generated by choosing the problem, the algorithm, and whether or not the model is used).

You may view the plotted results for the individual trials for each algorithm by clicking on the algorithm name in the "Name" column. For problems with only two variables, plots of the experimental iterates in the decision-variable space are given for particularly insightful illustrations - only their values are plotted for problems with three or more variables. For plots of the decision-variable space, green regions denote those that are feasible (where all constraints are satisfied), while red denotes the infeasible regions. Red dots denote the individual experimental iterates, while the green dot denotes the true optimum that the algorithm aims to find. Constant dotted lines are used to plot the optimal values in the three-or-more variable case. For the cost function value plots, the constant black line denotes the value at the true optimum.


Test Problem #1: Maximizing Profit in the Williams-Otto Reactor

Original code provided by: S. Costello

Problem Description

Test File Specs: algotest([4.8 77],40,0.5,[],[3 70],[6 100],[4.79 89.7],100,[],[],algonum)

Main Files: phipeval.m (required), phimod.m (model)

Auxiliary Files: plantbalancesT.m (required), modelbalances2.m (model)

Results:

Algorithm
Average Suboptimality
Violations
Suboptimality at Final Experiment
Iterations to Convergence
Time
Trials

#
Name
M1 (λ := 1)
M2 (λ := 10)
M3 (λ := 100)
M4
M5 (λ := 1)
M6 (λ := 10)
M7 (λ := 100)
M8 (50%)
M9 (70%)
M10 (90%)
M11
 

0
nothing
0.5186
0.5186
0.5186
0
0.5186
0.5186
0.5186
NA
NA
NA
0
 

------ Model-Free Algorithms ------

1
SCFOv7
0.0455 ± 0.0118
0.0455 ± 0.0118
0.0455 ± 0.0118
0 ± 0
0.0023 ± 0.0031
0.0023 ± 0.0031
0.0023 ± 0.0031
3.63 ± 0.4828 (100%)
4.37 ± 0.9017 (100%)
6.61 ± 1.949 (100%)
15.63 ± 2.811
100

1.01
SCFOv8
0.051 ± 0.0125
0.051 ± 0.0125
0.051 ± 0.0125
0 ± 0
0.0035 ± 0.0041
0.0035 ± 0.0041
0.0035 ± 0.0041
3.81 ± 0.3927 (100%)
4.619 ± 0.95 (100%)
6.857 ± 1.082 (100%)
25.3 ± 9.668
21

1.02
SCFOv9
0.046 ± 0.0115
0.046 ± 0.0115
0.046 ± 0.0115
0 ± 0
0.0018 ± 0.0026
0.0018 ± 0.0026
0.0018 ± 0.0026
3.71 ± 0.4539 (100%)
4.355 ± 0.9688 (100%)
6.742 ± 2.016 (100%)
43.46 ± 8.657
31

2
SCFOv7f
0.0481 ± 0.0131
0.0481 ± 0.0131
0.0481 ± 0.0131
0 ± 0
0.0041 ± 0.0046
0.0041 ± 0.0046
0.0041 ± 0.0046
3.66 ± 0.4737 (100%)
4.19 ± 0.6737 (100%)
6.87 ± 2.524 (100%)
2.289 ± 0.6446
100

2.01
SCFOv8f
0.0481 ± 0.0126
0.0481 ± 0.0126
0.0481 ± 0.0126
0 ± 0
0.0034 ± 0.004
0.0034 ± 0.004
0.0034 ± 0.004
3.677 ± 0.4675 (100%)
4.28 ± 0.6111 (100%)
6.86 ± 2.593 (100%)
4.354 ± 3.327
93

2.02
SCFOv9f
0.0483 ± 0.013
0.0483 ± 0.013
0.0483 ± 0.013
0 ± 0
0.0033 ± 0.004
0.0033 ± 0.004
0.0033 ± 0.004
3.729 ± 0.4444 (100%)
4.271 ± 0.8098 (100%)
6.625 ± 2.579 (100%)
10.04 ± 2.334
48

3
RSOCCD
0.1993 ± 0.0003
0.1993 ± 0.0003
0.1993 ± 0.0003
0 ± 0
0.0049 ± 0.0004
0.0049 ± 0.0004
0.0049 ± 0.0004
10 ± 0 (100%)
10 ± 0 (100%)
10 ± 0 (100%)
0.1472 ± 0.0949
100

4
SimplexB
0.0733 ± 0.0192
0.0733 ± 0.0192
0.0733 ± 0.0192
0 ± 0
0.0025 ± 0.0048
0.0025 ± 0.0048
0.0025 ± 0.0048
6.51 ± 2.488 (100%)
9.67 ± 2.761 (100%)
14.66 ± 3.55 (100%)
0.0221 ± 0.0116
100

------ Algorithms Employing a Model ------

1
SCFOv7
0.0472 ± 0.0123
0.0472 ± 0.0123
0.0472 ± 0.0123
0 ± 0
0.0019 ± 0.0029
0.0019 ± 0.0029
0.0019 ± 0.0029
3.77 ± 0.4659 (100%)
4.35 ± 0.7124 (100%)
6.25 ± 1.417 (100%)
15.38 ± 2.242
100

1.01
SCFOv8
0.0475 ± 0.0131
0.0475 ± 0.0131
0.0475 ± 0.0131
0 ± 0
0.0017 ± 0.0027
0.0017 ± 0.0027
0.0017 ± 0.0027
3.55 ± 0.4975 (100%)
4.45 ± 0.8646 (100%)
5.8 ± 1.249 (100%)
33.91 ± 21.05
20

1.02
SCFOv9
0.047 ± 0.016
0.047 ± 0.016
0.047 ± 0.016
0 ± 0
0.0044 ± 0.0061
0.0044 ± 0.0061
0.0044 ± 0.0061
3.56 ± 0.4964 (100%)
4.36 ± 1.054 (100%)
6.64 ± 2.018 (100%)
43.48 ± 6.72
25

2
SCFOv7f
0.0527 ± 0.0135
0.0527 ± 0.0135
0.0527 ± 0.0135
0 ± 0
0.0035 ± 0.004
0.0035 ± 0.004
0.0035 ± 0.004
3.77 ± 0.4208 (100%)
4.54 ± 0.8297 (100%)
7.03 ± 3.756 (100%)
2.567 ± 0.579
100

2.01
SCFOv8f
0.0495 ± 0.0126
0.0495 ± 0.0126
0.0495 ± 0.0126
0 ± 0
0.0036 ± 0.0045
0.0036 ± 0.0045
0.0036 ± 0.0045
3.72 ± 0.449 (100%)
4.41 ± 0.8378 (100%)
6.79 ± 3.769 (100%)
6.816 ± 7.891
100

2.02
SCFOv9f
0.0506 ± 0.014
0.0506 ± 0.014
0.0506 ± 0.014
0 ± 0
0.0037 ± 0.0049
0.0037 ± 0.0049
0.0037 ± 0.0049
3.75 ± 0.433 (100%)
4.518 ± 0.8862 (100%)
6.429 ± 1.689 (100%)
10.63 ± 2.042
56


Test Problem #2: Maximizing Profit in the Williams-Otto Reactor (Constrained)

Original code provided by: S. Costello

Problem Description

Test File Specs: algotest([3.5 72],40,0.5,5e-4,[3 70],[6 100],[4.97 84.3],100,.1,[],algonum)

Main Files: phipeval.m (required), gpeval.m (required), phimod.m (model), gmod.m (model)

Auxiliary Files: plantbalancesT.m (required), modelbalances2.m (model)

Results:

Algorithm
Average Suboptimality
Violations
Suboptimality at Final Experiment
Iterations to Convergence
Time
Trials

#
Name
M1 (λ := 1)
M2 (λ := 10)
M3 (λ := 100)
M4
M5 (λ := 1)
M6 (λ := 10)
M7 (λ := 100)
M8 (50%)
M9 (70%)
M10 (90%)
M11
 

0
nothing
0.4051
0.4051
0.4051
0
0.4051
0.4051
0.4051
NA
NA
NA
0
 

------ Model-Free Algorithms ------

1
SCFOv7
0.1861 ± 0.0618
0.2147 ± 0.0721
0.501 ± 0.4092
5.23 ± 7.339
0.1572 ± 0.0639
0.1725 ± 0.0786
0.3255 ± 0.5731
18.84 ± 16.71 (67%)
29.79 ± 15.45 (39%)
41 ± 0 (0%)
24.3 ± 6.888
100

1.01
SCFOv8
0.1876 ± 0.0468
0.1919 ± 0.0469
0.2351 ± 0.0549
1.8 ± 2.098
0.1598 ± 0.0531
0.1609 ± 0.0538
0.1722 ± 0.0841
19.92 ± 15.05 (76%)
31.92 ± 14.31 (32%)
41 ± 0 (0%)
32.84 ± 14.67
25

1.02
SCFOv9
0.1798 ± 0.0471
0.1867 ± 0.0473
0.2561 ± 0.0947
2.684 ± 3.974
0.152 ± 0.053
0.152 ± 0.053
0.152 ± 0.053
17.89 ± 15.41 (74%)
31.84 ± 14.34 (32%)
41 ± 0 (0%)
54.02 ± 13.36
19

2
SCFOv7f
0.2138 ± 0.046
0.2399 ± 0.0487
0.5005 ± 0.233
4.89 ± 6.439
0.188 ± 0.0492
0.1947 ± 0.0526
0.2608 ± 0.2727
24.07 ± 16.98 (58%)
40.47 ± 3.708 (4%)
41 ± 0 (0%)
4.177 ± 1.321
100

2.01
SCFOv8f
0.2188 ± 0.0539
0.2245 ± 0.0528
0.2819 ± 0.0772
1.959 ± 2.382
0.1904 ± 0.0557
0.1907 ± 0.0563
0.1939 ± 0.0702
22.03 ± 16.68 (59%)
40.33 ± 3.475 (5%)
41 ± 0 (0%)
8.608 ± 6.848
98

2.02
SCFOv9f
0.2191 ± 0.0485
0.2238 ± 0.0483
0.2707 ± 0.0664
2.396 ± 3.414
0.193 ± 0.0512
0.1932 ± 0.0512
0.1956 ± 0.0525
25.56 ± 16.95 (50%)
39.79 ± 5.795 (4%)
41 ± 0 (0%)
15.75 ± 4.904
48

3
RSOCCD
0.2616 ± 0.0006
1.143 ± 0.0235
9.953 ± 0.2596
31.35 ± 11.07
0.002 ± 0.0008
0.0418 ± 0.031
0.4407 ± 0.3433
10 ± 0 (100%)
10 ± 0 (100%)
10 ± 0 (100%)
0.1729 ± 0.04
100

------ Algorithms Employing a Model ------

1
SCFOv7
0.2339 ± 0.0256
0.2339 ± 0.0256
0.234 ± 0.0256
0.01 ± 0.0995
0.19 ± 0.0311
0.19 ± 0.0311
0.19 ± 0.0311
26.09 ± 11.71 (73%)
40.93 ± 0.6965 (1%)
41 ± 0 (0%)
25.41 ± 5.737
100

1.01
SCFOv8
0.2335 ± 0.029
0.2335 ± 0.029
0.2335 ± 0.029
0.0476 ± 0.213
0.1899 ± 0.0346
0.1899 ± 0.0346
0.1899 ± 0.0346
28.05 ± 13.03 (62%)
39.76 ± 5.537 (5%)
41 ± 0 (0%)
35.55 ± 16.66
21

1.02
SCFOv9
0.2361 ± 0.0249
0.2362 ± 0.025
0.2371 ± 0.0262
0.1 ± 0.4359
0.1934 ± 0.0312
0.1934 ± 0.0312
0.1934 ± 0.0312
26.35 ± 12.89 (65%)
41 ± 0 (0%)
41 ± 0 (0%)
54.37 ± 8.946
20

2
SCFOv7f
0.2716 ± 0.0209
0.2716 ± 0.0209
0.2716 ± 0.0209
0 ± 0
0.2354 ± 0.0233
0.2354 ± 0.0233
0.2354 ± 0.0233
39.95 ± 4.879 (7%)
41 ± 0 (0%)
41 ± 0 (0%)
4.551 ± 1.164
100

2.01
SCFOv8f
0.2707 ± 0.0159
0.2707 ± 0.0159
0.2712 ± 0.0168
0.0515 ± 0.505
0.2339 ± 0.0216
0.2339 ± 0.0216
0.2339 ± 0.0216
39.9 ± 5.021 (6%)
41 ± 0 (0%)
41 ± 0 (0%)
10.34 ± 11.21
97

2.02
SCFOv9f
0.2726 ± 0.0208
0.2726 ± 0.0208
0.273 ± 0.0209
0.0952 ± 0.7499
0.2352 ± 0.0255
0.2352 ± 0.0255
0.2352 ± 0.0255
39.68 ± 5.203 (8%)
41 ± 0 (0%)
41 ± 0 (0%)
16.95 ± 3.936
63

5
ConAdapt
0.3156 ± 0.0022
0.3156 ± 0.0022
0.3156 ± 0.0022
0 ± 0
0.281 ± 0
0.281 ± 0
0.281 ± 0
41 ± 0 (0%)
41 ± 0 (0%)
41 ± 0 (0%)
2.846 ± 0.5429
100


Test Problem #3: Minimizing the Batch Time of Polystyrene Production

Original code provided by: G. François

Problem Description

Test File Specs: algotest([242.39 945.30],40,60,1e4,[50 600],[450 1000],[363.06 875.60],10000,1e6,[],algonum)

Main Files: phipeval.m (required), gpeval.m (required), phimod.m (model), gmod.m (model)

Auxiliary Files: integstyrbis.m (required), linint.m (required)

Results:

Algorithm
Average Suboptimality
Violations
Suboptimality at Final Experiment
Iterations to Convergence
Time
Trials

#
Name
M1 (λ := 1)
M2 (λ := 10)
M3 (λ := 100)
M4
M5 (λ := 1)
M6 (λ := 10)
M7 (λ := 100)
M8 (50%)
M9 (70%)
M10 (90%)
M11
 

0
nothing
0.1979
0.1979
0.1979
0
0.1979
0.1979
0.1979
NA
NA
NA
0
 

------ Model-Free Algorithms ------

1
SCFOv7
0.0296 ± 0.0058
0.0296 ± 0.0058
0.0296 ± 0.0058
0 ± 0
0.0092 ± 0.0039
0.0092 ± 0.0039
0.0092 ± 0.0039
4.21 ± 0.7911 (100%)
5.6 ± 1.393 (100%)
16.42 ± 9.593 (97%)
25.12 ± 6.884
100

1.01
SCFOv8
0.0304 ± 0.0054
0.0304 ± 0.0054
0.0304 ± 0.0054
0 ± 0
0.0095 ± 0.004
0.0095 ± 0.004
0.0095 ± 0.004
4.429 ± 0.6598 (100%)
5.714 ± 1.278 (100%)
15.9 ± 7.037 (100%)
34.02 ± 12.27
21

1.02
SCFOv9
0.029 ± 0.0054
0.029 ± 0.0054
0.029 ± 0.0054
0 ± 0
0.009 ± 0.0043
0.009 ± 0.0043
0.009 ± 0.0043
4.04 ± 0.8237 (100%)
5.24 ± 0.9912 (100%)
15.52 ± 7.643 (100%)
57.82 ± 15.52
25

2
SCFOv7f
0.0287 ± 0.0055
0.0287 ± 0.0055
0.0287 ± 0.0055
0 ± 0
0.0092 ± 0.004
0.0092 ± 0.004
0.0092 ± 0.004
4.27 ± 0.8468 (100%)
5.48 ± 1.403 (100%)
15.01 ± 8.352 (98%)
4.467 ± 1.417
100

2.01
SCFOv8f
0.0292 ± 0.0057
0.0292 ± 0.0057
0.0292 ± 0.0057
0 ± 0
0.0091 ± 0.0044
0.0091 ± 0.0044
0.0091 ± 0.0044
4.306 ± 1.074 (100%)
5.494 ± 1.081 (100%)
17.36 ± 10.68 (94%)
7.599 ± 5.235
85

2.02
SCFOv9f
0.0297 ± 0.0058
0.0297 ± 0.0058
0.0297 ± 0.0058
0 ± 0
0.0085 ± 0.0033
0.0085 ± 0.0033
0.0085 ± 0.0033
4.258 ± 0.8222 (100%)
5.53 ± 1.294 (100%)
14.86 ± 7.806 (100%)
17.36 ± 5.101
66

3
RSOCCD
0.104 ± 0.0009
0.5588 ± 0.0009
5.106 ± 0.0009
4 ± 0
0.0155 ± 0.0012
0.0155 ± 0.0012
0.0155 ± 0.0012
10 ± 0 (100%)
11 ± 0 (100%)
11 ± 0 (100%)
0.2738 ± 0.0533
100

------ Algorithms Employing a Model ------

1
SCFOv7
0.1015 ± 0.0409
0.1015 ± 0.0409
0.1015 ± 0.0409
0 ± 0
0.0485 ± 0.0364
0.0485 ± 0.0364
0.0485 ± 0.0364
21.13 ± 14 (82%)
31.33 ± 10.38 (57%)
38.81 ± 4.046 (28%)
33.87 ± 11.99
100

1.01
SCFOv8
0.094 ± 0.0348
0.094 ± 0.0348
0.094 ± 0.0348
0 ± 0
0.0435 ± 0.0296
0.0435 ± 0.0296
0.0435 ± 0.0296
19.42 ± 11.67 (92%)
30.25 ± 10.17 (67%)
38.83 ± 2.911 (42%)
61.01 ± 73.23
12

1.02
SCFOv9
0.1133 ± 0.0365
0.1133 ± 0.0365
0.1133 ± 0.0365
0 ± 0
0.0534 ± 0.0372
0.0534 ± 0.0372
0.0534 ± 0.0372
25.24 ± 12.29 (88%)
33.8 ± 8.518 (48%)
39.48 ± 3.336 (24%)
66.83 ± 11.07
25

2
SCFOv7f
0.1131 ± 0.0372
0.1131 ± 0.0372
0.1131 ± 0.0372
0 ± 0
0.0519 ± 0.0364
0.0519 ± 0.0364
0.0519 ± 0.0364
25.49 ± 12.28 (81%)
34.29 ± 7.672 (56%)
39.83 ± 2.642 (21%)
17.78 ± 3.502
100

2.01
SCFOv8f
0.1101 ± 0.0382
0.1101 ± 0.0382
0.1101 ± 0.0382
0 ± 0
0.052 ± 0.0352
0.052 ± 0.0352
0.052 ± 0.0352
23.82 ± 12.69 (87%)
33.95 ± 8.552 (52%)
39.65 ± 3.465 (20%)
65.18 ± 90
92

2.02
SCFOv9f
0.1149 ± 0.0376
0.1149 ± 0.0376
0.1149 ± 0.0376
0 ± 0
0.0538 ± 0.0335
0.0538 ± 0.0335
0.0538 ± 0.0335
25.65 ± 12.82 (81%)
34.73 ± 8.039 (50%)
39.75 ± 3.143 (19%)
53.07 ± 9.349
52

5
ConAdapt
0.0229 ± 0.0033
0.1318 ± 0.0225
1.221 ± 0.2184
23.3 ± 2.022
0.0231 ± 0.0187
0.1801 ± 0.2337
1.75 ± 2.407
2 ± 0 (100%)
3.29 ± 5.933 (99%)
33.43 ± 9.512 (64%)
56.46 ± 12.3
100


Test Problem #4: Maximizing Electrical Efficiency in a Solid Oxide Fuel Cell Stack

Original code provided by: A. Gopalakrishnan

Problem Description

Test File Specs: algotest([2e-3 7e-3 26],60,6e-4,[0.1 0.1 1e-3 0.03],[1e-3 1.5e-3 1],[1e-2 3.5e-2 30],[1e-3 1.898e-3 23.6687],.1,[50 50 .1 50],[.01 .01 .01],algonum)

Main Files: phipeval.m (required), gpeval.m (required), geval.m (required), phimod.m (model), gmod.m (model)

Auxiliary Files: model_real.m (required), stack.m (required), model.m (model)

Results:

Algorithm
Average Suboptimality
Violations
Suboptimality at Final Experiment
Iterations to Convergence
Time
Trials

#
Name
M1 (λ := 1)
M2 (λ := 10)
M3 (λ := 100)
M4
M5 (λ := 1)
M6 (λ := 10)
M7 (λ := 100)
M8 (50%)
M9 (70%)
M10 (90%)
M11
 

0
nothing
1.642
1.642
1.642
0
1.642
1.642
1.642
NA
NA
NA
0
 

------ Model-Free Algorithms ------

1
SCFOv7
0.4251 ± 0.1246
0.9484 ± 0.504
6.181 ± 5.556
33.22 ± 13.18
0.3077 ± 0.11
1.086 ± 0.9399
8.874 ± 10.46
6.73 ± 5.223 (100%)
15.43 ± 16.13 (96%)
50.75 ± 17.33 (29%)
77.28 ± 24.94
100

1.01
SCFOv8
0.5283 ± 0.0933
0.54 ± 0.0899
0.6569 ± 0.1868
3.3 ± 5.64
0.4223 ± 0.1797
0.4245 ± 0.1777
0.4464 ± 0.1706
30.6 ± 20.89 (100%)
61 ± 0 (0%)
61 ± 0 (0%)
125.2 ± 45.35
10

1.02
SCFOv9
0.5477 ± 0.0975
0.5759 ± 0.1004
0.8579 ± 0.2362
2.52 ± 1.769
0.3696 ± 0.1397
0.3696 ± 0.1397
0.3696 ± 0.1397
19.12 ± 18.39 (96%)
59.52 ± 3.667 (20%)
60.96 ± 0.196 (0%)
167.5 ± 18.45
25

2
SCFOv7f
0.4341 ± 0.1151
0.8585 ± 0.5041
5.103 ± 5.326
30.25 ± 13.37
0.3204 ± 0.1096
0.9128 ± 0.9955
6.837 ± 10.85
6.57 ± 3.827 (100%)
14.52 ± 14.51 (95%)
54.91 ± 13.85 (24%)
23.86 ± 5.334
100

2.01
SCFOv8f
0.555 ± 0.1026
0.5771 ± 0.11
0.7981 ± 0.2787
2.152 ± 2.429
0.453 ± 0.1983
0.4549 ± 0.1974
0.474 ± 0.2331
26.19 ± 21.41 (94%)
60.35 ± 1.103 (18%)
61 ± 0 (0%)
44.73 ± 13.95
79

2.02
SCFOv9f
0.5358 ± 0.1134
0.5534 ± 0.1161
0.7286 ± 0.2128
1.735 ± 1.208
0.4458 ± 0.1838
0.4458 ± 0.1838
0.4458 ± 0.1838
24.73 ± 21.25 (92%)
59.59 ± 3.194 (24%)
61 ± 0 (0%)
70.17 ± 15
49

3
RSOCCD
2.497 ± 0.0228
7.862 ± 0.0228
61.52 ± 0.0228
21 ± 0
0.5928 ± 0.0309
0.5928 ± 0.0309
0.5928 ± 0.0309
17 ± 0 (100%)
61 ± 0 (0%)
61 ± 0 (0%)
3.328 ± 0.7407
100

------ Algorithms Employing a Model ------

1
SCFOv7
0.6865 ± 0.2016
0.7057 ± 0.1936
0.8985 ± 0.6854
5.12 ± 10.37
0.4142 ± 0.1171
0.4434 ± 0.1492
0.7357 ± 1.361
17.62 ± 11.03 (99%)
38.68 ± 21.48 (72%)
60.51 ± 3.814 (4%)
84.1 ± 22.95
100

1.01
SCFOv8
0.6404 ± 0.1532
0.6423 ± 0.1529
0.6614 ± 0.1549
0.8 ± 1.558
0.3391 ± 0.2431
0.3391 ± 0.2431
0.3391 ± 0.2431
30.67 ± 17.37 (100%)
59.6 ± 2.551 (33%)
60.93 ± 0.2494 (0%)
121.4 ± 44.83
15

1.02
SCFOv9
0.6518 ± 0.1605
0.6548 ± 0.1603
0.6842 ± 0.1681
0.76 ± 1.632
0.3896 ± 0.2394
0.3896 ± 0.2394
0.3896 ± 0.2394
32.64 ± 18.15 (96%)
59.84 ± 1.759 (28%)
60.72 ± 0.7222 (8%)
167.5 ± 20.86
25

2
SCFOv7f
0.8029 ± 0.2806
0.8383 ± 0.2496
1.192 ± 1.183
4.88 ± 11.07
0.4437 ± 0.1474
0.5186 ± 0.2196
1.268 ± 2.589
24.42 ± 14.31 (98%)
46.21 ± 18.18 (62%)
59.85 ± 6.842 (3%)
19.2 ± 4.378
100

2.01
SCFOv8f
0.6266 ± 0.1484
0.6294 ± 0.1479
0.6579 ± 0.1515
0.6341 ± 1.065
0.3018 ± 0.2259
0.3018 ± 0.2259
0.3018 ± 0.2259
31.78 ± 17.03 (93%)
58.78 ± 2.604 (41%)
60.56 ± 0.8848 (11%)
47.83 ± 26.76
82

2.02
SCFOv9f
0.6136 ± 0.1517
0.615 ± 0.151
0.6298 ± 0.1467
0.4921 ± 0.99
0.35 ± 0.2133
0.35 ± 0.2133
0.35 ± 0.2133
29.08 ± 16.83 (95%)
58.16 ± 5.171 (51%)
60.83 ± 0.6053 (5%)
65.38 ± 12.04
63

5
ConAdapt
0.4985 ± 0.0027
0.526 ± 0.0043
0.8003 ± 0.0448
15.22 ± 4.836
0.4818 ± 0.0148
0.4899 ± 0.0199
0.5711 ± 0.2085
2 ± 0 (100%)
48.46 ± 14.44 (69%)
61 ± 0 (0%)
3.315 ± 0.6327
100


Test Problem #5: Minimizing the Overall Pumping Effort in the "Trois Bacs"

Original code provided by: G. A. Bunin

Problem Description

Test File Specs: algotest([6 6.2],40,0,[0.2 0.2 0.2 0.2 0.2 0.2],[0 0],[8 8],[4.9481 4.4160],40,[20 20 20 20 20 20],[],algonum)

Main Files: phieval.m (required), gpeval.m (required), gmod.m (model)

Results:

Algorithm
Average Suboptimality
Violations
Suboptimality at Final Experiment
Iterations to Convergence
Time
Trials

#
Name
M1 (λ := 1)
M2 (λ := 10)
M3 (λ := 100)
M4
M5 (λ := 1)
M6 (λ := 10)
M7 (λ := 100)
M8 (50%)
M9 (70%)
M10 (90%)
M11
 

0
nothing
0.7614
0.7614
0.7614
0
0.7614
0.7614
0.7614
NA
NA
NA
0
 

------ Model-Free Algorithms ------

1
SCFOv7
0.1316 ± 0.0259
0.1318 ± 0.0258
0.1331 ± 0.0269
0.28 ± 1.898
0.0816 ± 0.0356
0.0822 ± 0.0352
0.0878 ± 0.0526
3 ± 0 (100%)
6.46 ± 6.002 (100%)
39.98 ± 2.293 (22%)
49.18 ± 10.74
100

1.01
SCFOv8
0.1266 ± 0.0253
0.1266 ± 0.0253
0.1266 ± 0.0253
0 ± 0
0.0688 ± 0.0332
0.0688 ± 0.0332
0.0688 ± 0.0332
3 ± 0 (100%)
4.85 ± 2.594 (100%)
39.1 ± 3.39 (35%)
48.69 ± 18.55
20

1.02
SCFOv9
0.1242 ± 0.0224
0.1242 ± 0.0224
0.1242 ± 0.0224
0 ± 0
0.0687 ± 0.0292
0.0687 ± 0.0292
0.0687 ± 0.0292
3 ± 0 (100%)
4.095 ± 0.2935 (100%)
39 ± 2.862 (38%)
71.04 ± 6.238
21

2
SCFOv7f
0.1302 ± 0.0294
0.1306 ± 0.0288
0.1341 ± 0.0308
0.56 ± 2.892
0.0786 ± 0.0376
0.0797 ± 0.0362
0.0904 ± 0.0633
3 ± 0 (100%)
5.42 ± 3.798 (100%)
39.12 ± 5.208 (26%)
15.17 ± 2.593
100

2.01
SCFOv8f
0.1264 ± 0.0273
0.1264 ± 0.0273
0.1265 ± 0.0272
0.0337 ± 0.1805
0.0774 ± 0.0366
0.0774 ± 0.0366
0.0774 ± 0.0366
3 ± 0 (100%)
4.663 ± 2.44 (100%)
38.34 ± 5.798 (37%)
39.52 ± 21.48
89

2.02
SCFOv9f
0.1235 ± 0.0281
0.124 ± 0.027
0.1285 ± 0.0318
0.5122 ± 2.931
0.0734 ± 0.0378
0.0734 ± 0.0378
0.0734 ± 0.0378
3 ± 0 (100%)
4.122 ± 0.3948 (100%)
37.8 ± 6.733 (29%)
70.89 ± 20.29
41

3
RSOCCD
0.1619 ± 0.0203
0.7506 ± 0.0185
6.638 ± 0.0644
18.48 ± 8.41
0.0354 ± 0.0269
0.0374 ± 0.0244
0.0572 ± 0.0851
10 ± 0 (100%)
10 ± 0 (100%)
12.17 ± 7.91 (93%)
0.1592 ± 0.0479
100

------ Algorithms Employing a Model ------

1
SCFOv7
0.1932 ± 0.2072
0.2014 ± 0.2028
0.2835 ± 0.2256
7.19 ± 10.01
0.146 ± 0.2082
0.1534 ± 0.2039
0.2273 ± 0.2657
8.05 ± 11.8 (89%)
14.2 ± 15.05 (82%)
35.28 ± 10.48 (36%)
55.93 ± 17.49
100

1.01
SCFOv8
0.2161 ± 0.1536
0.222 ± 0.1502
0.2812 ± 0.1535
5 ± 7.632
0.1396 ± 0.1726
0.143 ± 0.1713
0.1765 ± 0.1983
21.75 ± 16.48 (83%)
25.58 ± 15.14 (79%)
37.58 ± 5.484 (54%)
76.68 ± 34.37
24

1.02
SCFOv9
0.1764 ± 0.1582
0.1807 ± 0.1558
0.2239 ± 0.1571
5.333 ± 7.73
0.1337 ± 0.1645
0.138 ± 0.1613
0.1816 ± 0.1702
6.762 ± 11.11 (90%)
13.29 ± 14.76 (86%)
33.67 ± 12.2 (38%)
81.71 ± 15.48
21

2
SCFOv7f
0.1845 ± 0.18
0.1921 ± 0.1761
0.2679 ± 0.2038
6.44 ± 9.455
0.1323 ± 0.1757
0.1401 ± 0.1714
0.2179 ± 0.2565
9.05 ± 12.83 (88%)
12.99 ± 14.55 (81%)
35.55 ± 10.8 (32%)
16.65 ± 2.918
100

2.01
SCFOv8f
0.1886 ± 0.1602
0.1955 ± 0.1569
0.2646 ± 0.1747
5.667 ± 7.467
0.1454 ± 0.1786
0.1483 ± 0.177
0.1778 ± 0.1953
14 ± 15.35 (81%)
18.94 ± 16.02 (74%)
36.85 ± 9.505 (25%)
48.24 ± 38.34
81

2.02
SCFOv9f
0.2121 ± 0.2309
0.2208 ± 0.226
0.3079 ± 0.2222
7.915 ± 10.45
0.1605 ± 0.2285
0.1678 ± 0.2241
0.2406 ± 0.2605
10.74 ± 14.67 (83%)
13.53 ± 15.04 (79%)
33.68 ± 12.07 (40%)
68.79 ± 11.59
47

5
ConAdapt
0.1252 ± 0.0283
0.1817 ± 0.0292
0.7466 ± 0.1331
21.86 ± 3.302
0.1403 ± 0.1359
0.1741 ± 0.1203
0.5115 ± 0.5949
21.66 ± 14.56 (86%)
35.69 ± 6.148 (63%)
40.38 ± 1.377 (15%)
2.757 ± 0.511
100


Test Problem #6: Maximizing Production in a Continuous Stirred-Tank Reactor

Original code provided by: G. A. Bunin

Problem Description

Test File Specs: algotest([14.52 14.9],40,0.1,[0.03 0.03],[1 1],[50 50],[17.2 30.3],10,[1 1],[],algonum)

Main Files: phipeval.m (required), gpeval.m (required), phimod.m (model), gmod.m (model)

Results:

Algorithm
Average Suboptimality
Violations
Suboptimality at Final Experiment
Iterations to Convergence
Time
Trials

#
Name
M1 (λ := 1)
M2 (λ := 10)
M3 (λ := 100)
M4
M5 (λ := 1)
M6 (λ := 10)
M7 (λ := 100)
M8 (50%)
M9 (70%)
M10 (90%)
M11
 

0
nothing
0.6318
0.6318
0.6318
0
0.6318
0.6318
0.6318
NA
NA
NA
0
 

------ Model-Free Algorithms ------

1
SCFOv7
0.1313 ± 0.051
0.1732 ± 0.0943
0.5921 ± 0.6962
1.39 ± 1.984
0.0371 ± 0.0202
0.0446 ± 0.0598
0.1192 ± 0.5476
5.85 ± 3.623 (100%)
8.79 ± 5.93 (100%)
29.13 ± 10.7 (81%)
35.67 ± 8.848
100

1.01
SCFOv8
0.1564 ± 0.057
0.2252 ± 0.1298
0.9134 ± 0.9648
1.158 ± 0.6699
0.0478 ± 0.0181
0.0478 ± 0.0181
0.0478 ± 0.0181
7.158 ± 4.463 (100%)
11.95 ± 7.193 (100%)
32.11 ± 9.403 (58%)
58.16 ± 19.74
19

1.02
SCFOv9
0.1307 ± 0.0623
0.1599 ± 0.1003
0.4527 ± 0.6137
0.8387 ± 0.7228
0.0408 ± 0.0265
0.0408 ± 0.0265
0.0408 ± 0.0265
5.71 ± 5.043 (100%)
9.226 ± 7.564 (100%)
28.06 ± 11.14 (74%)
70.2 ± 4.428
31

2
SCFOv7f
0.15 ± 0.065
0.2058 ± 0.1147
0.7629 ± 0.7844
1.3 ± 1.204
0.0422 ± 0.0297
0.0422 ± 0.0297
0.0422 ± 0.0297
7.02 ± 4.461 (100%)
10.86 ± 7.746 (100%)
29.05 ± 11.09 (72%)
3.289 ± 0.6527
100

2.01
SCFOv8f
0.1546 ± 0.0672
0.2005 ± 0.1131
0.6599 ± 0.7348
1.115 ± 1.143
0.0473 ± 0.0342
0.0473 ± 0.0342
0.0473 ± 0.0342
7.128 ± 4.589 (100%)
12.46 ± 9.266 (100%)
30.14 ± 10.55 (65%)
6.137 ± 3.104
78

2.02
SCFOv9f
0.1438 ± 0.0544
0.1703 ± 0.0758
0.4348 ± 0.4221
1.178 ± 0.9727
0.0456 ± 0.0292
0.0456 ± 0.0292
0.0456 ± 0.0292
5.844 ± 3.881 (100%)
9.933 ± 7.558 (98%)
30.4 ± 10.68 (67%)
11.42 ± 1.614
45

3
RSOCCD
0.8144 ± 0.0066
3.743 ± 0.1388
33.02 ± 1.566
37 ± 0
0.2797 ± 0.0087
0.9416 ± 0.1836
7.56 ± 2.071
11 ± 0 (100%)
33.2 ± 13.16 (26%)
41 ± 0 (0%)
0.2417 ± 0.0519
100

------ Algorithms Employing a Model ------

1
SCFOv7
0.3513 ± 0.1476
0.3513 ± 0.1476
0.3515 ± 0.1474
0.02 ± 0.14
0.1711 ± 0.164
0.1711 ± 0.164
0.1711 ± 0.164
21.63 ± 12.37 (81%)
29.57 ± 11.52 (63%)
39.13 ± 4.326 (23%)
35.44 ± 8.884
100

1.01
SCFOv8
0.2705 ± 0.0502
0.2705 ± 0.0502
0.2705 ± 0.0502
0 ± 0
0.1119 ± 0.045
0.1119 ± 0.045
0.1119 ± 0.045
13.64 ± 5.05 (100%)
26.64 ± 8.321 (91%)
40.14 ± 3.958 (5%)
55.9 ± 22.87
22

1.02
SCFOv9
0.2637 ± 0.0758
0.2637 ± 0.0758
0.2637 ± 0.0758
0 ± 0
0.0929 ± 0.0616
0.0929 ± 0.0616
0.0929 ± 0.0616
15.5 ± 10.08 (100%)
25 ± 10.01 (86%)
39.64 ± 3.014 (21%)
81.71 ± 15.29
14

2
SCFOv7f
0.3189 ± 0.1334
0.3189 ± 0.1334
0.3193 ± 0.133
0.04 ± 0.2417
0.136 ± 0.1333
0.136 ± 0.1333
0.136 ± 0.1333
19.43 ± 11.29 (87%)
26.12 ± 11.44 (75%)
39.01 ± 5.11 (21%)
5.491 ± 1.23
100

2.01
SCFOv8f
0.2891 ± 0.109
0.2891 ± 0.109
0.2891 ± 0.109
0 ± 0
0.1268 ± 0.1092
0.1268 ± 0.1092
0.1268 ± 0.1092
17.34 ± 11.3 (90%)
24.77 ± 10.96 (77%)
39.83 ± 3.464 (16%)
14.7 ± 14.22
100

2.02
SCFOv9f
0.3077 ± 0.1261
0.3077 ± 0.1261
0.3077 ± 0.1261
0 ± 0
0.1365 ± 0.139
0.1365 ± 0.139
0.1365 ± 0.139
18.25 ± 10.47 (92%)
26.2 ± 10.97 (76%)
39.75 ± 3.4 (16%)
20.23 ± 4.29
51

5
ConAdapt
0.3611 ± 0
0.3611 ± 0
0.3611 ± 0
0 ± 0
0.3543 ± 0
0.3543 ± 0
0.3543 ± 0
41 ± 0 (0%)
41 ± 0 (0%)
41 ± 0 (0%)
4.369 ± 0.805
100


Test Problem #7: Maximizing Production in a Batch Reactor with a Reversible Reaction

Original code provided by: G. A. Bunin

Problem Description

Test File Specs: algotest([0 0],40,0.01,[],[-1 -1],[1 1],[-0.2884 -1],0.1,[],[],algonum)

Main Files: phipeval.m (required), phimod.m (model)

Results:

Algorithm
Average Suboptimality
Violations
Suboptimality at Final Experiment
Iterations to Convergence
Time
Trials

#
Name
M1 (λ := 1)
M2 (λ := 10)
M3 (λ := 100)
M4
M5 (λ := 1)
M6 (λ := 10)
M7 (λ := 100)
M8 (50%)
M9 (70%)
M10 (90%)
M11
 

0
nothing
0.873
0.873
0.873
0
0.873
0.873
0.873
NA
NA
NA
0
 

------ Model-Free Algorithms ------

1
SCFOv7
0.185 ± 0.0557
0.185 ± 0.0557
0.185 ± 0.0557
0.01 ± 0.0995
0.0502 ± 0.0497
0.0502 ± 0.0497
0.0502 ± 0.0497
7.48 ± 3.976 (100%)
11.52 ± 6.579 (99%)
34.47 ± 8.06 (57%)
17.27 ± 3.774
100

1.01
SCFOv8
0.1964 ± 0.0568
0.1964 ± 0.0568
0.1964 ± 0.0568
0 ± 0
0.0527 ± 0.043
0.0527 ± 0.043
0.0527 ± 0.043
7.526 ± 3.574 (100%)
10.42 ± 4.345 (100%)
35.42 ± 9.326 (42%)
28.4 ± 9.71
19

1.02
SCFOv9
0.192 ± 0.0823
0.192 ± 0.0823
0.192 ± 0.0823
0 ± 0
0.0578 ± 0.064
0.0578 ± 0.064
0.0578 ± 0.064
7.125 ± 4.146 (100%)
12.58 ± 8.756 (100%)
35.71 ± 7.971 (46%)
38.96 ± 3.21
24

2
SCFOv7f
0.2037 ± 0.077
0.2037 ± 0.077
0.2037 ± 0.077
0.04 ± 0.196
0.0584 ± 0.0572
0.0584 ± 0.0572
0.0584 ± 0.0572
8.31 ± 5.692 (100%)
13.53 ± 7.866 (97%)
36.7 ± 6.485 (47%)
2.418 ± 0.5273
100

2.01
SCFOv8f
0.1932 ± 0.0703
0.1932 ± 0.0703
0.1932 ± 0.0703
0 ± 0
0.0487 ± 0.0506
0.0487 ± 0.0506
0.0487 ± 0.0506
7.582 ± 4.363 (100%)
12.58 ± 7.219 (100%)
36.01 ± 6.121 (63%)
4.326 ± 3.066
91

2.02
SCFOv9f
0.1854 ± 0.0684
0.1854 ± 0.0684
0.1854 ± 0.0684
0.0926 ± 0.3479
0.0486 ± 0.05
0.0486 ± 0.05
0.0486 ± 0.05
7.852 ± 5.042 (100%)
12.22 ± 8.098 (98%)
35.46 ± 6.978 (65%)
10.62 ± 1.691
54

3
RSOCCD
0.3242 ± 0.0453
0.3242 ± 0.0453
0.3242 ± 0.0453
0 ± 0
0.0733 ± 0.0599
0.0733 ± 0.0599
0.0733 ± 0.0599
11 ± 0 (100%)
11 ± 0 (100%)
24.2 ± 14.89 (56%)
0.1048 ± 0.0418
100

4
SimplexB
0.263 ± 0.1955
0.263 ± 0.1955
0.263 ± 0.1955
0 ± 0
0.1626 ± 0.2301
0.1626 ± 0.2301
0.1626 ± 0.2301
12.09 ± 12.08 (86%)
13.91 ± 12.13 (84%)
24.54 ± 14.08 (60%)
0.0202 ± 0.0076
100

------ Algorithms Employing a Model ------

1
SCFOv7
0.1717 ± 0.0216
0.1717 ± 0.0216
0.1717 ± 0.0216
0.06 ± 0.3693
0.1077 ± 0.0314
0.1077 ± 0.0314
0.1077 ± 0.0314
4.9 ± 0.6083 (100%)
5.96 ± 0.6621 (100%)
40.63 ± 1.653 (8%)
18.3 ± 4.283
100

1.01
SCFOv8
0.1735 ± 0.0299
0.1735 ± 0.0299
0.1735 ± 0.0299
0.0741 ± 0.2619
0.0804 ± 0.0379
0.0804 ± 0.0379
0.0804 ± 0.0379
5.074 ± 0.9399 (100%)
6.259 ± 0.8858 (100%)
39.93 ± 1.999 (22%)
26.06 ± 21.07
27

1.02
SCFOv9
0.171 ± 0.0247
0.171 ± 0.0247
0.171 ± 0.0247
0.0606 ± 0.2386
0.0909 ± 0.0414
0.0909 ± 0.0414
0.0909 ± 0.0414
4.848 ± 0.4998 (100%)
5.879 ± 0.537 (100%)
39.67 ± 3.461 (15%)
39.4 ± 7.107
33

2
SCFOv7f
0.1317 ± 0.0234
0.1317 ± 0.0234
0.1317 ± 0.0234
0.01 ± 0.0995
0.0354 ± 0.0375
0.0354 ± 0.0375
0.0354 ± 0.0375
5.28 ± 1.068 (100%)
7.46 ± 3.887 (100%)
33.52 ± 9.064 (70%)
8.457 ± 2.415
100

2.01
SCFOv8f
0.135 ± 0.0354
0.135 ± 0.0354
0.135 ± 0.0354
0.023 ± 0.1499
0.0337 ± 0.0363
0.0337 ± 0.0363
0.0337 ± 0.0363
5.609 ± 1.593 (100%)
7.356 ± 2.112 (100%)
32.25 ± 9.2 (72%)
23.44 ± 27.47
87

2.02
SCFOv9f
0.1291 ± 0.0303
0.1291 ± 0.0303
0.1291 ± 0.0303
0.0377 ± 0.1906
0.0411 ± 0.0421
0.0411 ± 0.0421
0.0411 ± 0.0421
5.151 ± 1.365 (100%)
7.132 ± 3.905 (100%)
33.34 ± 9.691 (62%)
26.13 ± 7.733
53


Test Problem #8: Maximizing Production in a Fed-Batch Reactor with Three Reactions

Original code provided by: G. A. Bunin

Problem Description

Test File Specs: algotest([0 0 0],60,0.01,[],[-1 -0.5 -1.5],[1 0.5 1.5],[0.1953 0.3770 -1.0027],0.1,[],[1 1 1 1],algonum)

Main Files: phipeval.m (required), geval.m (required), phimod.m (model)

Results:

Algorithm
Average Suboptimality
Violations
Suboptimality at Final Experiment
Iterations to Convergence
Time
Trials

#
Name
M1 (λ := 1)
M2 (λ := 10)
M3 (λ := 100)
M4
M5 (λ := 1)
M6 (λ := 10)
M7 (λ := 100)
M8 (50%)
M9 (70%)
M10 (90%)
M11
 

0
nothing
0.5821
0.5821
0.5821
0
0.5821
0.5821
0.5821
NA
NA
NA
0
 

------ Model-Free Algorithms ------

1
SCFOv7
0.1734 ± 0.0331
0.1734 ± 0.0331
0.1734 ± 0.0331
0 ± 0
0.1424 ± 0.0653
0.1424 ± 0.0653
0.1424 ± 0.0653
30.24 ± 21.28 (92%)
57.1 ± 9.224 (43%)
60.93 ± 0.4529 (2%)
18.91 ± 2.506
100

1.01
SCFOv8
0.1815 ± 0.0447
0.1815 ± 0.0447
0.1815 ± 0.0447
0 ± 0
0.1401 ± 0.0726
0.1401 ± 0.0726
0.1401 ± 0.0726
30.56 ± 18.53 (94%)
54.67 ± 14.66 (33%)
61 ± 0 (0%)
32.43 ± 13.48
18

1.02
SCFOv9
0.1781 ± 0.0325
0.1781 ± 0.0325
0.1781 ± 0.0325
0 ± 0
0.1432 ± 0.0591
0.1432 ± 0.0591
0.1432 ± 0.0591
32.88 ± 20.31 (100%)
54.88 ± 11.65 (38%)
61 ± 0 (0%)
45.39 ± 10.22
26

2
SCFOv7f
0.1757 ± 0.0373
0.1757 ± 0.0373
0.1757 ± 0.0373
0 ± 0
0.1329 ± 0.068
0.1329 ± 0.068
0.1329 ± 0.068
28.63 ± 20.58 (93%)
56.95 ± 7.896 (48%)
60.96 ± 0.3137 (1%)
3.465 ± 0.9567
100

2.01
SCFOv8f
0.1717 ± 0.0343
0.1717 ± 0.0343
0.1717 ± 0.0343
0 ± 0
0.1399 ± 0.0674
0.1399 ± 0.0674
0.1399 ± 0.0674
29.27 ± 21.78 (88%)
57.19 ± 8.93 (42%)
60.98 ± 0.1474 (0%)
6.124 ± 4.557
90

2.02
SCFOv9f
0.1841 ± 0.0392
0.1841 ± 0.0392
0.1841 ± 0.0392
0 ± 0
0.1357 ± 0.0586
0.1357 ± 0.0586
0.1357 ± 0.0586
27.39 ± 21.7 (95%)
57.32 ± 8.64 (37%)
60.98 ± 0.1543 (0%)
12.47 ± 2.356
41

3
RSOCCD
1.314 ± 0.0042
6.826 ± 0.0042
61.95 ± 0.0042
17 ± 0
0.179 ± 0.0058
0.179 ± 0.0058
0.179 ± 0.0058
17 ± 0 (100%)
50.88 ± 18.52 (23%)
61 ± 0 (0%)
0.1574 ± 0.0618
100

------ Algorithms Employing a Model ------

1
SCFOv7
0.188 ± 0.0392
0.188 ± 0.0392
0.188 ± 0.0392
0 ± 0
0.1464 ± 0.0546
0.1464 ± 0.0546
0.1464 ± 0.0546
22 ± 20.9 (95%)
58.13 ± 6.28 (38%)
61 ± 0 (0%)
22.57 ± 3.86
100

1.01
SCFOv8
0.1864 ± 0.0337
0.1864 ± 0.0337
0.1864 ± 0.0337
0 ± 0
0.1356 ± 0.0366
0.1356 ± 0.0366
0.1356 ± 0.0366
21.25 ± 20.08 (100%)
58.67 ± 2.593 (50%)
61 ± 0 (0%)
62.29 ± 47.19
12

1.02
SCFOv9
0.1888 ± 0.0412
0.1888 ± 0.0412
0.1888 ± 0.0412
0 ± 0
0.14 ± 0.044
0.14 ± 0.044
0.14 ± 0.044
22.43 ± 18.17 (96%)
58.71 ± 5.737 (29%)
61 ± 0 (0%)
47.27 ± 6.856
28

2
SCFOv7f
0.186 ± 0.0435
0.186 ± 0.0435
0.186 ± 0.0435
0 ± 0
0.1333 ± 0.0516
0.1333 ± 0.0516
0.1333 ± 0.0516
23.23 ± 20.92 (94%)
53.92 ± 14.32 (41%)
60.99 ± 0.0995 (0%)
8.779 ± 1.5
100

2.01
SCFOv8f
0.175 ± 0.0345
0.175 ± 0.0345
0.175 ± 0.0345
0 ± 0
0.1382 ± 0.0532
0.1382 ± 0.0532
0.1382 ± 0.0532
17.81 ± 18.99 (96%)
52.79 ± 15.02 (53%)
61 ± 0 (0%)
29.29 ± 38.2
70

2.02
SCFOv9f
0.1845 ± 0.0351
0.1845 ± 0.0351
0.1845 ± 0.0351
0 ± 0
0.1377 ± 0.0466
0.1377 ± 0.0466
0.1377 ± 0.0466
21.85 ± 20.72 (93%)
55.1 ± 12.73 (43%)
61 ± 0 (0%)
23.54 ± 3.126
40


Test Problem #9: Batch-to-Batch Tuning of a Temperature-Tracking Model-Predictive Controller

Original code provided by: G. A. Bunin

Problem Description

Test File Specs: algotest([0 0.7],40,0.1,[],[-2 0],[2 1],[0.9492 0.6416],1,[],[],algonum)

Main Files: phipeval.m (required), phimod.m (model)

Results:

Algorithm
Average Suboptimality
Violations
Suboptimality at Final Experiment
Iterations to Convergence
Time
Trials

#
Name
M1 (λ := 1)
M2 (λ := 10)
M3 (λ := 100)
M4
M5 (λ := 1)
M6 (λ := 10)
M7 (λ := 100)
M8 (50%)
M9 (70%)
M10 (90%)
M11
 

0
nothing
0.9232
0.9232
0.9232
0
0.9232
0.9232
0.9232
NA
NA
NA
0
 

------ Model-Free Algorithms ------

1
SCFOv7
0.1454 ± 0.0398
0.1454 ± 0.0398
0.1454 ± 0.0398
0 ± 0
0.0826 ± 0.0968
0.0826 ± 0.0968
0.0826 ± 0.0968
9.72 ± 8.888 (100%)
25.92 ± 12.39 (86%)
39.19 ± 2.982 (33%)
19.9 ± 6.998
100

1.01
SCFOv8
0.1323 ± 0.0413
0.1323 ± 0.0413
0.1323 ± 0.0413
0 ± 0
0.0613 ± 0.089
0.0613 ± 0.089
0.0613 ± 0.089
8.794 ± 7.557 (100%)
25.65 ± 12.95 (85%)
38.94 ± 2.028 (50%)
26.6 ± 10.5
34

1.02
SCFOv9
0.1406 ± 0.0439
0.1406 ± 0.0439
0.1406 ± 0.0439
0 ± 0
0.045 ± 0.0604
0.045 ± 0.0604
0.045 ± 0.0604
8.153 ± 7.32 (100%)
26.98 ± 10.92 (92%)
38.54 ± 3.072 (47%)
41.36 ± 6.586
59

2
SCFOv7f
0.1422 ± 0.0348
0.1422 ± 0.0348
0.1422 ± 0.0348
0 ± 0
0.0707 ± 0.1021
0.0707 ± 0.1021
0.0707 ± 0.1021
10.78 ± 9.759 (97%)
28.94 ± 10.31 (86%)
39.3 ± 2.456 (40%)
2.242 ± 0.4342
100

2.01
SCFOv8f
0.1416 ± 0.0393
0.1416 ± 0.0393
0.1416 ± 0.0393
0 ± 0
0.0605 ± 0.0867
0.0605 ± 0.0867
0.0605 ± 0.0867
8.63 ± 8.511 (100%)
30.36 ± 10.19 (83%)
39.41 ± 2.367 (37%)
2.651 ± 0.6351
100

2.02
SCFOv9f
0.1443 ± 0.0335
0.1443 ± 0.0335
0.1443 ± 0.0335
0.01 ± 0.0995
0.074 ± 0.094
0.074 ± 0.094
0.074 ± 0.094
8.93 ± 8.612 (100%)
29.69 ± 10.32 (87%)
39.3 ± 3.372 (32%)
10.7 ± 2.503
100

3
RSOCCD
3.783 ± 0.0014
3.783 ± 0.0014
3.783 ± 0.0014
0 ± 0
0.0253 ± 0.0018
0.0253 ± 0.0018
0.0253 ± 0.0018
11 ± 0 (100%)
11 ± 0 (100%)
11 ± 0 (100%)
0.0782 ± 0.0141
100

4
SimplexB
0.1445 ± 0.0507
0.1445 ± 0.0507
0.1445 ± 0.0507
0 ± 0
0.0333 ± 0.0331
0.0333 ± 0.0331
0.0333 ± 0.0331
8.17 ± 3.265 (100%)
10.39 ± 3.146 (100%)
16.88 ± 8.749 (91%)
0.0227 ± 0.0068
100

------ Algorithms Employing a Model ------

1
SCFOv7
0.4129 ± 0.1624
0.4129 ± 0.1624
0.4129 ± 0.1624
0 ± 0
0.2417 ± 0.2175
0.2417 ± 0.2175
0.2417 ± 0.2175
28.79 ± 12.33 (72%)
36.51 ± 6.198 (51%)
40.52 ± 1.389 (11%)
45.09 ± 10.63
100

1.01
SCFOv8
0.4115 ± 0.1549
0.4115 ± 0.1549
0.4115 ± 0.1549
0 ± 0
0.2126 ± 0.1906
0.2126 ± 0.1906
0.2126 ± 0.1906
27.63 ± 11.66 (81%)
37.48 ± 4.628 (44%)
40.08 ± 2.07 (21%)
155.8 ± 360
48

1.02
SCFOv9
0.4395 ± 0.1507
0.4395 ± 0.1507
0.4395 ± 0.1507
0 ± 0
0.2033 ± 0.1754
0.2033 ± 0.1754
0.2033 ± 0.1754
28.32 ± 11.37 (76%)
37.32 ± 5.394 (48%)
40.2 ± 1.913 (16%)
104.2 ± 14.73
100

2
SCFOv7f
0.3737 ± 0.1831
0.3737 ± 0.1831
0.3737 ± 0.1831
0 ± 0
0.0626 ± 0.0749
0.0626 ± 0.0749
0.0626 ± 0.0749
20.65 ± 11.76 (96%)
30.55 ± 9.785 (81%)
38.17 ± 3.852 (47%)
29.75 ± 4.193
100

2.01
SCFOv8f
0.4058 ± 0.1979
0.4058 ± 0.1979
0.4058 ± 0.1979
0 ± 0
0.1081 ± 0.1343
0.1081 ± 0.1343
0.1081 ± 0.1343
21.88 ± 10.96 (97%)
30.29 ± 9.233 (86%)
38.67 ± 3.441 (42%)
37.18 ± 8.17
100

2.02
SCFOv9f
0.4657 ± 0.1616
0.4657 ± 0.1616
0.4657 ± 0.1616
0 ± 0
0.1145 ± 0.1962
0.1145 ± 0.1962
0.1145 ± 0.1962
26.62 ± 10.12 (88%)
33.5 ± 7.348 (75%)
38.46 ± 3.827 (43%)
84.73 ± 14.07
100


Test Problem #10: Iterative Tuning of a Fixed-Order Controller in a Torsional System

Original code provided by: G. A. Bunin

Problem Description

Test File Specs: algotest([1 2.77 -2.6 1 0.5],100,0.001,[],[0 0 -5 -2 -1],[5 5 5 2 1],[4.812 0.3826 -2.9531 0.1292 -0.5116],0.05,[],[2 2],algonum)

Main Files: phipeval.m (required), geval.m (required), phimod.m (model)

Results:

Algorithm
Average Suboptimality
Violations
Suboptimality at Final Experiment
Iterations to Convergence
Time
Trials

#
Name
M1 (λ := 1)
M2 (λ := 10)
M3 (λ := 100)
M4
M5 (λ := 1)
M6 (λ := 10)
M7 (λ := 100)
M8 (50%)
M9 (70%)
M10 (90%)
M11
 

0
nothing
0.7401
0.7401
0.7401
0
0.7401
0.7401
0.7401
NA
NA
NA
0
 

------ Model-Free Algorithms ------

1
SCFOv7
0.3809 ± 0.4178
0.3809 ± 0.4178
0.3809 ± 0.4178
0 ± 0
0.1502 ± 0.6047
0.1502 ± 0.6047
0.1502 ± 0.6047
38.09 ± 33.63 (95%)
49.62 ± 32.81 (91%)
82.23 ± 24.75 (63%)
42.63 ± 9.744
100

1.01
SCFOv8
0.5191 ± 0.5044
0.5191 ± 0.5044
0.5191 ± 0.5044
0 ± 0
0.2563 ± 0.9208
0.2563 ± 0.9208
0.2563 ± 0.9208
29.2 ± 27.23 (95%)
34.9 ± 25.43 (95%)
64.4 ± 30.71 (80%)
58.88 ± 16.39
20

1.02
SCFOv9
0.707 ± 0.9858
0.707 ± 0.9858
0.707 ± 0.9858
0 ± 0
1.954 ± 7.223
1.954 ± 7.223
1.954 ± 7.223
44.29 ± 31.98 (92%)
49.58 ± 31.17 (92%)
73.88 ± 25.52 (75%)
72.84 ± 10.08
24

2
SCFOv7f
0.6102 ± 0.8314
0.6102 ± 0.8314
0.6102 ± 0.8314
0 ± 0
0.2406 ± 1.277
0.2406 ± 1.277
0.2406 ± 1.277
46.93 ± 35.79 (89%)
55.28 ± 35.27 (83%)
87 ± 20.15 (55%)
4.003 ± 1.089
100

2.01
SCFOv8f
0.6563 ± 0.99
0.6563 ± 0.99
0.6563 ± 0.99
0 ± 0
0.5994 ± 3.739
0.5994 ± 3.739
0.5994 ± 3.739
47.01 ± 37.71 (91%)
56.71 ± 36.04 (86%)
86.98 ± 18.11 (62%)
6.499 ± 3.551
95

2.02
SCFOv9f
0.7211 ± 0.9676
0.7211 ± 0.9676
0.7211 ± 0.9676
0 ± 0
0.7371 ± 4.849
0.7371 ± 4.849
0.7371 ± 4.849
46.43 ± 34.04 (96%)
52.47 ± 33.82 (94%)
80.18 ± 23.43 (71%)
13.95 ± 3.406
51

3
RSOCCD
8.726 ± 0.0007
10.24 ± 0.0007
25.39 ± 0.0007
10 ± 0
0.4623 ± 0.001
0.4623 ± 0.001
0.4623 ± 0.001
101 ± 0 (0%)
101 ± 0 (0%)
101 ± 0 (0%)
0.2423 ± 0.0489
100

------ Algorithms Employing a Model ------

1
SCFOv7
0.2606 ± 0.095
0.2606 ± 0.095
0.2606 ± 0.095
0 ± 0
0.1954 ± 0.0683
0.1954 ± 0.0683
0.1954 ± 0.0683
14.99 ± 24.25 (98%)
57.63 ± 42.44 (67%)
100.9 ± 0.9763 (2%)
42.04 ± 7.952
100

1.01
SCFOv8
0.2573 ± 0.0834
0.2573 ± 0.0834
0.2573 ± 0.0834
0 ± 0
0.1896 ± 0.0648
0.1896 ± 0.0648
0.1896 ± 0.0648
13.77 ± 21.5 (100%)
55.36 ± 42.41 (68%)
101 ± 0 (0%)
94.52 ± 71.56
22

1.02
SCFOv9
0.2592 ± 0.1005
0.2592 ± 0.1005
0.2592 ± 0.1005
0 ± 0
0.1924 ± 0.0763
0.1924 ± 0.0763
0.1924 ± 0.0763
17.05 ± 26.64 (100%)
60.48 ± 44.3 (57%)
98.95 ± 9.157 (5%)
77.95 ± 11.87
21

2
SCFOv7f
0.2505 ± 0.0974
0.2505 ± 0.0974
0.2505 ± 0.0974
0 ± 0
0.1846 ± 0.0703
0.1846 ± 0.0703
0.1846 ± 0.0703
13.21 ± 23.44 (98%)
53.78 ± 43.49 (70%)
100.8 ± 1.511 (3%)
5.298 ± 1.137
100

2.01
SCFOv8f
0.2402 ± 0.0796
0.2402 ± 0.0796
0.2402 ± 0.0796
0 ± 0
0.1945 ± 0.1754
0.1945 ± 0.1754
0.1945 ± 0.1754
11.49 ± 20.92 (98%)
52.08 ± 41.39 (75%)
100.9 ± 0.7777 (2%)
24.34 ± 46.33
84

2.02
SCFOv9f
0.2764 ± 0.1209
0.2764 ± 0.1209
0.2764 ± 0.1209
0 ± 0
0.1985 ± 0.0835
0.1985 ± 0.0835
0.1985 ± 0.0835
19.91 ± 30.1 (95%)
57.5 ± 43.05 (61%)
101 ± 0 (0%)
15.99 ± 4.341
56


Test Problem #11: Minimizing the Steady-State Production Cost of a Gold Cyanidation Leaching Process

Original code provided by: Zhang Jun (张俊)

Problem Description

Test File Specs: algotest([52 18],40,2,.001,[10 5],[80 20],[31.0563 11.6218],200,.04,[],algonum)

Main Files: phipeval.m (required), gpeval.m (required), phimod.m (model), gmod.m (model)

Auxiliary Files: model_single.m (required)

Results:

Algorithm
Average Suboptimality
Violations
Suboptimality at Final Experiment
Iterations to Convergence
Time
Trials

#
Name
M1 (λ := 1)
M2 (λ := 10)
M3 (λ := 100)
M4
M5 (λ := 1)
M6 (λ := 10)
M7 (λ := 100)
M8 (50%)
M9 (70%)
M10 (90%)
M11
 

0
nothing
0.687
0.687
0.687
0
0.687
0.687
0.687
NA
NA
NA
0
 

------ Model-Free Algorithms ------

1
SCFOv7
0.0751 ± 0.0191
0.0753 ± 0.0193
0.077 ± 0.0248
0.05 ± 0.2958
0.0053 ± 0.0053
0.0061 ± 0.01
0.0139 ± 0.0871
4.07 ± 0.7906 (100%)
4.92 ± 0.8565 (100%)
9.03 ± 4.412 (100%)
23.35 ± 1.918
100

1.01
SCFOv8
0.083 ± 0.0199
0.0836 ± 0.0207
0.0904 ± 0.0456
0.1481 ± 0.7554
0.0056 ± 0.0056
0.0056 ± 0.0056
0.0056 ± 0.0056
4.074 ± 0.7662 (100%)
5.148 ± 1.008 (100%)
9.889 ± 4.833 (100%)
31.23 ± 8.918
27

1.02
SCFOv9
0.0794 ± 0.0253
0.0796 ± 0.0252
0.0817 ± 0.0257
0.0909 ± 0.2875
0.0058 ± 0.0075
0.0058 ± 0.0075
0.0058 ± 0.0075
4.136 ± 0.8144 (100%)
5 ± 0.7977 (100%)
9.682 ± 4.865 (100%)
55.74 ± 3.97
22

2
SCFOv7f
0.0724 ± 0.0172
0.0725 ± 0.0172
0.0727 ± 0.0173
0.01 ± 0.0995
0.0089 ± 0.0136
0.0089 ± 0.0136
0.0089 ± 0.0136
4.11 ± 0.7469 (100%)
4.96 ± 0.7864 (100%)
10.6 ± 8.615 (99%)
4.103 ± 1.424
100

2.01
SCFOv8f
0.072 ± 0.0149
0.0723 ± 0.0149
0.0749 ± 0.0207
0.1111 ± 0.5666
0.0064 ± 0.0066
0.0067 ± 0.0075
0.0104 ± 0.0407
4.04 ± 0.6951 (100%)
4.919 ± 0.7477 (100%)
10.41 ± 8.767 (98%)
7.71 ± 5.792
99

2.02
SCFOv9f
0.0742 ± 0.02
0.0744 ± 0.0199
0.0759 ± 0.0205
0.0508 ± 0.2197
0.0086 ± 0.0123
0.0086 ± 0.0123
0.0086 ± 0.0123
4.119 ± 0.761 (100%)
4.966 ± 0.8227 (100%)
11.05 ± 8.879 (98%)
14.4 ± 3.562
59

3
RSOCCD
0.7568 ± 0.0011
1.963 ± 0.0011
14.02 ± 0.0011
4 ± 0
0.3259 ± 0.0015
0.3259 ± 0.0015
0.3259 ± 0.0015
10 ± 0 (100%)
41 ± 0 (0%)
41 ± 0 (0%)
0.1798 ± 0.073
100

------ Algorithms Employing a Model ------

1
SCFOv7
0.3164 ± 0.0442
0.3164 ± 0.0442
0.3164 ± 0.0442
0 ± 0
0.0608 ± 0.0288
0.0608 ± 0.0288
0.0608 ± 0.0288
17.03 ± 2.92 (100%)
24.39 ± 3.952 (100%)
37.21 ± 4.318 (56%)
25.6 ± 3.534
100

1.01
SCFOv8
0.275 ± 0.0432
0.275 ± 0.0432
0.275 ± 0.0432
0 ± 0
0.0443 ± 0.0224
0.0443 ± 0.0224
0.0443 ± 0.0224
14.33 ± 2.749 (100%)
21.15 ± 3.788 (100%)
34.26 ± 5.866 (70%)
44.53 ± 22.73
27

1.02
SCFOv9
0.2945 ± 0.0308
0.2945 ± 0.0308
0.2945 ± 0.0308
0 ± 0
0.0574 ± 0.0237
0.0574 ± 0.0237
0.0574 ± 0.0237
15.43 ± 2.223 (100%)
22.96 ± 3.532 (100%)
37.26 ± 4.757 (48%)
61.87 ± 9.689
23

2
SCFOv7f
0.3618 ± 0.0441
0.3618 ± 0.0441
0.3618 ± 0.0441
0 ± 0
0.1052 ± 0.0399
0.1052 ± 0.0399
0.1052 ± 0.0399
20.42 ± 3.439 (100%)
30.26 ± 4.77 (95%)
40.41 ± 1.656 (12%)
3.11 ± 0.6163
100

2.01
SCFOv8f
0.3355 ± 0.0515
0.3355 ± 0.0515
0.3355 ± 0.0515
0 ± 0
0.0914 ± 0.0391
0.0914 ± 0.0391
0.0914 ± 0.0391
18.23 ± 3.833 (100%)
27.98 ± 5.671 (98%)
39.2 ± 4.291 (20%)
10.09 ± 13.36
86

2.02
SCFOv9f
0.3453 ± 0.0354
0.3453 ± 0.0354
0.3453 ± 0.0354
0 ± 0
0.0951 ± 0.0326
0.0951 ± 0.0326
0.0951 ± 0.0326
18.77 ± 2.978 (100%)
28.83 ± 3.847 (100%)
40.48 ± 1.352 (13%)
10.8 ± 2.373
52

5
ConAdapt
0.6616 ± 0
0.6616 ± 0
0.6616 ± 0
0 ± 0
0.661 ± 0
0.661 ± 0
0.661 ± 0
41 ± 0 (0%)
41 ± 0 (0%)
41 ± 0 (0%)
6.963 ± 1.81
100