The following are the test problems currently making up the database, together with the performances obtained by the different tested algorithms. To make it easier to pair up problems with algorithms and to analyze the results, the problems are broken up into three classes, with Class 1 problems denoting those with only bound constraints, Class 2 problems as those with general numerical constraints but no experimental constraints, and Class 3 problems being those that possess experimental constraints.

Class 1: P1, P7, P9

Class 2: P8, P10

Class 3: P2, P3, P4, P5, P6, P11

In reporting the metric values, both the average (left) and the standard error (right) are given, with the plus-minus (±) sign used to separate the two. For Metrics 8-10, which attempt to gauge convergence speed, the percentage of the trials that achieved the specified convergence is also given. In the case that convergence was not achieved for any of the trials, the letters "NA" (not available) are reported. Currently, testing is done automatically by dedicated computers, with a maximum of 100 trials done for each testing combination (generated by choosing the problem, the algorithm, and whether or not the model is used).

You may view the plotted results for the individual trials for each algorithm by clicking on the algorithm name in the "Name" column. For problems with only two variables, plots of the experimental iterates in the decision-variable space are given for particularly insightful illustrations - only their values are plotted for problems with three or more variables. For plots of the decision-variable space, green regions denote those that are feasible (where all constraints are satisfied), while red denotes the infeasible regions. Red dots denote the individual experimental iterates, while the green dot denotes the true optimum that the algorithm aims to find. Constant dotted lines are used to plot the optimal values in the three-or-more variable case. For the cost function value plots, the constant black line denotes the value at the true optimum.


Test Problem #1: Maximizing Profit in the Williams-Otto Reactor

Original code provided by: S. Costello

Problem Description

Test File Specs: algotest([4.8 77],40,0.5,[],[3 70],[6 100],[4.79 89.7],100,[],[],algonum)

Main Files: phipeval.m (required), phimod.m (model)

Auxiliary Files: plantbalancesT.m (required), modelbalances2.m (model)

Results:

Algorithm
Average Suboptimality
Violations
Suboptimality at Final Experiment
Iterations to Convergence
Time
Trials

#
Name
M1 (λ := 1)
M2 (λ := 10)
M3 (λ := 100)
M4
M5 (λ := 1)
M6 (λ := 10)
M7 (λ := 100)
M8 (50%)
M9 (70%)
M10 (90%)
M11
 

0
nothing
0.5186
0.5186
0.5186
0
0.5186
0.5186
0.5186
NA
NA
NA
0
 

------ Model-Free Algorithms ------

1
SCFOv7
0.0455 ± 0.0118
0.0455 ± 0.0118
0.0455 ± 0.0118
0 ± 0
0.0023 ± 0.0031
0.0023 ± 0.0031
0.0023 ± 0.0031
3.63 ± 0.4828 (100%)
4.37 ± 0.9017 (100%)
6.61 ± 1.949 (100%)
15.63 ± 2.811
100

1.01
SCFOv8
0.0511 ± 0.0129
0.0511 ± 0.0129
0.0511 ± 0.0129
0 ± 0
0.0036 ± 0.0043
0.0036 ± 0.0043
0.0036 ± 0.0043
3.789 ± 0.4077 (100%)
4.632 ± 0.9846 (100%)
6.737 ± 1.018 (100%)
23.5 ± 8.265
19

1.02
SCFOv9
0.0492 ± 0.0102
0.0492 ± 0.0102
0.0492 ± 0.0102
0 ± 0
0.0026 ± 0.0032
0.0026 ± 0.0032
0.0026 ± 0.0032
3.882 ± 0.3222 (100%)
4.412 ± 1.032 (100%)
7.294 ± 2.19 (100%)
44.57 ± 10.01
17

2
SCFOv7f
0.0481 ± 0.0131
0.0481 ± 0.0131
0.0481 ± 0.0131
0 ± 0
0.0041 ± 0.0046
0.0041 ± 0.0046
0.0041 ± 0.0046
3.66 ± 0.4737 (100%)
4.19 ± 0.6737 (100%)
6.87 ± 2.524 (100%)
2.289 ± 0.6446
100

2.01
SCFOv8f
0.0478 ± 0.0126
0.0478 ± 0.0126
0.0478 ± 0.0126
0 ± 0
0.0036 ± 0.0041
0.0036 ± 0.0041
0.0036 ± 0.0041
3.671 ± 0.47 (100%)
4.259 ± 0.6165 (100%)
6.847 ± 2.564 (100%)
3.847 ± 2.956
85

2.02
SCFOv9f
0.0502 ± 0.0134
0.0502 ± 0.0134
0.0502 ± 0.0134
0 ± 0
0.0034 ± 0.0041
0.0034 ± 0.0041
0.0034 ± 0.0041
3.75 ± 0.433 (100%)
4.393 ± 0.9 (100%)
6.679 ± 2.45 (100%)
10.04 ± 1.831
28

3
RSOCCD
0.1993 ± 0.0003
0.1993 ± 0.0003
0.1993 ± 0.0003
0 ± 0
0.0049 ± 0.0004
0.0049 ± 0.0004
0.0049 ± 0.0004
10 ± 0 (100%)
10 ± 0 (100%)
10 ± 0 (100%)
0.1472 ± 0.0949
100

4
SimplexB
0.0733 ± 0.0192
0.0733 ± 0.0192
0.0733 ± 0.0192
0 ± 0
0.0025 ± 0.0048
0.0025 ± 0.0048
0.0025 ± 0.0048
6.51 ± 2.488 (100%)
9.67 ± 2.761 (100%)
14.66 ± 3.55 (100%)
0.0221 ± 0.0116
100

------ Algorithms Employing a Model ------

1
SCFOv7
0.0472 ± 0.0123
0.0472 ± 0.0123
0.0472 ± 0.0123
0 ± 0
0.0019 ± 0.0029
0.0019 ± 0.0029
0.0019 ± 0.0029
3.77 ± 0.4659 (100%)
4.35 ± 0.7124 (100%)
6.25 ± 1.417 (100%)
15.38 ± 2.242
100

1.01
SCFOv8
0.048 ± 0.0132
0.048 ± 0.0132
0.048 ± 0.0132
0 ± 0
0.0018 ± 0.0028
0.0018 ± 0.0028
0.0018 ± 0.0028
3.556 ± 0.4969 (100%)
4.5 ± 0.8975 (100%)
5.778 ± 1.272 (100%)
29.7 ± 16.4
18

1.02
SCFOv9
0.0472 ± 0.0158
0.0472 ± 0.0158
0.0472 ± 0.0158
0 ± 0
0.0043 ± 0.0057
0.0043 ± 0.0057
0.0043 ± 0.0057
3.588 ± 0.4922 (100%)
4.294 ± 0.7487 (100%)
6.647 ± 2.113 (100%)
43.88 ± 6.544
17

2
SCFOv7f
0.0527 ± 0.0135
0.0527 ± 0.0135
0.0527 ± 0.0135
0 ± 0
0.0035 ± 0.004
0.0035 ± 0.004
0.0035 ± 0.004
3.77 ± 0.4208 (100%)
4.54 ± 0.8297 (100%)
7.03 ± 3.756 (100%)
2.567 ± 0.579
100

2.01
SCFOv8f
0.0494 ± 0.0125
0.0494 ± 0.0125
0.0494 ± 0.0125
0 ± 0
0.0037 ± 0.0045
0.0037 ± 0.0045
0.0037 ± 0.0045
3.717 ± 0.4504 (100%)
4.404 ± 0.8399 (100%)
6.788 ± 3.788 (100%)
6.631 ± 7.713
99

2.02
SCFOv9f
0.0513 ± 0.0145
0.0513 ± 0.0145
0.0513 ± 0.0145
0 ± 0
0.0029 ± 0.0029
0.0029 ± 0.0029
0.0029 ± 0.0029
3.737 ± 0.4403 (100%)
4.579 ± 0.9356 (100%)
6.553 ± 1.758 (100%)
10.99 ± 2.327
38


Test Problem #2: Maximizing Profit in the Williams-Otto Reactor (Constrained)

Original code provided by: S. Costello

Problem Description

Test File Specs: algotest([3.5 72],40,0.5,5e-4,[3 70],[6 100],[4.97 84.3],100,.1,[],algonum)

Main Files: phipeval.m (required), gpeval.m (required), phimod.m (model), gmod.m (model)

Auxiliary Files: plantbalancesT.m (required), modelbalances2.m (model)

Results:

Algorithm
Average Suboptimality
Violations
Suboptimality at Final Experiment
Iterations to Convergence
Time
Trials

#
Name
M1 (λ := 1)
M2 (λ := 10)
M3 (λ := 100)
M4
M5 (λ := 1)
M6 (λ := 10)
M7 (λ := 100)
M8 (50%)
M9 (70%)
M10 (90%)
M11
 

0
nothing
0.4051
0.4051
0.4051
0
0.4051
0.4051
0.4051
NA
NA
NA
0
 

------ Model-Free Algorithms ------

1
SCFOv7
0.1861 ± 0.0618
0.2147 ± 0.0721
0.501 ± 0.4092
5.23 ± 7.339
0.1572 ± 0.0639
0.1725 ± 0.0786
0.3255 ± 0.5731
18.84 ± 16.71 (67%)
29.79 ± 15.45 (39%)
41 ± 0 (0%)
24.3 ± 6.888
100

1.01
SCFOv8
0.189 ± 0.0472
0.1932 ± 0.0474
0.2352 ± 0.056
1.833 ± 2.134
0.1619 ± 0.0531
0.1631 ± 0.0539
0.1748 ± 0.0848
20.5 ± 15.08 (75%)
31.83 ± 14.6 (29%)
41 ± 0 (0%)
32.28 ± 14.72
24

1.02
SCFOv9
0.1926 ± 0.039
0.2022 ± 0.0363
0.299 ± 0.096
4.1 ± 5.009
0.1674 ± 0.0419
0.1674 ± 0.0419
0.1674 ± 0.0419
19.8 ± 15.48 (70%)
37.8 ± 9.6 (10%)
41 ± 0 (0%)
58.82 ± 16.41
10

2
SCFOv7f
0.2138 ± 0.046
0.2399 ± 0.0487
0.5005 ± 0.233
4.89 ± 6.439
0.188 ± 0.0492
0.1947 ± 0.0526
0.2608 ± 0.2727
24.07 ± 16.98 (58%)
40.47 ± 3.708 (4%)
41 ± 0 (0%)
4.177 ± 1.321
100

2.01
SCFOv8f
0.2194 ± 0.055
0.2249 ± 0.0539
0.2798 ± 0.0768
1.955 ± 2.449
0.1908 ± 0.0572
0.1911 ± 0.0578
0.1946 ± 0.0726
22.21 ± 16.75 (58%)
40.26 ± 3.64 (6%)
41 ± 0 (0%)
7.646 ± 5.779
89

2.02
SCFOv9f
0.2147 ± 0.046
0.2201 ± 0.0452
0.2733 ± 0.0634
2.531 ± 3.881
0.1892 ± 0.0472
0.1894 ± 0.0472
0.1913 ± 0.0487
24.53 ± 17.25 (53%)
40.09 ± 5.046 (3%)
41 ± 0 (0%)
15.99 ± 4.934
32

3
RSOCCD
0.2616 ± 0.0006
1.143 ± 0.0235
9.953 ± 0.2596
31.35 ± 11.07
0.002 ± 0.0008
0.0418 ± 0.031
0.4407 ± 0.3433
10 ± 0 (100%)
10 ± 0 (100%)
10 ± 0 (100%)
0.1729 ± 0.04
100

------ Algorithms Employing a Model ------

1
SCFOv7
0.2339 ± 0.0256
0.2339 ± 0.0256
0.234 ± 0.0256
0.01 ± 0.0995
0.19 ± 0.0311
0.19 ± 0.0311
0.19 ± 0.0311
26.09 ± 11.71 (73%)
40.93 ± 0.6965 (1%)
41 ± 0 (0%)
25.41 ± 5.737
100

1.01
SCFOv8
0.2336 ± 0.0297
0.2336 ± 0.0297
0.2336 ± 0.0297
0.05 ± 0.2179
0.1894 ± 0.0354
0.1894 ± 0.0354
0.1894 ± 0.0354
27.4 ± 13.01 (65%)
39.7 ± 5.667 (5%)
41 ± 0 (0%)
33.77 ± 14.98
20

1.02
SCFOv9
0.232 ± 0.0235
0.2322 ± 0.0237
0.2337 ± 0.0258
0.1538 ± 0.5329
0.1878 ± 0.0284
0.1878 ± 0.0284
0.1878 ± 0.0284
22.92 ± 12.06 (77%)
41 ± 0 (0%)
41 ± 0 (0%)
56.63 ± 10.33
13

2
SCFOv7f
0.2716 ± 0.0209
0.2716 ± 0.0209
0.2716 ± 0.0209
0 ± 0
0.2354 ± 0.0233
0.2354 ± 0.0233
0.2354 ± 0.0233
39.95 ± 4.879 (7%)
41 ± 0 (0%)
41 ± 0 (0%)
4.551 ± 1.164
100

2.01
SCFOv8f
0.2707 ± 0.0163
0.2707 ± 0.0163
0.2712 ± 0.0172
0.0543 ± 0.5184
0.2339 ± 0.0221
0.2339 ± 0.0221
0.2339 ± 0.0221
39.84 ± 5.148 (7%)
41 ± 0 (0%)
41 ± 0 (0%)
9.288 ± 10.45
92

2.02
SCFOv9f
0.2729 ± 0.0183
0.2729 ± 0.0183
0.2734 ± 0.0184
0.1395 ± 0.9043
0.2346 ± 0.0223
0.2346 ± 0.0223
0.2346 ± 0.0223
40.05 ± 3.596 (7%)
41 ± 0 (0%)
41 ± 0 (0%)
17.33 ± 4.005
43

5
ConAdapt
0.3156 ± 0.0022
0.3156 ± 0.0022
0.3156 ± 0.0022
0 ± 0
0.281 ± 0
0.281 ± 0
0.281 ± 0
41 ± 0 (0%)
41 ± 0 (0%)
41 ± 0 (0%)
2.846 ± 0.5429
100


Test Problem #3: Minimizing the Batch Time of Polystyrene Production

Original code provided by: G. François

Problem Description

Test File Specs: algotest([242.39 945.30],40,60,1e4,[50 600],[450 1000],[363.06 875.60],10000,1e6,[],algonum)

Main Files: phipeval.m (required), gpeval.m (required), phimod.m (model), gmod.m (model)

Auxiliary Files: integstyrbis.m (required), linint.m (required)

Results:

Algorithm
Average Suboptimality
Violations
Suboptimality at Final Experiment
Iterations to Convergence
Time
Trials

#
Name
M1 (λ := 1)
M2 (λ := 10)
M3 (λ := 100)
M4
M5 (λ := 1)
M6 (λ := 10)
M7 (λ := 100)
M8 (50%)
M9 (70%)
M10 (90%)
M11
 

0
nothing
0.1979
0.1979
0.1979
0
0.1979
0.1979
0.1979
NA
NA
NA
0
 

------ Model-Free Algorithms ------

1
SCFOv7
0.0296 ± 0.0058
0.0296 ± 0.0058
0.0296 ± 0.0058
0 ± 0
0.0092 ± 0.0039
0.0092 ± 0.0039
0.0092 ± 0.0039
4.21 ± 0.7911 (100%)
5.6 ± 1.393 (100%)
16.42 ± 9.593 (97%)
25.12 ± 6.884
100

1.01
SCFOv8
0.0303 ± 0.0052
0.0303 ± 0.0052
0.0303 ± 0.0052
0 ± 0
0.0097 ± 0.0041
0.0097 ± 0.0041
0.0097 ± 0.0041
4.421 ± 0.674 (100%)
5.632 ± 1.223 (100%)
15.37 ± 7.073 (100%)
31.45 ± 9.418
19

1.02
SCFOv9
0.0296 ± 0.0051
0.0296 ± 0.0051
0.0296 ± 0.0051
0 ± 0
0.0087 ± 0.0044
0.0087 ± 0.0044
0.0087 ± 0.0044
4.133 ± 0.8055 (100%)
5.267 ± 0.9286 (100%)
17.33 ± 8.3 (100%)
62.17 ± 18.67
15

2
SCFOv7f
0.0287 ± 0.0055
0.0287 ± 0.0055
0.0287 ± 0.0055
0 ± 0
0.0092 ± 0.004
0.0092 ± 0.004
0.0092 ± 0.004
4.27 ± 0.8468 (100%)
5.48 ± 1.403 (100%)
15.01 ± 8.352 (98%)
4.467 ± 1.417
100

2.01
SCFOv8f
0.0291 ± 0.0058
0.0291 ± 0.0058
0.0291 ± 0.0058
0 ± 0
0.0091 ± 0.0044
0.0091 ± 0.0044
0.0091 ± 0.0044
4.235 ± 1.009 (100%)
5.457 ± 1.078 (100%)
17.6 ± 10.87 (94%)
7.087 ± 4.409
81

2.02
SCFOv9f
0.0296 ± 0.0061
0.0296 ± 0.0061
0.0296 ± 0.0061
0 ± 0
0.0082 ± 0.0029
0.0082 ± 0.0029
0.0082 ± 0.0029
4.184 ± 0.7559 (100%)
5.553 ± 1.39 (100%)
14.32 ± 7.851 (100%)
18.63 ± 5.565
38

3
RSOCCD
0.104 ± 0.0009
0.5588 ± 0.0009
5.106 ± 0.0009
4 ± 0
0.0155 ± 0.0012
0.0155 ± 0.0012
0.0155 ± 0.0012
10 ± 0 (100%)
11 ± 0 (100%)
11 ± 0 (100%)
0.2738 ± 0.0533
100

------ Algorithms Employing a Model ------

1
SCFOv7
0.1015 ± 0.0409
0.1015 ± 0.0409
0.1015 ± 0.0409
0 ± 0
0.0485 ± 0.0364
0.0485 ± 0.0364
0.0485 ± 0.0364
21.13 ± 14 (82%)
31.33 ± 10.38 (57%)
38.81 ± 4.046 (28%)
33.87 ± 11.99
100

1.01
SCFOv8
0.094 ± 0.0348
0.094 ± 0.0348
0.094 ± 0.0348
0 ± 0
0.0435 ± 0.0296
0.0435 ± 0.0296
0.0435 ± 0.0296
19.42 ± 11.67 (92%)
30.25 ± 10.17 (67%)
38.83 ± 2.911 (42%)
61.01 ± 73.23
12

1.02
SCFOv9
0.1103 ± 0.039
0.1103 ± 0.039
0.1103 ± 0.039
0 ± 0
0.0492 ± 0.0399
0.0492 ± 0.0399
0.0492 ± 0.0399
24.21 ± 12.77 (89%)
32.32 ± 8.742 (58%)
39 ± 3.699 (32%)
67.1 ± 9.806
19

2
SCFOv7f
0.1131 ± 0.0372
0.1131 ± 0.0372
0.1131 ± 0.0372
0 ± 0
0.0519 ± 0.0364
0.0519 ± 0.0364
0.0519 ± 0.0364
25.49 ± 12.28 (81%)
34.29 ± 7.672 (56%)
39.83 ± 2.642 (21%)
17.78 ± 3.502
100

2.01
SCFOv8f
0.1093 ± 0.0379
0.1093 ± 0.0379
0.1093 ± 0.0379
0 ± 0
0.0516 ± 0.0352
0.0516 ± 0.0352
0.0516 ± 0.0352
23.66 ± 12.72 (87%)
33.86 ± 8.653 (52%)
39.6 ± 3.551 (20%)
54.7 ± 80.78
87

2.02
SCFOv9f
0.1202 ± 0.0364
0.1202 ± 0.0364
0.1202 ± 0.0364
0 ± 0
0.0592 ± 0.0328
0.0592 ± 0.0328
0.0592 ± 0.0328
27.32 ± 12.53 (76%)
35.92 ± 7.303 (45%)
40.16 ± 2.621 (13%)
53.14 ± 9.71
38

5
ConAdapt
0.0229 ± 0.0033
0.1318 ± 0.0225
1.221 ± 0.2184
23.3 ± 2.022
0.0231 ± 0.0187
0.1801 ± 0.2337
1.75 ± 2.407
2 ± 0 (100%)
3.29 ± 5.933 (99%)
33.43 ± 9.512 (64%)
56.46 ± 12.3
100


Test Problem #4: Maximizing Electrical Efficiency in a Solid Oxide Fuel Cell Stack

Original code provided by: A. Gopalakrishnan

Problem Description

Test File Specs: algotest([2e-3 7e-3 26],60,6e-4,[0.1 0.1 1e-3 0.03],[1e-3 1.5e-3 1],[1e-2 3.5e-2 30],[1e-3 1.898e-3 23.6687],.1,[50 50 .1 50],[.01 .01 .01],algonum)

Main Files: phipeval.m (required), gpeval.m (required), geval.m (required), phimod.m (model), gmod.m (model)

Auxiliary Files: model_real.m (required), stack.m (required), model.m (model)

Results:

Algorithm
Average Suboptimality
Violations
Suboptimality at Final Experiment
Iterations to Convergence
Time
Trials

#
Name
M1 (λ := 1)
M2 (λ := 10)
M3 (λ := 100)
M4
M5 (λ := 1)
M6 (λ := 10)
M7 (λ := 100)
M8 (50%)
M9 (70%)
M10 (90%)
M11
 

0
nothing
1.642
1.642
1.642
0
1.642
1.642
1.642
NA
NA
NA
0
 

------ Model-Free Algorithms ------

1
SCFOv7
0.4251 ± 0.1246
0.9484 ± 0.504
6.181 ± 5.556
33.22 ± 13.18
0.3077 ± 0.11
1.086 ± 0.9399
8.874 ± 10.46
6.73 ± 5.223 (100%)
15.43 ± 16.13 (96%)
50.75 ± 17.33 (29%)
77.28 ± 24.94
100

1.01
SCFOv8
0.5193 ± 0.0941
0.5312 ± 0.0907
0.6507 ± 0.1959
3.556 ± 5.89
0.4372 ± 0.1835
0.4396 ± 0.1811
0.4639 ± 0.1711
28.89 ± 21.34 (100%)
61 ± 0 (0%)
61 ± 0 (0%)
123.3 ± 47.4
9

1.02
SCFOv9
0.5409 ± 0.098
0.5714 ± 0.1021
0.8765 ± 0.2554
2.75 ± 1.479
0.3845 ± 0.1315
0.3845 ± 0.1315
0.3845 ± 0.1315
17.75 ± 17.26 (94%)
59.06 ± 4.479 (19%)
60.94 ± 0.2421 (0%)
171.9 ± 13.47
16

2
SCFOv7f
0.4341 ± 0.1151
0.8585 ± 0.5041
5.103 ± 5.326
30.25 ± 13.37
0.3204 ± 0.1096
0.9128 ± 0.9955
6.837 ± 10.85
6.57 ± 3.827 (100%)
14.52 ± 14.51 (95%)
54.91 ± 13.85 (24%)
23.86 ± 5.334
100

2.01
SCFOv8f
0.5587 ± 0.1004
0.5811 ± 0.1078
0.806 ± 0.2779
2.233 ± 2.491
0.4473 ± 0.2009
0.4494 ± 0.1999
0.4701 ± 0.2383
26.88 ± 21.65 (93%)
60.36 ± 1.065 (18%)
61 ± 0 (0%)
42.87 ± 12.8
73

2.02
SCFOv9f
0.537 ± 0.1108
0.5576 ± 0.1125
0.7635 ± 0.214
1.788 ± 1.174
0.4423 ± 0.1829
0.4423 ± 0.1829
0.4423 ± 0.1829
23.97 ± 20.75 (91%)
59.48 ± 3.726 (21%)
61 ± 0 (0%)
71.44 ± 17.02
33

3
RSOCCD
2.497 ± 0.0228
7.862 ± 0.0228
61.52 ± 0.0228
21 ± 0
0.5928 ± 0.0309
0.5928 ± 0.0309
0.5928 ± 0.0309
17 ± 0 (100%)
61 ± 0 (0%)
61 ± 0 (0%)
3.328 ± 0.7407
100

------ Algorithms Employing a Model ------

1
SCFOv7
0.6865 ± 0.2016
0.7057 ± 0.1936
0.8985 ± 0.6854
5.12 ± 10.37
0.4142 ± 0.1171
0.4434 ± 0.1492
0.7357 ± 1.361
17.62 ± 11.03 (99%)
38.68 ± 21.48 (72%)
60.51 ± 3.814 (4%)
84.1 ± 22.95
100

1.01
SCFOv8
0.6537 ± 0.15
0.6558 ± 0.1494
0.6762 ± 0.1497
0.8571 ± 1.597
0.3611 ± 0.2367
0.3611 ± 0.2367
0.3611 ± 0.2367
32.21 ± 16.95 (100%)
60.21 ± 1.145 (29%)
61 ± 0 (0%)
116.6 ± 42.53
14

1.02
SCFOv9
0.6418 ± 0.1549
0.6453 ± 0.1549
0.68 ± 0.1656
1.071 ± 2.052
0.3651 ± 0.2227
0.3651 ± 0.2227
0.3651 ± 0.2227
31.64 ± 17.27 (100%)
59.71 ± 1.79 (36%)
60.57 ± 0.9035 (14%)
177.3 ± 22.86
14

2
SCFOv7f
0.8029 ± 0.2806
0.8383 ± 0.2496
1.192 ± 1.183
4.88 ± 11.07
0.4437 ± 0.1474
0.5186 ± 0.2196
1.268 ± 2.589
24.42 ± 14.31 (98%)
46.21 ± 18.18 (62%)
59.85 ± 6.842 (3%)
19.2 ± 4.378
100

2.01
SCFOv8f
0.6294 ± 0.1494
0.632 ± 0.1489
0.6575 ± 0.1517
0.6316 ± 1.098
0.3056 ± 0.2303
0.3056 ± 0.2303
0.3056 ± 0.2303
31.57 ± 17.17 (92%)
58.88 ± 2.651 (37%)
60.58 ± 0.8924 (11%)
43.82 ± 23.41
76

2.02
SCFOv9f
0.6373 ± 0.1361
0.6388 ± 0.1351
0.6545 ± 0.129
0.5526 ± 1.117
0.3218 ± 0.1895
0.3218 ± 0.1895
0.3218 ± 0.1895
30 ± 16.89 (95%)
58.63 ± 2.559 (61%)
60.74 ± 0.7499 (8%)
66.37 ± 14.28
38

5
ConAdapt
0.4985 ± 0.0027
0.526 ± 0.0043
0.8003 ± 0.0448
15.22 ± 4.836
0.4818 ± 0.0148
0.4899 ± 0.0199
0.5711 ± 0.2085
2 ± 0 (100%)
48.46 ± 14.44 (69%)
61 ± 0 (0%)
3.315 ± 0.6327
100


Test Problem #5: Minimizing the Overall Pumping Effort in the "Trois Bacs"

Original code provided by: G. A. Bunin

Problem Description

Test File Specs: algotest([6 6.2],40,0,[0.2 0.2 0.2 0.2 0.2 0.2],[0 0],[8 8],[4.9481 4.4160],40,[20 20 20 20 20 20],[],algonum)

Main Files: phieval.m (required), gpeval.m (required), gmod.m (model)

Results:

Algorithm
Average Suboptimality
Violations
Suboptimality at Final Experiment
Iterations to Convergence
Time
Trials

#
Name
M1 (λ := 1)
M2 (λ := 10)
M3 (λ := 100)
M4
M5 (λ := 1)
M6 (λ := 10)
M7 (λ := 100)
M8 (50%)
M9 (70%)
M10 (90%)
M11
 

0
nothing
0.7614
0.7614
0.7614
0
0.7614
0.7614
0.7614
NA
NA
NA
0
 

------ Model-Free Algorithms ------

1
SCFOv7
0.1316 ± 0.0259
0.1318 ± 0.0258
0.1331 ± 0.0269
0.28 ± 1.898
0.0816 ± 0.0356
0.0822 ± 0.0352
0.0878 ± 0.0526
3 ± 0 (100%)
6.46 ± 6.002 (100%)
39.98 ± 2.293 (22%)
49.18 ± 10.74
100

1.01
SCFOv8
0.1266 ± 0.0253
0.1266 ± 0.0253
0.1266 ± 0.0253
0 ± 0
0.0688 ± 0.0332
0.0688 ± 0.0332
0.0688 ± 0.0332
3 ± 0 (100%)
4.85 ± 2.594 (100%)
39.1 ± 3.39 (35%)
48.69 ± 18.55
20

1.02
SCFOv9
0.1159 ± 0.0222
0.1159 ± 0.0222
0.1159 ± 0.0222
0 ± 0
0.0523 ± 0.0326
0.0523 ± 0.0326
0.0523 ± 0.0326
3 ± 0 (100%)
4.111 ± 0.3143 (100%)
37.78 ± 2.529 (67%)
72.01 ± 5.236
9

2
SCFOv7f
0.1302 ± 0.0294
0.1306 ± 0.0288
0.1341 ± 0.0308
0.56 ± 2.892
0.0786 ± 0.0376
0.0797 ± 0.0362
0.0904 ± 0.0633
3 ± 0 (100%)
5.42 ± 3.798 (100%)
39.12 ± 5.208 (26%)
15.17 ± 2.593
100

2.01
SCFOv8f
0.1272 ± 0.0279
0.1273 ± 0.0279
0.1274 ± 0.0277
0.038 ± 0.1911
0.0785 ± 0.0369
0.0785 ± 0.0369
0.0785 ± 0.0369
3 ± 0 (100%)
4.595 ± 2.303 (100%)
38.24 ± 6.074 (38%)
36.78 ± 21.29
79

2.02
SCFOv9f
0.1286 ± 0.0258
0.1286 ± 0.0258
0.1287 ± 0.0257
0.087 ± 0.2818
0.0801 ± 0.0383
0.0801 ± 0.0383
0.0801 ± 0.0383
3 ± 0 (100%)
4.087 ± 0.2818 (100%)
38.87 ± 3.882 (26%)
73.22 ± 19.98
23

3
RSOCCD
0.1619 ± 0.0203
0.7506 ± 0.0185
6.638 ± 0.0644
18.48 ± 8.41
0.0354 ± 0.0269
0.0374 ± 0.0244
0.0572 ± 0.0851
10 ± 0 (100%)
10 ± 0 (100%)
12.17 ± 7.91 (93%)
0.1592 ± 0.0479
100

------ Algorithms Employing a Model ------

1
SCFOv7
0.1932 ± 0.2072
0.2014 ± 0.2028
0.2835 ± 0.2256
7.19 ± 10.01
0.146 ± 0.2082
0.1534 ± 0.2039
0.2273 ± 0.2657
8.05 ± 11.8 (89%)
14.2 ± 15.05 (82%)
35.28 ± 10.48 (36%)
55.93 ± 17.49
100

1.01
SCFOv8
0.2206 ± 0.1673
0.2272 ± 0.1635
0.2928 ± 0.1651
5.75 ± 8.148
0.1381 ± 0.1765
0.1421 ± 0.175
0.1822 ± 0.206
21.35 ± 16.8 (85%)
24.95 ± 15.96 (80%)
37.4 ± 5.851 (55%)
70.43 ± 34.39
20

1.02
SCFOv9
0.1856 ± 0.1757
0.188 ± 0.1742
0.2125 ± 0.162
5 ± 6.801
0.1442 ± 0.1851
0.1485 ± 0.1818
0.1911 ± 0.1886
7.938 ± 12.5 (88%)
14.56 ± 15.66 (81%)
31.88 ± 13.44 (38%)
80.19 ± 11.8
16

2
SCFOv7f
0.1845 ± 0.18
0.1921 ± 0.1761
0.2679 ± 0.2038
6.44 ± 9.455
0.1323 ± 0.1757
0.1401 ± 0.1714
0.2179 ± 0.2565
9.05 ± 12.83 (88%)
12.99 ± 14.55 (81%)
35.55 ± 10.8 (32%)
16.65 ± 2.918
100

2.01
SCFOv8f
0.1882 ± 0.167
0.1952 ± 0.1636
0.2651 ± 0.1824
5.581 ± 7.73
0.1434 ± 0.1785
0.1466 ± 0.1768
0.1789 ± 0.1967
12.15 ± 14.45 (84%)
17.2 ± 15.66 (76%)
36.5 ± 9.871 (27%)
43.08 ± 36.05
74

2.02
SCFOv9f
0.201 ± 0.2105
0.2084 ± 0.2064
0.2817 ± 0.2057
6.71 ± 9.47
0.1456 ± 0.2053
0.1506 ± 0.2024
0.2001 ± 0.2202
9.903 ± 13.96 (84%)
12.35 ± 13.76 (84%)
36.16 ± 9.323 (32%)
71.28 ± 13.45
31

5
ConAdapt
0.1252 ± 0.0283
0.1817 ± 0.0292
0.7466 ± 0.1331
21.86 ± 3.302
0.1403 ± 0.1359
0.1741 ± 0.1203
0.5115 ± 0.5949
21.66 ± 14.56 (86%)
35.69 ± 6.148 (63%)
40.38 ± 1.377 (15%)
2.757 ± 0.511
100


Test Problem #6: Maximizing Production in a Continuous Stirred-Tank Reactor

Original code provided by: G. A. Bunin

Problem Description

Test File Specs: algotest([14.52 14.9],40,0.1,[0.03 0.03],[1 1],[50 50],[17.2 30.3],10,[1 1],[],algonum)

Main Files: phipeval.m (required), gpeval.m (required), phimod.m (model), gmod.m (model)

Results:

Algorithm
Average Suboptimality
Violations
Suboptimality at Final Experiment
Iterations to Convergence
Time
Trials

#
Name
M1 (λ := 1)
M2 (λ := 10)
M3 (λ := 100)
M4
M5 (λ := 1)
M6 (λ := 10)
M7 (λ := 100)
M8 (50%)
M9 (70%)
M10 (90%)
M11
 

0
nothing
0.6318
0.6318
0.6318
0
0.6318
0.6318
0.6318
NA
NA
NA
0
 

------ Model-Free Algorithms ------

1
SCFOv7
0.1313 ± 0.051
0.1732 ± 0.0943
0.5921 ± 0.6962
1.39 ± 1.984
0.0371 ± 0.0202
0.0446 ± 0.0598
0.1192 ± 0.5476
5.85 ± 3.623 (100%)
8.79 ± 5.93 (100%)
29.13 ± 10.7 (81%)
35.67 ± 8.848
100

1.01
SCFOv8
0.1496 ± 0.0552
0.233 ± 0.1317
1.066 ± 0.9997
1.364 ± 0.481
0.0387 ± 0.0134
0.0387 ± 0.0134
0.0387 ± 0.0134
6.727 ± 4.223 (100%)
11.36 ± 7.474 (100%)
29.91 ± 9.298 (73%)
46.24 ± 15.84
11

1.02
SCFOv9
0.1385 ± 0.0587
0.1642 ± 0.0924
0.4213 ± 0.555
0.8095 ± 0.7315
0.0397 ± 0.0244
0.0397 ± 0.0244
0.0397 ± 0.0244
6.333 ± 5.549 (100%)
9.571 ± 6.786 (100%)
30.29 ± 9.958 (71%)
71.13 ± 4.774
21

2
SCFOv7f
0.15 ± 0.065
0.2058 ± 0.1147
0.7629 ± 0.7844
1.3 ± 1.204
0.0422 ± 0.0297
0.0422 ± 0.0297
0.0422 ± 0.0297
7.02 ± 4.461 (100%)
10.86 ± 7.746 (100%)
29.05 ± 11.09 (72%)
3.289 ± 0.6527
100

2.01
SCFOv8f
0.1528 ± 0.0661
0.198 ± 0.1129
0.6507 ± 0.7422
1.105 ± 1.154
0.0472 ± 0.0346
0.0472 ± 0.0346
0.0472 ± 0.0346
6.947 ± 4.365 (100%)
12.3 ± 9.207 (100%)
29.96 ± 10.6 (66%)
6.009 ± 3.038
76

2.02
SCFOv9f
0.1488 ± 0.0502
0.1751 ± 0.0591
0.4386 ± 0.3759
1.25 ± 1.01
0.0493 ± 0.0214
0.0493 ± 0.0214
0.0493 ± 0.0214
6.208 ± 4.193 (100%)
10.38 ± 6.75 (100%)
31 ± 10.48 (63%)
11.17 ± 1.236
24

3
RSOCCD
0.8144 ± 0.0066
3.743 ± 0.1388
33.02 ± 1.566
37 ± 0
0.2797 ± 0.0087
0.9416 ± 0.1836
7.56 ± 2.071
11 ± 0 (100%)
33.2 ± 13.16 (26%)
41 ± 0 (0%)
0.2417 ± 0.0519
100

------ Algorithms Employing a Model ------

1
SCFOv7
0.3513 ± 0.1476
0.3513 ± 0.1476
0.3515 ± 0.1474
0.02 ± 0.14
0.1711 ± 0.164
0.1711 ± 0.164
0.1711 ± 0.164
21.63 ± 12.37 (81%)
29.57 ± 11.52 (63%)
39.13 ± 4.326 (23%)
35.44 ± 8.884
100

1.01
SCFOv8
0.2709 ± 0.0525
0.2709 ± 0.0525
0.2709 ± 0.0525
0 ± 0
0.1087 ± 0.0451
0.1087 ± 0.0451
0.1087 ± 0.0451
13.65 ± 5.275 (100%)
26.55 ± 8.721 (90%)
40.05 ± 4.141 (5%)
52.37 ± 20.94
20

1.02
SCFOv9
0.2712 ± 0.0762
0.2712 ± 0.0762
0.2712 ± 0.0762
0 ± 0
0.1017 ± 0.0718
0.1017 ± 0.0718
0.1017 ± 0.0718
16.13 ± 9.943 (100%)
27.38 ± 10.14 (88%)
39.5 ± 3.279 (25%)
85.39 ± 18.13
8

2
SCFOv7f
0.3189 ± 0.1334
0.3189 ± 0.1334
0.3193 ± 0.133
0.04 ± 0.2417
0.136 ± 0.1333
0.136 ± 0.1333
0.136 ± 0.1333
19.43 ± 11.29 (87%)
26.12 ± 11.44 (75%)
39.01 ± 5.11 (21%)
5.491 ± 1.23
100

2.01
SCFOv8f
0.2899 ± 0.1111
0.2899 ± 0.1111
0.2899 ± 0.1111
0 ± 0
0.1294 ± 0.1107
0.1294 ± 0.1107
0.1294 ± 0.1107
17.49 ± 11.49 (90%)
24.99 ± 11.1 (76%)
39.79 ± 3.529 (17%)
13.62 ± 13.43
96

2.02
SCFOv9f
0.3027 ± 0.1184
0.3027 ± 0.1184
0.3027 ± 0.1184
0 ± 0
0.1304 ± 0.1271
0.1304 ± 0.1271
0.1304 ± 0.1271
17.39 ± 9.858 (94%)
26.53 ± 10.96 (78%)
39.56 ± 3.804 (17%)
21.1 ± 4.497
36

5
ConAdapt
0.3611 ± 0
0.3611 ± 0
0.3611 ± 0
0 ± 0
0.3543 ± 0
0.3543 ± 0
0.3543 ± 0
41 ± 0 (0%)
41 ± 0 (0%)
41 ± 0 (0%)
4.369 ± 0.805
100


Test Problem #7: Maximizing Production in a Batch Reactor with a Reversible Reaction

Original code provided by: G. A. Bunin

Problem Description

Test File Specs: algotest([0 0],40,0.01,[],[-1 -1],[1 1],[-0.2884 -1],0.1,[],[],algonum)

Main Files: phipeval.m (required), phimod.m (model)

Results:

Algorithm
Average Suboptimality
Violations
Suboptimality at Final Experiment
Iterations to Convergence
Time
Trials

#
Name
M1 (λ := 1)
M2 (λ := 10)
M3 (λ := 100)
M4
M5 (λ := 1)
M6 (λ := 10)
M7 (λ := 100)
M8 (50%)
M9 (70%)
M10 (90%)
M11
 

0
nothing
0.873
0.873
0.873
0
0.873
0.873
0.873
NA
NA
NA
0
 

------ Model-Free Algorithms ------

1
SCFOv7
0.185 ± 0.0557
0.185 ± 0.0557
0.185 ± 0.0557
0.01 ± 0.0995
0.0502 ± 0.0497
0.0502 ± 0.0497
0.0502 ± 0.0497
7.48 ± 3.976 (100%)
11.52 ± 6.579 (99%)
34.47 ± 8.06 (57%)
17.27 ± 3.774
100

1.01
SCFOv8
0.1964 ± 0.0553
0.1964 ± 0.0553
0.1964 ± 0.0553
0 ± 0
0.0576 ± 0.0454
0.0576 ± 0.0454
0.0576 ± 0.0454
7.5 ± 3.978 (100%)
10.64 ± 4.76 (100%)
36.29 ± 8.101 (36%)
24.98 ± 8.885
14

1.02
SCFOv9
0.197 ± 0.093
0.197 ± 0.093
0.197 ± 0.093
0 ± 0
0.0705 ± 0.0685
0.0705 ± 0.0685
0.0705 ± 0.0685
7.176 ± 4.743 (100%)
12.06 ± 7.742 (100%)
35.71 ± 7.629 (47%)
38.84 ± 2.56
17

2
SCFOv7f
0.2037 ± 0.077
0.2037 ± 0.077
0.2037 ± 0.077
0.04 ± 0.196
0.0584 ± 0.0572
0.0584 ± 0.0572
0.0584 ± 0.0572
8.31 ± 5.692 (100%)
13.53 ± 7.866 (97%)
36.7 ± 6.485 (47%)
2.418 ± 0.5273
100

2.01
SCFOv8f
0.1932 ± 0.0685
0.1932 ± 0.0685
0.1932 ± 0.0685
0 ± 0
0.0481 ± 0.0485
0.0481 ± 0.0485
0.0481 ± 0.0485
7.506 ± 4.2 (100%)
12.64 ± 7.193 (100%)
35.95 ± 6.185 (64%)
3.921 ± 2.741
85

2.02
SCFOv9f
0.1996 ± 0.0759
0.1996 ± 0.0759
0.1996 ± 0.0759
0.1212 ± 0.4088
0.0568 ± 0.0524
0.0568 ± 0.0524
0.0568 ± 0.0524
8.818 ± 6.002 (100%)
14 ± 9.277 (97%)
36.55 ± 6.12 (52%)
10.65 ± 1.854
33

3
RSOCCD
0.3242 ± 0.0453
0.3242 ± 0.0453
0.3242 ± 0.0453
0 ± 0
0.0733 ± 0.0599
0.0733 ± 0.0599
0.0733 ± 0.0599
11 ± 0 (100%)
11 ± 0 (100%)
24.2 ± 14.89 (56%)
0.1048 ± 0.0418
100

4
SimplexB
0.263 ± 0.1955
0.263 ± 0.1955
0.263 ± 0.1955
0 ± 0
0.1626 ± 0.2301
0.1626 ± 0.2301
0.1626 ± 0.2301
12.09 ± 12.08 (86%)
13.91 ± 12.13 (84%)
24.54 ± 14.08 (60%)
0.0202 ± 0.0076
100

------ Algorithms Employing a Model ------

1
SCFOv7
0.1717 ± 0.0216
0.1717 ± 0.0216
0.1717 ± 0.0216
0.06 ± 0.3693
0.1077 ± 0.0314
0.1077 ± 0.0314
0.1077 ± 0.0314
4.9 ± 0.6083 (100%)
5.96 ± 0.6621 (100%)
40.63 ± 1.653 (8%)
18.3 ± 4.283
100

1.01
SCFOv8
0.1714 ± 0.0273
0.1714 ± 0.0273
0.1714 ± 0.0273
0.08 ± 0.2713
0.0786 ± 0.038
0.0786 ± 0.038
0.0786 ± 0.038
4.92 ± 0.56 (100%)
6.12 ± 0.7111 (100%)
39.88 ± 2.065 (24%)
21.59 ± 14.49
25

1.02
SCFOv9
0.1681 ± 0.0261
0.1681 ± 0.0261
0.1681 ± 0.0261
0.087 ± 0.2818
0.0877 ± 0.043
0.0877 ± 0.043
0.0877 ± 0.043
4.826 ± 0.4802 (100%)
5.87 ± 0.536 (100%)
39.26 ± 4.035 (17%)
41.36 ± 7.591
23

2
SCFOv7f
0.1317 ± 0.0234
0.1317 ± 0.0234
0.1317 ± 0.0234
0.01 ± 0.0995
0.0354 ± 0.0375
0.0354 ± 0.0375
0.0354 ± 0.0375
5.28 ± 1.068 (100%)
7.46 ± 3.887 (100%)
33.52 ± 9.064 (70%)
8.457 ± 2.415
100

2.01
SCFOv8f
0.1307 ± 0.0326
0.1307 ± 0.0326
0.1307 ± 0.0326
0.0247 ± 0.1552
0.0348 ± 0.0372
0.0348 ± 0.0372
0.0348 ± 0.0372
5.42 ± 1.465 (100%)
7.099 ± 1.928 (100%)
31.72 ± 9.31 (74%)
19.76 ± 24.77
81

2.02
SCFOv9f
0.1269 ± 0.0278
0.1269 ± 0.0278
0.1269 ± 0.0278
0.0588 ± 0.2353
0.0372 ± 0.0342
0.0372 ± 0.0342
0.0372 ± 0.0342
5.324 ± 1.604 (100%)
7.206 ± 4.451 (100%)
33.15 ± 9.595 (62%)
27.51 ± 7.075
34


Test Problem #8: Maximizing Production in a Fed-Batch Reactor with Three Reactions

Original code provided by: G. A. Bunin

Problem Description

Test File Specs: algotest([0 0 0],60,0.01,[],[-1 -0.5 -1.5],[1 0.5 1.5],[0.1953 0.3770 -1.0027],0.1,[],[1 1 1 1],algonum)

Main Files: phipeval.m (required), geval.m (required), phimod.m (model)

Results:

Algorithm
Average Suboptimality
Violations
Suboptimality at Final Experiment
Iterations to Convergence
Time
Trials

#
Name
M1 (λ := 1)
M2 (λ := 10)
M3 (λ := 100)
M4
M5 (λ := 1)
M6 (λ := 10)
M7 (λ := 100)
M8 (50%)
M9 (70%)
M10 (90%)
M11
 

0
nothing
0.5821
0.5821
0.5821
0
0.5821
0.5821
0.5821
NA
NA
NA
0
 

------ Model-Free Algorithms ------

1
SCFOv7
0.1734 ± 0.0331
0.1734 ± 0.0331
0.1734 ± 0.0331
0 ± 0
0.1424 ± 0.0653
0.1424 ± 0.0653
0.1424 ± 0.0653
30.24 ± 21.28 (92%)
57.1 ± 9.224 (43%)
60.93 ± 0.4529 (2%)
18.91 ± 2.506
100

1.01
SCFOv8
0.1815 ± 0.0447
0.1815 ± 0.0447
0.1815 ± 0.0447
0 ± 0
0.1401 ± 0.0726
0.1401 ± 0.0726
0.1401 ± 0.0726
30.56 ± 18.53 (94%)
54.67 ± 14.66 (33%)
61 ± 0 (0%)
32.43 ± 13.48
18

1.02
SCFOv9
0.1708 ± 0.0343
0.1708 ± 0.0343
0.1708 ± 0.0343
0 ± 0
0.1485 ± 0.0593
0.1485 ± 0.0593
0.1485 ± 0.0593
31.25 ± 20.04 (100%)
55.06 ± 11.83 (38%)
61 ± 0 (0%)
45.77 ± 12.2
16

2
SCFOv7f
0.1757 ± 0.0373
0.1757 ± 0.0373
0.1757 ± 0.0373
0 ± 0
0.1329 ± 0.068
0.1329 ± 0.068
0.1329 ± 0.068
28.63 ± 20.58 (93%)
56.95 ± 7.896 (48%)
60.96 ± 0.3137 (1%)
3.465 ± 0.9567
100

2.01
SCFOv8f
0.1733 ± 0.0342
0.1733 ± 0.0342
0.1733 ± 0.0342
0 ± 0
0.1417 ± 0.068
0.1417 ± 0.068
0.1417 ± 0.068
29.89 ± 21.61 (88%)
57.35 ± 8.895 (41%)
60.98 ± 0.1516 (0%)
5.636 ± 4.178
85

2.02
SCFOv9f
0.1869 ± 0.0403
0.1869 ± 0.0403
0.1869 ± 0.0403
0 ± 0
0.141 ± 0.0657
0.141 ± 0.0657
0.141 ± 0.0657
29.56 ± 21.67 (96%)
55.81 ± 10.29 (41%)
60.96 ± 0.1889 (0%)
12.3 ± 2.412
27

3
RSOCCD
1.314 ± 0.0042
6.826 ± 0.0042
61.95 ± 0.0042
17 ± 0
0.179 ± 0.0058
0.179 ± 0.0058
0.179 ± 0.0058
17 ± 0 (100%)
50.88 ± 18.52 (23%)
61 ± 0 (0%)
0.1574 ± 0.0618
100

------ Algorithms Employing a Model ------

1
SCFOv7
0.188 ± 0.0392
0.188 ± 0.0392
0.188 ± 0.0392
0 ± 0
0.1464 ± 0.0546
0.1464 ± 0.0546
0.1464 ± 0.0546
22 ± 20.9 (95%)
58.13 ± 6.28 (38%)
61 ± 0 (0%)
22.57 ± 3.86
100

1.01
SCFOv8
0.1864 ± 0.0337
0.1864 ± 0.0337
0.1864 ± 0.0337
0 ± 0
0.1356 ± 0.0366
0.1356 ± 0.0366
0.1356 ± 0.0366
21.25 ± 20.08 (100%)
58.67 ± 2.593 (50%)
61 ± 0 (0%)
62.29 ± 47.19
12

1.02
SCFOv9
0.192 ± 0.0419
0.192 ± 0.0419
0.192 ± 0.0419
0 ± 0
0.1397 ± 0.0447
0.1397 ± 0.0447
0.1397 ± 0.0447
22.36 ± 18.7 (95%)
59.09 ± 5.977 (23%)
61 ± 0 (0%)
47.72 ± 7.254
22

2
SCFOv7f
0.186 ± 0.0435
0.186 ± 0.0435
0.186 ± 0.0435
0 ± 0
0.1333 ± 0.0516
0.1333 ± 0.0516
0.1333 ± 0.0516
23.23 ± 20.92 (94%)
53.92 ± 14.32 (41%)
60.99 ± 0.0995 (0%)
8.779 ± 1.5
100

2.01
SCFOv8f
0.1756 ± 0.0351
0.1756 ± 0.0351
0.1756 ± 0.0351
0 ± 0
0.139 ± 0.0541
0.139 ± 0.0541
0.139 ± 0.0541
17.91 ± 19.23 (96%)
52.61 ± 15.33 (51%)
61 ± 0 (0%)
25.98 ± 35.61
67

2.02
SCFOv9f
0.1846 ± 0.0321
0.1846 ± 0.0321
0.1846 ± 0.0321
0 ± 0
0.1368 ± 0.0492
0.1368 ± 0.0492
0.1368 ± 0.0492
16.09 ± 17.04 (95%)
54.23 ± 14.11 (45%)
61 ± 0 (0%)
24.18 ± 2.254
22


Test Problem #9: Batch-to-Batch Tuning of a Temperature-Tracking Model-Predictive Controller

Original code provided by: G. A. Bunin

Problem Description

Test File Specs: algotest([0 0.7],40,0.1,[],[-2 0],[2 1],[0.9492 0.6416],1,[],[],algonum)

Main Files: phipeval.m (required), phimod.m (model)

Results:

Algorithm
Average Suboptimality
Violations
Suboptimality at Final Experiment
Iterations to Convergence
Time
Trials

#
Name
M1 (λ := 1)
M2 (λ := 10)
M3 (λ := 100)
M4
M5 (λ := 1)
M6 (λ := 10)
M7 (λ := 100)
M8 (50%)
M9 (70%)
M10 (90%)
M11
 

0
nothing
0.9232
0.9232
0.9232
0
0.9232
0.9232
0.9232
NA
NA
NA
0
 

------ Model-Free Algorithms ------

1
SCFOv7
0.1454 ± 0.0398
0.1454 ± 0.0398
0.1454 ± 0.0398
0 ± 0
0.0826 ± 0.0968
0.0826 ± 0.0968
0.0826 ± 0.0968
9.72 ± 8.888 (100%)
25.92 ± 12.39 (86%)
39.19 ± 2.982 (33%)
19.9 ± 6.998
100

1.01
SCFOv8
0.132 ± 0.0416
0.132 ± 0.0416
0.132 ± 0.0416
0 ± 0
0.059 ± 0.0835
0.059 ± 0.0835
0.059 ± 0.0835
9.1 ± 7.613 (100%)
25.8 ± 13.1 (87%)
38.97 ± 2.041 (50%)
25.01 ± 10.1
30

1.02
SCFOv9
0.1439 ± 0.0511
0.1439 ± 0.0511
0.1439 ± 0.0511
0 ± 0
0.0437 ± 0.0566
0.0437 ± 0.0566
0.0437 ± 0.0566
7.833 ± 6.656 (100%)
26.96 ± 10.2 (92%)
38.71 ± 3.323 (38%)
43.8 ± 8.459
24

2
SCFOv7f
0.1422 ± 0.0348
0.1422 ± 0.0348
0.1422 ± 0.0348
0 ± 0
0.0707 ± 0.1021
0.0707 ± 0.1021
0.0707 ± 0.1021
10.78 ± 9.759 (97%)
28.94 ± 10.31 (86%)
39.3 ± 2.456 (40%)
2.242 ± 0.4342
100

2.01
SCFOv8f
0.1416 ± 0.0393
0.1416 ± 0.0393
0.1416 ± 0.0393
0 ± 0
0.0605 ± 0.0867
0.0605 ± 0.0867
0.0605 ± 0.0867
8.63 ± 8.511 (100%)
30.36 ± 10.19 (83%)
39.41 ± 2.367 (37%)
2.651 ± 0.6351
100

2.02
SCFOv9f
0.1472 ± 0.0333
0.1472 ± 0.0333
0.1472 ± 0.0333
0.0122 ± 0.1098
0.0708 ± 0.0902
0.0708 ± 0.0902
0.0708 ± 0.0902
9.22 ± 8.42 (100%)
29.83 ± 10.16 (87%)
39.18 ± 3.596 (33%)
10.92 ± 2.658
82

3
RSOCCD
3.783 ± 0.0014
3.783 ± 0.0014
3.783 ± 0.0014
0 ± 0
0.0253 ± 0.0018
0.0253 ± 0.0018
0.0253 ± 0.0018
11 ± 0 (100%)
11 ± 0 (100%)
11 ± 0 (100%)
0.0782 ± 0.0141
100

4
SimplexB
0.1445 ± 0.0507
0.1445 ± 0.0507
0.1445 ± 0.0507
0 ± 0
0.0333 ± 0.0331
0.0333 ± 0.0331
0.0333 ± 0.0331
8.17 ± 3.265 (100%)
10.39 ± 3.146 (100%)
16.88 ± 8.749 (91%)
0.0227 ± 0.0068
100

------ Algorithms Employing a Model ------

1
SCFOv7
0.4129 ± 0.1624
0.4129 ± 0.1624
0.4129 ± 0.1624
0 ± 0
0.2417 ± 0.2175
0.2417 ± 0.2175
0.2417 ± 0.2175
28.79 ± 12.33 (72%)
36.51 ± 6.198 (51%)
40.52 ± 1.389 (11%)
45.09 ± 10.63
100

1.01
SCFOv8
0.4115 ± 0.1549
0.4115 ± 0.1549
0.4115 ± 0.1549
0 ± 0
0.2126 ± 0.1906
0.2126 ± 0.1906
0.2126 ± 0.1906
27.63 ± 11.66 (81%)
37.48 ± 4.628 (44%)
40.08 ± 2.07 (21%)
155.8 ± 360
48

1.02
SCFOv9
0.3895 ± 0.1411
0.3895 ± 0.1411
0.3895 ± 0.1411
0 ± 0
0.1466 ± 0.103
0.1466 ± 0.103
0.1466 ± 0.103
26.3 ± 12.01 (80%)
36.55 ± 6.289 (55%)
39.8 ± 2.112 (25%)
112.4 ± 14.74
20

2
SCFOv7f
0.3737 ± 0.1831
0.3737 ± 0.1831
0.3737 ± 0.1831
0 ± 0
0.0626 ± 0.0749
0.0626 ± 0.0749
0.0626 ± 0.0749
20.65 ± 11.76 (96%)
30.55 ± 9.785 (81%)
38.17 ± 3.852 (47%)
29.75 ± 4.193
100

2.01
SCFOv8f
0.4058 ± 0.1979
0.4058 ± 0.1979
0.4058 ± 0.1979
0 ± 0
0.1081 ± 0.1343
0.1081 ± 0.1343
0.1081 ± 0.1343
21.88 ± 10.96 (97%)
30.29 ± 9.233 (86%)
38.67 ± 3.441 (42%)
37.18 ± 8.17
100

2.02
SCFOv9f
0.4657 ± 0.1616
0.4657 ± 0.1616
0.4657 ± 0.1616
0 ± 0
0.1145 ± 0.1962
0.1145 ± 0.1962
0.1145 ± 0.1962
26.62 ± 10.12 (88%)
33.5 ± 7.348 (75%)
38.46 ± 3.827 (43%)
84.73 ± 14.07
100


Test Problem #10: Iterative Tuning of a Fixed-Order Controller in a Torsional System

Original code provided by: G. A. Bunin

Problem Description

Test File Specs: algotest([1 2.77 -2.6 1 0.5],100,0.001,[],[0 0 -5 -2 -1],[5 5 5 2 1],[4.812 0.3826 -2.9531 0.1292 -0.5116],0.05,[],[2 2],algonum)

Main Files: phipeval.m (required), geval.m (required), phimod.m (model)

Results:

Algorithm
Average Suboptimality
Violations
Suboptimality at Final Experiment
Iterations to Convergence
Time
Trials

#
Name
M1 (λ := 1)
M2 (λ := 10)
M3 (λ := 100)
M4
M5 (λ := 1)
M6 (λ := 10)
M7 (λ := 100)
M8 (50%)
M9 (70%)
M10 (90%)
M11
 

0
nothing
0.7401
0.7401
0.7401
0
0.7401
0.7401
0.7401
NA
NA
NA
0
 

------ Model-Free Algorithms ------

1
SCFOv7
0.3809 ± 0.4178
0.3809 ± 0.4178
0.3809 ± 0.4178
0 ± 0
0.1502 ± 0.6047
0.1502 ± 0.6047
0.1502 ± 0.6047
38.09 ± 33.63 (95%)
49.62 ± 32.81 (91%)
82.23 ± 24.75 (63%)
42.63 ± 9.744
100

1.01
SCFOv8
0.5401 ± 0.5089
0.5401 ± 0.5089
0.5401 ± 0.5089
0 ± 0
0.2679 ± 0.9433
0.2679 ± 0.9433
0.2679 ± 0.9433
30.42 ± 27.4 (95%)
35.58 ± 25.91 (95%)
62.58 ± 30.43 (79%)
58.11 ± 16.46
19

1.02
SCFOv9
0.5229 ± 0.504
0.5229 ± 0.504
0.5229 ± 0.504
0 ± 0
2.237 ± 8.473
2.237 ± 8.473
2.237 ± 8.473
47.13 ± 30.33 (94%)
51.5 ± 28.92 (94%)
73.63 ± 24.36 (81%)
73.62 ± 10
16

2
SCFOv7f
0.6102 ± 0.8314
0.6102 ± 0.8314
0.6102 ± 0.8314
0 ± 0
0.2406 ± 1.277
0.2406 ± 1.277
0.2406 ± 1.277
46.93 ± 35.79 (89%)
55.28 ± 35.27 (83%)
87 ± 20.15 (55%)
4.003 ± 1.089
100

2.01
SCFOv8f
0.6497 ± 0.9924
0.6497 ± 0.9924
0.6497 ± 0.9924
0 ± 0
0.6353 ± 3.86
0.6353 ± 3.86
0.6353 ± 3.86
45.79 ± 37.01 (91%)
55.07 ± 35.99 (87%)
86.21 ± 18.45 (63%)
6.091 ± 3.254
89

2.02
SCFOv9f
0.6276 ± 0.7282
0.6276 ± 0.7282
0.6276 ± 0.7282
0 ± 0
1.023 ± 5.747
1.023 ± 5.747
1.023 ± 5.747
49.11 ± 33.3 (94%)
56.11 ± 33.13 (92%)
82.81 ± 22.73 (69%)
14.28 ± 3.67
36

3
RSOCCD
8.726 ± 0.0007
10.24 ± 0.0007
25.39 ± 0.0007
10 ± 0
0.4623 ± 0.001
0.4623 ± 0.001
0.4623 ± 0.001
101 ± 0 (0%)
101 ± 0 (0%)
101 ± 0 (0%)
0.2423 ± 0.0489
100

------ Algorithms Employing a Model ------

1
SCFOv7
0.2606 ± 0.095
0.2606 ± 0.095
0.2606 ± 0.095
0 ± 0
0.1954 ± 0.0683
0.1954 ± 0.0683
0.1954 ± 0.0683
14.99 ± 24.25 (98%)
57.63 ± 42.44 (67%)
100.9 ± 0.9763 (2%)
42.04 ± 7.952
100

1.01
SCFOv8
0.2602 ± 0.0842
0.2602 ± 0.0842
0.2602 ± 0.0842
0 ± 0
0.1906 ± 0.0662
0.1906 ± 0.0662
0.1906 ± 0.0662
14.29 ± 21.87 (100%)
57.76 ± 41.93 (67%)
101 ± 0 (0%)
90.54 ± 70.84
21

1.02
SCFOv9
0.2681 ± 0.1173
0.2681 ± 0.1173
0.2681 ± 0.1173
0 ± 0
0.2003 ± 0.0867
0.2003 ± 0.0867
0.2003 ± 0.0867
21.08 ± 30.26 (100%)
58.25 ± 45.52 (58%)
97.42 ± 11.88 (8%)
77.05 ± 7.692
12

2
SCFOv7f
0.2505 ± 0.0974
0.2505 ± 0.0974
0.2505 ± 0.0974
0 ± 0
0.1846 ± 0.0703
0.1846 ± 0.0703
0.1846 ± 0.0703
13.21 ± 23.44 (98%)
53.78 ± 43.49 (70%)
100.8 ± 1.511 (3%)
5.298 ± 1.137
100

2.01
SCFOv8f
0.243 ± 0.081
0.243 ± 0.081
0.243 ± 0.081
0 ± 0
0.1974 ± 0.1801
0.1974 ± 0.1801
0.1974 ± 0.1801
11.92 ± 21.5 (97%)
54.06 ± 41.19 (73%)
100.9 ± 0.8014 (3%)
17.63 ± 39.05
79

2.02
SCFOv9f
0.274 ± 0.1221
0.274 ± 0.1221
0.274 ± 0.1221
0 ± 0
0.1972 ± 0.0834
0.1972 ± 0.0834
0.1972 ± 0.0834
20.15 ± 30.67 (95%)
53.9 ± 44.17 (60%)
101 ± 0 (0%)
16.66 ± 4.767
40


Test Problem #11: Minimizing the Steady-State Production Cost of a Gold Cyanidation Leaching Process

Original code provided by: Zhang Jun (张俊)

Problem Description

Test File Specs: algotest([52 18],40,2,.001,[10 5],[80 20],[31.0563 11.6218],200,.04,[],algonum)

Main Files: phipeval.m (required), gpeval.m (required), phimod.m (model), gmod.m (model)

Auxiliary Files: model_single.m (required)

Results:

Algorithm
Average Suboptimality
Violations
Suboptimality at Final Experiment
Iterations to Convergence
Time
Trials

#
Name
M1 (λ := 1)
M2 (λ := 10)
M3 (λ := 100)
M4
M5 (λ := 1)
M6 (λ := 10)
M7 (λ := 100)
M8 (50%)
M9 (70%)
M10 (90%)
M11
 

0
nothing
0.687
0.687
0.687
0
0.687
0.687
0.687
NA
NA
NA
0
 

------ Model-Free Algorithms ------

1
SCFOv7
0.0751 ± 0.0191
0.0753 ± 0.0193
0.077 ± 0.0248
0.05 ± 0.2958
0.0053 ± 0.0053
0.0061 ± 0.01
0.0139 ± 0.0871
4.07 ± 0.7906 (100%)
4.92 ± 0.8565 (100%)
9.03 ± 4.412 (100%)
23.35 ± 1.918
100

1.01
SCFOv8
0.083 ± 0.0199
0.0836 ± 0.0207
0.0904 ± 0.0456
0.1481 ± 0.7554
0.0056 ± 0.0056
0.0056 ± 0.0056
0.0056 ± 0.0056
4.074 ± 0.7662 (100%)
5.148 ± 1.008 (100%)
9.889 ± 4.833 (100%)
31.23 ± 8.918
27

1.02
SCFOv9
0.0817 ± 0.0273
0.082 ± 0.0271
0.0853 ± 0.0275
0.1429 ± 0.3499
0.0051 ± 0.0077
0.0051 ± 0.0077
0.0051 ± 0.0077
4.214 ± 0.7726 (100%)
5.143 ± 0.833 (100%)
10.57 ± 5.729 (100%)
56.25 ± 4.54
14

2
SCFOv7f
0.0724 ± 0.0172
0.0725 ± 0.0172
0.0727 ± 0.0173
0.01 ± 0.0995
0.0089 ± 0.0136
0.0089 ± 0.0136
0.0089 ± 0.0136
4.11 ± 0.7469 (100%)
4.96 ± 0.7864 (100%)
10.6 ± 8.615 (99%)
4.103 ± 1.424
100

2.01
SCFOv8f
0.0719 ± 0.0151
0.0722 ± 0.0151
0.0751 ± 0.0213
0.1196 ± 0.5869
0.006 ± 0.0063
0.0064 ± 0.0074
0.0104 ± 0.0422
4.022 ± 0.7068 (100%)
4.913 ± 0.7753 (100%)
10.27 ± 8.502 (99%)
6.98 ± 5.161
92

2.02
SCFOv9f
0.0748 ± 0.0211
0.0749 ± 0.0211
0.0752 ± 0.021
0.027 ± 0.1622
0.0093 ± 0.0145
0.0093 ± 0.0145
0.0093 ± 0.0145
4.189 ± 0.7292 (100%)
5 ± 0.8383 (100%)
10.43 ± 9.122 (97%)
14.72 ± 3.903
37

3
RSOCCD
0.7568 ± 0.0011
1.963 ± 0.0011
14.02 ± 0.0011
4 ± 0
0.3259 ± 0.0015
0.3259 ± 0.0015
0.3259 ± 0.0015
10 ± 0 (100%)
41 ± 0 (0%)
41 ± 0 (0%)
0.1798 ± 0.073
100

------ Algorithms Employing a Model ------

1
SCFOv7
0.3164 ± 0.0442
0.3164 ± 0.0442
0.3164 ± 0.0442
0 ± 0
0.0608 ± 0.0288
0.0608 ± 0.0288
0.0608 ± 0.0288
17.03 ± 2.92 (100%)
24.39 ± 3.952 (100%)
37.21 ± 4.318 (56%)
25.6 ± 3.534
100

1.01
SCFOv8
0.2815 ± 0.0388
0.2815 ± 0.0388
0.2815 ± 0.0388
0 ± 0
0.0461 ± 0.0215
0.0461 ± 0.0215
0.0461 ± 0.0215
14.67 ± 2.609 (100%)
21.79 ± 3.266 (100%)
34.92 ± 5.074 (71%)
39.8 ± 19.48
24

1.02
SCFOv9
0.2885 ± 0.031
0.2885 ± 0.031
0.2885 ± 0.031
0 ± 0
0.0534 ± 0.0252
0.0534 ± 0.0252
0.0534 ± 0.0252
15 ± 2.148 (100%)
22.38 ± 3.52 (100%)
36.46 ± 5.329 (54%)
64.38 ± 11.65
13

2
SCFOv7f
0.3618 ± 0.0441
0.3618 ± 0.0441
0.3618 ± 0.0441
0 ± 0
0.1052 ± 0.0399
0.1052 ± 0.0399
0.1052 ± 0.0399
20.42 ± 3.439 (100%)
30.26 ± 4.77 (95%)
40.41 ± 1.656 (12%)
3.11 ± 0.6163
100

2.01
SCFOv8f
0.3386 ± 0.0489
0.3386 ± 0.0489
0.3386 ± 0.0489
0 ± 0
0.0931 ± 0.0381
0.0931 ± 0.0381
0.0931 ± 0.0381
18.43 ± 3.71 (100%)
28.28 ± 5.465 (98%)
39.45 ± 3.967 (18%)
9.097 ± 12.5
83

2.02
SCFOv9f
0.3389 ± 0.0369
0.3389 ± 0.0369
0.3389 ± 0.0369
0 ± 0
0.0893 ± 0.0321
0.0893 ± 0.0321
0.0893 ± 0.0321
18.23 ± 2.839 (100%)
28.2 ± 3.97 (100%)
40.23 ± 1.587 (20%)
11.22 ± 2.763
35

5
ConAdapt
0.6616 ± 0
0.6616 ± 0
0.6616 ± 0
0 ± 0
0.661 ± 0
0.661 ± 0
0.661 ± 0
41 ± 0 (0%)
41 ± 0 (0%)
41 ± 0 (0%)
6.963 ± 1.81
100