The following are the test problems currently making up the database, together with the performances obtained by the different tested algorithms. To make it easier to pair up problems with algorithms and to analyze the results, the problems are broken up into three classes, with Class 1 problems denoting those with only bound constraints, Class 2 problems as those with general numerical constraints but no experimental constraints, and Class 3 problems being those that possess experimental constraints.

Class 1: P1, P7, P9

Class 2: P8, P10

Class 3: P2, P3, P4, P5, P6, P11

In reporting the metric values, both the average (left) and the standard error (right) are given, with the plus-minus (±) sign used to separate the two. For Metrics 8-10, which attempt to gauge convergence speed, the percentage of the trials that achieved the specified convergence is also given. In the case that convergence was not achieved for any of the trials, the letters "NA" (not available) are reported. Currently, testing is done automatically by dedicated computers, with a maximum of 100 trials done for each testing combination (generated by choosing the problem, the algorithm, and whether or not the model is used).

You may view the plotted results for the individual trials for each algorithm by clicking on the algorithm name in the "Name" column. For problems with only two variables, plots of the experimental iterates in the decision-variable space are given for particularly insightful illustrations - only their values are plotted for problems with three or more variables. For plots of the decision-variable space, green regions denote those that are feasible (where all constraints are satisfied), while red denotes the infeasible regions. Red dots denote the individual experimental iterates, while the green dot denotes the true optimum that the algorithm aims to find. Constant dotted lines are used to plot the optimal values in the three-or-more variable case. For the cost function value plots, the constant black line denotes the value at the true optimum.


Test Problem #1: Maximizing Profit in the Williams-Otto Reactor

Original code provided by: S. Costello

Problem Description

Test File Specs: algotest([4.8 77],40,0.5,[],[3 70],[6 100],[4.79 89.7],100,[],[],algonum)

Main Files: phipeval.m (required), phimod.m (model)

Auxiliary Files: plantbalancesT.m (required), modelbalances2.m (model)

Results:

Algorithm
Average Suboptimality
Violations
Suboptimality at Final Experiment
Iterations to Convergence
Time
Trials

#
Name
M1 (λ := 1)
M2 (λ := 10)
M3 (λ := 100)
M4
M5 (λ := 1)
M6 (λ := 10)
M7 (λ := 100)
M8 (50%)
M9 (70%)
M10 (90%)
M11
 

0
nothing
0.5186
0.5186
0.5186
0
0.5186
0.5186
0.5186
NA
NA
NA
0
 

------ Model-Free Algorithms ------

1
SCFOv7
0.0455 ± 0.0118
0.0455 ± 0.0118
0.0455 ± 0.0118
0 ± 0
0.0023 ± 0.0031
0.0023 ± 0.0031
0.0023 ± 0.0031
3.63 ± 0.4828 (100%)
4.37 ± 0.9017 (100%)
6.61 ± 1.949 (100%)
15.63 ± 2.811
100

1.01
SCFOv8
0.0508 ± 0.012
0.0508 ± 0.012
0.0508 ± 0.012
0 ± 0
0.0032 ± 0.0044
0.0032 ± 0.0044
0.0032 ± 0.0044
3.765 ± 0.4242 (100%)
4.647 ± 1.026 (100%)
6.882 ± 0.963 (100%)
21.32 ± 5.515
17

1.02
SCFOv9
0.0451 ± 0.007
0.0451 ± 0.007
0.0451 ± 0.007
0 ± 0
0.0027 ± 0.0039
0.0027 ± 0.0039
0.0027 ± 0.0039
4 ± 0 (100%)
4.167 ± 0.3727 (100%)
6.5 ± 1.803 (100%)
40.57 ± 1.214
6

2
SCFOv7f
0.0481 ± 0.0131
0.0481 ± 0.0131
0.0481 ± 0.0131
0 ± 0
0.0041 ± 0.0046
0.0041 ± 0.0046
0.0041 ± 0.0046
3.66 ± 0.4737 (100%)
4.19 ± 0.6737 (100%)
6.87 ± 2.524 (100%)
2.289 ± 0.6446
100

2.01
SCFOv8f
0.0478 ± 0.0125
0.0478 ± 0.0125
0.0478 ± 0.0125
0 ± 0
0.0038 ± 0.0042
0.0038 ± 0.0042
0.0038 ± 0.0042
3.684 ± 0.4648 (100%)
4.25 ± 0.6097 (100%)
6.934 ± 2.672 (100%)
3.096 ± 1.967
76

2.02
SCFOv9f
0.0496 ± 0.0165
0.0496 ± 0.0165
0.0496 ± 0.0165
0 ± 0
0.0013 ± 0.0015
0.0013 ± 0.0015
0.0013 ± 0.0015
3.571 ± 0.4949 (100%)
4 ± 0.9258 (100%)
7.429 ± 4.03 (100%)
9.074 ± 0.9535
7

3
RSOCCD
0.1993 ± 0.0003
0.1993 ± 0.0003
0.1993 ± 0.0003
0 ± 0
0.0049 ± 0.0004
0.0049 ± 0.0004
0.0049 ± 0.0004
10 ± 0 (100%)
10 ± 0 (100%)
10 ± 0 (100%)
0.1472 ± 0.0949
100

4
SimplexB
0.0733 ± 0.0192
0.0733 ± 0.0192
0.0733 ± 0.0192
0 ± 0
0.0025 ± 0.0048
0.0025 ± 0.0048
0.0025 ± 0.0048
6.51 ± 2.488 (100%)
9.67 ± 2.761 (100%)
14.66 ± 3.55 (100%)
0.0221 ± 0.0116
100

------ Algorithms Employing a Model ------

1
SCFOv7
0.0472 ± 0.0123
0.0472 ± 0.0123
0.0472 ± 0.0123
0 ± 0
0.0019 ± 0.0029
0.0019 ± 0.0029
0.0019 ± 0.0029
3.77 ± 0.4659 (100%)
4.35 ± 0.7124 (100%)
6.25 ± 1.417 (100%)
15.38 ± 2.242
100

1.01
SCFOv8
0.047 ± 0.0129
0.047 ± 0.0129
0.047 ± 0.0129
0 ± 0
0.0018 ± 0.0029
0.0018 ± 0.0029
0.0018 ± 0.0029
3.529 ± 0.4991 (100%)
4.471 ± 0.9151 (100%)
5.706 ± 1.273 (100%)
28.18 ± 15.6
17

1.02
SCFOv9
0.0495 ± 0.017
0.0495 ± 0.017
0.0495 ± 0.017
0 ± 0
0.0052 ± 0.0065
0.0052 ± 0.0065
0.0052 ± 0.0065
3.667 ± 0.4714 (100%)
4.333 ± 0.7454 (100%)
6.333 ± 1.7 (100%)
40.07 ± 1.819
6

2
SCFOv7f
0.0527 ± 0.0135
0.0527 ± 0.0135
0.0527 ± 0.0135
0 ± 0
0.0035 ± 0.004
0.0035 ± 0.004
0.0035 ± 0.004
3.77 ± 0.4208 (100%)
4.54 ± 0.8297 (100%)
7.03 ± 3.756 (100%)
2.567 ± 0.579
100

2.01
SCFOv8f
0.0498 ± 0.0129
0.0498 ± 0.0129
0.0498 ± 0.0129
0 ± 0
0.0038 ± 0.0047
0.0038 ± 0.0047
0.0038 ± 0.0047
3.709 ± 0.4541 (100%)
4.395 ± 0.8391 (100%)
6.802 ± 4.031 (100%)
4.055 ± 3.968
86

2.02
SCFOv9f
0.0541 ± 0.012
0.0541 ± 0.012
0.0541 ± 0.012
0 ± 0
0.0037 ± 0.0036
0.0037 ± 0.0036
0.0037 ± 0.0036
3.857 ± 0.3499 (100%)
4.857 ± 0.6389 (100%)
6.143 ± 1.125 (100%)
10.17 ± 0.6192
7


Test Problem #2: Maximizing Profit in the Williams-Otto Reactor (Constrained)

Original code provided by: S. Costello

Problem Description

Test File Specs: algotest([3.5 72],40,0.5,5e-4,[3 70],[6 100],[4.97 84.3],100,.1,[],algonum)

Main Files: phipeval.m (required), gpeval.m (required), phimod.m (model), gmod.m (model)

Auxiliary Files: plantbalancesT.m (required), modelbalances2.m (model)

Results:

Algorithm
Average Suboptimality
Violations
Suboptimality at Final Experiment
Iterations to Convergence
Time
Trials

#
Name
M1 (λ := 1)
M2 (λ := 10)
M3 (λ := 100)
M4
M5 (λ := 1)
M6 (λ := 10)
M7 (λ := 100)
M8 (50%)
M9 (70%)
M10 (90%)
M11
 

0
nothing
0.4051
0.4051
0.4051
0
0.4051
0.4051
0.4051
NA
NA
NA
0
 

------ Model-Free Algorithms ------

1
SCFOv7
0.1861 ± 0.0618
0.2147 ± 0.0721
0.501 ± 0.4092
5.23 ± 7.339
0.1572 ± 0.0639
0.1725 ± 0.0786
0.3255 ± 0.5731
18.84 ± 16.71 (67%)
29.79 ± 15.45 (39%)
41 ± 0 (0%)
24.3 ± 6.888
100

1.01
SCFOv8
0.1888 ± 0.0482
0.1935 ± 0.048
0.2396 ± 0.0523
2.158 ± 2.277
0.1616 ± 0.055
0.1631 ± 0.056
0.1779 ± 0.0922
20.74 ± 15.16 (74%)
30.37 ± 15.72 (32%)
41 ± 0 (0%)
25.26 ± 3.287
19

1.02
SCFOv9
0.1469 ± 0.0197
0.1615 ± 0.032
0.3067 ± 0.1545
1 ± 0
0.1179 ± 0.025
0.1179 ± 0.025
0.1179 ± 0.025
5.5 ± 0.5 (100%)
25 ± 16 (50%)
41 ± 0 (0%)
47.44 ± 0.0431
2

2
SCFOv7f
0.2138 ± 0.046
0.2399 ± 0.0487
0.5005 ± 0.233
4.89 ± 6.439
0.188 ± 0.0492
0.1947 ± 0.0526
0.2608 ± 0.2727
24.07 ± 16.98 (58%)
40.47 ± 3.708 (4%)
41 ± 0 (0%)
4.177 ± 1.321
100

2.01
SCFOv8f
0.216 ± 0.0527
0.2215 ± 0.0521
0.2765 ± 0.0765
2.101 ± 2.644
0.1875 ± 0.0551
0.1879 ± 0.056
0.1925 ± 0.0752
21.59 ± 16.51 (61%)
40.04 ± 4.109 (7%)
41 ± 0 (0%)
5.438 ± 3.623
69

2.02
SCFOv9f
0.2147 ± 0.0498
0.2195 ± 0.0514
0.2677 ± 0.082
2.455 ± 2.536
0.1867 ± 0.0536
0.1873 ± 0.0537
0.193 ± 0.0575
20.82 ± 16.6 (64%)
38.36 ± 8.337 (9%)
41 ± 0 (0%)
15.61 ± 4.323
11

3
RSOCCD
0.2616 ± 0.0006
1.143 ± 0.0235
9.953 ± 0.2596
31.35 ± 11.07
0.002 ± 0.0008
0.0418 ± 0.031
0.4407 ± 0.3433
10 ± 0 (100%)
10 ± 0 (100%)
10 ± 0 (100%)
0.1729 ± 0.04
100

------ Algorithms Employing a Model ------

1
SCFOv7
0.2339 ± 0.0256
0.2339 ± 0.0256
0.234 ± 0.0256
0.01 ± 0.0995
0.19 ± 0.0311
0.19 ± 0.0311
0.19 ± 0.0311
26.09 ± 11.71 (73%)
40.93 ± 0.6965 (1%)
41 ± 0 (0%)
25.41 ± 5.737
100

1.01
SCFOv8
0.2322 ± 0.0274
0.2322 ± 0.0274
0.2322 ± 0.0274
0.0556 ± 0.2291
0.1878 ± 0.0318
0.1878 ± 0.0318
0.1878 ± 0.0318
27.56 ± 12.76 (67%)
39.56 ± 5.956 (6%)
41 ± 0 (0%)
29.81 ± 9.565
18

1.02
SCFOv9
0.2386 ± 0.0226
0.2391 ± 0.0229
0.244 ± 0.0279
0.5 ± 0.866
0.1976 ± 0.0313
0.1976 ± 0.0313
0.1976 ± 0.0313
27.75 ± 13.37 (50%)
41 ± 0 (0%)
41 ± 0 (0%)
51.54 ± 1.366
4

2
SCFOv7f
0.2716 ± 0.0209
0.2716 ± 0.0209
0.2716 ± 0.0209
0 ± 0
0.2354 ± 0.0233
0.2354 ± 0.0233
0.2354 ± 0.0233
39.95 ± 4.879 (7%)
41 ± 0 (0%)
41 ± 0 (0%)
4.551 ± 1.164
100

2.01
SCFOv8f
0.2708 ± 0.0167
0.2709 ± 0.0167
0.2714 ± 0.0176
0.0581 ± 0.536
0.2344 ± 0.0221
0.2344 ± 0.0221
0.2344 ± 0.0221
39.79 ± 5.314 (6%)
41 ± 0 (0%)
41 ± 0 (0%)
7.445 ± 7.686
86

2.02
SCFOv9f
0.2726 ± 0.0213
0.2727 ± 0.0213
0.2739 ± 0.0215
0.375 ± 1.452
0.2349 ± 0.0261
0.2349 ± 0.0261
0.2349 ± 0.0261
39.38 ± 4.526 (13%)
41 ± 0 (0%)
41 ± 0 (0%)
15.71 ± 2.041
16

5
ConAdapt
0.3156 ± 0.0022
0.3156 ± 0.0022
0.3156 ± 0.0022
0 ± 0
0.281 ± 0
0.281 ± 0
0.281 ± 0
41 ± 0 (0%)
41 ± 0 (0%)
41 ± 0 (0%)
2.846 ± 0.5429
100


Test Problem #3: Minimizing the Batch Time of Polystyrene Production

Original code provided by: G. François

Problem Description

Test File Specs: algotest([242.39 945.30],40,60,1e4,[50 600],[450 1000],[363.06 875.60],10000,1e6,[],algonum)

Main Files: phipeval.m (required), gpeval.m (required), phimod.m (model), gmod.m (model)

Auxiliary Files: integstyrbis.m (required), linint.m (required)

Results:

Algorithm
Average Suboptimality
Violations
Suboptimality at Final Experiment
Iterations to Convergence
Time
Trials

#
Name
M1 (λ := 1)
M2 (λ := 10)
M3 (λ := 100)
M4
M5 (λ := 1)
M6 (λ := 10)
M7 (λ := 100)
M8 (50%)
M9 (70%)
M10 (90%)
M11
 

0
nothing
0.1979
0.1979
0.1979
0
0.1979
0.1979
0.1979
NA
NA
NA
0
 

------ Model-Free Algorithms ------

1
SCFOv7
0.0296 ± 0.0058
0.0296 ± 0.0058
0.0296 ± 0.0058
0 ± 0
0.0092 ± 0.0039
0.0092 ± 0.0039
0.0092 ± 0.0039
4.21 ± 0.7911 (100%)
5.6 ± 1.393 (100%)
16.42 ± 9.593 (97%)
25.12 ± 6.884
100

1.01
SCFOv8
0.0301 ± 0.0054
0.0301 ± 0.0054
0.0301 ± 0.0054
0 ± 0
0.0097 ± 0.0043
0.0097 ± 0.0043
0.0097 ± 0.0043
4.353 ± 0.6809 (100%)
5.647 ± 1.281 (100%)
15.59 ± 7.285 (100%)
29 ± 6.418
17

1.02
SCFOv9
0.0285 ± 0.0034
0.0285 ± 0.0034
0.0285 ± 0.0034
0 ± 0
0.009 ± 0.004
0.009 ± 0.004
0.009 ± 0.004
4 ± 0.7071 (100%)
5.5 ± 1.118 (100%)
12.75 ± 3.562 (100%)
51.05 ± 1.698
4

2
SCFOv7f
0.0287 ± 0.0055
0.0287 ± 0.0055
0.0287 ± 0.0055
0 ± 0
0.0092 ± 0.004
0.0092 ± 0.004
0.0092 ± 0.004
4.27 ± 0.8468 (100%)
5.48 ± 1.403 (100%)
15.01 ± 8.352 (98%)
4.467 ± 1.417
100

2.01
SCFOv8f
0.0292 ± 0.0058
0.0292 ± 0.0058
0.0292 ± 0.0058
0 ± 0
0.0091 ± 0.0044
0.0091 ± 0.0044
0.0091 ± 0.0044
4.243 ± 0.97 (100%)
5.446 ± 1.042 (100%)
17.66 ± 10.87 (93%)
6.321 ± 3.53
74

2.02
SCFOv9f
0.03 ± 0.0056
0.03 ± 0.0056
0.03 ± 0.0056
0 ± 0
0.0083 ± 0.0028
0.0083 ± 0.0028
0.0083 ± 0.0028
4.286 ± 0.589 (100%)
5.429 ± 0.8207 (100%)
16.21 ± 9.792 (100%)
16.57 ± 2.878
14

3
RSOCCD
0.104 ± 0.0009
0.5588 ± 0.0009
5.106 ± 0.0009
4 ± 0
0.0155 ± 0.0012
0.0155 ± 0.0012
0.0155 ± 0.0012
10 ± 0 (100%)
11 ± 0 (100%)
11 ± 0 (100%)
0.2738 ± 0.0533
100

------ Algorithms Employing a Model ------

1
SCFOv7
0.1015 ± 0.0409
0.1015 ± 0.0409
0.1015 ± 0.0409
0 ± 0
0.0485 ± 0.0364
0.0485 ± 0.0364
0.0485 ± 0.0364
21.13 ± 14 (82%)
31.33 ± 10.38 (57%)
38.81 ± 4.046 (28%)
33.87 ± 11.99
100

1.01
SCFOv8
0.0941 ± 0.0363
0.0941 ± 0.0363
0.0941 ± 0.0363
0 ± 0
0.0445 ± 0.0307
0.0445 ± 0.0307
0.0445 ± 0.0307
19.45 ± 12.18 (91%)
29.82 ± 10.51 (64%)
38.64 ± 2.963 (45%)
39.04 ± 7.346
11

1.02
SCFOv9
0.0924 ± 0.032
0.0924 ± 0.032
0.0924 ± 0.032
0 ± 0
0.0332 ± 0.0306
0.0332 ± 0.0306
0.0332 ± 0.0306
19.6 ± 11.72 (100%)
29 ± 7.348 (80%)
39.2 ± 3.6 (20%)
63.41 ± 4.293
5

2
SCFOv7f
0.1131 ± 0.0372
0.1131 ± 0.0372
0.1131 ± 0.0372
0 ± 0
0.0519 ± 0.0364
0.0519 ± 0.0364
0.0519 ± 0.0364
25.49 ± 12.28 (81%)
34.29 ± 7.672 (56%)
39.83 ± 2.642 (21%)
17.78 ± 3.502
100

2.01
SCFOv8f
0.1102 ± 0.0386
0.1102 ± 0.0386
0.1102 ± 0.0386
0 ± 0
0.0523 ± 0.0356
0.0523 ± 0.0356
0.0523 ± 0.0356
23.74 ± 12.88 (86%)
33.89 ± 8.792 (51%)
39.55 ± 3.674 (19%)
35.43 ± 48.86
80

2.02
SCFOv9f
0.1163 ± 0.0353
0.1163 ± 0.0353
0.1163 ± 0.0353
0 ± 0
0.0502 ± 0.0321
0.0502 ± 0.0321
0.0502 ± 0.0321
26.71 ± 11.13 (71%)
34.79 ± 6.909 (57%)
39.79 ± 3.121 (21%)
51.21 ± 7.454
14

5
ConAdapt
0.0229 ± 0.0033
0.1318 ± 0.0225
1.221 ± 0.2184
23.3 ± 2.022
0.0231 ± 0.0187
0.1801 ± 0.2337
1.75 ± 2.407
2 ± 0 (100%)
3.29 ± 5.933 (99%)
33.43 ± 9.512 (64%)
56.46 ± 12.3
100


Test Problem #4: Maximizing Electrical Efficiency in a Solid Oxide Fuel Cell Stack

Original code provided by: A. Gopalakrishnan

Problem Description

Test File Specs: algotest([2e-3 7e-3 26],60,6e-4,[0.1 0.1 1e-3 0.03],[1e-3 1.5e-3 1],[1e-2 3.5e-2 30],[1e-3 1.898e-3 23.6687],.1,[50 50 .1 50],[.01 .01 .01],algonum)

Main Files: phipeval.m (required), gpeval.m (required), geval.m (required), phimod.m (model), gmod.m (model)

Auxiliary Files: model_real.m (required), stack.m (required), model.m (model)

Results:

Algorithm
Average Suboptimality
Violations
Suboptimality at Final Experiment
Iterations to Convergence
Time
Trials

#
Name
M1 (λ := 1)
M2 (λ := 10)
M3 (λ := 100)
M4
M5 (λ := 1)
M6 (λ := 10)
M7 (λ := 100)
M8 (50%)
M9 (70%)
M10 (90%)
M11
 

0
nothing
1.642
1.642
1.642
0
1.642
1.642
1.642
NA
NA
NA
0
 

------ Model-Free Algorithms ------

1
SCFOv7
0.4251 ± 0.1246
0.9484 ± 0.504
6.181 ± 5.556
33.22 ± 13.18
0.3077 ± 0.11
1.086 ± 0.9399
8.874 ± 10.46
6.73 ± 5.223 (100%)
15.43 ± 16.13 (96%)
50.75 ± 17.33 (29%)
77.28 ± 24.94
100

1.01
SCFOv8
0.5049 ± 0.1074
0.5195 ± 0.105
0.6659 ± 0.2381
1.5 ± 1.118
0.4594 ± 0.1885
0.4594 ± 0.1885
0.4594 ± 0.1885
23.33 ± 18.21 (100%)
61 ± 0 (0%)
61 ± 0 (0%)
92.66 ± 10.17
6

1.02
SCFOv9
0.4975 ± 0.081
0.532 ± 0.1091
0.8775 ± 0.4606
3 ± 0.8165
0.3638 ± 0.2392
0.3638 ± 0.2392
0.3638 ± 0.2392
8.667 ± 3.859 (100%)
60.67 ± 0.4714 (0%)
60.67 ± 0.4714 (0%)
168.7 ± 13.09
3

2
SCFOv7f
0.4341 ± 0.1151
0.8585 ± 0.5041
5.103 ± 5.326
30.25 ± 13.37
0.3204 ± 0.1096
0.9128 ± 0.9955
6.837 ± 10.85
6.57 ± 3.827 (100%)
14.52 ± 14.51 (95%)
54.91 ± 13.85 (24%)
23.86 ± 5.334
100

2.01
SCFOv8f
0.551 ± 0.1004
0.5732 ± 0.1074
0.7954 ± 0.2753
1.938 ± 1.968
0.4378 ± 0.207
0.4402 ± 0.2061
0.4634 ± 0.2485
25.75 ± 21.7 (92%)
60.37 ± 1.061 (17%)
61 ± 0 (0%)
40.06 ± 10.41
65

2.02
SCFOv9f
0.4959 ± 0.0932
0.5176 ± 0.0834
0.7348 ± 0.166
1.583 ± 0.9538
0.4114 ± 0.1759
0.4114 ± 0.1759
0.4114 ± 0.1759
15.58 ± 16.5 (92%)
58.33 ± 5.265 (33%)
61 ± 0 (0%)
67.08 ± 4.67
12

3
RSOCCD
2.497 ± 0.0228
7.862 ± 0.0228
61.52 ± 0.0228
21 ± 0
0.5928 ± 0.0309
0.5928 ± 0.0309
0.5928 ± 0.0309
17 ± 0 (100%)
61 ± 0 (0%)
61 ± 0 (0%)
3.328 ± 0.7407
100

------ Algorithms Employing a Model ------

1
SCFOv7
0.6865 ± 0.2016
0.7057 ± 0.1936
0.8985 ± 0.6854
5.12 ± 10.37
0.4142 ± 0.1171
0.4434 ± 0.1492
0.7357 ± 1.361
17.62 ± 11.03 (99%)
38.68 ± 21.48 (72%)
60.51 ± 3.814 (4%)
84.1 ± 22.95
100

1.01
SCFOv8
0.6535 ± 0.1635
0.6544 ± 0.1635
0.6632 ± 0.1658
0.8182 ± 1.749
0.38 ± 0.2305
0.38 ± 0.2305
0.38 ± 0.2305
30.27 ± 18.56 (100%)
60.09 ± 1.24 (36%)
61 ± 0 (0%)
95 ± 11.27
11

1.02
SCFOv9
0.7165 ± 0.0806
0.7165 ± 0.0806
0.7165 ± 0.0806
0 ± 0
0.5407 ± 0.1447
0.5407 ± 0.1447
0.5407 ± 0.1447
38.67 ± 8.576 (100%)
61 ± 0 (0%)
61 ± 0 (0%)
159.4 ± 9.384
3

2
SCFOv7f
0.8029 ± 0.2806
0.8383 ± 0.2496
1.192 ± 1.183
4.88 ± 11.07
0.4437 ± 0.1474
0.5186 ± 0.2196
1.268 ± 2.589
24.42 ± 14.31 (98%)
46.21 ± 18.18 (62%)
59.85 ± 6.842 (3%)
19.2 ± 4.378
100

2.01
SCFOv8f
0.6264 ± 0.1538
0.6287 ± 0.1531
0.6518 ± 0.1532
0.6 ± 1.061
0.3047 ± 0.2345
0.3047 ± 0.2345
0.3047 ± 0.2345
32.07 ± 17.56 (91%)
58.86 ± 2.674 (39%)
60.56 ± 0.9202 (11%)
38.52 ± 15.12
70

2.02
SCFOv9f
0.6393 ± 0.1232
0.6405 ± 0.1226
0.6526 ± 0.1218
0.2857 ± 0.589
0.38 ± 0.2221
0.38 ± 0.2221
0.38 ± 0.2221
33.5 ± 17.83 (93%)
58.07 ± 3.474 (64%)
60.57 ± 0.9035 (14%)
66.03 ± 15.14
14

5
ConAdapt
0.4985 ± 0.0027
0.526 ± 0.0043
0.8003 ± 0.0448
15.22 ± 4.836
0.4818 ± 0.0148
0.4899 ± 0.0199
0.5711 ± 0.2085
2 ± 0 (100%)
48.46 ± 14.44 (69%)
61 ± 0 (0%)
3.315 ± 0.6327
100


Test Problem #5: Minimizing the Overall Pumping Effort in the "Trois Bacs"

Original code provided by: G. A. Bunin

Problem Description

Test File Specs: algotest([6 6.2],40,0,[0.2 0.2 0.2 0.2 0.2 0.2],[0 0],[8 8],[4.9481 4.4160],40,[20 20 20 20 20 20],[],algonum)

Main Files: phieval.m (required), gpeval.m (required), gmod.m (model)

Results:

Algorithm
Average Suboptimality
Violations
Suboptimality at Final Experiment
Iterations to Convergence
Time
Trials

#
Name
M1 (λ := 1)
M2 (λ := 10)
M3 (λ := 100)
M4
M5 (λ := 1)
M6 (λ := 10)
M7 (λ := 100)
M8 (50%)
M9 (70%)
M10 (90%)
M11
 

0
nothing
0.7614
0.7614
0.7614
0
0.7614
0.7614
0.7614
NA
NA
NA
0
 

------ Model-Free Algorithms ------

1
SCFOv7
0.1316 ± 0.0259
0.1318 ± 0.0258
0.1331 ± 0.0269
0.28 ± 1.898
0.0816 ± 0.0356
0.0822 ± 0.0352
0.0878 ± 0.0526
3 ± 0 (100%)
6.46 ± 6.002 (100%)
39.98 ± 2.293 (22%)
49.18 ± 10.74
100

1.01
SCFOv8
0.1271 ± 0.0251
0.1271 ± 0.0251
0.1271 ± 0.0251
0 ± 0
0.0675 ± 0.0332
0.0675 ± 0.0332
0.0675 ± 0.0332
3 ± 0 (100%)
4.941 ± 2.796 (100%)
38.88 ± 3.612 (35%)
41.91 ± 8.598
17

1.02
SCFOv9
0.104 ± 0.0119
0.104 ± 0.0119
0.104 ± 0.0119
0 ± 0
0.0288 ± 0.0056
0.0288 ± 0.0056
0.0288 ± 0.0056
3 ± 0 (100%)
4 ± 0 (100%)
35.5 ± 1.5 (100%)
68.26 ± 0.1119
2

2
SCFOv7f
0.1302 ± 0.0294
0.1306 ± 0.0288
0.1341 ± 0.0308
0.56 ± 2.892
0.0786 ± 0.0376
0.0797 ± 0.0362
0.0904 ± 0.0633
3 ± 0 (100%)
5.42 ± 3.798 (100%)
39.12 ± 5.208 (26%)
15.17 ± 2.593
100

2.01
SCFOv8f
0.1273 ± 0.0277
0.1273 ± 0.0277
0.1273 ± 0.0276
0.0303 ± 0.1714
0.0777 ± 0.0346
0.0777 ± 0.0346
0.0777 ± 0.0346
3 ± 0 (100%)
4.682 ± 2.506 (100%)
38.41 ± 6.053 (38%)
29.45 ± 12.36
66

2.02
SCFOv9f
0.1283 ± 0.0262
0.1284 ± 0.0261
0.1287 ± 0.0258
0.25 ± 0.433
0.0805 ± 0.046
0.0805 ± 0.046
0.0805 ± 0.046
3 ± 0 (100%)
4.25 ± 0.433 (100%)
38.38 ± 3.16 (38%)
67.96 ± 18.79
8

3
RSOCCD
0.1619 ± 0.0203
0.7506 ± 0.0185
6.638 ± 0.0644
18.48 ± 8.41
0.0354 ± 0.0269
0.0374 ± 0.0244
0.0572 ± 0.0851
10 ± 0 (100%)
10 ± 0 (100%)
12.17 ± 7.91 (93%)
0.1592 ± 0.0479
100

------ Algorithms Employing a Model ------

1
SCFOv7
0.1932 ± 0.2072
0.2014 ± 0.2028
0.2835 ± 0.2256
7.19 ± 10.01
0.146 ± 0.2082
0.1534 ± 0.2039
0.2273 ± 0.2657
8.05 ± 11.8 (89%)
14.2 ± 15.05 (82%)
35.28 ± 10.48 (36%)
55.93 ± 17.49
100

1.01
SCFOv8
0.2241 ± 0.1907
0.2309 ± 0.186
0.298 ± 0.1817
6 ± 8.944
0.1656 ± 0.1947
0.1699 ± 0.1929
0.2126 ± 0.2271
15.93 ± 16.07 (87%)
20.73 ± 16.36 (80%)
37 ± 6.633 (53%)
55.53 ± 26.16
15

1.02
SCFOv9
0.0769 ± 0.0269
0.0812 ± 0.0274
0.1244 ± 0.0518
9 ± 8.062
0.0403 ± 0.0249
0.0403 ± 0.0249
0.0403 ± 0.0249
3 ± 0 (100%)
4 ± 0 (100%)
26.25 ± 15.01 (50%)
70.62 ± 0.583
4

2
SCFOv7f
0.1845 ± 0.18
0.1921 ± 0.1761
0.2679 ± 0.2038
6.44 ± 9.455
0.1323 ± 0.1757
0.1401 ± 0.1714
0.2179 ± 0.2565
9.05 ± 12.83 (88%)
12.99 ± 14.55 (81%)
35.55 ± 10.8 (32%)
16.65 ± 2.918
100

2.01
SCFOv8f
0.1891 ± 0.179
0.1961 ± 0.1754
0.2661 ± 0.1952
5.469 ± 8.201
0.1361 ± 0.1711
0.1397 ± 0.1692
0.1755 ± 0.1948
9.828 ± 13.35 (88%)
15.03 ± 15.22 (80%)
35.89 ± 10.47 (30%)
30.76 ± 16.09
64

2.02
SCFOv9f
0.15 ± 0.0513
0.1624 ± 0.0383
0.2861 ± 0.1267
9.4 ± 12.22
0.0757 ± 0.0397
0.085 ± 0.0338
0.1777 ± 0.186
5 ± 4 (100%)
9.8 ± 5.075 (100%)
40.8 ± 0.4 (0%)
68.55 ± 11.1
5

5
ConAdapt
0.1252 ± 0.0283
0.1817 ± 0.0292
0.7466 ± 0.1331
21.86 ± 3.302
0.1403 ± 0.1359
0.1741 ± 0.1203
0.5115 ± 0.5949
21.66 ± 14.56 (86%)
35.69 ± 6.148 (63%)
40.38 ± 1.377 (15%)
2.757 ± 0.511
100


Test Problem #6: Maximizing Production in a Continuous Stirred-Tank Reactor

Original code provided by: G. A. Bunin

Problem Description

Test File Specs: algotest([14.52 14.9],40,0.1,[0.03 0.03],[1 1],[50 50],[17.2 30.3],10,[1 1],[],algonum)

Main Files: phipeval.m (required), gpeval.m (required), phimod.m (model), gmod.m (model)

Results:

Algorithm
Average Suboptimality
Violations
Suboptimality at Final Experiment
Iterations to Convergence
Time
Trials

#
Name
M1 (λ := 1)
M2 (λ := 10)
M3 (λ := 100)
M4
M5 (λ := 1)
M6 (λ := 10)
M7 (λ := 100)
M8 (50%)
M9 (70%)
M10 (90%)
M11
 

0
nothing
0.6318
0.6318
0.6318
0
0.6318
0.6318
0.6318
NA
NA
NA
0
 

------ Model-Free Algorithms ------

1
SCFOv7
0.1313 ± 0.051
0.1732 ± 0.0943
0.5921 ± 0.6962
1.39 ± 1.984
0.0371 ± 0.0202
0.0446 ± 0.0598
0.1192 ± 0.5476
5.85 ± 3.623 (100%)
8.79 ± 5.93 (100%)
29.13 ± 10.7 (81%)
35.67 ± 8.848
100

1.01
SCFOv8
0.1523 ± 0.0572
0.2435 ± 0.1337
1.155 ± 1.007
1.4 ± 0.4899
0.0366 ± 0.0122
0.0366 ± 0.0122
0.0366 ± 0.0122
7 ± 4.336 (100%)
11.8 ± 7.705 (100%)
29.1 ± 9.375 (70%)
43.81 ± 14.52
10

1.02
SCFOv9
0.1292 ± 0.0279
0.1317 ± 0.0307
0.1575 ± 0.0693
0.8 ± 1.166
0.0314 ± 0.0152
0.0314 ± 0.0152
0.0314 ± 0.0152
4.4 ± 1.356 (100%)
7.4 ± 2.577 (100%)
29.4 ± 6.468 (100%)
69.23 ± 2.821
5

2
SCFOv7f
0.15 ± 0.065
0.2058 ± 0.1147
0.7629 ± 0.7844
1.3 ± 1.204
0.0422 ± 0.0297
0.0422 ± 0.0297
0.0422 ± 0.0297
7.02 ± 4.461 (100%)
10.86 ± 7.746 (100%)
29.05 ± 11.09 (72%)
3.289 ± 0.6527
100

2.01
SCFOv8f
0.1492 ± 0.0683
0.1973 ± 0.1165
0.6775 ± 0.7531
1.154 ± 1.218
0.0475 ± 0.0368
0.0475 ± 0.0368
0.0475 ± 0.0368
6.631 ± 4.446 (100%)
12.09 ± 9.636 (100%)
29.72 ± 10.63 (65%)
5.004 ± 1.847
65

2.02
SCFOv9f
0.1539 ± 0.0386
0.1805 ± 0.0456
0.4465 ± 0.3034
1.167 ± 0.3727
0.0647 ± 0.0209
0.0647 ± 0.0209
0.0647 ± 0.0209
6.167 ± 2.967 (100%)
10.5 ± 4.787 (100%)
35.5 ± 6.265 (50%)
11.23 ± 1.13
6

3
RSOCCD
0.8144 ± 0.0066
3.743 ± 0.1388
33.02 ± 1.566
37 ± 0
0.2797 ± 0.0087
0.9416 ± 0.1836
7.56 ± 2.071
11 ± 0 (100%)
33.2 ± 13.16 (26%)
41 ± 0 (0%)
0.2417 ± 0.0519
100

------ Algorithms Employing a Model ------

1
SCFOv7
0.3513 ± 0.1476
0.3513 ± 0.1476
0.3515 ± 0.1474
0.02 ± 0.14
0.1711 ± 0.164
0.1711 ± 0.164
0.1711 ± 0.164
21.63 ± 12.37 (81%)
29.57 ± 11.52 (63%)
39.13 ± 4.326 (23%)
35.44 ± 8.884
100

1.01
SCFOv8
0.2699 ± 0.0553
0.2699 ± 0.0553
0.2699 ± 0.0553
0 ± 0
0.1089 ± 0.0474
0.1089 ± 0.0474
0.1089 ± 0.0474
13.89 ± 5.486 (100%)
26.5 ± 8.971 (89%)
39.94 ± 4.352 (6%)
47.52 ± 15.87
18

1.02
SCFOv9
0.2995 ± 0.0975
0.2995 ± 0.0975
0.2995 ± 0.0975
0 ± 0
0.1253 ± 0.1002
0.1253 ± 0.1002
0.1253 ± 0.1002
18.67 ± 13.02 (100%)
29.67 ± 10.34 (67%)
37.67 ± 4.714 (33%)
80.75 ± 6.043
3

2
SCFOv7f
0.3189 ± 0.1334
0.3189 ± 0.1334
0.3193 ± 0.133
0.04 ± 0.2417
0.136 ± 0.1333
0.136 ± 0.1333
0.136 ± 0.1333
19.43 ± 11.29 (87%)
26.12 ± 11.44 (75%)
39.01 ± 5.11 (21%)
5.491 ± 1.23
100

2.01
SCFOv8f
0.298 ± 0.1176
0.298 ± 0.1176
0.298 ± 0.1176
0 ± 0
0.1373 ± 0.1172
0.1373 ± 0.1172
0.1373 ± 0.1172
18.56 ± 12.07 (88%)
25.65 ± 11.59 (72%)
39.71 ± 3.76 (18%)
8.981 ± 7.605
82

2.02
SCFOv9f
0.2798 ± 0.1038
0.2798 ± 0.1038
0.2798 ± 0.1038
0 ± 0
0.115 ± 0.1024
0.115 ± 0.1024
0.115 ± 0.1024
17.1 ± 11.15 (100%)
24 ± 12.47 (70%)
39.1 ± 3.176 (30%)
19.35 ± 3.756
10

5
ConAdapt
0.3611 ± 0
0.3611 ± 0
0.3611 ± 0
0 ± 0
0.3543 ± 0
0.3543 ± 0
0.3543 ± 0
41 ± 0 (0%)
41 ± 0 (0%)
41 ± 0 (0%)
4.369 ± 0.805
100


Test Problem #7: Maximizing Production in a Batch Reactor with a Reversible Reaction

Original code provided by: G. A. Bunin

Problem Description

Test File Specs: algotest([0 0],40,0.01,[],[-1 -1],[1 1],[-0.2884 -1],0.1,[],[],algonum)

Main Files: phipeval.m (required), phimod.m (model)

Results:

Algorithm
Average Suboptimality
Violations
Suboptimality at Final Experiment
Iterations to Convergence
Time
Trials

#
Name
M1 (λ := 1)
M2 (λ := 10)
M3 (λ := 100)
M4
M5 (λ := 1)
M6 (λ := 10)
M7 (λ := 100)
M8 (50%)
M9 (70%)
M10 (90%)
M11
 

0
nothing
0.873
0.873
0.873
0
0.873
0.873
0.873
NA
NA
NA
0
 

------ Model-Free Algorithms ------

1
SCFOv7
0.185 ± 0.0557
0.185 ± 0.0557
0.185 ± 0.0557
0.01 ± 0.0995
0.0502 ± 0.0497
0.0502 ± 0.0497
0.0502 ± 0.0497
7.48 ± 3.976 (100%)
11.52 ± 6.579 (99%)
34.47 ± 8.06 (57%)
17.27 ± 3.774
100

1.01
SCFOv8
0.1947 ± 0.0571
0.1947 ± 0.0571
0.1947 ± 0.0571
0 ± 0
0.061 ± 0.048
0.061 ± 0.048
0.061 ± 0.048
7.25 ± 4.166 (100%)
10.08 ± 4.481 (100%)
37.33 ± 7.792 (25%)
22.44 ± 6.851
12

1.02
SCFOv9
0.2145 ± 0.1182
0.2145 ± 0.1182
0.2145 ± 0.1182
0 ± 0
0.077 ± 0.0805
0.077 ± 0.0805
0.077 ± 0.0805
8.222 ± 6.124 (100%)
13.78 ± 9.307 (100%)
33.11 ± 9.207 (56%)
37.57 ± 1.284
9

2
SCFOv7f
0.2037 ± 0.077
0.2037 ± 0.077
0.2037 ± 0.077
0.04 ± 0.196
0.0584 ± 0.0572
0.0584 ± 0.0572
0.0584 ± 0.0572
8.31 ± 5.692 (100%)
13.53 ± 7.866 (97%)
36.7 ± 6.485 (47%)
2.418 ± 0.5273
100

2.01
SCFOv8f
0.191 ± 0.07
0.191 ± 0.07
0.191 ± 0.07
0 ± 0
0.0459 ± 0.0468
0.0459 ± 0.0468
0.0459 ± 0.0468
7.526 ± 4.314 (100%)
12.13 ± 6.412 (100%)
35.82 ± 6.298 (65%)
3.25 ± 1.551
78

2.02
SCFOv9f
0.1963 ± 0.0815
0.1963 ± 0.0815
0.1963 ± 0.0815
0.2222 ± 0.6285
0.0426 ± 0.0454
0.0426 ± 0.0454
0.0426 ± 0.0454
9.889 ± 8.225 (100%)
13.33 ± 8.327 (100%)
34.67 ± 8.705 (56%)
10.21 ± 1.177
9

3
RSOCCD
0.3242 ± 0.0453
0.3242 ± 0.0453
0.3242 ± 0.0453
0 ± 0
0.0733 ± 0.0599
0.0733 ± 0.0599
0.0733 ± 0.0599
11 ± 0 (100%)
11 ± 0 (100%)
24.2 ± 14.89 (56%)
0.1048 ± 0.0418
100

4
SimplexB
0.263 ± 0.1955
0.263 ± 0.1955
0.263 ± 0.1955
0 ± 0
0.1626 ± 0.2301
0.1626 ± 0.2301
0.1626 ± 0.2301
12.09 ± 12.08 (86%)
13.91 ± 12.13 (84%)
24.54 ± 14.08 (60%)
0.0202 ± 0.0076
100

------ Algorithms Employing a Model ------

1
SCFOv7
0.1717 ± 0.0216
0.1717 ± 0.0216
0.1717 ± 0.0216
0.06 ± 0.3693
0.1077 ± 0.0314
0.1077 ± 0.0314
0.1077 ± 0.0314
4.9 ± 0.6083 (100%)
5.96 ± 0.6621 (100%)
40.63 ± 1.653 (8%)
18.3 ± 4.283
100

1.01
SCFOv8
0.1723 ± 0.0274
0.1723 ± 0.0274
0.1723 ± 0.0274
0.0833 ± 0.2764
0.0764 ± 0.0372
0.0764 ± 0.0372
0.0764 ± 0.0372
4.917 ± 0.5713 (100%)
6.083 ± 0.7022 (100%)
39.83 ± 2.095 (25%)
18.72 ± 3.528
24

1.02
SCFOv9
0.1682 ± 0.0118
0.1682 ± 0.0118
0.1682 ± 0.0118
0 ± 0
0.099 ± 0.0164
0.099 ± 0.0164
0.099 ± 0.0164
4.5 ± 0.5 (100%)
5.5 ± 0.5 (100%)
41 ± 0 (0%)
36 ± 1.413
4

2
SCFOv7f
0.1317 ± 0.0234
0.1317 ± 0.0234
0.1317 ± 0.0234
0.01 ± 0.0995
0.0354 ± 0.0375
0.0354 ± 0.0375
0.0354 ± 0.0375
5.28 ± 1.068 (100%)
7.46 ± 3.887 (100%)
33.52 ± 9.064 (70%)
8.457 ± 2.415
100

2.01
SCFOv8f
0.126 ± 0.029
0.126 ± 0.029
0.126 ± 0.029
0.0278 ± 0.1643
0.0364 ± 0.0377
0.0364 ± 0.0377
0.0364 ± 0.0377
5.125 ± 1.117 (100%)
6.708 ± 1.594 (100%)
31.85 ± 9.748 (72%)
12.35 ± 13.82
72

2.02
SCFOv9f
0.1167 ± 0.0155
0.1167 ± 0.0155
0.1167 ± 0.0155
0.1667 ± 0.3727
0.032 ± 0.0266
0.032 ± 0.0266
0.032 ± 0.0266
4.667 ± 0.4714 (100%)
6.333 ± 1.374 (100%)
36 ± 3.109 (83%)
27.79 ± 7.141
6


Test Problem #8: Maximizing Production in a Fed-Batch Reactor with Three Reactions

Original code provided by: G. A. Bunin

Problem Description

Test File Specs: algotest([0 0 0],60,0.01,[],[-1 -0.5 -1.5],[1 0.5 1.5],[0.1953 0.3770 -1.0027],0.1,[],[1 1 1 1],algonum)

Main Files: phipeval.m (required), geval.m (required), phimod.m (model)

Results:

Algorithm
Average Suboptimality
Violations
Suboptimality at Final Experiment
Iterations to Convergence
Time
Trials

#
Name
M1 (λ := 1)
M2 (λ := 10)
M3 (λ := 100)
M4
M5 (λ := 1)
M6 (λ := 10)
M7 (λ := 100)
M8 (50%)
M9 (70%)
M10 (90%)
M11
 

0
nothing
0.5821
0.5821
0.5821
0
0.5821
0.5821
0.5821
NA
NA
NA
0
 

------ Model-Free Algorithms ------

1
SCFOv7
0.1734 ± 0.0331
0.1734 ± 0.0331
0.1734 ± 0.0331
0 ± 0
0.1424 ± 0.0653
0.1424 ± 0.0653
0.1424 ± 0.0653
30.24 ± 21.28 (92%)
57.1 ± 9.224 (43%)
60.93 ± 0.4529 (2%)
18.91 ± 2.506
100

1.01
SCFOv8
0.1844 ± 0.0475
0.1844 ± 0.0475
0.1844 ± 0.0475
0 ± 0
0.1396 ± 0.0742
0.1396 ± 0.0742
0.1396 ± 0.0742
28.14 ± 17.17 (93%)
53.57 ± 16.4 (29%)
61 ± 0 (0%)
27.77 ± 11.34
14

1.02
SCFOv9
0.1676 ± 0.0288
0.1676 ± 0.0288
0.1676 ± 0.0288
0 ± 0
0.1218 ± 0.0388
0.1218 ± 0.0388
0.1218 ± 0.0388
30.5 ± 24.21 (100%)
49.75 ± 18.91 (25%)
61 ± 0 (0%)
40.71 ± 2.002
4

2
SCFOv7f
0.1757 ± 0.0373
0.1757 ± 0.0373
0.1757 ± 0.0373
0 ± 0
0.1329 ± 0.068
0.1329 ± 0.068
0.1329 ± 0.068
28.63 ± 20.58 (93%)
56.95 ± 7.896 (48%)
60.96 ± 0.3137 (1%)
3.465 ± 0.9567
100

2.01
SCFOv8f
0.1728 ± 0.0338
0.1728 ± 0.0338
0.1728 ± 0.0338
0 ± 0
0.147 ± 0.068
0.147 ± 0.068
0.147 ± 0.068
29.76 ± 21.55 (88%)
57.53 ± 9.136 (39%)
60.99 ± 0.1147 (0%)
4.549 ± 2.674
75

2.02
SCFOv9f
0.1826 ± 0.0485
0.1826 ± 0.0485
0.1826 ± 0.0485
0 ± 0
0.1272 ± 0.0397
0.1272 ± 0.0397
0.1272 ± 0.0397
30.78 ± 21.02 (100%)
54.44 ± 11.18 (67%)
61 ± 0 (0%)
10.96 ± 1.836
9

3
RSOCCD
1.314 ± 0.0042
6.826 ± 0.0042
61.95 ± 0.0042
17 ± 0
0.179 ± 0.0058
0.179 ± 0.0058
0.179 ± 0.0058
17 ± 0 (100%)
50.88 ± 18.52 (23%)
61 ± 0 (0%)
0.1574 ± 0.0618
100

------ Algorithms Employing a Model ------

1
SCFOv7
0.188 ± 0.0392
0.188 ± 0.0392
0.188 ± 0.0392
0 ± 0
0.1464 ± 0.0546
0.1464 ± 0.0546
0.1464 ± 0.0546
22 ± 20.9 (95%)
58.13 ± 6.28 (38%)
61 ± 0 (0%)
22.57 ± 3.86
100

1.01
SCFOv8
0.1842 ± 0.0347
0.1842 ± 0.0347
0.1842 ± 0.0347
0 ± 0
0.1392 ± 0.0386
0.1392 ± 0.0386
0.1392 ± 0.0386
18.2 ± 18.77 (100%)
58.3 ± 2.685 (60%)
61 ± 0 (0%)
47.93 ± 37.86
10

1.02
SCFOv9
0.1907 ± 0.0349
0.1907 ± 0.0349
0.1907 ± 0.0349
0 ± 0
0.1698 ± 0.04
0.1698 ± 0.04
0.1698 ± 0.04
8 ± 5 (100%)
59.5 ± 1.5 (50%)
61 ± 0 (0%)
44.27 ± 0.854
2

2
SCFOv7f
0.186 ± 0.0435
0.186 ± 0.0435
0.186 ± 0.0435
0 ± 0
0.1333 ± 0.0516
0.1333 ± 0.0516
0.1333 ± 0.0516
23.23 ± 20.92 (94%)
53.92 ± 14.32 (41%)
60.99 ± 0.0995 (0%)
8.779 ± 1.5
100

2.01
SCFOv8f
0.1761 ± 0.0355
0.1761 ± 0.0355
0.1761 ± 0.0355
0 ± 0
0.1386 ± 0.0549
0.1386 ± 0.0549
0.1386 ± 0.0549
17.6 ± 19.35 (95%)
52.13 ± 15.82 (52%)
61 ± 0 (0%)
18.67 ± 25.12
62

2.02
SCFOv9f
0.1829 ± 0.0115
0.1829 ± 0.0115
0.1829 ± 0.0115
0 ± 0
0.1449 ± 0.0447
0.1449 ± 0.0447
0.1449 ± 0.0447
12.44 ± 14.24 (100%)
51.89 ± 14.71 (78%)
61 ± 0 (0%)
23.61 ± 1.432
9


Test Problem #9: Batch-to-Batch Tuning of a Temperature-Tracking Model-Predictive Controller

Original code provided by: G. A. Bunin

Problem Description

Test File Specs: algotest([0 0.7],40,0.1,[],[-2 0],[2 1],[0.9492 0.6416],1,[],[],algonum)

Main Files: phipeval.m (required), phimod.m (model)

Results:

Algorithm
Average Suboptimality
Violations
Suboptimality at Final Experiment
Iterations to Convergence
Time
Trials

#
Name
M1 (λ := 1)
M2 (λ := 10)
M3 (λ := 100)
M4
M5 (λ := 1)
M6 (λ := 10)
M7 (λ := 100)
M8 (50%)
M9 (70%)
M10 (90%)
M11
 

0
nothing
0.9232
0.9232
0.9232
0
0.9232
0.9232
0.9232
NA
NA
NA
0
 

------ Model-Free Algorithms ------

1
SCFOv7
0.1454 ± 0.0398
0.1454 ± 0.0398
0.1454 ± 0.0398
0 ± 0
0.0826 ± 0.0968
0.0826 ± 0.0968
0.0826 ± 0.0968
9.72 ± 8.888 (100%)
25.92 ± 12.39 (86%)
39.19 ± 2.982 (33%)
19.9 ± 6.998
100

1.01
SCFOv8
0.1291 ± 0.0413
0.1291 ± 0.0413
0.1291 ± 0.0413
0 ± 0
0.0634 ± 0.0898
0.0634 ± 0.0898
0.0634 ± 0.0898
9.04 ± 7.856 (100%)
24.92 ± 12.9 (84%)
39.44 ± 1.768 (40%)
21.65 ± 7.139
25

1.02
SCFOv9
0.1151 ± 0.0281
0.1151 ± 0.0281
0.1151 ± 0.0281
0 ± 0
0.0136 ± 0.0084
0.0136 ± 0.0084
0.0136 ± 0.0084
4.6 ± 2.245 (100%)
24.2 ± 7.756 (100%)
35.8 ± 4.214 (80%)
40.43 ± 6.672
5

2
SCFOv7f
0.1422 ± 0.0348
0.1422 ± 0.0348
0.1422 ± 0.0348
0 ± 0
0.0707 ± 0.1021
0.0707 ± 0.1021
0.0707 ± 0.1021
10.78 ± 9.759 (97%)
28.94 ± 10.31 (86%)
39.3 ± 2.456 (40%)
2.242 ± 0.4342
100

2.01
SCFOv8f
0.1416 ± 0.0393
0.1416 ± 0.0393
0.1416 ± 0.0393
0 ± 0
0.0605 ± 0.0867
0.0605 ± 0.0867
0.0605 ± 0.0867
8.63 ± 8.511 (100%)
30.36 ± 10.19 (83%)
39.41 ± 2.367 (37%)
2.651 ± 0.6351
100

2.02
SCFOv9f
0.1594 ± 0.035
0.1594 ± 0.035
0.1594 ± 0.035
0 ± 0
0.0222 ± 0.0223
0.0222 ± 0.0223
0.0222 ± 0.0223
14.8 ± 8.557 (100%)
33.73 ± 6.865 (87%)
39.53 ± 1.586 (40%)
10.04 ± 2.37
15

3
RSOCCD
3.783 ± 0.0014
3.783 ± 0.0014
3.783 ± 0.0014
0 ± 0
0.0253 ± 0.0018
0.0253 ± 0.0018
0.0253 ± 0.0018
11 ± 0 (100%)
11 ± 0 (100%)
11 ± 0 (100%)
0.0782 ± 0.0141
100

4
SimplexB
0.1445 ± 0.0507
0.1445 ± 0.0507
0.1445 ± 0.0507
0 ± 0
0.0333 ± 0.0331
0.0333 ± 0.0331
0.0333 ± 0.0331
8.17 ± 3.265 (100%)
10.39 ± 3.146 (100%)
16.88 ± 8.749 (91%)
0.0227 ± 0.0068
100

------ Algorithms Employing a Model ------

1
SCFOv7
0.4129 ± 0.1624
0.4129 ± 0.1624
0.4129 ± 0.1624
0 ± 0
0.2417 ± 0.2175
0.2417 ± 0.2175
0.2417 ± 0.2175
28.79 ± 12.33 (72%)
36.51 ± 6.198 (51%)
40.52 ± 1.389 (11%)
45.09 ± 10.63
100

1.01
SCFOv8
0.4184 ± 0.157
0.4184 ± 0.157
0.4184 ± 0.157
0 ± 0
0.1993 ± 0.1691
0.1993 ± 0.1691
0.1993 ± 0.1691
27.31 ± 11.84 (82%)
37.44 ± 4.65 (44%)
40.04 ± 2.129 (22%)
62.97 ± 18.89
45

1.02
SCFOv9
0.3589 ± 0.0588
0.3589 ± 0.0588
0.3589 ± 0.0588
0 ± 0
0.1011 ± 0.049
0.1011 ± 0.049
0.1011 ± 0.049
16.25 ± 2.278 (100%)
33 ± 6.364 (100%)
38.5 ± 3.775 (25%)
107.6 ± 6.002
4

2
SCFOv7f
0.3737 ± 0.1831
0.3737 ± 0.1831
0.3737 ± 0.1831
0 ± 0
0.0626 ± 0.0749
0.0626 ± 0.0749
0.0626 ± 0.0749
20.65 ± 11.76 (96%)
30.55 ± 9.785 (81%)
38.17 ± 3.852 (47%)
29.75 ± 4.193
100

2.01
SCFOv8f
0.4058 ± 0.1979
0.4058 ± 0.1979
0.4058 ± 0.1979
0 ± 0
0.1081 ± 0.1343
0.1081 ± 0.1343
0.1081 ± 0.1343
21.88 ± 10.96 (97%)
30.29 ± 9.233 (86%)
38.67 ± 3.441 (42%)
37.18 ± 8.17
100

2.02
SCFOv9f
0.4838 ± 0.1427
0.4838 ± 0.1427
0.4838 ± 0.1427
0 ± 0
0.1402 ± 0.2382
0.1402 ± 0.2382
0.1402 ± 0.2382
27.87 ± 8.82 (89%)
33.87 ± 6.546 (76%)
39.16 ± 2.925 (37%)
79.62 ± 9.497
62


Test Problem #10: Iterative Tuning of a Fixed-Order Controller in a Torsional System

Original code provided by: G. A. Bunin

Problem Description

Test File Specs: algotest([1 2.77 -2.6 1 0.5],100,0.001,[],[0 0 -5 -2 -1],[5 5 5 2 1],[4.812 0.3826 -2.9531 0.1292 -0.5116],0.05,[],[2 2],algonum)

Main Files: phipeval.m (required), geval.m (required), phimod.m (model)

Results:

Algorithm
Average Suboptimality
Violations
Suboptimality at Final Experiment
Iterations to Convergence
Time
Trials

#
Name
M1 (λ := 1)
M2 (λ := 10)
M3 (λ := 100)
M4
M5 (λ := 1)
M6 (λ := 10)
M7 (λ := 100)
M8 (50%)
M9 (70%)
M10 (90%)
M11
 

0
nothing
0.7401
0.7401
0.7401
0
0.7401
0.7401
0.7401
NA
NA
NA
0
 

------ Model-Free Algorithms ------

1
SCFOv7
0.3809 ± 0.4178
0.3809 ± 0.4178
0.3809 ± 0.4178
0 ± 0
0.1502 ± 0.6047
0.1502 ± 0.6047
0.1502 ± 0.6047
38.09 ± 33.63 (95%)
49.62 ± 32.81 (91%)
82.23 ± 24.75 (63%)
42.63 ± 9.744
100

1.01
SCFOv8
0.5089 ± 0.4737
0.5089 ± 0.4737
0.5089 ± 0.4737
0 ± 0
0.2961 ± 0.9934
0.2961 ± 0.9934
0.2961 ± 0.9934
30.65 ± 28.33 (94%)
34.12 ± 27.01 (94%)
63.29 ± 32.1 (76%)
55.12 ± 13.27
17

1.02
SCFOv9
0.245 ± 0.1229
0.245 ± 0.1229
0.245 ± 0.1229
0 ± 0
0.0621 ± 0.0596
0.0621 ± 0.0596
0.0621 ± 0.0596
33 ± 17.45 (100%)
33.17 ± 17.18 (100%)
68 ± 21.37 (83%)
72.27 ± 7.5
6

2
SCFOv7f
0.6102 ± 0.8314
0.6102 ± 0.8314
0.6102 ± 0.8314
0 ± 0
0.2406 ± 1.277
0.2406 ± 1.277
0.2406 ± 1.277
46.93 ± 35.79 (89%)
55.28 ± 35.27 (83%)
87 ± 20.15 (55%)
4.003 ± 1.089
100

2.01
SCFOv8f
0.6754 ± 1.022
0.6754 ± 1.022
0.6754 ± 1.022
0 ± 0
0.6767 ± 3.994
0.6767 ± 3.994
0.6767 ± 3.994
46.55 ± 37.17 (90%)
55.45 ± 36.12 (86%)
86.05 ± 18.92 (63%)
5.558 ± 2.596
83

2.02
SCFOv9f
0.941 ± 0.8471
0.941 ± 0.8471
0.941 ± 0.8471
0 ± 0
0.0491 ± 0.0459
0.0491 ± 0.0459
0.0491 ± 0.0459
55.23 ± 33.15 (92%)
61.08 ± 30.2 (92%)
75.46 ± 26.23 (69%)
13.04 ± 1.782
13

3
RSOCCD
8.726 ± 0.0007
10.24 ± 0.0007
25.39 ± 0.0007
10 ± 0
0.4623 ± 0.001
0.4623 ± 0.001
0.4623 ± 0.001
101 ± 0 (0%)
101 ± 0 (0%)
101 ± 0 (0%)
0.2423 ± 0.0489
100

------ Algorithms Employing a Model ------

1
SCFOv7
0.2606 ± 0.095
0.2606 ± 0.095
0.2606 ± 0.095
0 ± 0
0.1954 ± 0.0683
0.1954 ± 0.0683
0.1954 ± 0.0683
14.99 ± 24.25 (98%)
57.63 ± 42.44 (67%)
100.9 ± 0.9763 (2%)
42.04 ± 7.952
100

1.01
SCFOv8
0.2431 ± 0.0823
0.2431 ± 0.0823
0.2431 ± 0.0823
0 ± 0
0.1771 ± 0.0636
0.1771 ± 0.0636
0.1771 ± 0.0636
12.82 ± 20.43 (100%)
48.41 ± 41.28 (76%)
101 ± 0 (0%)
61.7 ± 40.22
17

1.02
SCFOv9
0.1863 ± 0.0961
0.1863 ± 0.0961
0.1863 ± 0.0961
0 ± 0
0.126 ± 0.0772
0.126 ± 0.0772
0.126 ± 0.0772
3.5 ± 0.5 (100%)
46 ± 41 (100%)
79.5 ± 21.5 (50%)
70.38 ± 2.768
2

2
SCFOv7f
0.2505 ± 0.0974
0.2505 ± 0.0974
0.2505 ± 0.0974
0 ± 0
0.1846 ± 0.0703
0.1846 ± 0.0703
0.1846 ± 0.0703
13.21 ± 23.44 (98%)
53.78 ± 43.49 (70%)
100.8 ± 1.511 (3%)
5.298 ± 1.137
100

2.01
SCFOv8f
0.2436 ± 0.0831
0.2436 ± 0.0831
0.2436 ± 0.0831
0 ± 0
0.1982 ± 0.1848
0.1982 ± 0.1848
0.1982 ± 0.1848
12.36 ± 21.98 (97%)
54.35 ± 41.11 (72%)
100.9 ± 0.8219 (3%)
10.04 ± 21.3
75

2.02
SCFOv9f
0.3059 ± 0.1676
0.3059 ± 0.1676
0.3059 ± 0.1676
0 ± 0
0.2215 ± 0.1185
0.2215 ± 0.1185
0.2215 ± 0.1185
38.75 ± 44.26 (88%)
65.38 ± 45.99 (38%)
101 ± 0 (0%)
13.06 ± 0.9261
8


Test Problem #11: Minimizing the Steady-State Production Cost of a Gold Cyanidation Leaching Process

Original code provided by: Zhang Jun (张俊)

Problem Description

Test File Specs: algotest([52 18],40,2,.001,[10 5],[80 20],[31.0563 11.6218],200,.04,[],algonum)

Main Files: phipeval.m (required), gpeval.m (required), phimod.m (model), gmod.m (model)

Auxiliary Files: model_single.m (required)

Results:

Algorithm
Average Suboptimality
Violations
Suboptimality at Final Experiment
Iterations to Convergence
Time
Trials

#
Name
M1 (λ := 1)
M2 (λ := 10)
M3 (λ := 100)
M4
M5 (λ := 1)
M6 (λ := 10)
M7 (λ := 100)
M8 (50%)
M9 (70%)
M10 (90%)
M11
 

0
nothing
0.687
0.687
0.687
0
0.687
0.687
0.687
NA
NA
NA
0
 

------ Model-Free Algorithms ------

1
SCFOv7
0.0751 ± 0.0191
0.0753 ± 0.0193
0.077 ± 0.0248
0.05 ± 0.2958
0.0053 ± 0.0053
0.0061 ± 0.01
0.0139 ± 0.0871
4.07 ± 0.7906 (100%)
4.92 ± 0.8565 (100%)
9.03 ± 4.412 (100%)
23.35 ± 1.918
100

1.01
SCFOv8
0.0822 ± 0.0199
0.0829 ± 0.0208
0.09 ± 0.0464
0.1538 ± 0.7692
0.0058 ± 0.0056
0.0058 ± 0.0056
0.0058 ± 0.0056
4.038 ± 0.7586 (100%)
5.154 ± 1.026 (100%)
9.538 ± 4.576 (100%)
30.14 ± 7.138
26

1.02
SCFOv9
0.0897 ± 0.018
0.0899 ± 0.0178
0.0918 ± 0.0162
0.2 ± 0.4
0.007 ± 0.0118
0.007 ± 0.0118
0.007 ± 0.0118
4.8 ± 0.4 (100%)
5.2 ± 0.4 (100%)
10 ± 5.06 (100%)
54.31 ± 1.352
5

2
SCFOv7f
0.0724 ± 0.0172
0.0725 ± 0.0172
0.0727 ± 0.0173
0.01 ± 0.0995
0.0089 ± 0.0136
0.0089 ± 0.0136
0.0089 ± 0.0136
4.11 ± 0.7469 (100%)
4.96 ± 0.7864 (100%)
10.6 ± 8.615 (99%)
4.103 ± 1.424
100

2.01
SCFOv8f
0.0721 ± 0.0155
0.0724 ± 0.0154
0.0755 ± 0.022
0.122 ± 0.6127
0.0052 ± 0.005
0.0052 ± 0.005
0.0052 ± 0.005
4.037 ± 0.7062 (100%)
4.939 ± 0.8016 (100%)
9.902 ± 7.853 (100%)
5.657 ± 2.903
82

2.02
SCFOv9f
0.0653 ± 0.0145
0.0654 ± 0.0145
0.0666 ± 0.015
0.0909 ± 0.2875
0.006 ± 0.0064
0.006 ± 0.0064
0.006 ± 0.0064
3.818 ± 0.7158 (100%)
4.636 ± 0.6428 (100%)
7.818 ± 2.622 (100%)
13.6 ± 3.56
11

3
RSOCCD
0.7568 ± 0.0011
1.963 ± 0.0011
14.02 ± 0.0011
4 ± 0
0.3259 ± 0.0015
0.3259 ± 0.0015
0.3259 ± 0.0015
10 ± 0 (100%)
41 ± 0 (0%)
41 ± 0 (0%)
0.1798 ± 0.073
100

------ Algorithms Employing a Model ------

1
SCFOv7
0.3164 ± 0.0442
0.3164 ± 0.0442
0.3164 ± 0.0442
0 ± 0
0.0608 ± 0.0288
0.0608 ± 0.0288
0.0608 ± 0.0288
17.03 ± 2.92 (100%)
24.39 ± 3.952 (100%)
37.21 ± 4.318 (56%)
25.6 ± 3.534
100

1.01
SCFOv8
0.2815 ± 0.0388
0.2815 ± 0.0388
0.2815 ± 0.0388
0 ± 0
0.0461 ± 0.0215
0.0461 ± 0.0215
0.0461 ± 0.0215
14.67 ± 2.609 (100%)
21.79 ± 3.266 (100%)
34.92 ± 5.074 (71%)
39.8 ± 19.48
24

1.02
SCFOv9
0.3029 ± 0.0395
0.3029 ± 0.0395
0.3029 ± 0.0395
0 ± 0
0.0708 ± 0.029
0.0708 ± 0.029
0.0708 ± 0.029
15.8 ± 2.315 (100%)
23.8 ± 4.622 (100%)
38.4 ± 5.2 (20%)
57.88 ± 1.396
5

2
SCFOv7f
0.3618 ± 0.0441
0.3618 ± 0.0441
0.3618 ± 0.0441
0 ± 0
0.1052 ± 0.0399
0.1052 ± 0.0399
0.1052 ± 0.0399
20.42 ± 3.439 (100%)
30.26 ± 4.77 (95%)
40.41 ± 1.656 (12%)
3.11 ± 0.6163
100

2.01
SCFOv8f
0.3502 ± 0.0371
0.3502 ± 0.0371
0.3502 ± 0.0371
0 ± 0
0.1004 ± 0.0333
0.1004 ± 0.0333
0.1004 ± 0.0333
19.22 ± 3.081 (100%)
29.5 ± 4.363 (97%)
40.42 ± 2.365 (9%)
5.574 ± 7.777
74

2.02
SCFOv9f
0.3303 ± 0.051
0.3303 ± 0.051
0.3303 ± 0.051
0 ± 0
0.0877 ± 0.0371
0.0877 ± 0.0371
0.0877 ± 0.0371
18 ± 3.712 (100%)
27.22 ± 5.072 (100%)
39.56 ± 2.362 (33%)
10.27 ± 0.8897
9

5
ConAdapt
0.6616 ± 0
0.6616 ± 0
0.6616 ± 0
0 ± 0
0.661 ± 0
0.661 ± 0
0.661 ± 0
41 ± 0 (0%)
41 ± 0 (0%)
41 ± 0 (0%)
6.963 ± 1.81
100