Experiment F: Total Run Time


Purpose

Comparing the run time of likelihood computations with the new optimization algorithm for determining an elimination order (v1.4) to the run time with the previous optimization algorithm (v1.3).

Description of the problem

We compared between the new optimization algorithm for determining an elimination order (v1.4) to the previous optimization algorithm (v1.3).  The run time includes both optimization time and inference time.  We tested 50 datasets, chosen so that the run times on the new version are above 10 seconds and below 10 hours.  Some of these datasets were randomly simulated, and some are based on real pedigree data.  On each dataset, v1.3 was run for up to a couple of hours, and the total run time was estimated according to the percentage of the computation that was completed at that time.

Experimental Results

The timing was improved in 48 out of the 51 datasets, often by an order of magnitude.

Detailed Results

The best time results are in bold. There are links to all files of simulated datasets.  The datasets that are based on real pedigree data are confidential and are therefore not posted.
 

input files
Partial Run Time 
Superlink 1.3 (% of computation performed)
Estimated Run Time 
Superlink 1.3 
Run Time 
Superlink 1.4
Estimated Speedup
data1 / ped1
53.72 (100%)
53.72
17.82
3.015
graph 2
306.28 (100%)
306.28
31.37
9.763
graph 3
1015.96  (87.5%)
1161.09
51.82
22.406
graph 4
353.57 (100%)
353.57
53.77
6.576
graph 5
2749.85 (50%)
5499.7
114.10
48.201
graph 6
1862.17 (22%)
8464.4
114.90
73.668
data7 / ped7
1015.05 (0.03%)
3383500
119.10
28408.900
graph 8
918.88 (100%)
918.88 
124.87
 7.359
data9 / ped9
1471.84 (0.002%)
73592000
134.38
547641.018
graph 10
1532.94 (100%)
1532.94
140.25
10.930
graph 11
3468.47 (100%)
3468.47 
150.16
23.098
graph 12
1877.40 (100%)
1877.40
160.48
11.699
data13 / ped13
500 (0.001%)
50000000
161.62
309367.653
graph 14
2572.42 (100%)
2572.42
171.14
 15.031
graph 15
10018.20 (100%)
10018.20
175.81
56.983
graph 16
1287.64 (15%)
8584.26
245.40
34.981
graph 17
2823.31 (100%)
2823.31
247.82
11.393
data18 / ped18
795.85 (100%)
795.85
248.11
3.208
data19 / ped19
278.63 (100%)
278.63
257.98
1.080
data20 / ped20
476.52 (100%)
476.52
266.47
1.788
graph 21
4617.52 (74%)
6239.89
357.37
17.461
graph 22
3694.38 (38%)
9722.05
379.35
25.628
data23 / ped23
654.93 (100%)
654.93
502.02
1.305
graph 24
321.33 (100%)
321.33
570.36
0.563
data25 / ped25
4606.07 (100%)
4606.07
646.69
7.123
graph 26
3647.95 (28%)
13510.93
676.23
19.98
graph 27
3722.11 (38%)
9795.02
686.65
14.265
graph 28
5471.72 (100%)
5471.72
776.30
7.048
graph 29
5396.03 (100%)
5396.03
785.58
6.869
data30 / ped30
2243.83 (100%)
2243.83
815.33
2.752
data31 / ped31
5028.64 (0.00001%)
50286400000
920.57
54625286.507
graph 32
3916.04 (20%)
19580.2
981.03
19.959
data33 / ped33
1067.05 (100%)
1067.05
1046.72
1.019
data34 / ped34
1832 (0.001%)
183200000
1047.21
174941.034
graph 35
2862.33 (9%)
31803.66
1132.40
28.085
graph 36
1648.06 (100%)
1648.06
1163.87
1.416
data37 / ped37
3408.30 (100%)
3408.30
1381.61
2.467
data38 / ped38
2338.48 (100%)
2338.48
1604.73
1.457
data39 / ped39
6575.03 (100%)
6575.03
1784.25
3.685
data40 / ped40
11236.4 (4%)
280910
1874.66
149.846
data41 / ped41
10413.5 (20%)
52067.5
2102.77
24.761
data42 / ped42
14537.50 (100%)
14537.50
2167.61
6.707
graph 43
3485.99 (100%)
3485.99 
2417.04
1.442
data44 / ped44
3217.02 (100%)
3217.02
2866.07
1.122
graph 45
4373.16 (100%)
4373.16
2951.57
1.482
graph 46
17881.4 (38%)
47056.31
4419.40
10.648
graph 47
4429.17 (100%)
4429.17
5880.5
0.753
graph 48
4429.59 (100%)
4429.59
15188.81
0.292
graph 49
30218.03 (7%)
431686.14
15921.33
27.114
data50 / ped50
48591.29 (76%)
63935.91
23895.85
2.676
data51 / ped51
52380.7 (0.1%)
52380700
26309.54
1990.939