Neural Graph Matching Network:
Learning Lawler's Quadratic Assignment Problem with Extension to Hypergraph and Multiple-graph Matching

This project page includes demo, code, pre-trained models and additional results of our paper published in TPAMI.

Abstract

Abstract

Graph matching involves combinatorial optimization based on edge-to-edge affinity matrix, which can be generally formulated as Lawler’s Quadratic Assignment Problem (QAP). This paper presents a QAP network directly learning with the affinity matrix (equivalently the association graph) whereby the matching problem is translated into a vertex classification task. The association graph is learned by an embedding network for vertex classification, followed by Sinkhorn normalization and a cross-entropy loss for end-to-end learning. We further improve the embedding model on association graph by introducing Sinkhorn based matching-aware constraint, as well as dummy nodes to deal with unequal sizes of graphs. To our best knowledge, this is the first network to directly learn with the general Lawler’s QAP. In contrast, recent deep matching methods focus on the learning of node and edge features in two graphs respectively. We also show how to extend our network to hypergraph matching, and matching of multiple graphs. Experimental results on both synthetic graphs and real-world images show its effectiveness. For pure QAP tasks on synthetic data and QAPLIB benchmark, our method can perform competitively and even surpass state-of-the-art graph matching and QAP solvers with notable less time cost.

Authors
Paper & Code
Video Demo
Approach
We propose NGM for two-graph matching and equivalently Lawler's Quadratic Assignment Problem (Lawler's QAP), NHGM for hypergraph matching, and NMGM for multi-graph matching. Here is an overview of our proposed approaches:


Our methods are based on the so-called Association Graph for two-graph matching and multi-graph matching, and Association Hypergraph for hyper-graph matching. The original matching problem is equivalent to the vertex classification problem on the association graphs. Association graphs are illustrated as:


The network architecture of our proposed NGM model is shown as follows. We also propose an improved version namely NGM-v2 exploiting SplineConv:
Image Matching Result

Two families of models are proposed in this research project:

  • NGM which is proposed in the prelimiary preprint of this paper based on the popular VGG16 backbone at the time when NGM is developed in 2019;
  • NGM-v2 which a same pipeline with NGM, while the VGG16 backbone is replaced with the improved VGG16+SplineConv proposed in ECCV 2020. Based on the new backbone, our NGM-v2 achieves state-of-the-art accuracy on all benchmarks.

Experimental result on Pascal VOC Keypoint dataset:

method aero bike bird boat bottle bus car cat chair cow table dog horse mbkie person plant sheep sofa train tv mean
GMN (ZanfirCVPR18) 41.6 59.6 60.3 48.0 79.2 70.2 67.4 64.9 39.2 61.3 66.9 59.8 61.1 59.8 37.2 78.2 68.0 49.9 84.2 91.4 62.4
PCA-GM (WangICCV19) 49.8 61.9 65.3 57.2 78.8 75.6 64.7 69.7 41.6 63.4 50.7 67.1 66.7 61.6 44.5 81.2 67.8 59.2 78.5 90.4 64.8
IPCA-GM (WangPAMI20) 53.8 66.2 67.1 61.2 80.4 75.3 72.6 72.5 44.6 65.2 54.3 67.2 67.9 64.2 47.9 84.4 70.8 64.0 83.8 90.8 67.7
CIE-H (YuICLR20) 49.9 63.1 70.7 53.0 82.4 75.4 67.7 72.3 42.4 66.9 69.9 69.5 70.7 62.0 46.7 85.0 70.0 61.8 80.2 91.8 67.6
LCS (WangCVPR20) 46.9 58.0 63.6 69.9 87.8 79.8 71.8 60.3 44.8 64.3 79.4 57.5 64.4 57.6 52.4 96.1 62.9 65.8 94.4 92.0 68.5
BBGM (RolinkECCV20) 61.9 71.1 79.7 79.0 87.4 94.0 89.5 80.2 56.8 79.1 64.6 78.9 76.2 75.1 65.2 98.2 77.3 77.0 94.9 93.9 79.0
NGM (ours) 50.1 63.5 57.9 53.4 79.8 77.1 73.6 68.2 41.1 66.4 40.8 60.3 61.9 63.5 45.6 77.1 69.3 65.5 79.2 88.2 64.1
NHGM (ours) 52.4 62.2 58.3 55.7 78.7 77.7 74.4 70.7 42.0 64.6 53.8 61.0 61.9 60.8 46.8 79.1 66.8 55.1 80.9 88.7 64.6
NGM-v2 (ours) 61.8 71.2 77.6 78.8 87.3 93.6 87.7 79.8 55.4 77.8 89.5 78.8 80.1 79.2 62.6 97.7 77.7 75.7 96.7 93.2 80.1
NHGM-v2 (ours) 59.9 71.5 77.2 79.0 87.7 94.6 89.0 81.8 60.0 81.3 87.0 78.1 76.5 77.5 64.4 98.7 77.8 75.4 97.9 92.8 80.4
It is worth noting that the result of BBGM is different from the result from their paper. By looking into the code released by BBGM, we find that the authors of BBGM filter out keypoints which are out of the bounding box but we do not. Therefore, the number of nodes of graphs in BBGM is smaller than ours, and the graph matching problem is less challenging than ours. In this paper, we modify BBGM to fit into our setting, and we report our reproduced result for fair comparison.

Experimental result on Willow Object Class dataset:

method # graphs Car Duck Face Motorbike Winebottle mean
GMN (ZanfirCVPR18) 2 67.9 76.7 99.8 69.2 83.1 79.3
PCA-GM (WangICCV19) 2 87.6 83.6 100.0 77.6 88.4 87.4
IPCA-GM (WangPAMI20) 2 90.4 88.6 100.0 83.0 88.3 90.1
BBGM (RolienkECCV20) 2 96.8 89.9 100.0 99.8 99.4 97.2
LCS (WangCVPR20) 2 91.2 86.2 100.0 99.4 97.9 94.9
NGM (ours) 2 84.2 77.6 99.4 76.8 88.3 85.3
NHGM (ours) 2 86.5 72.2 99.9 79.3 89.4 85.5
NGM-v2 (ours) 2 97.4 93.4 100.0 98.6 98.3 97.5
NHGM-v2 (ours) 2 97.4 93.9 100.0 98.6 98.9 97.8
HiPPI (BernardICCV19) ≥40 74.0 88.0 100.0 84.0 95.0 88.2
MGM-Floyd (JiangPAMI20) 32 85.0 79.3 100.0 84.3 93.1 88.3
NMGM (ours) 10 78.5 92.1 100.0 78.7 94.8 88.8
NMGM-v2 (ours) 10 97.6 94.5 100.0 100.0 99.0 98.2

QAPLIB Result

Experimental result including objective scores and running time on QAPLIB dataset:

problem objective score running time (sec)
Upper Bound SM RRWM Sinkhorn-JA NGM NGM-G5 NGM-G50 NGM-G500 NGM-G5k SM RRWM Sinkhorn-JA NGM NGM-G5 NGM-G50 NGM-G500 NGM-G5k
bur26a
5426670 6533340 6663181 5688893 5684628 5828287 5650343 5650343 5621774 0.02 0.15 309.9 0.02 0.02 0.19 1.85 19.34
bur26b
3817852 4690772 4741283 4053243 4063246 4110185 4066986 3943769 3927943 0.01 0.15 191.7 0.02 0.02 0.19 1.88 19.21
bur26c
5426795 6537412 6474996 5639665 5638641 5892078 5728542 5671528 5608065 0.01 0.15 136.9 0.02 0.02 0.19 1.90 18.74
bur26d
3821225 4649645 4678974 3985052 3994147 4137709 4077089 3965083 3962317 0.02 0.16 276.6 0.02 0.02 0.18 1.89 18.78
bur26e
5386879 6711029 6619788 5539241 5666202 5856035 5655088 5638619 5536142 0.01 0.16 52.9 0.03 0.02 0.19 1.87 18.62
bur26f
3782044 4723824 4814298 3979071 3954977 4082092 4050459 3949064 3949711 0.01 0.15 173.6 0.02 0.02 0.20 1.86 18.59
bur26g
10117172 12168111 12336830 10624776 10855165 10694274 10694272 10477104 10433439 0.01 0.15 292.8 0.02 0.02 0.19 1.87 18.55
bur26h
7098658 8753694 8772077 7453329 7670546 7493907 7493907 7417478 7348866 0.01 0.15 330.4 0.02 0.02 0.18 1.87 18.64
chr12a
9552 50732 43624 9552 27556 32670 29780 20138 14940 0.01 0.14 75.7 0.01 0.01 0.12 1.15 11.26
chr12b
9742 46386 73860 9742 29396 21528 21528 24734 14984 0.01 0.14 75.1 0.01 0.01 0.12 1.16 11.34
chr12c
11156 57404 50130 11156 34344 31602 31560 22174 16346 0.01 0.14 97.9 0.01 0.02 0.12 1.12 11.20
chr15a
9896 77094 90870 11616 50272 51746 39414 31478 20442 0.01 0.14 683.6 0.01 0.02 0.13 1.26 12.60
chr15b
7990 77430 115556 7990 52082 52066 31936 28546 22048 0.01 0.14 461.9 0.01 0.01 0.13 1.26 12.61
chr15c
9504 64198 70738 9504 38568 48318 34932 26548 24190 0.01 0.14 214.1 0.01 0.02 0.13 1.25 12.59
chr18a
11098 94806 115328 11948 83026 53612 50814 41254 33124 0.01 0.14 781.5 0.01 0.02 0.14 1.37 14.00
chr18b
1534 4054 3852 2690 4810 4330 3430 3056 2504 0.01 0.14 52.1 0.01 0.02 0.14 1.48 13.97
chr20a
2192 11154 13970 4624 10728 9516 8382 6626 5178 0.02 0.14 1285.8 0.01 0.02 0.16 1.51 15.02
chr20b
2298 9664 14168 3400 9962 8522 8196 5426 5766 0.02 0.14 911.3 0.01 0.02 0.16 1.66 14.96
chr20c
14142 112406 195572 40464 115128 103040 66182 66382 49770 0.02 0.14 945.0 0.01 0.02 0.15 1.68 14.93
chr22a
6156 16732 15892 9258 16410 15394 10778 10160 9348 0.02 0.14 1488.4 0.01 0.02 0.17 1.75 16.88
chr22b
6194 13294 13658 6634 15876 11882 10330 9156 9006 0.02 0.14 1005.3 0.01 0.02 0.17 1.76 16.10
chr25a
3796 21526 32060 5152 18950 16704 13758 13162 11648 0.03 0.15 2553.2 0.01 0.02 0.19 2.04 17.93
els19
17212548 33807116 74662642 18041490 34880280 53830864 31247564 28600336 27029748 0.01 0.14 700.0 0.01 0.02 0.15 1.47 14.49
esc16a
68 98 80 100 88 84 86 82 78 0.00 0.14 12.8 0.01 0.02 0.14 1.27 13.00
esc16b
292 318 294 304 308 310 306 296 292 0.00 0.14 4.6 0.01 0.02 0.13 1.29 12.97
esc16c
160 276 204 266 184 216 210 186 174 0.01 0.14 7.7 0.01 0.01 0.13 1.30 13.00
esc16d
16 48 44 58 40 32 40 26 20 0.01 0.14 14.6 0.01 0.02 0.13 1.30 12.95
esc16e
28 52 50 44 48 46 42 38 32 0.00 0.14 13.5 0.01 0.02 0.14 1.31 13.00
esc16f
0 0 0 0 0 0 0 0 0 0.00 0.14 0.9 0.01 0.02 0.14 1.30 12.98
esc16g
26 44 52 52 50 48 48 38 32 0.00 0.14 17.1 0.01 0.02 0.14 1.30 13.04
esc16h
996 1292 1002 1282 1036 1448 1146 1076 1004 0.00 0.14 15.1 0.01 0.02 0.13 1.29 12.95
esc16i
14 54 28 36 26 50 32 26 18 0.02 0.14 5625.6 0.01 0.02 0.13 1.31 12.97
esc16j
8 22 18 18 16 28 16 14 8 0.01 0.14 13.0 0.01 0.02 0.14 1.30 13.04
esc32a
130 426 240 456 428 316 370 308 298 0.01 0.15 91.8 0.02 0.03 0.23 2.25 22.91
esc32b
168 460 400 416 424 412 404 356 368 0.00 0.15 28.9 0.01 0.03 0.24 2.36 22.87
esc32c
642 770 650 886 844 814 754 802 754 0.00 0.15 112.4 0.02 0.02 0.25 2.38 22.95
esc32d
200 360 224 356 288 356 328 284 284 0.00 0.14 68.4 0.02 0.02 0.23 2.41 22.84
esc32e
2 68 6 46 42 32 22 2 2 0.02 0.15 9661.4 0.02 0.03 0.23 2.42 22.88
esc32g
6 36 10 46 28 30 14 14 10 0.01 0.15 52135.2 0.02 0.03 0.24 2.35 22.82
esc32h
438 602 506 - 592 592 612 568 534 0.00 0.15 - 0.02 0.02 0.23 2.45 22.79
esc64a
116 254 124 276 250 248 244 220 200 0.01 0.20 225.8 0.03 0.08 0.65 6.00 61.55
esc128
64 202 78 - 238 302 282 266 242 0.08 1.34 - 0.57 0.61 3.48 29.53 297.84
had12
1652 1894 2090 - 1790 1808 1792 1722 1700 0.01 0.14 - 0.01 0.02 0.11 1.13 11.28
had14
2724 3310 3494 2916 2922 3098 2940 2928 2866 0.01 0.14 102.2 0.01 0.01 0.12 1.20 12.13
had16
3720 4390 4646 3978 4150 4070 4064 3960 3902 0.01 0.14 56.7 0.01 0.02 0.13 1.27 12.88
had18
5358 6172 6540 5736 5780 5950 5670 5662 5558 0.01 0.15 271.4 0.01 0.02 0.14 1.37 13.90
had20
6922 8154 8550 7464 7334 7592 7430 7362 7300 0.01 0.14 328.4 0.01 0.02 0.15 1.51 14.97
kra30a
88900 148690 136830 125290 114410 128920 122660 122780 114410 0.01 0.14 491.6 0.01 0.03 0.21 2.22 21.36
kra30b
91420 150760 141550 126980 118130 130940 128880 124590 118130 0.01 0.14 489.9 0.02 0.02 0.21 2.28 21.35
kra32
88700 145310 148730 128120 121340 133290 129080 125530 120930 0.01 0.15 479.6 0.02 0.03 0.23 2.42 22.96
lipa20a
3683 3956 3940 3683 3929 3904 3891 3864 3853 0.01 0.14 271.1 0.01 0.02 0.15 1.64 14.89
lipa20b
27076 36502 38236 27076 33907 34970 33902 33815 33125 0.01 0.14 73.3 0.01 0.02 0.15 1.65 15.04
lipa30a
13178 13861 13786 13178 13841 13756 13742 13660 13631 0.01 0.15 191.9 0.02 0.02 0.22 2.24 21.18
lipa30b
151426 198434 201775 151426 192356 191633 191034 189057 187607 0.03 0.15 160.5 0.02 0.03 0.22 2.28 21.35
lipa40a
31538 32736 32686 31538 32666 32658 32521 32504 32454 0.01 0.14 183.2 0.02 0.03 0.31 3.11 30.09
lipa40b
476581 628272 647295 476581 616656 620456 610699 604766 601848 0.04 0.17 369.3 0.04 0.04 0.31 3.11 30.10
lipa50a
62093 64070 64162 62642 64100 63886 63816 63705 63671 0.01 0.16 275.2 0.03 0.05 0.42 4.20 41.04
lipa50b
1210244 1589128 1591109 1210244 1543264 1552533 1530231 1531102 1523856 0.08 0.20 763.5 0.08 0.05 0.44 4.02 41.25
lipa60a
107218 109861 110468 108456 110094 110068 109884 109748 109595 0.03 0.17 551.5 0.05 0.07 0.58 5.46 54.70
lipa60b
2520135 3303961 3300291 2520135 3269504 3246252 3231895 3219131 3208501 0.10 0.22 1796.2 0.10 0.09 0.58 5.39 55.37
lipa70a
169755 173649 173569 172504 173862 173577 173419 173218 173220 0.01 0.17 565.8 0.08 0.11 0.81 7.19 72.61
lipa70b
4603200 6055613 6063182 4603200 5978316 5986328 5889358 5908729 5890161 0.15 0.25 3592.8 0.15 0.11 0.80 7.11 72.90
lipa80a
253195 258345 258608 257395 258402 258108 257997 257767 257663 0.01 0.17 1023.4 0.13 0.14 1.01 9.38 94.93
lipa80b
7763962 10231797 10223697 7763962 10173155 10094877 10059724 10007233 9983040 0.20 0.27 4158.0 0.20 0.14 0.99 9.29 95.20
lipa90a
360630 367384 367370 366649 367193 367085 366926 366795 366508 0.08 0.24 1889.5 0.16 0.20 1.31 12.06 122.14
lipa90b
12490441 16291267 16514577 12490441 16194745 16232027 16176035 16143880 16076956 0.26 0.33 5544.5 0.25 0.21 1.34 12.01 122.29
nug12
578 886 1038 682 720 772 716 676 634 0.01 0.14 11.4 0.01 0.02 0.11 1.12 11.29
nug14
1014 1450 1720 - 1210 1308 1212 1156 1156 0.01 0.14 - 0.01 0.01 0.13 1.20 12.10
nug15
1150 1668 2004 1448 1482 1500 1312 1360 1318 0.01 0.14 69.6 0.01 0.02 0.13 1.24 12.56
nug16a
1610 2224 2626 1940 1836 2140 1882 1888 1836 0.01 0.14 118.7 0.01 0.01 0.13 1.31 13.01
nug16b
1240 1862 2192 1492 1580 1696 1610 1468 1396 0.01 0.15 66.8 0.01 0.01 0.14 1.45 12.96
nug17
1732 2452 2934 2010 2004 2256 2110 2062 1980 0.01 0.14 181.6 0.01 0.02 0.14 1.49 13.44
nug18
1930 2688 3188 2192 2312 2384 2340 2322 2242 0.01 0.15 155.2 0.01 0.02 0.14 1.58 13.88
nug20
2570 3450 4174 3254 2936 3260 3174 3068 2936 0.01 0.15 146.7 0.01 0.02 0.16 1.68 14.87
nug21
2438 3702 4228 3064 2916 3346 3132 2954 2916 0.01 0.14 256.8 0.01 0.02 0.16 1.74 15.52
nug22
3596 5896 6382 3988 4616 5092 4616 4160 4298 0.01 0.14 382.6 0.01 0.02 0.16 1.73 16.09
nug24
3488 4928 5720 4424 4234 4568 4514 4278 4234 0.01 0.14 202.6 0.01 0.02 0.18 1.76 17.34
nug25
3744 5332 5712 4302 4420 4822 4788 4510 4420 0.01 0.14 478.7 0.01 0.03 0.18 1.97 17.95
nug27
5234 7802 8626 6244 6332 6860 6546 6276 6208 0.01 0.14 360.3 0.01 0.03 0.20 2.12 19.34
nug28
5166 7418 8324 6298 6128 6764 6332 6424 6128 0.01 0.14 339.6 0.01 0.03 0.21 2.16 19.96
nug30
6124 8956 10034 7242 7608 7666 7780 7530 7294 0.01 0.14 330.7 0.02 0.03 0.22 2.32 21.28
rou12
235528 325404 377168 276446 321082 301728 275876 252156 264898 0.01 0.14 41.9 0.01 0.01 0.12 1.31 11.35
rou15
354210 489350 546526 390810 469592 453260 430000 413294 403872 0.01 0.14 66.2 0.01 0.01 0.13 1.44 12.49
rou20
725522 950018 1010554 823298 897348 876282 867682 845494 817776 0.01 0.14 115.1 0.01 0.02 0.15 1.66 14.97
scr12
31410 71392 95134 45334 44400 38228 42014 40908 36292 0.01 0.14 20.8 0.01 0.01 0.12 1.29 11.33
scr15
51140 104308 101714 74632 81344 77376 75746 75224 68768 0.02 0.14 117.1 0.01 0.02 0.13 1.45 12.62
scr20
110030 263058 350528 171260 182882 212432 175534 171666 154636 0.01 0.14 220.8 0.01 0.02 0.16 1.68 14.96
sko42
15812 20770 23612 19058 20192 19274 19340 19218 18716 0.03 0.18 1342.5 0.03 0.03 0.31 3.16 32.01
sko49
23386 29616 34548 27160 28712 28584 27718 27238 27554 0.03 0.17 1849.2 0.04 0.05 0.42 3.86 39.80
sko56
34458 44594 49650 40954 42182 42476 41218 40474 40684 0.05 0.19 3318.1 0.05 0.06 0.50 4.89 48.94
sko64
48498 60878 65540 55738 60368 58274 57594 56392 56222 0.06 0.21 4533.6 0.07 0.09 0.63 6.01 61.82
sko72
66256 82156 89264 76332 79716 78956 78092 77320 76870 0.09 0.22 8845.2 0.11 0.11 0.80 7.61 77.14
sko81
90998 112838 118372 105246 107588 108404 105776 105418 104710 0.10 0.28 15863.8 0.14 0.15 1.03 9.58 97.28
sko90
115534 140840 148784 133818 137402 136624 134472 133056 132942 0.16 0.32 16796.6 0.19 0.19 1.28 12.15 122.47
sko100a
152002 185738 184854 176626 180972 176336 174818 173980 172810 0.18 0.33 18370.8 0.26 0.27 1.70 15.80 155.41
sko100b
153890 185366 189502 177398 180774 177600 177832 176538 175588 0.17 0.33 15432.1 0.26 0.24 1.70 15.59 155.32
sko100c
147862 178710 188756 169566 175740 172940 171062 170784 169806 0.17 0.38 13000.4 0.27 0.27 1.68 15.29 155.69
sko100d
149576 181328 186086 170648 175096 173170 172968 171586 170816 0.17 0.33 17350.9 0.26 0.26 1.72 15.27 155.58
sko100e
149150 180062 192342 171656 176010 175448 172216 171232 170958 0.17 0.37 16240.4 0.26 0.28 1.69 15.27 155.24
sko100f
149036 177518 189284 171296 173552 173194 171608 171232 169986 0.17 0.37 19155.6 0.25 0.27 1.69 15.33 155.17
ste36a
9526 30030 33294 17938 16648 20452 19506 18320 16768 0.02 0.15 2415.2 0.02 0.03 0.26 2.54 26.21
ste36b
15852 176526 193046 47616 43248 63332 58752 53426 43248 0.02 0.16 3718.0 0.02 0.03 0.27 2.59 26.32
ste36c
8239110 24530792 28908062 14212212 12988352 15819606 15824300 14055568 12988352 0.02 0.15 1312.1 0.02 0.03 0.27 2.61 26.42
tai12a
224416 318032 392004 245012 259014 284106 280362 272610 255158 0.01 0.14 27.1 0.01 0.01 0.11 1.13 11.38
tai12b
39464925 96190153 124497790 81727424 65138752 53817560 53707944 50142456 47252044 0.01 0.14 225.1 0.01 0.03 0.12 1.15 11.35
tai15a
388214 514304 571952 471272 467812 488870 450056 450586 436968 0.01 0.14 28.2 0.01 0.02 0.13 1.26 12.51
tai15b
51765268 702925159 702292926 52585356 495479040 234006816 53608232 52900096 52871608 0.01 0.14 29.0 0.02 0.01 0.13 1.24 12.56
tai17a
491812 669712 738566 598716 630644 608616 594994 574052 544754 0.01 0.14 52.4 0.01 0.02 0.14 1.42 13.94
tai20a
703482 976236 1012228 849082 896518 884076 838930 831358 806382 0.01 0.14 82.6 0.01 0.02 0.15 1.63 14.91
tai20b
122455319 394836310 602903767 220470588 237607744 275857568 160806544 159867648 140704160 0.02 0.15 489.9 0.01 0.02 0.15 1.62 14.89
tai25a
1167256 1485502 1536172 1341104 1393248 1427052 1405630 1364288 1352912 0.02 0.14 116.0 0.02 0.02 0.19 1.92 18.03
tai25b
344355646 764920942 1253946482 798113083 730775168 773628544 638959616 600683456 518647040 0.02 0.14 1040.0 0.01 0.02 0.19 1.93 17.95
tai30a
1818146 2210304 2305048 2072218 2065706 2208012 2125220 2101282 2065706 0.03 0.15 175.3 0.01 0.03 0.21 2.27 21.35
tai30b
637117113 1008164383 1766978330 1114514832 1359600384 1193240320 1082828032 972068480 896379008 0.03 0.15 3464.2 0.02 0.02 0.22 2.23 21.32
tai35a
2422002 3030184 3100748 2820060 2886132 2874582 2861898 2826342 2786748 0.03 0.15 221.1 0.03 0.03 0.27 2.55 25.35
tai35b
283315445 454981851 574511546 446783959 455718176 472176896 440893216 415469760 377687744 0.03 0.15 3440.6 0.03 0.03 0.26 2.53 25.36
tai40a
3139370 3825396 3985684 3547918 3610604 3755024 3672066 3660256 3610604 0.04 0.16 1121.6 0.04 0.04 0.31 2.90 30.20
tai40b
637250948 1165811212 1423772477 1019672934 1053339520 1096534400 1022449792 952554880 917498816 0.04 0.15 6646.7 0.04 0.03 0.31 2.91 29.92
tai50a
4938796 6078426 6203546 5569952 5891066 5788660 5704692 5714682 5677282 0.07 0.19 1418.5 0.04 0.05 0.43 4.08 41.38
tai50b
458821517 796553600 790688128 696556852 764856128 767252544 718563200 676292800 614638528 0.08 0.18 12552.0 0.04 0.05 0.43 3.98 41.11
tai60a
7205962 8614998 8731620 8243624 8596094 8449862 8397220 8341650 8281996 0.11 0.21 3121.1 0.07 0.08 0.60 5.40 55.35
tai60b
608215054 1089964672 1279537664 978843717 994559424 902721984 942202560 909268288 862969152 0.12 0.20 18385.7 0.07 0.07 0.60 5.40 55.34
tai64c
1855928 5893540 6363888 3189566 5703540 2348902 2324478 2072176 2133738 0.01 0.21 373.4 0.08 0.08 0.64 6.00 61.70
tai80a
13499184 15665790 16069786 15352662 15648708 15608216 15417076 15383582 15283138 0.20 0.28 4745.2 0.14 0.14 0.97 9.31 95.09
tai80b
818415043 1338090880 1410723456 1215586531 1275809408 1209990912 1159786624 1163362432 1120577408 0.25 0.24 35995.4 0.13 0.15 1.00 9.24 94.92
tai100a
21052466 24176962 24446982 23787764 24077728 23977908 23840360 23759392 23644528 0.34 0.39 5447.5 0.27 0.28 1.67 15.38 156.06
tai100b
1185996137 1990209280 2192130048 1589275900 1853681152 1736472576 1680677888 1654308224 1612020992 0.54 0.34 130312.5 0.24 0.27 1.69 15.22 156.20
tai150b
498896643 662657408 755505920 - 653429440 641556480 637699712 629012864 628349568 1.35 1.60 - 1.55 1.25 5.20 44.96 433.41
tho30
149936 230828 267194 202844 187062 207424 198456 196072 185622 0.01 0.14 739.1 0.02 0.03 0.21 2.13 21.38
tho40
240516 375154 440146 314070 313026 323808 311780 318188 304878 0.02 0.15 1407.0 0.03 0.04 0.30 2.97 30.18
tho150
8133398 10000616 10689758 9508422 9702946 9720102 9649596 9569826 9557766 0.68 0.82 99778.2 1.12 1.09 4.86 43.62 443.22
wil50
48816 56588 60420 54030 55390 54962 54552 54086 53418 0.04 0.18 1867.0 0.05 0.04 0.43 4.02 42.11
wil100
273038 305030 307258 292118 295418 299162 297012 295952 294172 0.17 0.34 12315.5 0.28 0.28 1.68 15.43 154.60

Visualization
Visualization result is provided on the testing set of 20 VOC categories.
Keypoints with the same color stands for predicted correspondence. Green lines mean correct matching and red lines mean wrong matching. (Click the image to zoom in)