Results of HyperSeg for Panoptic Segmentation are Low

[feivellau]

I used the pretrained HyperSeg-3B model weights to predict coco val, which can be downloaded from HyperSeg-3B, but my results were poor.

prefix_inst = 'This is an image <image>, Please do Panoptic Segmentation.'
category = '<cls>, ' * (num_class - 1) + '<cls>.'
sources_value = f'\nThis is all the candidate categories: {category}\n'

OrderedDict([('panoptic_seg', {'PQ': 34.023268236220815, 'SQ': 75.32845313071545, 'RQ': 40.32891362260387, 'PQ_th': 51.342259135233824, 'SQ_th': 84.3932910658531, 'RQ_th': 60.575261888713584, 'PQ_st': 7.88139518110686, 'SQ_st': 61.645678888998134, 'RQ_st': 9.768387937910006})])
OrderedDict([('sem_seg', {'mIoU': 47.10610613400771, 'fwIoU': 40.78556216027967, 'IoU-person': 76.86523415799827, 'BoundaryIoU-person': 74.84097143237388, 'min(IoU, B-Iou)-person': 74.84097143237388, 'IoU-bicycle': 49.77444606760769, 'BoundaryIoU-bicycle': 5.699380393612183, 'min(IoU, B-Iou)-bicycle': 5.699380393612183, 'IoU-car': 57.80286595159511, 'BoundaryIoU-car': 0.0, 'min(IoU, B-Iou)-car': 0.0, 'IoU-motorcycle': 71.47153572213224, 'BoundaryIoU-motorcycle': 0.0, 'min(IoU, B-Iou)-motorcycle': 0.0, 'IoU-airplane': 49.142987915073135, 'BoundaryIoU-airplane': 0.0, 'min(IoU, B-Iou)-airplane': 0.0, 'IoU-bus': 61.2487032377756, 'BoundaryIoU-bus': 0.0, 'min(IoU, B-Iou)-bus': 0.0, 'IoU-train': 67.13720266901521, 'BoundaryIoU-train': 0.0, 'min(IoU, B-Iou)-train': 0.0, 'IoU-truck': 54.71939907917906, 'BoundaryIoU-truck': 0.0, 'min(IoU, B-Iou)-truck': 0.0, 'IoU-boat': 56.29437937362363, 'BoundaryIoU-boat': 0.0, 'min(IoU, B-Iou)-boat': 0.0, 'IoU-traffic light': 34.694706951694144, 'BoundaryIoU-traffic light': 0.0, 'min(IoU, B-Iou)-traffic light': 0.0, 'IoU-fire hydrant': 87.8720450426131, 'BoundaryIoU-fire hydrant': 0.0, 'min(IoU, B-Iou)-fire hydrant': 0.0, 'IoU-stop sign': 71.04186789964257, 'BoundaryIoU-stop sign': 0.0, 'min(IoU, B-Iou)-stop sign': 0.0, 'IoU-parking meter': 74.84417437302405, 'BoundaryIoU-parking meter': 0.0, 'min(IoU, B-Iou)-parking meter': 0.0, 'IoU-bench': 39.77754995430617, 'BoundaryIoU-bench': 0.0, 'min(IoU, B-Iou)-bench': 0.0, 'IoU-bird': 64.80214751277941, 'BoundaryIoU-bird': 0.0, 'min(IoU, B-Iou)-bird': 0.0, 'IoU-cat': 76.57665692520456, 'BoundaryIoU-cat': 0.0, 'min(IoU, B-Iou)-cat': 0.0, 'IoU-dog': 74.33893164092119, 'BoundaryIoU-dog': 0.0, 'min(IoU, B-Iou)-dog': 0.0, 'IoU-horse': 69.94640743384667, 'BoundaryIoU-horse': 0.0, 'min(IoU, B-Iou)-horse': 0.0, 'IoU-sheep': 57.58684078428801, 'BoundaryIoU-sheep': 0.0, 'min(IoU, B-Iou)-sheep': 0.0, 'IoU-cow': 62.39062831618466, 'BoundaryIoU-cow': 0.0, 'min(IoU, B-Iou)-cow': 0.0, 'IoU-elephant': 66.34796806323143, 'BoundaryIoU-elephant': 0.0, 'min(IoU, B-Iou)-elephant': 0.0, 'IoU-bear': 85.09103694992183, 'BoundaryIoU-bear': 0.0, 'min(IoU, B-Iou)-bear': 0.0, 'IoU-zebra': 88.37305415161954, 'BoundaryIoU-zebra': 0.0, 'min(IoU, B-Iou)-zebra': 0.0, 'IoU-giraffe': 80.19038301642311, 'BoundaryIoU-giraffe': 0.0, 'min(IoU, B-Iou)-giraffe': 0.0, 'IoU-backpack': 26.699875586180493, 'BoundaryIoU-backpack': 0.0, 'min(IoU, B-Iou)-backpack': 0.0, 'IoU-umbrella': 77.03734068431235, 'BoundaryIoU-umbrella': 0.0, 'min(IoU, B-Iou)-umbrella': 0.0, 'IoU-handbag': 24.628525950230575, 'BoundaryIoU-handbag': 0.0, 'min(IoU, B-Iou)-handbag': 0.0, 'IoU-tie': 77.60750086249855, 'BoundaryIoU-tie': 0.0, 'min(IoU, B-Iou)-tie': 0.0, 'IoU-suitcase': 75.69701661460661, 'BoundaryIoU-suitcase': 0.0, 'min(IoU, B-Iou)-suitcase': 0.0, 'IoU-frisbee': 72.22820858187416, 'BoundaryIoU-frisbee': 0.0, 'min(IoU, B-Iou)-frisbee': 0.0, 'IoU-skis': 50.21887236594916, 'BoundaryIoU-skis': 0.0, 'min(IoU, B-Iou)-skis': 0.0, 'IoU-snowboard': 53.55720765353536, 'BoundaryIoU-snowboard': 0.0, 'min(IoU, B-Iou)-snowboard': 0.0, 'IoU-sports ball': 56.306842256910905, 'BoundaryIoU-sports ball': 0.0, 'min(IoU, B-Iou)-sports ball': 0.0, 'IoU-kite': 46.19773374355344, 'BoundaryIoU-kite': 0.0, 'min(IoU, B-Iou)-kite': 0.0, 'IoU-baseball bat': 44.07106603232785, 'BoundaryIoU-baseball bat': 0.0, 'min(IoU, B-Iou)-baseball bat': 0.0, 'IoU-baseball glove': 69.48372927310076, 'BoundaryIoU-baseball glove': 0.0, 'min(IoU, B-Iou)-baseball glove': 0.0, 'IoU-skateboard': 80.99768039060832, 'BoundaryIoU-skateboard': 0.0, 'min(IoU, B-Iou)-skateboard': 0.0, 'IoU-surfboard': 54.81512637080505, 'BoundaryIoU-surfboard': 0.0, 'min(IoU, B-Iou)-surfboard': 0.0, 'IoU-tennis racket': 78.13116145884727, 'BoundaryIoU-tennis racket': 0.0, 'min(IoU, B-Iou)-tennis racket': 0.0, 'IoU-bottle': 67.37866481345955, 'BoundaryIoU-bottle': 0.0, 'min(IoU, B-Iou)-bottle': 0.0, 'IoU-wine glass': 72.66044500269987, 'BoundaryIoU-wine glass': 0.0, 'min(IoU, B-Iou)-wine glass': 0.0, 'IoU-cup': 65.13784359639868, 'BoundaryIoU-cup': 0.0, 'min(IoU, B-Iou)-cup': 0.0, 'IoU-fork': 56.65397658444559, 'BoundaryIoU-fork': 0.0, 'min(IoU, B-Iou)-fork': 0.0, 'IoU-knife': 59.623859150811384, 'BoundaryIoU-knife': 0.0, 'min(IoU, B-Iou)-knife': 0.0, 'IoU-spoon': 54.97753959314954, 'BoundaryIoU-spoon': 0.0, 'min(IoU, B-Iou)-spoon': 0.0, 'IoU-bowl': 62.611849981228964, 'BoundaryIoU-bowl': 0.0, 'min(IoU, B-Iou)-bowl': 0.0, 'IoU-banana': 81.1013121101312, 'BoundaryIoU-banana': 0.0, 'min(IoU, B-Iou)-banana': 0.0, 'IoU-apple': 63.17411233094903, 'BoundaryIoU-apple': 0.0, 'min(IoU, B-Iou)-apple': 0.0, 'IoU-sandwich': 64.42898084689158, 'BoundaryIoU-sandwich': 0.0, 'min(IoU, B-Iou)-sandwich': 0.0, 'IoU-orange': 78.48189921775514, 'BoundaryIoU-orange': 0.0, 'min(IoU, B-Iou)-orange': 0.0, 'IoU-broccoli': 66.22564496018761, 'BoundaryIoU-broccoli': 0.0, 'min(IoU, B-Iou)-broccoli': 0.0, 'IoU-carrot': 60.35010153885314, 'BoundaryIoU-carrot': 0.0, 'min(IoU, B-Iou)-carrot': 0.0, 'IoU-hot dog': 55.73567435189432, 'BoundaryIoU-hot dog': 0.0, 'min(IoU, B-Iou)-hot dog': 0.0, 'IoU-pizza': 67.80903651176567, 'BoundaryIoU-pizza': 0.0, 'min(IoU, B-Iou)-pizza': 0.0, 'IoU-donut': 25.059437347051734, 'BoundaryIoU-donut': 0.0, 'min(IoU, B-Iou)-donut': 0.0, 'IoU-cake': 72.96160059529242, 'BoundaryIoU-cake': 0.0, 'min(IoU, B-Iou)-cake': 0.0, 'IoU-chair': 54.12171363235975, 'BoundaryIoU-chair': 0.0, 'min(IoU, B-Iou)-chair': 0.0, 'IoU-couch': 64.1829192314708, 'BoundaryIoU-couch': 0.0, 'min(IoU, B-Iou)-couch': 0.0, 'IoU-potted plant': 42.0164928475888, 'BoundaryIoU-potted plant': 0.0, 'min(IoU, B-Iou)-potted plant': 0.0, 'IoU-bed': 54.16774404758002, 'BoundaryIoU-bed': 0.0, 'min(IoU, B-Iou)-bed': 0.0, 'IoU-dining table': 47.439250514632285, 'BoundaryIoU-dining table': 0.0, 'min(IoU, B-Iou)-dining table': 0.0, 'IoU-toilet': 66.40673719196576, 'BoundaryIoU-toilet': 0.0, 'min(IoU, B-Iou)-toilet': 0.0, 'IoU-tv': 72.48945527567759, 'BoundaryIoU-tv': 0.0, 'min(IoU, B-Iou)-tv': 0.0, 'IoU-laptop': 73.67629473246434, 'BoundaryIoU-laptop': 0.0, 'min(IoU, B-Iou)-laptop': 0.0, 'IoU-mouse': 64.57721387644865, 'BoundaryIoU-mouse': 0.0, 'min(IoU, B-Iou)-mouse': 0.0, 'IoU-remote': 68.8754347412764, 'BoundaryIoU-remote': 0.0, 'min(IoU, B-Iou)-remote': 0.0, 'IoU-keyboard': 65.21193613419645, 'BoundaryIoU-keyboard': 0.0, 'min(IoU, B-Iou)-keyboard': 0.0, 'IoU-cell phone': 52.77889082603513, 'BoundaryIoU-cell phone': 0.0, 'min(IoU, B-Iou)-cell phone': 0.0, 'IoU-microwave': 61.67172908239672, 'BoundaryIoU-microwave': 0.0, 'min(IoU, B-Iou)-microwave': 0.0, 'IoU-oven': 64.96056024532633, 'BoundaryIoU-oven': 0.0, 'min(IoU, B-Iou)-oven': 0.0, 'IoU-toaster': 3.12835235975122, 'BoundaryIoU-toaster': 0.0, 'min(IoU, B-Iou)-toaster': 0.0, 'IoU-sink': 73.27783874778979, 'BoundaryIoU-sink': 0.0, 'min(IoU, B-Iou)-sink': 0.0, 'IoU-refrigerator': 66.810363504348, 'BoundaryIoU-refrigerator': 0.0, 'min(IoU, B-Iou)-refrigerator': 0.0, 'IoU-book': 48.28309223325623, 'BoundaryIoU-book': 0.0, 'min(IoU, B-Iou)-book': 0.0, 'IoU-clock': 70.14829583332686, 'BoundaryIoU-clock': 0.0, 'min(IoU, B-Iou)-clock': 0.0, 'IoU-vase': 60.369625050522856, 'BoundaryIoU-vase': 0.0, 'min(IoU, B-Iou)-vase': 0.0, 'IoU-scissors': 36.24348935770208, 'BoundaryIoU-scissors': 0.0, 'min(IoU, B-Iou)-scissors': 0.0, 'IoU-teddy bear': 82.9862063479873, 'BoundaryIoU-teddy bear': 0.0, 'min(IoU, B-Iou)-teddy bear': 0.0, 'IoU-hair drier': 67.62024657058518, 'BoundaryIoU-hair drier': 0.0, 'min(IoU, B-Iou)-hair drier': 0.0, 'IoU-toothbrush': 63.4218383535985, 'BoundaryIoU-toothbrush': 0.0, 'min(IoU, B-Iou)-toothbrush': 0.0, 'IoU-banner': 16.366803082913712, 'BoundaryIoU-banner': 0.0, 'min(IoU, B-Iou)-banner': 0.0, 'IoU-blanket': 12.141469201603563, 'BoundaryIoU-blanket': 0.0, 'min(IoU, B-Iou)-blanket': 0.0, 'IoU-bridge': 18.090647329150926, 'BoundaryIoU-bridge': 0.0, 'min(IoU, B-Iou)-bridge': 0.0, 'IoU-cardboard': 5.818590596700507, 'BoundaryIoU-cardboard': 0.0, 'min(IoU, B-Iou)-cardboard': 0.0, 'IoU-counter': 18.777919388749133, 'BoundaryIoU-counter': 0.0, 'min(IoU, B-Iou)-counter': 0.0, 'IoU-curtain': 46.023302120189044, 'BoundaryIoU-curtain': 0.0, 'min(IoU, B-Iou)-curtain': 0.0, 'IoU-door-stuff': 30.059889591477052, 'BoundaryIoU-door-stuff': 0.0, 'min(IoU, B-Iou)-door-stuff': 0.0, 'IoU-floor-wood': 18.985570130504943, 'BoundaryIoU-floor-wood': 0.0, 'min(IoU, B-Iou)-floor-wood': 0.0, 'IoU-flower': 1.1843733261241562, 'BoundaryIoU-flower': 0.0, 'min(IoU, B-Iou)-flower': 0.0, 'IoU-fruit': 9.3089751195928, 'BoundaryIoU-fruit': 0.0, 'min(IoU, B-Iou)-fruit': 0.0, 'IoU-gravel': 3.3143502763556696, 'BoundaryIoU-gravel': 0.0, 'min(IoU, B-Iou)-gravel': 0.0, 'IoU-house': 17.078757870045795, 'BoundaryIoU-house': 0.0, 'min(IoU, B-Iou)-house': 0.0, 'IoU-light': 12.843147533629008, 'BoundaryIoU-light': 0.0, 'min(IoU, B-Iou)-light': 0.0, 'IoU-mirror-stuff': 55.87075382325077, 'BoundaryIoU-mirror-stuff': 0.0, 'min(IoU, B-Iou)-mirror-stuff': 0.0, 'IoU-net': 17.812074463879394, 'BoundaryIoU-net': 0.0, 'min(IoU, B-Iou)-net': 0.0, 'IoU-pillow': 11.370176522847542, 'BoundaryIoU-pillow': 0.0, 'min(IoU, B-Iou)-pillow': 0.0, 'IoU-platform': 21.148287002158696, 'BoundaryIoU-platform': 0.0, 'min(IoU, B-Iou)-platform': 0.0, 'IoU-playingfield': 34.720717480468004, 'BoundaryIoU-playingfield': 0.0, 'min(IoU, B-Iou)-playingfield': 0.0, 'IoU-railroad': 48.60876935744891, 'BoundaryIoU-railroad': 0.0, 'min(IoU, B-Iou)-railroad': 0.0, 'IoU-river': 32.21103319333702, 'BoundaryIoU-river': 0.0, 'min(IoU, B-Iou)-river': 0.0, 'IoU-road': 13.423525950675202, 'BoundaryIoU-road': 0.0, 'min(IoU, B-Iou)-road': 0.0, 'IoU-roof': 4.353851378015987, 'BoundaryIoU-roof': 0.0, 'min(IoU, B-Iou)-roof': 0.0, 'IoU-sand': 34.98816966557299, 'BoundaryIoU-sand': 0.0, 'min(IoU, B-Iou)-sand': 0.0, 'IoU-sea': 66.31424460686594, 'BoundaryIoU-sea': 0.0, 'min(IoU, B-Iou)-sea': 0.0, 'IoU-shelf': 17.004052629308575, 'BoundaryIoU-shelf': 0.0, 'min(IoU, B-Iou)-shelf': 0.0, 'IoU-snow': 65.26723217742621, 'BoundaryIoU-snow': 0.0, 'min(IoU, B-Iou)-snow': 0.0, 'IoU-stairs': 16.16188304426952, 'BoundaryIoU-stairs': 0.0, 'min(IoU, B-Iou)-stairs': 0.0, 'IoU-tent': 5.0381400855035245, 'BoundaryIoU-tent': 0.0, 'min(IoU, B-Iou)-tent': 0.0, 'IoU-towel': 36.78781812569149, 'BoundaryIoU-towel': 0.0, 'min(IoU, B-Iou)-towel': 0.0, 'IoU-wall-brick': 24.383870153104965, 'BoundaryIoU-wall-brick': 0.0, 'min(IoU, B-Iou)-wall-brick': 0.0, 'IoU-wall-stone': 11.483984912712858, 'BoundaryIoU-wall-stone': 0.0, 'min(IoU, B-Iou)-wall-stone': 0.0, 'IoU-wall-tile': 57.4456727970032, 'BoundaryIoU-wall-tile': 0.0, 'min(IoU, B-Iou)-wall-tile': 0.0, 'IoU-wall-wood': 26.898167108398884, 'BoundaryIoU-wall-wood': 0.0, 'min(IoU, B-Iou)-wall-wood': 0.0, 'IoU-water-other': 12.450092970015849, 'BoundaryIoU-water-other': 0.0, 'min(IoU, B-Iou)-water-other': 0.0, 'IoU-window-blind': 19.93955482524407, 'BoundaryIoU-window-blind': 0.0, 'min(IoU, B-Iou)-window-blind': 0.0, 'IoU-window-other': 31.297134348309736, 'BoundaryIoU-window-other': 0.0, 'min(IoU, B-Iou)-window-other': 0.0, 'IoU-tree-merged': 41.771692042962364, 'BoundaryIoU-tree-merged': 0.0, 'min(IoU, B-Iou)-tree-merged': 0.0, 'IoU-fence-merged': 26.284588311063413, 'BoundaryIoU-fence-merged': 0.0, 'min(IoU, B-Iou)-fence-merged': 0.0, 'IoU-ceiling-merged': 9.508832156345404, 'BoundaryIoU-ceiling-merged': 0.0, 'min(IoU, B-Iou)-ceiling-merged': 0.0, 'IoU-sky-other-merged': 1.5533691454671286, 'BoundaryIoU-sky-other-merged': 0.0, 'min(IoU, B-Iou)-sky-other-merged': 0.0, 'IoU-cabinet-merged': 15.077960230930513, 'BoundaryIoU-cabinet-merged': 0.0, 'min(IoU, B-Iou)-cabinet-merged': 0.0, 'IoU-table-merged': 31.669745873570644, 'BoundaryIoU-table-merged': 0.0, 'min(IoU, B-Iou)-table-merged': 0.0, 'IoU-floor-other-merged': 5.380311444997086, 'BoundaryIoU-floor-other-merged': 0.0, 'min(IoU, B-Iou)-floor-other-merged': 0.0, 'IoU-pavement-merged': 36.15755984302191, 'BoundaryIoU-pavement-merged': 0.0, 'min(IoU, B-Iou)-pavement-merged': 0.0, 'IoU-mountain-merged': 23.84024358608279, 'BoundaryIoU-mountain-merged': 0.0, 'min(IoU, B-Iou)-mountain-merged': 0.0, 'IoU-grass-merged': 12.333251714715528, 'BoundaryIoU-grass-merged': 0.0, 'min(IoU, B-Iou)-grass-merged': 0.0, 'IoU-dirt-merged': 29.13928092562565, 'BoundaryIoU-dirt-merged': 0.0, 'min(IoU, B-Iou)-dirt-merged': 0.0, 'IoU-paper-merged': 13.260273743636514, 'BoundaryIoU-paper-merged': 0.0, 'min(IoU, B-Iou)-paper-merged': 0.0, 'IoU-food-other-merged': 28.392264782973186, 'BoundaryIoU-food-other-merged': 0.0, 'min(IoU, B-Iou)-food-other-merged': 0.0, 'IoU-building-other-merged': 30.684724860606888, 'BoundaryIoU-building-other-merged': 0.0, 'min(IoU, B-Iou)-building-other-merged': 0.0, 'IoU-rock-merged': 41.308264427283746, 'BoundaryIoU-rock-merged': 0.0, 'min(IoU, B-Iou)-rock-merged': 0.0, 'IoU-wall-other-merged': 38.40018202164418, 'BoundaryIoU-wall-other-merged': 0.0, 'min(IoU, B-Iou)-wall-other-merged': 0.0, 'IoU-rug-merged': 40.12982823725847, 'BoundaryIoU-rug-merged': 0.0, 'min(IoU, B-Iou)-rug-merged': 0.0, 'mACC': 66.4192026909059, 'pACC': 54.43542068775747, 'ACC-person': 95.05528930598234, 'ACC-bicycle': 93.14750240112139, 'ACC-car': 89.5359041345989, 'ACC-motorcycle': 95.45923321566325, 'ACC-airplane': 96.8139303181935, 'ACC-bus': 96.83341162213762, 'ACC-train': 88.68513933486025, 'ACC-truck': 72.43799381102515, 'ACC-boat': 89.34170236008761, 'ACC-traffic light': 94.620605802914, 'ACC-fire hydrant': 97.29873989589515, 'ACC-stop sign': 94.8557856290381, 'ACC-parking meter': 81.8545988489108, 'ACC-bench': 80.26353738479445, 'ACC-bird': 81.06553763716981, 'ACC-cat': 79.9148552139132, 'ACC-dog': 85.06110172360111, 'ACC-horse': 93.9747897919071, 'ACC-sheep': 63.03878392273757, 'ACC-cow': 92.57027401460437, 'ACC-elephant': 69.81962572625041, 'ACC-bear': 93.65463349524074, 'ACC-zebra': 93.90806838171292, 'ACC-giraffe': 90.78229197310881, 'ACC-backpack': 61.12720309199744, 'ACC-umbrella': 93.50957063670076, 'ACC-handbag': 73.5302348861449, 'ACC-tie': 88.59761526079922, 'ACC-suitcase': 90.04067117163137, 'ACC-frisbee': 95.54872727272728, 'ACC-skis': 75.59600268486561, 'ACC-snowboard': 85.25238176875413, 'ACC-sports ball': 85.85191387312516, 'ACC-kite': 84.1109592523515, 'ACC-baseball bat': 78.91345555896875, 'ACC-baseball glove': 84.57375903388744, 'ACC-skateboard': 91.49281164858544, 'ACC-surfboard': 91.3071413225452, 'ACC-tennis racket': 84.25470187573094, 'ACC-bottle': 84.67162563281948, 'ACC-wine glass': 85.27460970789288, 'ACC-cup': 84.70586412754925, 'ACC-fork': 80.03651479993383, 'ACC-knife': 73.21737378178355, 'ACC-spoon': 77.1409018606888, 'ACC-bowl': 79.88311083409735, 'ACC-banana': 90.88682632767468, 'ACC-apple': 83.39498137281704, 'ACC-sandwich': 80.9831069457288, 'ACC-orange': 90.51145132104247, 'ACC-broccoli': 84.40645232830839, 'ACC-carrot': 79.36864059568633, 'ACC-hot dog': 62.23232844835476, 'ACC-pizza': 94.8419024732751, 'ACC-donut': 84.36436953780344, 'ACC-cake': 90.43349052974985, 'ACC-chair': 75.88454224288583, 'ACC-couch': 81.55375309828973, 'ACC-potted plant': 79.51161019490506, 'ACC-bed': 61.32455384878398, 'ACC-dining table': 78.8468070583829, 'ACC-toilet': 68.4079170587306, 'ACC-tv': 86.51166235023551, 'ACC-laptop': 95.53735250810934, 'ACC-mouse': 91.60092204798545, 'ACC-remote': 77.1620957820929, 'ACC-keyboard': 75.27702633038919, 'ACC-cell phone': 91.43335533130788, 'ACC-microwave': 84.71952867763525, 'ACC-oven': 92.15120462689302, 'ACC-toaster': 3.452216398198962, 'ACC-sink': 81.14838964018654, 'ACC-refrigerator': 72.92916070329547, 'ACC-book': 77.25389453923958, 'ACC-clock': 88.42585496214234, 'ACC-vase': 83.09929800719512, 'ACC-scissors': 87.62108732978909, 'ACC-teddy bear': 93.48854877654766, 'ACC-hair drier': 88.67446560214044, 'ACC-toothbrush': 68.82470465601112, 'ACC-banner': 45.221316592956036, 'ACC-blanket': 15.441774333872685, 'ACC-bridge': 50.594887396767675, 'ACC-cardboard': 69.94305656536281, 'ACC-counter': 59.12053370124213, 'ACC-curtain': 55.65989790431895, 'ACC-door-stuff': 38.56086214996913, 'ACC-floor-wood': 20.779704477773393, 'ACC-flower': 48.652790295410554, 'ACC-fruit': 45.12162195757208, 'ACC-gravel': 33.79529181860695, 'ACC-house': 25.892302087432977, 'ACC-light': 31.50629952532251, 'ACC-mirror-stuff': 64.0731246136504, 'ACC-net': 22.475130417046334, 'ACC-pillow': 18.650555844484984, 'ACC-platform': 28.355778790516737, 'ACC-playingfield': 39.7997534432527, 'ACC-railroad': 73.5316575579415, 'ACC-river': 72.82964686765358, 'ACC-road': 15.973242027963913, 'ACC-roof': 5.02568992001695, 'ACC-sand': 40.53622916867475, 'ACC-sea': 70.31233244508047, 'ACC-shelf': 59.21630612128499, 'ACC-snow': 76.16073613364071, 'ACC-stairs': 37.457091241404484, 'ACC-tent': 6.496407526211676, 'ACC-towel': 39.0476035737432, 'ACC-wall-brick': 77.64945076612756, 'ACC-wall-stone': 18.582257917880845, 'ACC-wall-tile': 72.57652189024904, 'ACC-wall-wood': 34.25104626873994, 'ACC-water-other': 15.05100661295521, 'ACC-window-blind': 22.893819447821855, 'ACC-window-other': 61.2582476447626, 'ACC-tree-merged': 46.93543191053634, 'ACC-fence-merged': 36.804158000951915, 'ACC-ceiling-merged': 10.347765764182382, 'ACC-sky-other-merged': 1.5670221612189772, 'ACC-cabinet-merged': 16.57083987199708, 'ACC-table-merged': 39.27968574630232, 'ACC-floor-other-merged': 5.438802666368119, 'ACC-pavement-merged': 45.802890804272565, 'ACC-mountain-merged': 27.231370484125648, 'ACC-grass-merged': 12.830829534178331, 'ACC-dirt-merged': 72.61972966636603, 'ACC-paper-merged': 14.96078822906238, 'ACC-food-other-merged': 39.42774283106674, 'ACC-building-other-merged': 75.60908565451042, 'ACC-rock-merged': 87.06290237000889, 'ACC-wall-other-merged': 45.016156995925314, 'ACC-rug-merged': 78.860792470837})])

[feivellau]

@congvvc

[temiwale88]

@feivellau - can you please share your script - or repo - for how you use this model on a single image? I want to try it on a single image with prompt. Thanks!

[y9miao]

@feivellau I am also looking into using HyperSeg for panoptic segmentation. But it seems the code for panoptic segmentation is missing, so I wonder how you implement that. Are you using similar code from PSALM, on which the HyperSeg is based on. Any hints would be very helpful.

Thanks!

[feivellau]

@feivellau I am also looking into using HyperSeg for panoptic segmentation. But it seems the code for panoptic segmentation is missing, so I wonder how you implement that. Are you using similar code from PSALM, on which the HyperSeg is based on. Any hints would be very helpful.

Thanks!

Yes, I replicated the code based on PSALM.

[TangSuger7]

I've supplemented the example segmentation code with a reference to PSALM and the results are not satisfactory, how can I troubleshoot this issue? I noticed that the mask2former you guys are referencing is for instance tasks, does this still apply to panopric tasks?

[TangSuger7]

[feivellau]

You can see that my results are also very poor, and I don't know what's causing this.

[y9miao]

I observed similar thing.
Can I kindly ask the author @congvvc to release the training and eval code on panoptic segmentation?
Otherwise I will have to re-implement it by myself and report the number.