Member Search
|
|
Search
|
Watch Thread | My Post History | My Watches | User Settings View: Flat (newest first) | Threaded | Tree Previous Thread | Next Thread |
||
Forums
Marathon Matches
SpaceNet RoadDetector
Differences in APLS metric
|
||
| Differences in APLS metric | Reply | |
| Hi, I am having trouble reproducing the APLS metric reported by the testing server on sample data. Specifically, I converted an output graph for Sample Vegas 1454 from pixel coordinates to longitude/latitude coordinates in geojson format, and placed the output graph in sample_data/sample_geojson/test0_prop.geojson and the ground truth graph from AOI_2_Vegas_Roads_Sample/geojson/spacenetroads/spacenetroads_AOI_2_Vegas_img1454.geojson in sample_data/sample_geojson/test0_gt.geojson. But I get this from apls.py: C_gt_onto_prop, C_prop_onto_gt: 0.560342761102 0.575001651265 Total APLS Metric = Mean( 0.56 + 0.58 ) = 0.57 ... APLS Metric = 0.567577572758 But the website reports this result: Image id:AOI_2_Vegas_img1454 G1->G2: 0.45312575737285765 G2->G1: 0.3884143548897544 Score: 0.41828201928653147 Any idea what the discrepancy could be? I checked the default parameters in apls.py main function, but only linestring_delta is mentioned in the problem statement, and it is already correctly set to 50. Note that the apls.py that I am using is the latest commit c74da04022c26ae13dc11895c68429c3a5993c0a from https://github.com/CosmiQ/apls/. Here is my CSV file submitted to website:https://pastebin.com/SRRsBSGs And here is the same graph converted to geojson: https://pastebin.com/KXNixp92 |
|
| Re: Differences in APLS metric (response to post by fbastani) | Reply | |
| "I checked the default parameters in apls.py main function, but only linestring_delta is mentioned in the problem statement, and it is already correctly set to 50." There is also the maximum Euclidian distance used when projecting points from one graph to another, this is 4 meters in the problem specification, but the Python version uses 5 meters by default. If you change this, you will get much closer results, but even then they will not be identical. One of the reasons can be the errors introduced by the conversion from pixel coordinates to geographical coordinates, but there may be other differences. I checked your sample file with the scorer published in the visualizer tool and this gives exactly the same results as you received from the online scorer. Please treat the Python version as a tool that helps your work, but as the problem statement says: "For a definitive and detailed algorithm see the source code of the visualizer tool." |
|
| Re: Differences in APLS metric (response to post by walrus71) | Reply | |
| Ah, didn't realize there was an implementation in the visualizer too, thought it meant the Python one was the definitive one. Thanks for the quick reply! | |
|
Forums
Marathon Matches
SpaceNet RoadDetector
Differences in APLS metric
Previous Thread | Next Thread |
 |