JOIN
Get Time
forums   
Search | Watch Thread  |  My Post History  |  My Watches  |  User Settings
View: Flat (oldest first)  | Threaded  | Tree
Previous Thread  |  Next Thread
Re: Differences in APLS metric (response to post by walrus71) | Reply
Ah, didn't realize there was an implementation in the visualizer too, thought it meant the Python one was the definitive one. Thanks for the quick reply!
Re: Differences in APLS metric (response to post by fbastani) | Reply
"I checked the default parameters in apls.py main function, but only linestring_delta is mentioned in the problem statement, and it is already correctly set to 50."

There is also the maximum Euclidian distance used when projecting points from one graph to another, this is 4 meters in the problem specification, but the Python version uses 5 meters by default. If you change this, you will get much closer results, but even then they will not be identical. One of the reasons can be the errors introduced by the conversion from pixel coordinates to geographical coordinates, but there may be other differences. I checked your sample file with the scorer published in the visualizer tool and this gives exactly the same results as you received from the online scorer. Please treat the Python version as a tool that helps your work, but as the problem statement says: "For a definitive and detailed algorithm see the source code of the visualizer tool."
Differences in APLS metric | Reply
Hi,

I am having trouble reproducing the APLS metric reported by the testing server on sample data. Specifically, I converted an output graph for Sample Vegas 1454 from pixel coordinates to longitude/latitude coordinates in geojson format, and placed the output graph in sample_data/sample_geojson/test0_prop.geojson and the ground truth graph from AOI_2_Vegas_Roads_Sample/geojson/spacenetroads/spacenetroads_AOI_2_Vegas_img1454.geojson in sample_data/sample_geojson/test0_gt.geojson. But I get this from apls.py:

C_gt_onto_prop, C_prop_onto_gt: 0.560342761102 0.575001651265
Total APLS Metric = Mean( 0.56 + 0.58 ) = 0.57
...
APLS Metric = 0.567577572758

But the website reports this result:

Image id:AOI_2_Vegas_img1454
G1->G2: 0.45312575737285765
G2->G1: 0.3884143548897544
Score: 0.41828201928653147

Any idea what the discrepancy could be? I checked the default parameters in apls.py main function, but only linestring_delta is mentioned in the problem statement, and it is already correctly set to 50.

Note that the apls.py that I am using is the latest commit c74da04022c26ae13dc11895c68429c3a5993c0a from https://github.com/CosmiQ/apls/.

Here is my CSV file submitted to website:https://pastebin.com/SRRsBSGs
And here is the same graph converted to geojson: https://pastebin.com/KXNixp92
RSS