-
Notifications
You must be signed in to change notification settings - Fork 10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
GPX export elevation data inconsistency #160
Comments
Hello, First of all, thank you for the very detailed report. This is something I can work with! Yes, altitude... I will have to dive into this topic again a bit deeper, but here is how it is currently implemented in Sky Dolly. And just to let you know that I got your issue and will certainly look into this in more detail. So yes, there are different ways to measure altitude. From the top of my head:
Then there is the question what the "system" (here: MSFS) actually measures. Sky Dolly currently records two altitude values ("simulation variables") from MSFS (one is "for information / export" purposes only). Unfortunatelly the SimConnect documentation is a bit sparse about the exact definition, but from practical experiments we get:
I just noticed that there is also a "standard pressure altitude" variable, but Sky Dolly is not currently recording it (yet):
This might become useful when trying to align the values with other tools (such as Little Navmap). And finally there is the question what each file format requires the altitude to be - and in the end what the final application makes out of it (Google Earth Pro has options how to interpret the altitude). From the GPX format description (e.g. from https://en.wikipedia.org/wiki/GPS_Exchange_Format) "Latitude and longitude are expressed in decimal degrees, and elevation in meters, both using the WGS 84 datum.[6] Dates and times are expressed in Coordinated Universal Time (UTC) using ISO 8601 format.[1]" So as the GPX format is mostly produced by "GPS trackers" the altitude is measured in relation to the WGS 84 reference ellipsoid (aka "GPS altitude"). Currently the above "PLANE ALTITUDE" simulation variable comes closest to this definition (but it is not clear at all what "PLANE ALTITUDE" actually is: could be some MSFS specific, internal altitude measurement). But this is the altitude not depending on pressure at all, so at the time I chose this as export value for the GPX format... uh... wait a second ;) I just said that I assumed PLANE ALTITUDE (the value reported by MSFS) to be "height above some reference ellipsoid" (aka "GPS altitude" - it is even documented as such in my code): https://github.com/till213/SkyDolly/blob/main/src/Model/include/Model/PositionData.h struct MODEL_API PositionData final : public TimeVariableData
{
// Position
double latitude {0.0};
double longitude {0.0};
// GPS altitude
double altitude {0.0};
// Indicated pressure altitude (analytical purposes only)
double indicatedAltitude {0.0};
...
}; However I actually convert this altitude - again - from (wrongly assumed?) 'orthometric altitude' (EGM = earth gravity model - taking earth gravity undulation values into account) to reference altitude: // Convert height above EGM geoid to height above WGS84 ellipsoid (HAE) [meters]
const double heightAboveEllipsoid = d->convert.egmToWgs84Ellipsoid(Convert::feetToMeters(positionData.altitude), positionData.latitude, positionData.longitude);
const QString trackPoint =
" <trkpt lat=\"" % Export::formatCoordinate(positionData.latitude) % "\" lon=\"" % Export::formatCoordinate(positionData.longitude) % "\">\n"
" <ele>" % Export::formatNumber(heightAboveEllipsoid).toUtf8() % "</ele>\n"
" <time>" % dateTimeUtc.toString(Qt::ISODate) % "</time>\n"
" </trkpt>\n"; -> You could try and remove the "undulation values" file from the Sky Dolly With that "undulation values" resource file (which is really optional: Sky Dolly properly checks its existence) removed the heights are not converted at all. Like this you would get the "raw PLANE ALTITUDE" value in the exported GPX file. And yes, in case of the KML export I indeed simply export the "PLANE ALTITUDE" value "as is" (and mark it as "absolute altitude" value in the KML file). So if this gives the expected value ("-1.5 metres") then I will adjust the GPX export plugin as well! Is the exported height then more sensible? Again, I am not quite sure what PLANE ALTITUDE really is ("GPS altitude" or "orthometric altitude above the geoid"), but I remember having had discussions about it with some other developer... but I cannot remember its outcome. But when you say that the expected altitude at that point is really "-1.5 metres" (instead of "+42 metres") then this conversion is probably wrong |
So I looked into this a bit further: It is actually quite hard to get information about the GPX format, specifically about its actual units. For instance the official documentation https://www.topografix.com/GPX/1/1/ is mostly concerned about the document structure (XML), but does not say a word about "WGS84" or anything related to units. So the only source that I have right now is the English Wikipedia entry: Latitude and longitude are expressed in decimal degrees, and elevation in meters, both using the WGS 84 datum.[6] Dates and times are expressed in Coordinated Universal Time (UTC) using ISO 8601 format.[1] Or in other words: the elevation (altitude) values in the GPX format are meant to be "alitude above the WGS84 reference ellipsoid" (that is my understanding anyway). That being said, I now believe that the simulation variable "PLANE ALTITDUE" (as reported by MSFS) is not really a "GPS altitude", but rather an "altitude above mean sea level" (but pressure independent). This is supported by the fact that it reports -1.5 metres for the airport in Rotherdam, which is slightly below sea level. So that makes sense. However when we measure the altitude with reference to the WGS84 reference ellipsoid at this coordinate (as given in your example GXP output above) we get indeed: https://geographiclib.sourceforge.io/cgi-bin/GeoidEval?input=51.949519+4.426761&option=Submit Geoid height:
And here we have (roughly) our "42 metres" (here: 43 metres), and this is (to my understanding) the "altitude of the geoid ("altitude above mean sea level") above the WGS84 reference ellipsoid". Or in other words: those "42 (43) metres" is the elevation that we should report in the GPX file, according to its description given on Wikipedia anyway. Now the altitude system in KML (Google Earth) does work a bit different: it does not take the WGS84 reference ellipsoid into account at all (and so it does not convert the imported GPX elevation values using some "undulation map", as it should). In fact, it has its own "reference system", as follows: https://developers.google.com/kml/documentation/altitudemode?hl=en
Now while all those "altitude modes" have a very practical use when it comes to defining "waypoint markers" none of those modes is directly applicable ("compatible") with the WGS84 reference ellipsoid altitudes. So when you import a GPX file and choose "ABSOLUTE" this is probably your best choice, but it will take the elevation values "as is" (without conversion from the WGS84 reference ellispoid back to the geoid altitude), as it seems. Long story short: while Sky Dolly seems to do the right thing when it comes to convert "AIRPLANE ALTITUDE" ("altitude above sea level" - to be confirmed by practical experiments) to WGS84 reference ellipsoid altitudes this seems to have only "academic value" (and comes at the cost of an almost 20 MiB large For the very least I might provide an export option for the altitudes for the GPX exporter (and perhaps also the IGC exporter, see below):
By the way, the initial reason to export proper WGS84 altitudes was the IGC = International Gliding Commision file format that also specifices WGS84 altitudes - but even there I have seen files that clearly do not follow this specificiation, but rather provide "standard pressure altitudes". -> As a current "workaround" you can simply remove that earth gravity model file I will have to think of a proper, user-friendly and practical solution here (perhaps getting rid of this earth gravity model file/conversion altogether, also saving 20 MiB worth of disk space again...) |
Marking this issue as "enhancement" rather than "bug", because currently Sky Dolly does "what it is supposed (designed) to do" - but that does not give satisfactory results in practice and hence should be improved (rather than "fixed" ;)). |
Fun fact: there are other tools with the exact same "altitude conversion problem", e.g. here: googlemaps/android-maps-utils#704 Quote: "On Android, the Location API provides location altitude in WGS84 (*) However, this is problematic if you want to display this altitude to end users, who are more commonly familiar with MSL altitude. In fact, if you display WGS84 altitude to users, many of them will report this as an error in altitude." (*) Which makes absolute sense: the GPS system is built around the World Geodetic System (WGS84), so the reported altitudes are "altitude above the reference ellipsoid". And since GPX is a format for GPS data exchange, it is just a natural extension that the reported values are also in relation to the WGS84 reference ellipsoid - and hence not "altitude above mean sea level" (= values which are more practical for end users, and are hence also used in tools such as Google Earth). Anyway, I will think of a more practical, user-friendly solution. For now you may also just use the Sky Dolly KML exporter: the Sky Dolly KML exporter does export the "AIRPLANE ALTITUDE" as-is, without any conversion (and marks the altitude values as "ABSOLUTE", which comes closest to the MSFS altitude model, I believe). Summary:
|
Hi Oliver, thank you for the exhaustive and clear explanation. First of all, I'm not familiar with all of these standards, I did a few searches on the web to try to get to the bottom of it myself too. I really do appreciate that this software tries to be accurate even in these difficult conditions (where documentation and standards don't really exist, or very difficult to find). Very difficult, and there is no ultimate right answer. Please correct me if I'm wrong, I really have very little experience with this information. Recommendation for resolution All in all, I think GPX files should still use the height above mean sea level (geoid) in the There are two messages from Dan Foster (who maintains the reference documentation at https://topografix.com), which are relevant here I think. He says he defined https://www.topografix.com/gpx_mailing_list.asp#[email protected]
https://www.topografix.com/gpx_mailing_list.asp#[email protected].
It's also worth reading some of the other messages there. About Wikipedia wrong? I think that wikipedia page should also be changed. I think the following quote from https://en.wikipedia.org/wiki/GPS_Exchange_Format#Units is wrong,
You can even see a discussion there, which also suggests it is the altitude above mean sea level. https://en.wikipedia.org/wiki/Talk:GPS_Exchange_Format#%3Cele%3E?
PLANE ALTITUDE reported by MSFS You're right, I agree, it's most probably something very close to the mean sea level altitude. https://eaip.lvnl.nl/web/2024-05-02-AIRAC/html/eAIP/EH-AD-2.EHRD-en-GB.html#ehrd-ad-2.2
https://eaip.lvnl.nl/web/2024-05-02-AIRAC/html/eAIP/EH-GEN-2.1-en-GB.html#gen-2-1-vertical
If I understand this correctly, EGM96 is one of the standards for mean sea level, and this suggests that around 43.6 m (143 ft) is the elevation above mean sea level there, so AIP also agrees with what you've said. Conclusion I really like that this software (and you) strive for a very high standard, and I can see why it's such a difficult topic. I also agree with you on changing this from "bug" to "enhancement", because this topic can be subjective. Even though I think Thanks, |
Hello Gábor, Wow! Thank you so much for all the research you have done: this helps a lot!
You are absolutely right! First, the (English) Wikipedia is also extremely sparse when it says "using the WGS 84 datum." The German page does not even mention that. But the post that you have found - again, thanks so much! - makes it absolutely clear:
-> So: (elevation) is meters above mean sea level - essentially the "AIRPLANE ALTITUDE" value (and what also Little Navmap is exporting) And the fact alone that the following variant exists (I wasn't even aware about this tag):
already illustrates that the "ele" tag value must be something different than the "geoid height". Where the "geoid height" is calculated based on the reference ellipsoid and the "undulation values, as done by the "GeographicLib" that Sky Dolly is using. Essentially this value that can also be evaluated online: https://geographiclib.sourceforge.io/cgi-bin/GeoidEval?input=51.949519+4.426761&option=Submit I will fix both the export and import (which I strongly assume does the inverse - and wrong - conversion as well). So there is no strong need for an option, except that I may think about adding an option which would still allow to export the "geoidheight" value - and I will also support this tag in the import (when present - and then the conversion would have to be applied, but only then). I will convert this issue into a bug issue again - because now it clearly is a bug. |
Hi Oliver, thanks for the quick reply! Great, then the GPX file will be consistent with everything I use! One thing here could be problematic (not for me, but for older users), if they try to reimport an already exported GPX file (which was exported by Sky Dolly), then this fix will be strange, that they exported, then imported something, and their plane is not on the ground. I don't know your software's legacy/deprecation policy. But as I've said, I'm perfectly happy if import/export only uses the mean sea levels only.
I don't fully understand this. If from now on the Thanks, |
Yes, that is a valid point. However:
But yes, in general I try to maintain backwards compatibility, even during this "preview" (alpha) phase. But when it comes to a "clean bug fix" then I prefer the "clean code" for now, even if this means some breaking behaviour change. As a side-note, there exists a related issue: This one is about the imported altitude being off by several metres as well. This is due to the fact that flightradar24.com and flightaware.com (and others) report their altitude values in "standard pressure altitude" values (only - at least in their free service. The paid subscription also provides "radar altitudes", or so I understand). In other words: the imported flights are not aligned on the ground either - so Sky Dolly users are currently used to that fact. This will be fixed by some "interactive wizard dialog" that aligns the aircraft on the ground, based on the actual ground altitude in MSFS (I have to place the aircraft at each of those coordinates - "interactively" - and align the aircraft "on ground").
That is correct. I was now talking about the subsequent
When importing a GPX file (that can really come from anywhere) I will check for the optional But again: |
Agree. Wish it was possible to do it everytime, in every codebase. :)
Not sure I agree. I think the (Or maybe we misunderstand each other, I think what you're saying is that the |
Again, you are absolutely spot on and correct. In fact, I just came to the exact same conclusion after doing some more research and was just about to write it down (again, as an "internal note"). In fact we have, from https://www.topografix.com/gpx_mailing_list.asp#[email protected]:
So it is absolutely as you say: the So:
|
An "export geoid height" option will be added, allowing to - optionally - also export the new The geoid height corresponds essentially to N1 and N2 in the following illustration: With this option enabled we get for instance: Waypoint: <wpt lat="51.950283" lon="4.428136">
<ele>-1.56</ele>
<time>2024-07-09T11:07:11.000Z</time>
<geoidheight>43.77</geoidheight>
<name>CUSTD</name>
</wpt> Trackpoint: <trkpt lat="51.950283" lon="4.428136">
<ele>-1.56</ele>
<time>2024-07-09T11:07:11.000Z</time>
<geoidheight>43.77</geoidheight>
</trkpt> Note: the time is also properly calculated now, specifically when exporting "all aircraft into single file" - including proper time zone suffix (= proper ISO 8601 date time format). The https://geographiclib.sourceforge.io/cgi-bin/GeoidEval?input=51.950283+4.428136&option=Submit |
Fix will be part of upcoming Sky Dolly v0.18 |
Thank you very much Oliver! Very nice and quick support. Great, looking forward to the compiled and built version, will download when ready! Very nice software and work. |
Thank you very much! You know what I like so much about it (besides the actual programming aspect): you buy a flight simulator, one moment later you browse through official A320 documents on the web and the next moment you're deep into geodesy, digging into undulations and flight physics (in order to better calculate the bank angle for imported flights - but that is a separate story yet to come :)) |
Exactly! I learn so much about a lot of things. Aviation really touches on a lot of subjects like engineering, international law, physics, geography, etc. 😃 |
Describe the bug
GPX export elevation data might be incorrect.
Not sure why, but it showed me +42 m at Rotterdam Airport at L apron using the default Cessna 152 aircraft. The elevation using different recording methods show around -1 m elevation there. (Officially the elevation there is -15 ft, but looks like every other software I tried and even the KML export says, that it is not +42 m there, but rather around -1 m, which is much closer to reality than the values in GPX file.)
Inconsistency with KML file.
The GPX elevation data is inconsistent with the KML file's elevation data. When I export the same flight in both KML and GPX format, and then import those files in Google Earth Pro, it shows me two paths, and the difference between them is around 40 m (120 ft). (When importing the data in Google Earth Pro, I used "Absolute" as altitude reference in the "Altitude" tab in "Edit Path" window.)
Inconsistency with SimFlightPath.
I also recorded a similar path with SimFlightPath (https://flightsim.to/file/15872/simflightpath). The GPX file's elevation data is clearly different. At the same stand, SimFlightPath's exported GPX shows around -1 m, where SkyDolly's exported GPX shows around +42 m.
Inconsistency with Little Navmap.
While Little Navmap (https://albar965.github.io/littlenavmap.html) was connected to the sim at the same location, it was recording an Aircraft Trail. (Does this recording automatically when it's connected to the simulator.) I exported this with "File", "GPS Exchange Format (GPX)", "Export Flight Plan and Trail as GPX", and it showed me inconsistent values with SkyDolly.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
The
<ele>
tag's value should show around -1 at the stand instead of the current +42, and the elevation at the stand should show around -1 m in the elevation profile in Google Earth Pro.Application version:
"Humble Hawker" (2024.07)
Version 0.17.5 (4cf6f06)
Thu Jul 4 12:32:10 2024
Additional context
The followings show the Cessna 152 on the ground exported by different applications. They're not exactly the same stand, but all of them are at the same L apron at EHRD.
SkyDolly, first waypoint in the GPX file exported is the following.
SkyDolly, first waypoint in the KML file exported is the following.
SimFlightPath, first waypoint in the GPX file exported is the following.
Little Navmap, first waypoint in the GPX file exported is the following.
The text was updated successfully, but these errors were encountered: