tileedit updates

[Face]

Does your format allow to be extended by additional tags (e.g. by adding more lines after the first)?
I might want to add more info, such as resolution level, latitude and longitude indices (or index ranges, if the PNG is a mosaic of multiple tiles), name of the PNG file, etc.

In essence it is an arbitrary serialization of the elevation file header values. As such, I never took string values into account, so the name of the PNG file might be a problem. The idea is to have a space separated tag list in the "name=value" form, all in one line. The one-liner approach makes it more probable that editors that support PNG tags keep them alive during round-trips, like the "Comment" tag in GIMP. That same string is then simply written to a file.

"ele2png" only takes that first line into account, so the "format" (a huge term for it, really) is indeed extendable.

Small note on the min/max/mean values there: my goal was to make the ele2png round-trip consistent, so that the chain ele->png->ele will produce exact binary matches on unedited PNGs. To achieve this, I had to store the ELE min/max/mean values as presented in the original file ("smin" etc.), but also as calculated in the conversion algorithm ("emin" etc.), which did not always match in my tests. Therefore I also stored the calculated values alongside the original values, and if the re-calculation on import yields the same values as the stored calculated ones, I used the originals to create the imported ELE header.

 

[Nikogori]

The elevation edits are written as soon as you navigate away from the edited tile. If you switch to a different tile after your edit, then switch back to the edited tile and the edits are still visible, they will have been written. I'll add some visual cue to indicate if a tile contains unsaved edits.

Edits are also propagated to ancestor tiles down to a given depth (currently hardcoded to 5 levels down from the edited tile resolution). However, I suspect that the propagation stops if an ancestor is missing in the chain (I haven't tested that yet). In your case, since you are creating a new tile at level 16, it is possible that the corresponding level-15 tile also doesn't exist, so the edit would be applied to level 16 only. You will only see it in orbiter when the LOD mechanism picks up that tile at exactly the correct resolution. Zooming in and out might trigger that.

Edits are currently not propagated to child tiles, since that could erase existing high-resolution content. I haven't quite figured out if and how to implement that (e.g. highpass-filtering the existing high-res tile, then adding the filtered high-frequency content to the low-frequency edit arriving from the ancestor tile).

Make sure that the tile load preference in Orbiter is set to "Try cache first, then compressed archive", since the edits are always written to the cache.

Thank you. It is working now.

It seems elevation data will not be saved unless the resolution is exactly 13.

 

[4throck]

One of the reasons I asked for PNG export and import is that it allows you to model terrain in Blender and then export as a height map.
Pictures are worth a thousand words:
https://blender.stackexchange.com/questions/105283/create-height-bitmap-from-mesh

Needless to say that the reverse is possible (import a heightmap and generate a terrain mesh in Blender).

Don't think that many users have access to GIF software.
So 16bit PNG is the best option I think. Just use 1 DN = 1 meter

Just my 2 cents

 

[martins]

In addition to PNG, I am also thinking about adding an export option for FITS format. It's more versatile than png (also allows output to floating point images), and I think that Gimp may be able to read it (at least the image elements - FITS can also store other types of data). Plus it's quite popular in the astronomy community.

Does this sound like a good idea?

 

[jacquesmomo]

I'd like to suggest a format like so: That said, I don't know if there are even users for ele2png anymore :lol: .

I'm afraid you are right !!! :lol::lol:

 

[Face]

I'm afraid you are right !!! :lol::lol:

Yep. Another lesson learned. :shrug:

 

[dgatsoulis]

I'd like to suggest a format like so: That said, I don't know if there are even users for ele2png anymore :lol:.

I'm afraid you are right !!! :lol::lol:

We exist!

 

[GLS]

Found 2 issues:
- when maximizing the window, it appears to change its size to that of the screen, but it isn't positioned correctly.
- the edit marker/pointer isn't that handy... could (at least) an option be added to choose between showing a center marker? That way we could know exactly which pixel is going to be edited.

---------- Post added at 05:21 PM ---------- Previous post was at 12:05 PM ----------

One suggestion: save the last path used to a texture folder, and also save the options in the Configure menu window.

 

[martins]

I have now implemented the PNG export/import functionality. I've uploaded a new trial version. The link is in the first post of this thread (I'll update this whenever new stuff is ready). Let me know if this works ok.

The PNG files are in 16-bit greyscale, so to modify them you need an editor which supports them. I'd suggest Gimp 2.10 or later.

The metafiles contain some additional information to Face's format, to allow export/import of multi-tile mosaics. However the original format should still work for importing a single-tile image into the current tile. Let me know if there are any problems with this.

Thanks for the suggestions - I'll work on them as soon as I get time.

- when maximizing the window, it appears to change its size to that of the screen, but it isn't positioned correctly.

Annoyingly, later Qt versions seem to have fixed this, but the only version I managed to compile statically is 5.6.3 which exhibits this problem. I'll see if I can correct this manually.

- the edit marker/pointer isn't that handy... could (at least) an option be added to choose between showing a center marker? That way we could know exactly which pixel is going to be edited.

Yes, I wanted to improve that - possibly also indicate the size of paintbrush, using a circle.

One suggestion: save the last path used to a texture folder, and also save the options in the Configure menu window.

Also on the to-do list. Qt has some facilities to store preferences, so should be fairly straightforward.

 

[4throck]

The PNG export looks nice, I have no problem reading the file. And the .hdr file is nicely readable on Notepad++
Great work!

I had a look here:
http://wms.lroc.asu.edu/lroc/view_rdr_product/NAC_DTM_APOLLO11
They offer geotiff as a file format, so perhaps that's a better option than FITS.

And they also have high resolution surface images with 50cm resolution to go along with this. That's why surface import makes sense I think:
http://wms.lroc.asu.edu/lroc/view_rdr/NAC_DTM_APOLLO11

 

[Face]

The metafiles contain some additional information to Face's format, to allow export/import of multi-tile mosaics. However the original format should still work for importing a single-tile image into the current tile. Let me know if there are any problems with this.

Many thanks for taking the ele2png format into account. The syntax is spot on, however the values are not corresponding with the image content.

The vmin and vmax values are the height values that the PNG format's low and high colors represent. A vmin of -1000 means that black represents -1000 height units, a vmax of e.g 1000 means that white represents 1000 height units. Values in between are scaled accordingly. Scale and offset of the tile are then used to convert the height units into absolute meters, i.e. hU*scale+offset . In that way, the color value is a direct representation of the ELE format data point value.

It seems like tileedit produces fixed scaled PNGs in the 16-bit grayscale mode, and according to my tests it starts with the lowest value as black, and from there 16-bit up to whatever meter value that would be. The values stored in vmin and vmax, though, only represent the internal range, not the picture range.

Therefore, while the syntax might be compatible, the semantics for vmin/vmax are different, so that import/export across the two tools will fail.

I'd suggest to offer a range option for the image creation. With this, the user can choose to use a nicer visual representation (auto-range) or a workable fixed range to have the possibility to compare color values across tiles.

---------- Post added at 17:21 ---------- Previous post was at 15:18 ----------

Small update:

I've made some round-trips now, and it looks like ele2png is not so tolerant on vmin/vmax being serialized as double values. It should be, though, and that is something I will fix.

In addition, the lat/lng values are serialized as radians instead of degrees, because it was not meant as input right from start. This is something that can also be fixed easily, but I don't know if it is worth the effort. Are those values inside the ELE format actually used anywhere?

Another thought: should we unify the extension to *.png.hdr, or is it good to have them separated? "ele2png" creates *.png.meta currently.

 

[martins]

The vmin and vmax values are the height values that the PNG format's low and high colors represent. A vmin of -1000 means that black represents -1000 height units, a vmax of e.g 1000 means that white represents 1000 height units. Values in between are scaled accordingly. Scale and offset of the tile are then used to convert the height units into absolute meters, i.e. hU*scale+offset . In that way, the color value is a direct representation of the ELE format data point value.

It seems like tileedit produces fixed scaled PNGs in the 16-bit grayscale mode, and according to my tests it starts with the lowest value as black, and from there 16-bit up to whatever meter value that would be. The values stored in vmin and vmax, though, only represent the internal range, not the picture range.

Therefore, while the syntax might be compatible, the semantics for vmin/vmax are different, so that import/export across the two tools will fail.

I'd suggest to offer a range option for the image creation. With this, the user can choose to use a nicer visual representation (auto-range) or a workable fixed range to have the possibility to compare color values across tiles.

Ok, it looks like I misinterpreted the meaning of the vmin and vmax values. So essentially you leave the scale and offset values alone, and add an additional scaling layer on top to map to the image range? I'll replicate that, and a user-defined mapping range is probably a good idea anyway.

In addition, the lat/lng values are serialized as radians instead of degrees, because it was not meant as input right from start. This is something that can also be fixed easily, but I don't know if it is worth the effort. Are those values inside the ELE format actually used anywhere?

I can cange it to radians. It's not normally used anywhere since the values can be inferred from the resolution and latitude/longitude indices if available, but could be used for validation to make sure the image is imported into the correct patch.

Another thought: should we unify the extension to *.png.hdr, or is it good to have them separated? "ele2png" creates *.png.meta currently.

I'd vote for unifying them. I used hdr, since .img/.hdr pairs seem to be quite common in scientific formats, but there's no problem changing it to .meta.

 

[Face]

So essentially you leave the scale and offset values alone, and add an additional scaling layer on top to map to the image range? I'll replicate that, and a user-defined mapping range is probably a good idea anyway.

Yes, scale and offset values are exactly what lands in the *.elv file, vmin and vmax define the scaling layer. It is a bit of a trade-off between exact representation of the *.elv data and usability as PNG image. Only the data point values as defined in the *.elv file are scaled to color values, not the resulting altitude values.

I can cange it to radians. It's not normally used anywhere since the values can be inferred from the resolution and latitude/longitude indices if available, but could be used for validation to make sure the image is imported into the correct patch.

I see. I also only serialized them in order to have a binary matching round-trip.

I'd vote for unifying them. I used hdr, since .img/.hdr pairs seem to be quite common in scientific formats, but there's no problem changing it to .meta.

If *.hdr is a common extension there, I'm all for using that instead. I'll change it accordingly.

 

[martins]

I have uploaded a new version (link in first post). This allows the user to specify the elevation data -> image value range manually during export. The vmin and vmax values in the metafile are now adjusted to hopefully comply with the discussion above.

Note that tileedit coverts the data in the elevation tiles to double on read, and only converts back to 16-bit values + scale + offset on write, so it is possible that a save would create a different file even if the physical values are unchanged (e.g. different offset, and correspondingly different 16-bit data). It does try to retain the scale, unless the 16-bit range would overflow.

For now I have kept the lat and lng values in degrees. It makes it a bit easier to interpret. Would you be ok with changing the ele2png output accordingly? Since the values are not actually used so for, this shouldn't cause any backward compatibility problems.

Let me know if this version now produces output that is compatible with ele2png.

 

[4throck]

Thanks, the export is much better like this

I tried to import some experimental elevation maps I created before (ex: Venus [ame="https://www.orbithangar.com/searchid.php?ID=7027"]Venus surface and heightmap tiles for Orbiter 2016[/ame] ) and I got a CTD.
I guess that's because my terrain doesn't have all the levels, but they work in Orbiter...

It would be interesting if TileEdit could generate and save empty elevation tiles (all pixels at zero meters) for planets that don't have them.
Would be useful for cases were we only have local DEM coverage, or when global coverage is of very low resolution.

 

[jacquesmomo]

It would be interesting if TileEdit could generate and save empty elevation tiles (all pixels at zero meters) for planets that don't have them.
Would be useful for cases were we only have local DEM coverage, or when global coverage is of very low resolution.

it should be a very good idea, indeed !!! :thumbup:

 

[Face]

I have uploaded a new version (link in first post). This allows the user to specify the elevation data -> image value range manually during export. The vmin and vmax values in the metafile are now adjusted to hopefully comply with the discussion above.

Note that tileedit coverts the data in the elevation tiles to double on read, and only converts back to 16-bit values + scale + offset on write, so it is possible that a save would create a different file even if the physical values are unchanged (e.g. different offset, and correspondingly different 16-bit data). It does try to retain the scale, unless the 16-bit range would overflow.

For now I have kept the lat and lng values in degrees. It makes it a bit easier to interpret. Would you be ok with changing the ele2png output accordingly? Since the values are not actually used so for, this shouldn't cause any backward compatibility problems.

Let me know if this version now produces output that is compatible with ele2png.

Looking very good now. :thumbup:
I've changed the discussed aspect: https://www.orbiter-forum.com/showthread.php?p=594001&postcount=209

 

[jacquesmomo]

I noticed just one thing:

My tiles for the (my) Kourou-CSG (French Guiana) area start at level 11 until level 19.
So there is both the "compressed" tiles delivered with Orbiter and therefore my tiles in the "Surf+Mask+Elev+Elev_mod" folders.

If I open "tileedit" everything goes well.
But if I want to set the option "cache files only" I must be with tileedit displayed tiles level 11 to 19. (surf-tiles).
If I am on a level 1 to 10 tile, I have a CTD.
I think it's normal, since there are no tiles from level 1 to 10 in my "cache-files"....
And no importance for Mask and Elev tuiles, even they are missing. It's only if Surf-tiles are missing.

This is not very serious, but a message like "attention etc ..." instead of a CTD would be better?
I do not know if it's possible ...

 

[martins]

New version uploaded:

- the edit marker/pointer isn't that handy... could (at least) an option be added to choose between showing a center marker? That way we could know exactly which pixel is going to be edited.

Position and size of the paint tool should now be indicated better. It may still need a bit more work - I am not quite happy with the precision of the marker yet.

One suggestion: save the last path used to a texture folder, and also save the options in the Configure menu window.

Tileset paths and options are now remembered across sessions.

I noticed just one thing:

My tiles for the (my) Kourou-CSG (French Guiana) area start at level 11 until level 19.
So there is both the "compressed" tiles delivered with Orbiter and therefore my tiles in the "Surf+Mask+Elev+Elev_mod" folders.

If I open "tileedit" everything goes well.
But if I want to set the option "cache files only" I must be with tileedit displayed tiles level 11 to 19. (surf-tiles).
If I am on a level 1 to 10 tile, I have a CTD.
I think it's normal, since there are no tiles from level 1 to 10 in my "cache-files"....
And no importance for Mask and Elev tuiles, even they are missing. It's only if Surf-tiles are missing.

This is not very serious, but a message like "attention etc ..." instead of a CTD would be better?
I do not know if it's possible ...

This should now work better. Non-existing tiles at the root of the quadtree which don't have an ancestor that can be queried for a subregion are now allowed. In addition, the user can now select not to synthesize tiles even if they do have an ancestor. Does this work ok for you now?

 

[jacquesmomo]

New version uploaded:
Position and size of the paint tool should now be indicated better. It may still need a bit more work - I am not quite happy with the precision of the marker yet.

for me that's fine !!

Tileset paths and options are now remembered across sessions.

Very very usefull : it's a good idea :thumbup:

New version uploaded:
Does this work ok for you now?

Yes it is perfect for me !
What an improvement over the first version of tileedit!

Thank you again, it's great !!!:tiphat: