New Orbiter SVN commit (r.71, Oct 14 2017)

[kuddel]

Oh. I always felt like adding node was the most busy phase, because my HDD is making noise, while the deflate phase is almost happening in idle. Was already wondering, what the hell it does so silently.

And that is the reason to pack'em into one file. The file I/O when "finding the file" is the bottleneck. Getting the data (sequentially) cannot be reduced; it has to be done anyhow.
As some other user mentioned earlier: just compare the time it takes copying the one compressed file with the time it takes copying hundreds of single files The amount of (net) data is the same.

 

[Urwumpe]

And that is the reason to pack'em into one file. The file I/O when "finding the file" is the bottleneck. Getting the data (sequentially) cannot be reduced; it has to be done anyhow.
As some other user mentioned earlier: just compare the time it takes copying the one compressed file with the time it takes copying hundreds of single files The amount of (net) data is the same.

Deleting the many tiny files is already annoying. :lol: NTFS isn't really effective for small files.

 

[kuddel]

Deleting the many tiny files is already annoying. :lol: NTFS isn't really effective for small files.

I haven't deleted mine yet... Now I remember why I hesitated :lol:

 

[jroly]

I did delete mine unfortuantly because I wanted to make sure Orbiter was infact using the archive. I did do some tests before I deleted and I didn't notice any difference, I used the senario "The Solar System".

I still have the downloaded zips thou so I can stil revert if need be.

---------- Post added at 11:37 ---------- Previous post was at 08:21 ----------

Can someone else can check if Brighton Beach senario is working with the archive? The base and DG are far above the surface atleast for me.

 

[jarmonik]

Here's a quick build of the D3D9Client for Orbiter rev 56. It should have a support for a packed archives (tested only with low resolution tiles) haven't had time to pack the texture files yet. To test it, remember to set "Archive only" option from the control panel. Otherwise, it will use cached (uncompressed) files.


There appears to be a tile problem on the Moon, which has existed a long time already (Not a new issue). Anyone else seeing that ? or any ideas where it's coming from ?

There's a black line surrounding a tiles, which is a result from elevated tiles. (Lng, Lat) coordinates for the vertices are correct but the elevation isn't. I don't recall seeing that on Earth. Only on Moon and Mars.

 

[jroly]

There appears to be a tile problem on the Moon, which has existed a long time already (Not a new issue). Anyone else seeing that ? or any ideas where it's coming from ?

There's a black line surrounding a tiles, which is a result from elevated tiles. (Lng, Lat) coordinates for the vertices are correct but the elevation isn't. I don't recall seeing that on Earth. Only on Moon and Mars.

Havn't seen that for awhile. From what I remember the cause of it was were people need to update their textures. Since you are only using the lower resolution, maybe that is why?

---------- Post added at 18:40 ---------- Previous post was at 17:34 ----------

Before you delete the uncompressed directory trees, could you give it a few tests regarding performance? In fact, if your hard disk space permits, it may be good to hang on to the original files for a while, just in case I need to make changes to the compressed format.

Martin, good thing I did delete the cache files because there is a problem. After I deleted I noticed all the surface bases are way above the surface including the vessels. I copied them back in and they returned to normal, deleted again and they are back up there. The Elevation looks fine, but Orbiter is not using the archive to calculate altitude, so Orbiter still is using the cache even when set to archive. So you will not see this problem unless you delete the cache.

 

[martins]

New Orbiter Beta Released (r.57, Jul 3 2016)

Change log:

1. ElevationManager: now supports loading elevation data from compressed archives

Should fix this:

Martin, good thing I did delete the cache files because there is a problem. After I deleted I noticed all the surface bases are way above the surface including the vessels. I copied them back in and they returned to normal, deleted again and they are back up there. The Elevation looks fine, but Orbiter is not using the archive to calculate altitude, so Orbiter still is using the cache even when set to archive. So you will not see this problem unless you delete the cache.

Thanks. Forgot to add archive load support to the elevation manager class. Can you check if the latest revision fixes the problem? No changes to OVP. The D3D9 build posted by Jarmo above should still work.

 

[jarmonik]

Something is wrong with the texpack.exe. It doesn't compress tiles beyond level 9, which is the highest global level I got right now. For an example the level 19 KSC doesn't get compressed.

 

[jroly]

Martin, It does work but there are problems with it. Moon and Mars is fine I think but with DG at KSC senario, the runway is hilly with the elevation. If you go up close it flattens but zoom out it appears hilly but regardless if the DG tries to use the runway it goes all over the place.

I know KSC is part of Elev_mod.tree and I checked that and it is only 1KB in size, so texpack did not compress it. (maybe it did compress, if it contains flattening data it would compess very small as there is data in it)

Anyway I copied the Elev_mod back in and that didn't fix it so it is something else.

 

[kuddel]

I can confirm the "bumpy KSC runway". Other than that, the D3D9Client (I run home-compiled trunk r697 of D3D9Client) seems to work well with the Archives.

 

[martins]

The problem is that currently texpack can't cope with gaps in the quadtree. Any tiles which don't have an unbroken path of existing tile files to the tree root will be missed. I'll need some time to think about how to fix this. In the meantime, I may provide working archive files (lowres global + highres KSC) as part of a release candidate.

 

[kuddel]

The problem is that currently texpack can't cope with gaps in the quadtree. Any tiles which don't have an unbroken path of existing tile files to the tree root will be missed. I'll need some time to think about how to fix this. In the meantime, I may provide working archive files (lowres global + highres KSC) as part of a release candidate.

I thought that I had all tiles...but maybe not.
Do I understand this correct, that those "bumps" resulted from a tile (leaf) that could not be routed back to root (via branches) and I just haven't all the "branches"? Which seems to be O.K. for the 'single file' algorithm, but is fatal to the 'packed algorithm', right?
And therefore I am not able to "fix" this at home, right?

Nevertheless, that general idea seems to work fine (minor deficiencies :thumbup: )

 

[4throck]

Windows has a built in compression option for any folder or file.
The big advantage is that you can still access stuff the same as before (no extra files are created).
Overall speed is dependent on processor/memory, but it can actually be faster if the system is limited by disk speed...
Perhaps it would be simpler to just use that?


I've tested compressing the moon terrain data and found that up to level 10 the gains are minimal (349MB original, 339MB windows compression).
I think that windows uses Zip (or similar) so I guess that the original data has little redundancy at lower resolutions.

 

[Enjo]

I've conducted an initial performance test, still without the Topographic Map MFD, using the scenario at the bottom, with the DG flying with Prograde AP. Results:

Cached data:
Tacc: 10x -> FPS: 15-18, Max disk usage: 3 MB/s
Tacc: 100x -> FPS: 20-50, Max disk usage: 12 MB/s

Zipped data:
Tacc: 10x -> FPS: 15-18, Max disk usage: 2 MB/s
Tacc: 100x -> FPS: 20-45, Max disk usage: 6 MB/s

Finds:
1) While using the zipped data, I noticed severe slowdowns at polar coordinates, and flying with the prograde AP at tacc. 1000x made the sim unstable. Nothing like this happened with cached data.
2) Greater FPS at higher timeframes results probably from the second thread not loading the data on time (in both cases)
3) I haven't visually noticed any difference in loading the highest patch level after slowing down from 100x down to 10x, between the two runs, but still see 1) for a more objective measure.

What I can't understand is that since the texture loading is done in the second thread, why is the main thread affected by slower texture load (point 1))? Isn't it so, that since the loading times of the archived versions is slower, therefore the queue gets longer and every element of the queue is in processed (or rather copied to) the main thread? If so, couldn't the first-in elements be ignored (i.e. not copied to the main thread), if the queue exceeds a certain reasonable size? This would mean that the vessel moves too fast to notice the first-in elements already.

And the scenario in question:

BEGIN_DESC
	DG in a martian polar orbit - testing cached vs archived planetary data.
END_DESC

BEGIN_ENVIRONMENT
  System Sol
  Date MJD 51982.5310760535
  Help CurrentState_img
END_ENVIRONMENT

BEGIN_FOCUS
  Ship GL-01
END_FOCUS

BEGIN_CAMERA
  TARGET GL-01
  MODE Cockpit
  FOV 50.00
END_CAMERA

BEGIN_HUD
  TYPE Surface
END_HUD

BEGIN_MFD Left
  TYPE Orbit
  PROJ Ship
  FRAME Equator
  ALT
  REF Mars
END_MFD

BEGIN_MFD Right
  TYPE User
  MODE TopoMapMFD
END_MFD

BEGIN_SHIPS
GL-01:Deltaglider
  STATUS Orbiting Mars
  RPOS 1735647.408 2962527.817 938553.374
  RVEL 164.0916 -1133.4380 3274.2263
  AROT -19.901 -4.152 29.278
  VROT -0.0569 0.0000 -0.0004
  AFCMODE 7
  PRPLEVEL 0:1 1:0.9
  NAVFREQ 0 0
END
END_SHIPS

 

[kuddel]

Is it just me or can someone else confirm this:
When I run "Welcome to Orbiter 2016" scenario with external client, the text annotations are very strange...Multiple repeating of one line.
I could not (yet) check if D3D7Client shows this behavior, but D3D9Client does.

Thanks for any feedback,
Kuddel

 

[jarmonik]

Is it just me or can someone else confirm this:
When I run "Welcome to Orbiter 2016" scenario with external client, the text annotations are very strange...Multiple repeating of one line.
I could not (yet) check if D3D7Client shows this behavior, but D3D9Client does.

I noticed that too. Well, it would be nice to have a pre-build binary of D3D7Client available.

 

[kuddel]

I tried to build D3D7Client, but I am unable to find "../GDIClient/GDIClient.h"...any idea?

---------- Post added at 22:22 ---------- Previous post was at 22:16 ----------

...found it, but due to missing <dplay.h> (DX7 SDK I think) I am unable to build

---------- Post added at 22:44 ---------- Previous post was at 22:22 ----------

...got it to build! But only by removing #include <dplay.h>, all used defines @ LogOut_DPErr and d3dim.lib.
It works, but I'd rather not give it to anyone

 

[jarmonik]

It works, but I'd rather not give it to anyone

If the text annotations works with it then the problem is in the D3D9. Do you look into it or should I look into it ?

 

[kuddel]

If the text annotations works with it then the problem is in the D3D9. Do you look into it or should I look into it ?

It looks like the text annotations work as expected in the D3D7Client...
The image update rate at my home computer is a bit slow with that debug build of D3D7Client, so it's not that easy to compare. But I didn't saw the problems with the 'multiple text'.

 

[kuddel]

@jarmo
In case you like to see / debug what D3D7Client is doin' different, I've attached a "bend" version of D3D7Client rev. 51 to compile with VS 2012 (@ home I've used VS 2015, but the project files / solutions should easily migrate)