Post 2024 API Cleanup Plan

[jarmonik]

Here's my current API cleanup plan:

Removal of the DirectX7 based renderer and DX7 compatibility requirements.
Those very awkward mesh configuration/override files in /Config/GC/ would be discontinued. (no support after Orbiter 2024)


Mesh Visibility Mode

There are known issues with mesh visibility flags and clipping distance. Documentation regarding the flags should be reviewed and solutions searched for the clipping. It would be possible to render the exterior mesh of the focus vessel using interior clipping distance, but if the focus vessel is docked to a station then the entire station should be rendered using interior clipping otherwise z-ordering/depth-buffer would fail to render the docking port correctly. If the clip distance is too short Z-fighting would occur. I guess this should be tested in a test configuration to see how short the distance can be until problems arise. There's an issue raised about this on GitHub https://github.com/orbitersim/orbiter/issues/363


New Mesh File Format

New mesh file format 'MSHX2' would be added to address several issues. The overall layout of the format would be identical to 'MSHX1' format which, of course, will remain supported.

Right now there is no way to explicitly identify Virtual Cockpit mesh from the rest which is required for rendering VC shadows and other interior effects. New entry is required to tell what the mesh actually is. Also the shader that's required for rendering the mesh need to be identified in a mesh file.

Mesh materials will depend on shader being used, so, material entries would be identified with "DIFFUSE", "SPECULAR", "SMOOTHNESS", etc, keywords to avoid mix-ups and to do sanity checks. Right now there's just an array of values with no ID what they mean.

Mesh flags entry would be added for a flags that applies to entire mesh not just a single group.

Mesh groups would be categorized either as 'static const' or 'dynamic'. oapiEditMeshGroup() can be used only in a 'dynamic' group, using it for a 'static' group would do nothing. Of course, 'static' mesh groups can be animated and manipulated via matrices, since these operations do not require altering vertex data in a buffer.

In a 'MSHX1' format all mesh groups are 'dynamic' as they have always been and the mesh would be rendered with DirectX7 compatibility shader by default with normal map support, self shadows, dynamic ambient light (i.e. Earth Glow). PBR shaders could be supported (unofficially) for a backwards compatibility if PBR textures are provided, but due to lack of PBR material definitions, mesh groups without texture would render with default material.

In a 'MSHX2' format all mesh groups are 'static' by default and must be flagged as 'dynamic' to gain access via oapiEditMeshGroup() if needed. All shaders supported. Build in mesh debugger will be able to convert and save 'MSHX1' meshes in 'MSHX2' format as well as manage and configure all mesh properties.


Animations
Incremental animations will be removed. There's been problems with drift, when animation state is frequently incremented from a previous state it will slowly lose it's position over time (sometimes in a few minutes). Also if animation state is altered in a wrong callback it could contribute in a drift. "Absolute animations" are always updated from a default state for every frame (no drift). But backwards compatibility is less than 100%.


Stock Vessels
There is a pull request for BakedVC and VC Lighting update. It contains a lot of changes those are not so well tested, therefore, it might be better not to merge it to main before the release of Orbiter 2024. There are no vessels using those features anyhow. After the release it would be great idea to give a graphics update for the stock vessels (DeltaGlider, Atlantis). Especially textures and meshes would need some work.


Graphics Options
Current graphics options are somewhat disorganized since some graphics settings are in the Options tab and other ones under the "Advanced" button. So, maybe moving all graphics options in the advanced panel would be good idea and then making that panel to open from the "Options" button.


Old Planetary Textures
If we can make a tool to convert old planetary textures into a new format (TileFormat 2) then we could cleanup the source code by discontinuing the support for old *.tex files.


Math Cleanup
Math routines are messy and need some cleaning up and standardization. As an preparation step for Vulkan translation plan is to decouple DirectX math libraries from the D3D9Client by using FVECTOR data types instead of D3DXVECTOR data types.

Other Things
virtual void clbkDestroyRenderWindow (bool fastclose) needs cleaning up, remove fastclose. clbkDestroyRenderWindow is called multiple times with different 'fastclose'
Module::clbkShutdown() callback is required for releasing resources, surfaces, fonts, etc...

 

[Urwumpe]

Would it be possible to give sun a volume in the course of the planet refactoring? AFAIR, its still a point in Orbiter.

 

[Gondos]

Would it be possible to give sun a volume in the course of the planet refactoring? AFAIR, its still a point in Orbiter.

I'm not sure the API prevents this, implementing a 3D sphere in VStar.cpp instead of a billboard may be possible already (as long as you don't plan on landing on the sun^^)

Regarding the API, could we plan a complete removal of the HDC stuff in the SDK and keep only the Sketchpad alternatives? Maybe merge the VESSEL/VESSEL2/VESSEL3/VESSEL4 classes to clean things up (same thing with MFD/MFD2)?

 

[Urwumpe]

(as long as you don't plan on landing on the sun^^)

Don't worry, I only do that at night.

 

[jarmonik]

It would be possible to improve the sun rendering but it would require an artist to create sun surface textures and there's also the question of time. The list of things that need doing is very very long. It's simply not possible to do everything, I'll have to prioritize.

Martin said something about re-working the MFD implementation, so, something is going to happen there, but I don't have details.

Complete removal of HDC stuff would force developers to move to a Sketchpad. As far as I can tell there is no HDC support on Vulkan, so, anything beyond simple MFD would be no-go. We probably should keep the current HDC support in D3D9Client because lot of applications are still depending on it. There wont be any D3D9 development after creation of VulkanClient. There is a change that Orbiter 2024 might be the last one with HDC support. So, add-on developers should start preparing for it. I would expect that HDC support is going to last at least a year. Hard to imagine things moving any faster than that.

 

[Gondos]

It would be possible to improve the sun rendering but it would require an artist to create sun surface textures and there's also the question of time. The list of things that need doing is very very long. It's simply not possible to do everything, I'll have to prioritize.

Martin said something about re-working the MFD implementation, so, something is going to happen there, but I don't have details.

Complete removal of HDC stuff would force developers to move to a Sketchpad. As far as I can tell there is no HDC support on Vulkan, so, anything beyond simple MFD would be no-go. We probably should keep the current HDC support in D3D9Client because lot of applications are still depending on it. There wont be any D3D9 development after creation of VulkanClient. There is a change that Orbiter 2024 might be the last one with HDC support. So, add-on developers should start preparing for it. I would expect that HDC support is going to last at least a year. Hard to imagine things moving any faster than that.

Of course, I'm not talking about removing it from the 2024 version, but for the next one.

 

[Urwumpe]

It would be possible to improve the sun rendering but it would require an artist to create sun surface textures and there's also the question of time. The list of things that need doing is very very long. It's simply not possible to do everything, I'll have to prioritize.

Then we should get it into to the next one. The textures where likely the smallest problem of many and depend on what the graphics engine side can offer.

 

[jarmonik]

Then we should get it into to the next one. The textures where likely the smallest problem of many and depend on what the graphics engine side can offer.

Maybe some kind of mathematical function could create the surface from a noise texture. It's easy to create a sphere are give it mathematically created surface color. But the function doesn't come ease.

 

[n72.75]

Maybe some kind of mathematical function could create the surface from a noise texture. It's easy to create a sphere are give it mathematically created surface color. But the function doesn't come ease.

Would something like Voronoi noise work? maybe with the right gradient applied to it.

On the main subject of this thread: how long do we want this process to take, or to ask another way, how big should the scope be? Would we want to be able to have an Orbiter 2025, or maybe give ourselves more breathing room for bigger changes. Or maybe we should just have a list of what we aren't doing in the release effort immediately following 2024.

Also, id like to thank you for taking on the roll of project/repository maintainer, in addition to all the graphicsl wizardry you do . This has definitely helped move things along.

 

[Urwumpe]

Yes, a big thank you from my side as well there.

Maybe we should know which ideas where there, before we decide on which we won't do for now. I can live with the sun being delayed after 20 years of flying past it and still I quite like the noise generator idea, it could even be done on a GPU shader, which makes it very attractive. But it is not important.

I would suggest deciding on some principles how development goes on and how we will decide according to those principles.

For example, looking through the requests, I would say quality of life improvements on the example vessels are rather low priority, despite the amount of wishes for them. They are popular and could make Orbiter more accessible, but would break their primary role as a teaching aid for Orbiter and its SDK and increase the testing effort for each Orbiter release. Also, nothing stops people from doing such improvements with add-ons which could still be later included in the default distribution. Or the people create their own distribution.

Also, I would prefer the principle of keeping Orbiter as a engine lean and mean in its basic release and leave everything else to add-ons and distributions.

I could imagine making a dedicated beginners Orbiter distro. Or a somebody with the interest creates a special ISS+Space Shuttle distribution that ensured perfect interoperatability between the various add-ons he picks (I could imagine some people doing that) All that doesn't lock out other add-ons or configurations, but could provide a valuable extra service to the users.

 

[Urwumpe]

This isn't a real cleanup task, but could you ensure, that the issues regarding x64 builds are assigned to the 64 bit project?

 

[jarmonik]

Would something like Voronoi noise work? maybe with the right gradient applied to it.

Yeah, I think so. It could be a good basis.

On the main subject of this thread: how long do we want this process to take, or to ask another way, how big should the scope be? Would we want to be able to have an Orbiter 2025, or maybe give ourselves more breathing room for bigger changes. Or maybe we should just have a list of what we aren't doing in the release effort immediately following 2024.

Good question. If we look at an early-access games on Steam they often have a regular interval releases containing what ever developers have. But Orbiter development happens on a spare time so it's not so simple. In my opinion it would be good idea to try to make a release once a year, but if it's a quiet year it doesn't hurt if the interval stretches to two years.

I don't expect this API cleanup to take long (at-least the things in the list). We would also need ImGUI and/or wxWidgets frameworks for a platform independent GUI. D3D9DebugControls would need to be remade but no point without solid GUI.

If the next goal is Orbiter 2025 then I don't expect Vulkan being ready by then.

 

[jarmonik]

This isn't a real cleanup task, but could you ensure, that the issues regarding x64 builds are assigned to the 64 bit project?

I don't think we have 64bit project yet and I have understood that the main project becomes 64 bit after removal of DX7.

 

[jarmonik]

Orbiter core uses data types such as Vector, Matrix, Matrix4 and the OrbiterAPI uses VECTOR3, VECTOR4, MATRIX3, MATRIX4. Should we keep them separated or standardize them (i.e. make the core use same data types as the API uses) ?

 

[GLS]

Orbiter core uses data types such as Vector, Matrix, Matrix4 and the OrbiterAPI uses VECTOR3, VECTOR4, MATRIX3, MATRIX4. Should we keep them separated or standardize them (i.e. make the core use same data types as the API uses) ?

That is a good question for Martin: why was it done like that?

 

[Thunder Chicken]

Orbiter core uses data types such as Vector, Matrix, Matrix4 and the OrbiterAPI uses VECTOR3, VECTOR4, MATRIX3, MATRIX4. Should we keep them separated or standardize them (i.e. make the core use same data types as the API uses) ?

EDIT: It seems that someone was/is already working on this?

Centralize basic math types and functions (part 6) by dimitry-ishenko · Pull Request #373 · orbitersim/orbiter

This is a sixth in a series of PRs meant clean up and centralize basic math constants, vectors and matrices. Currently these are scattered all over the project and many of them have overlapping fun...

github.com

That is a good question for Martin: why was it done like that?

I'm reaching in my memory a bit and am not familiar with the details, but I recall asking this many moons ago and IIRC I was told that some of these are DirectX and OpenGL structures.

 

[n72.75]

I'm reaching in my memory a bit and am not familiar with the details, but I recall asking this many moons ago and IIRC I was told that some of these are DirectX and OpenGL structures.

For some strange reason NASSP has two different versions of the vector and matrix types: one which is the normal OAPI kind, and another that looks suspiciously like the internal Orbiter types (circa 2003 code, that we haven't updated yet...).

is it perhaps a historical reason, like were these originally not part of Orbiter, internally but, still included in the SDK and later adopted by Orbiter?

Probably a question for someone who remembers using the SDK in 2002-2003 or someone who wants to dig through archived threads on archive.org

 

[jarmonik]

One possible reason is to keep the API simpler, where as the internal data structures can be more complex. I don't really see any problem keeping them separated.
I don't recall the internal data structures ever been exposed to end user. I made my fist add-on somewhere in 2003-2005