Project D3D11Client Development

[asmi]

back buffer only option makes all the MFDs display fine.

I highlighted what is off in the XR2 2d panels. It is essentially anything that changes on them. The switches that can be pressed, the numbers in the digital readouts are all displayed with a white layer on them. As if the transparency in the image is not being worked out right. And the standby artificial horizon is just not displayed at all.

Oh I see - I actually has never seen XR2 in a built-in client (I was using DGIV), so I thought that's the way they are supposed to look like :lol: Ok I'll see what I can do.

And it seems my XR-2 beauty shots are loosing quality and resolution as I post them up here. This might be a better one, with a bit more light on the hull, and the strobe lights on. Another fantastic effect indeed.

Yes, I've noticed the same thing, and there are two reasons for that - first of all, the forum engine reencodes them (probably due to image size restrictions), and the second one - it's static, while in the game you see it all in motion, which perceptually makes quite a difference

BTW I think I've identified what the issue might be with PP effects - that you've seen yesterday on your PC, so looks like I will be able to provide a fix tonight.

 

[Glider]

I've completed planet normal mapping, planet per-pixel lighting, fixed some bugs in tile texture loading code and added simple terrain on Mercury. Planet normal maps files must have names like these: Moon_norm.tex, Moon_tile_norm.bin, Moon_tile_norm.tex. All normal map resolutions should work including L > L8. Terrain currently availably only for Mercury, and it requires at least L8 texture. Terrain can work with any planet texture resolution above L8. Mercury terrain uses simple perlin-noise heightmap. Tile resolution set to L18 for all planets. DLL created with latest sources from asmi's fork attached. Sources will be added tomorrow when I'll figure out how to commit to bitbucket.

Planet height maps currently disabled, works only dynamic heghtmap generation for Mercury.

Next I'm going to add config for planets with terrain params, add support of real data heightmaps and finish terrain rendering by adding better shading and some ground microtextures. (currently terrain uses only planet diffuse texture).

 

[asmi]

I've completed planet normal mapping, planet per-pixel lighting, fixed some bugs in tile texture loading code and added simple terrain on Mercury. Planet normal maps files must have names like these: Moon_norm.tex, Moon_tile_norm.bin, Moon_tile_norm.tex. All normal map resolutions should work including L > L8. Terrain currently availably only for Mercury, and it requires at least L8 texture. Terrain can work with any planet texture resolution above L8. Mercury terrain uses simple perlin-noise heightmap. Tile resolution set to L18 for all planets. DLL created with latest sources from asmi's fork attached. Sources will be added tomorrow when I'll figure out how to commit to bitbucket.

Planet height maps currently disabled, works only dynamic heghtmap generation for Mercury.

Next I'm going to add config for planets with terrain params, add support of real data heightmaps and finish terrain rendering by adding better shading and some ground microtextures. (currently terrain uses only planet diffuse texture).

Works just fine:

P.S. What I like about athmosphereless planets - is that you can fly on 10Km orbit around it without any worries The screenshot above was made from 16 Km orbit.

Allthough I'd like to note two things
1. we NEED collision detection - it is a MUST now, as flying through planet looks just wrong
2. we NEED tesselation - right now terrain looks too bumpy. While this could be acceptable for athmosphereless celestial bodies, this is an absolute no-go for planets like Earth...

But anyways - great job!

For committing - first you need to clone current repository, then switch it to D3D11Client branch (you can just right-click latest version in the list of revisions and select "Update"), then you'll need to copy over your changes into that directory.
After that right-click on a repo root folder in Explorer->Commit. Enter notes and click "Commit". Then right click on the root folder again -> Hd Workbench. In this window click "Detect outgoing changes blah-blah-blah", it will show you your commit and offer you to push it to the server. Click push - and the process will start. As some point you will be asked for a user and pass, enter whatever user and pass you're chosen during registration. That's it!

 

[asmi]

I've checked in a fix for post-processing effects, including binary and shaders. Someone please test and confirm if it's working or not - as this issue is/was not reproducable on my system... Please note that current effects implementation is for DirectX 11-compilant video cards only.

 

[Veterok]

Great job Glider! :thumbup:

Mars is gorgeous normal mapped.
http://www.youtube.com/watch?v=G8uyHV3SBDA&feature=youtu.be

On the southern hemisphere of the moon the normals are reversed. It doesn't seem to be happening with Mars, but it's harder to tell due to the atmosphere, and shadows on the diffuse map.

Now that I look at the video the craters do look like they are bumping out from the surface.

 

[asmi]

OK, thanks to Cras I've localized a problem, and will try to fix it tomorrow. In the mean time I've checked in version with enabled bloom, but disabled tone mapping.

Cras, once again - thanks a lot for the help in pinning the issue down!

 

[Face]

I've completed planet normal mapping, planet per-pixel lighting, fixed some bugs in tile texture loading code and added simple terrain on Mercury.

Thanks. I've uploaded the content of this ZIP archive (despite the missing source code) as commit to my BB repo. Your Glider/master branch has been updated to derive from Asmi/master.

Sources will be added tomorrow when I'll figure out how to commit to bitbucket.

I don't know how much you know about DVCS, so please forgive me if this sounds cumbersome now.

With DVCS, the best way to approach it is to think of it as a fancy kind of ZIP archive manager. You pile up a stack of "ZIPs" (commiting) in your local repo, then "push" the difference in "ZIPs" to a remote stack. Consequently, you do not commit to BB, you commit to your local clone of the repo on your machine. Then you push to your clone of the repo on BB.

Therefore, you have to clone the repository first down to your machine. It doesn't really matter if you do it from Asmi's copy on BB or my copy on BB. You don't even need a registration on BB to do the clone.

Once you've got your local clone, you can commit to it whenever you wish, no network, no password needed. To publish your new commits (or "ZIPs", to stay with the mental picture), you have various ways. You can push it to another repository (if you have push rights), you can "bundle" it to a neat archive file that can be posted on O-F (and others can pull from this file) or you can email patches. Of course the best way is to do what Asmi does: create a BB account, fork/clone/copy the repository on BB, and push all commits from your local repo to this BB clone of yours.

But my offer still stands: if this would hinder your development time somewhat, I guess we'd all prefer you hacking on the code instead of fiddling with version control, so you can continue to upload ZIP-snapshots of your code and I'll do my best to put it into the repo under your name - just as I've done before.

regards,
Face

---------- Post added at 10:29 ---------- Previous post was at 10:06 ----------

@Asmi:

I would advice against putting the debug database file into the repository, at least not with every commit. The reason is, that binary files that size (7MB) - while not causing trouble otherwise - blow up the repo size somewhat if changed regulary. I estimate a commit with DLL and PDB in it to be around 1.5MBs (of course it is compressed). This would mean that every commit with different DLL and PDB in it adds 1.5MBs to the repo size of ca. 40MB. Without these binaries, a commit's size would be negligible.

Therefore, I'd like to propose a workflow as follows: keep up the fine-granular commits, but only commit the DLL from time to time, maybe once a certain state is ready to be announced on O-F. The PDB should not go in there at all, instead you can upload the PDB to BB's download section as single file. This way, the repo still represents a complete addon-package (but only at certain commit steps), and testers can download the most recent PDB for debug purposes on demand, without the huge file spoiling clone-time for new developers.

Of course we can remove the files from the repo later on, but that would rewrite history and therefore change the hash-codes...

 

[asmi]

Well generally speaking there shall be no binaries in the source control at all - there is a reason it's called source control As far as I'm concerned, whatever way is a good one as soon as we don't have to manually copy things back and forth. I still think it would be easier if we both will work in a single repo, but if you prefer something else - that good with me as long as you provide me with instructions on how to get other people's changes to my work copy witt minimal amount of hassle.
As for the binaries - I'm going to come up with some way to automate creation of installation zip from current sources - maybe it will be a batch file, maybe some simple program - we'll see. The idea is when someone needs to create that package, he(she) just runs it, and uploads the output to "Downloads" section on BB, or whatever else he(she) intends to do with it.

 

[Face]

Well generally speaking there shall be no binaries in the source control at all - there is a reason it's called source control
...
As for the binaries - I'm going to come up with some way to automate creation of installation zip from current sources - maybe it will be a batch file, maybe some simple program - we'll see. The idea is when someone needs to create that package, he(she) just runs it, and uploads the output to "Downloads" section on BB, or whatever else he(she) intends to do with it.

Well, of course you are right regarding binaries in source control, but in our case it is both no problem and comfortable to layout the repo itself as package. We have that already. Just the frequent update of the DLL would make it cumbersome. Of course you need to update it for publishing a functional package, but not for intermediate commits for developer interaction. The later will recompile it, anyway.
In addition, the PDB is not necessary for packaging at all. Of course it makes sense for debugging, but that will do as separate downloadable file IMHO.

A packaging script is certainly a good idea, but I don't think it is necessary at this point.

As far as I'm concerned, whatever way is a good one as soon as we don't have to manually copy things back and forth. I still think it would be easier if we both will work in a single repo, but if you prefer something else - that good with me as long as you provide me with instructions on how to get other people's changes to my work copy witt minimal amount of hassle.

Oh, hey, you can only mean Glider and yourself as "both", as I'm not doing anything worth of mentioning .

As for the single repo: if Glider commits to a copy of my repo, it already is in the "single" repo, as far as DVCS is concerned. I guess you mean "single branch", and that would mean merging or rebasing, like with the only usable workflow in SVN.

As you can see from my repo graph, there are actually 2 bookmarks that keep going: Asmi/master and Glider/master. My master is sitting there waiting for me to backport the code. If you two want, I can make the master bookmark the integration one, always merging in both your development if needed. The old master commit would then be renamed O2010P1, to indicate the goal of having it run for the old Orbiter version.

If done so, all you have to do is pulling master from me (or Glider's copy, if there is one) and merging it in to your development (or rebase your changes on top of the master). This way, you'll always have the "latest" integration code in your working copy. Still you are free to continue your work on your original commits without interference from e.g. Glider's code. At some point in time, of course, it needs to be integrated.

regards,
Face

 

[asmi]

Well, of course you are right regarding binaries in source control, but in our case it is both no problem and comfortable to layout the repo itself as package. We have that already. Just the frequent update of the DLL would make it cumbersome. Of course you need to update it for publishing a functional package, but not for intermediate commits for developer interaction. The later will recompile it, anyway.
In addition, the PDB is not necessary for packaging at all. Of course it makes sense for debugging, but that will do as separate downloadable file IMHO.

A packaging script is certainly a good idea, but I don't think it is necessary at this point.

Point taken.

Oh, hey, you can only mean Glider and yourself as "both", as I'm not doing anything worth of mentioning .

Well I didn't mean to say that it's a two person show and you are just hanging around - no, nothing like that. I was just thinking about the way to integrate my changes with his - that's why this word slipped out. I'm sorry if that insulted you - I didn't mean it. That's just my goal-oriented approach - right now there is a problem of integrating my changes with his - so I think of it as a two-person problem.

As for the single repo: if Glider commits to a copy of my repo, it already is in the "single" repo, as far as DVCS is concerned. I guess you mean "single branch", and that would mean merging or rebasing, like with the only usable workflow in SVN.

As you can see from my repo graph, there are actually 2 bookmarks that keep going: Asmi/master and Glider/master. My master is sitting there waiting for me to backport the code. If you two want, I can make the master bookmark the integration one, always merging in both your development if needed. The old master commit would then be renamed O2010P1, to indicate the goal of having it run for the old Orbiter version.

If done so, all you have to do is pulling master from me (or Glider's copy, if there is one) and merging it in to your development (or rebase your changes on top of the master). This way, you'll always have the "latest" integration code in your working copy. Still you are free to continue your work on your original commits without interference from e.g. Glider's code. At some point in time, of course, it needs to be integrated.

regards,
Face

Well I guess I'm just projecting the way we do this at work. I'm just used to that approach, and so it sounds like an easy and logical way to go. Let's try it your way and see how it goes. At the end of the day, there is absolutely nothing that stops us from creating a project somewhere in, let's say, codeplex, and turn on to approach that I'm used to in commercial development.

 

[Glider]

I've pushed all source code to asmi's fork. Also, I've fixed bug with inverted normals on south hemisphere. DLL attached.

 

[asmi]

I've pushed all source code to asmi's fork. Also, I've fixed bug with inverted normals on south hemisphere. DLL attached.

Can you please include all files for the libraries that you are using? They are non-standard, so I think they shall be included in SC as well.

 

[Cras]

After a good and long chat with asmi last night, he brought up a good point on two issues that are to be implemented at some point.

Plamsa re-entry effects
As many of you I think are aware, the re-entry "flames" as they are implemented in orbiter are not quite in line with what is seen in real life. Case in point, if you re-enter the Space Shuttle with the correct entry profile, you will never see flames. As you all probably have in in pictures and/or video, this is not at all the case. The Shuttle experienced re-entry flames, leaving that distinct trail in the sky, and engulfing the windows on the flight deck with that reddish glow. The Shuttle experienced this at an altitude much higher than you can normally get re-entry effects in Orbiter.

So the issue here is, what is the discrepancy? What is the actual effect seen on the Space Shuttle, how does this differ from what is in Orbiter now, and would it be possible to implement a more advanced/complete simulation of this effect in the D3D11 client, both in regard to the conditions for their appearance, and their appearance in general.

Lens Flare
asmi brought up adding lens flare, which would be tremendous. Anyone who has messed around in Space Engine would appreciate having the sun generate a flare and what that would add to the Orbiter graphical environment. While it is easy enough to explain the constituents and the mechanism in real terms of a lens-flare, we lack the mathematical concept. If anyone could shed light on how to mathematically describe the lens flare, how to generate a realistic looking one with Orbiter's partial concept of the camera view point, that would be tremendously helpful.

 

[Face]

Well I didn't mean to say that it's a two person show and you are just hanging around - no, nothing like that. I was just thinking about the way to integrate my changes with his - that's why this word slipped out. I'm sorry if that insulted you - I didn't mean it. That's just my goal-oriented approach - right now there is a problem of integrating my changes with his - so I think of it as a two-person problem.

No, no, I'm far from feeling insulted. My comment was merely a hint on me being already far off the development process and therefore my "decision" being not important to the project. Whatever fit your both needs is fine for me, too.

Well I guess I'm just projecting the way we do this at work. I'm just used to that approach, and so it sounds like an easy and logical way to go. Let's try it your way and see how it goes. At the end of the day, there is absolutely nothing that stops us from creating a project somewhere in, let's say, codeplex, and turn on to approach that I'm used to in commercial development.

We're using the fork/hack/merge approach at work for commercial development, too, so naturally I came up with it . Do you use SVN or TFS at work by any chance?

I've pushed all source code to asmi's fork. Also, I've fixed bug with inverted normals on south hemisphere. DLL attached.

So I see you are already up to speed, Glider. Nice! I'll put this all together to merge in your previous snapshots...

regards,
Face

 

[asmi]

We're using the fork/hack/merge approach at work for commercial development, too, so naturally I came up with it . Do you use SVN or TFS at work by any chance?

TFS it is

 

[Face]

I see. The mentioning of Codeplex kind of gave it away (as it is the only open source forge supporting TFS AFAIK) .

In this case you are used to an SVN-like workflow, where you work on a trunk and commit to the central server. There you always get your changes merged into the newest state on the server (if it changed in the meantime since last check-out) if you do a commit to the trunk. This means that you potentially loose a working state, if you do not stash/backup your changes before committing. Of course you can work on your own branch, but then you'd have to cope with the suboptimal merge algorithms of non-DAG systems like TFS.

We used to work this way, too, but only until we reached a specific count of developers concurrently working on the project (around 5). From this point on, CVS/SVN/TFS approaches just broke apart. If you ever spent a day in merge hell for a trivial branch, you know what I mean. Thus we tried DVCS systems like Git and Mercurial (with the later chosen due to easier user experience) and never looked back.

Working on your own repo without being disturbed by other people's checkins, and then preparing what should go public, together with selectively merging in collegues' work - backed by a great merging-algorithm - is just unbeatable IMHO.

BTW: I already have your master (without Glider's changes) running in O2010P1. I'll push that soon...

regards,
Face

P.S.: Do you join in on Orbiter-IRC occasionally? If so, maybe we can arrange a kind of "meeting" for the project, just like the SSU-guys do.

---------- Post added at 20:14 ---------- Previous post was at 19:06 ----------

I've pushed the O2010P1 merge of Asmi/master as version 0.1 to my repo. I've also updated the wiki there with installation section and new bookmark layout.

 

[asmi]

I see. The mentioning of Codeplex kind of gave it away (as it is the only open source forge supporting TFS AFAIK) .

Actually I told that in one of my first posts in this topic, so I ddin't know there was anytning to give up

In this case you are used to an SVN-like workflow, where you work on a trunk and commit to the central server. There you always get your changes merged into the newest state on the server (if it changed in the meantime since last check-out) if you do a commit to the trunk. This means that you potentially loose a working state, if you do not stash/backup your changes before committing. Of course you can work on your own branch, but then you'd have to cope with the suboptimal merge algorithms of non-DAG systems like TFS.

We used to work this way, too, but only until we reached a specific count of developers concurrently working on the project (around 5). From this point on, CVS/SVN/TFS approaches just broke apart. If you ever spent a day in merge hell for a trivial branch, you know what I mean. Thus we tried DVCS systems like Git and Mercurial (with the later chosen due to easier user experience) and never looked back.

Working on your own repo without being disturbed by other people's checkins, and then preparing what should go public, together with selectively merging in collegues' work - backed by a great merging-algorithm - is just unbeatable IMHO.

Well, that's whan team leads and PMs are for - to manage tasks so there will not be intersections, so no merging and if everyone takes and applies other's changes often enough (we here are doing this every day in the morning), there are no problems with merging either. As I understand DCVS approach is better when you work on bigger things - like when you hack around with the code for a few weeks before you commit that to SC. However we just require everyone to commit often enough, so even if there is a merging, WinMerge will easily handle that

BTW: I already have your master (without Glider's changes) running in O2010P1. I'll push that soon...

regards,
Face

Good, I hope to have some more fixes tonight.

P.S.: Do you join in on Orbiter-IRC occasionally? If so, maybe we can arrange a kind of "meeting" for the project, just like the SSU-guys do.

No I didn't, because I didn't know it existed I think this actually is a great idea to gather everyone involved and interested in the project to discuss it. I think it's going to be quite a challenge though since I live 9 time zones apart from Glider and 6 time zones from you... So we need to plan in advance.

 

[Face]

Well, that's whan team leads and PMs are for - to manage tasks so there will not be intersections, so no merging and if everyone takes and applies other's changes often enough (we here are doing this every day in the morning), there are no problems with merging either. As I understand DCVS approach is better when you work on bigger things - like when you hack around with the code for a few weeks before you commit that to SC. However we just require everyone to commit often enough, so even if there is a merging, WinMerge will easily handle that

Yeah, we thought so, too.
Unfortunately, reality taught us that it isn't so. Could be the type of projects we are doing, though: embedded Linux for Microcontrollers and associated configuration applications for Windows. It was close to impossible to fix the usual checkin-races with up to 10 devs working on the project concurrently. Even if one team is doing firmware and the other PC software. And loosing working states because of those races is really not funny.
Anyway, although we are still doing the continous integration work-flow, everything got better with using DVCS. Even the server-admin side. So I can only recommend it.

But there are certainly use-cases where central VCS is sufficient...

regards,
Face

 

[asmi]

Yeah, we thought so, too.
Unfortunately, reality taught us that it isn't so. Could be the type of projects we are doing, though: embedded Linux for Microcontrollers and associated configuration applications for Windows. It was close to impossible to fix the usual checkin-races with up to 10 devs working on the project concurrently. Even if one team is doing firmware and the other PC software. And loosing working states because of those races is really not funny.
Anyway, although we are still doing the continous integration work-flow, everything got better with using DVCS. Even the server-admin side. So I can only recommend it.

But there are certainly use-cases where central VCS is sufficient...

regards,
Face

Ok, this is an interesting discussion, allthough I think we're going way too far off-topic here, so let's wrap it up for now. Maybe we can discuss it somewhere else, but let's keep on-topic here - at the end of the day you are moderator so you're supposed to set an example for other members

---------- Post added at 22:37 ---------- Previous post was at 15:39 ----------

I've pushed another update which includes some PP effects tune-up, as well as bugfix for "strobe"-like artefacts that occured at very high FPS rates.

 

[DaveS]

Is there any way to tone down the PP blooming a bit? Right now I think it is a bit on the strong side as it washes out alot of the details.