New Release D3D9Client Development

[Ripley]

Will the MicroTextures be added to the D3D9client zip at some point, or will it remain a separate download?

 

[jedidia]

I hope this is jedidia after installing [email protected] lol

It is, as a matter of fact. Together with the code changes jarmonik did, this texture looks like good, sandy floor with some rocks in it up close, and leaves no intrusive tiling effects when zooming out. Great work!

 

[rstr]

jarmonik, can you briefly explain on texture size vs tile size plz.

- I.e.: is (A) the tile size fixed, and an increased texture size just adds more crisp details to the tile, or does (B) the shader automatically increase the tile area size with a bigger texture size ?

- If it is case(A), then: where is the tile size defined, and are there different sizes for _A, _B, and _C levels ?

- Are there currently fixed limits to the texture size, e.g. 2048x2048, or would 4096x4096 work as well without further modifications ? When I start to ceatethose normal maps via PovRay as mentioned above, I'd like to get an idea of what an optimal texture size for _A, _B, _C levels would be. Of course, once the PovRay code is written, it it no problem to generate any sizes, and we could experiment a bit, but maybe we already have an kmprwssion on what "optimal" might be.

- And how does the tile size map to the rendered area (physical dimensions) ?

Thx - Rob

 

[Ripley]

I get this view at Brigton Beach with D3D9 19b, and it does not look like jarmonik's screenshot at post #3499.

I'm sure I installed all the required LowRes and optional HiRes textures...
(reinstalling them now anyway)

:hmm:

 

[rstr]

D3D9_A.dds with Hipath Filter applied

I wonder if it would be worth high-pass filtering the microtexture (offline, not on the fly by the client!).

As promised in #3502, here is a hipath-filtered seamless D3D9_A.dds (V1.2); just replace the old one in your Textures folder by the attached, and be sure to use jarmonik's D3D3Client Beta 19 B for Rev 50. It's too dark yet, but I'm in a hurry today. A final release V1.3 will be published tomorrow.
I'll have the full release of the microtextures being updated as well by tomorrow. For the time being just use the attached replacement for testing.

Screenshot of "D3D9Moon_A" V1.2 hipath-filtered microtexture in the foreground (and _B microtexture seen at Mt. Hadley's slopes)

Hipath-filtered D3D9_A.dds closeup before landing:

Enjoy - Rob

 

[Marg]

Texture patterns themselves are good (B- classic lunar mid-range view), but shouldn't they be in that original format (RGB channels etc.) because there is some odd effect when changing direction of view (when angle to the sun changes). In previous format it looked (I think all this "normal map" thing) more realistic.
I remember textures then were reddish-bluish, when opening them in ordinary PAINT.NET program.

 

[martins]

It's too dark yet, but I'm in a hurry today. A final release V1.3 will be published tomorrow.

This doesn't sound right. The microtextures shouldn't have any noticeable effect on the overall surface albedo. This goes in particular for the high-pass filtered textures, which shouldn't contain any low-frequency content anymore (which carries the mean brightness information).

Maybe there is a mismatch between the convention you used to prepare the microtexture, and the way jarmonik blends the microtexture with the underlying global texture.

One sensible convention might be this: an RGB value of 0.5 in the microtexture represents the neutral intensity level. Any lower values darkens the blend result, any higher values brighten it.

If I remember correctly, in D3D7 this is achieved by a "Modulate2x" blend operation which multiplies the rgb values of the two textures, and then multiplies the result by 2.

So depending on the blend operation Jarmo is using, you would have to adjust the microtexture accordingly so that it doesn't introduce an overall albedo shift (i.e. zero-mean).

 

[rstr]

This doesn't sound right. The microtextures shouldn't have any noticeable effect on the overall surface albedo. This goes in particular for the high-pass filtered textures, which shouldn't contain any low-frequency content anymore (which carries the mean brightness information).

Maybe there is a mismatch between the convention you used to prepare the microtexture, and the way jarmonik blends the microtexture with the underlying global texture.

One sensible convention might be this: an RGB value of 0.5 in the microtexture represents the neutral intensity level. Any lower values darkens the blend result, any higher values brighten it.

No, it indeed doesn't sound right. I thought at first I've done a buzz job, but ...
- mean value is close to 0.5:

- V1.3 _A-microtexture has the same format as the previous V1.2 and earlier

Hence I'm puzzled - but I will recite "Shepard's prayer" to guard against a buzz job of mine (you sure know that story :rofl: ).
I'll re-install D3D9Client Beta 19, to check out whether there is any side effect from the new shader program in Beta 19b. Stand-by for news ....

Texture patterns themselves are good (B- classic lunar mid-range view), but shouldn't they be in that original format (RGB channels etc.) because there is some odd effect when changing direction of view (when angle to the sun changes). In previous format it looked (I think all this "normal map" thing) more realistic.

Dear Marg, this is a known issue for those textures based on Apollo / ALSJ image sources, but as a previous discussion and attempt in this thread has shown, it is not a prudent solution to try and make a normal map out of this material. This is why as parallel evolution of the microtextures ( => version 2.x) I am about to create an artifical set via Povray - with this, we can have true normal maps to realize your very welcome suggestions. I need to dig into my archives for Povray code I have used years ago to achieve a very basic crater-filled landscape ... I need some time to make this code generate fully realistic images ... in the meantime l try to freeze a stable V1.x texture set with the known limitations you've correctly noted (at least it makes folks like jedidia happy ).

Rob

 

[jedidia]

at least it makes folks like jededia happy

Hey, don't load any extra work on your shoulders because of me. I happen to be a very patient man :lol:

 

[martins]

No, it indeed doesn't sound right. I thought at first I've done a buzz job, but ...
- mean value is close to 0.5:

Okay, then, assuming that the surface patch in the image isn't actually that dark (you did compare the intensity with an image of the same scene without microtextures, right?), this would indicate that the blending operation was not RGB1*RGB2*2. We'll have to pass this on to Jarmo to clarify.

 

[jarmonik]

The blend equations is simply:

float3 cFnl = (cFar+0.5f) * (cMed+0.5f) * (cLow+0.5f);

So, the albedo from each texture is offset upwards by 0.5f resulting that each texture will range from 0.5f to 1.5f and then the results are multiplied.

If all channels of every texture is filled with 128 then the equation will become

1 * 1 * 1 = 1

Meaning that no change will occur to the color from a tile textures.

I noticed that some of the lowest mipmaps in _A texture were shifted about 7-10% towards dark. Fixing that cured the problem here. I'll take a look if we can do color balance corrections on a client side.

 

[martins]

The blend equations is simply:

float3 cFnl = (cFar+0.5f) * (cMed+0.5f) * (cLow+0.5f);

So, the albedo from each texture is offset upwards by 0.5f resulting that each texture will range from 0.5f to 1.5f and then the results are multiplied.

If all channels of every texture is filled with 128 then the equation will become

1 * 1 * 1 = 1

Meaning that no change will occur to the color from a tile textures.

I noticed that some of the lowest mipmaps in _A texture were shifted about 7-10% towards dark. Fixing that cured the problem here. I'll take a look if we can do color balance corrections on a client side.

I must admit that I didn't follow the preceding microtexture discussion in enough detail, so I'm not entirely sure how to interpret the equation.

Do cFar, cMed and cLow refer to three levels of microtexture, or does one of these refer to the global surface texture?

If it's the first, then I am missing a step: how is the result of blending the microtextures together (cFnl) blended with the global texture?

If it's the second (let's say cLow refers to the global texture), shouldn't it be

float3 cFnl = (cFar+0.5f) * (cMed+0.5f) * cLow;

so that if both cFar and cMed are at 0.5, the global albedo (cLow) is maintained?

 

[jarmonik]

- I.e.: is (A) the tile size fixed, and an increased texture size just adds more crisp details to the tile, or does (B) the shader automatically increase the tile area size with a bigger texture size ?

A) Yes, right now it makes more crisp detail. B) No.

- If it is case(A), then: where is the tile size defined, and are there different sizes for _A, _B, and _C levels ?

These three lines in Surface.fx

float3 cFar = tex2D(MicroCS, UVn*8.0f).rgb;            // High altitude micro texture C 
float3 cMed = tex2D(MicroBS, UVn*64.0f).rgb;        // Medimum altitude micro texture B 
float3 cLow = tex2D(MicroAS, UVr*512.0f).rgb;        // Low altitude micro texture A

- Are there currently fixed limits to the texture size, e.g. 2048x2048, or would 4096x4096 work as well without further modifications ?

No limits other than user hardware 16kx16k in my case. However, pushing resolution higher doesn't guarantee any better results. Random pattern of multiple smaller different textures might give better results that one big texture. I should run some test to see if it's applicable.

- And how does the tile size map to the rendered area (physical dimensions) ?

Depends about installed tile textures, tile latitude and possibly some other settings.

Important:
I'll add MicroTex.cfg to the upcoming version to address that problem. It will let you to specify pixels/meter resolution for each level A, B and C separately. After that tile resolution is constant regardless installed tile textures, latitude and other settings. NOTE: that, this will also reverse the answer to the first question you made. With a specified pixels/meter setting increase in texture size will increase a surface area coverage. If all goes well should be ready in next 6h.

---------- Post added at 00:15 ---------- Previous post was at 00:04 ----------

Do cFar, cMed and cLow refer to three levels of microtexture, or does one of these refer to the global surface texture?

It refers to three different levels of micro textures. The final step is:

// Apply luminance
cTex.rgb *= lerp(1.0f, cFnl.b, step1);

where "step1" is a sort of smooth on/off switch. So, depending the value of "step1" the eqation becomes.

cTex.rgb *= 1.0f or
cTex.rgb *= cFnl.b;

Luminance is stored in blue channel and per-pixel normal vector is stored in red and green channels. Normals are currently disabled.

Of course, the blend equation:
float3 cFnl = (cFar+0.5f) * (cMed+0.5f) * (cLow+0.5f);

is not ideal to process normals but works well enough.

 

[martins]

It refers to three different levels of micro textures. The final step is:

// Apply luminance
cTex.rgb *= lerp(1.0f, cFnl.b, step1);

where "step1" is a sort of smooth on/off switch. So, depending the value of "step1" the eqation becomes.

cTex.rgb *= 1.0f or
cTex.rgb *= cFnl.b;

That sounds ok to me. It certainly should maintain the global albedo if all three microtexture levels are at 0.5. So, rstr, are you really sure that the result of your 0.5-mean texture is too dark?

In fact, the one thing I am concerned about with the current blending equation is a possible bias towards high intensity:

Let's say you have two pixels next to each other, both of which have a value of 0.5 in all three microtextures. The mean result across both pixels is of course

cFnl = 1

Now assume that one pixel has a value of 0 in all microtextures, the other has a value of 1 in all microtextures. One would hope that their mean cFnl value is still 1, but it actually turns out as

cFnl = (0.5^3 + 1.5^3)/2 = 1.75

So a mean texture intensity of 0.5 will not guarantee an albedo-neutral microtexture blending.

What do you think about this alternative instead?

cFnl = 2/3 * (cFar + cMed + cLow)

 

[jarmonik]

The lunar surface shading is a bit problematic. I have tried to apply Lommel-Seeliger style shading but I am not really sure if it works as it should. It looks good from a distance but when viewing Brighton-beach from a close range the surface illuminance depends heavily from a camera angle. Top-down view gives a darker surface and shallow camera angle brighter view. One possibility is to switch towards lambertian style shading at lower altitudes but then again the surface would become darker.

Of course, these issues are not related to the reported texture issue.

Here's the full code surface shader:

float4 SurfaceTechPS(TileVS frg) : COLOR
{
	float3 cNgt = 0;
	float3 cSpe = 0;
	float4 cTex = tex2D(DiffTexS, frg.texUV.xy);
	float4 cMsk = tex2D(MaskTexS, frg.texUV.xy);

	float3 nrmW = frg.nrmW;					// Per-vertex surface normal vector
	float3 nvrW = frg.nrmW;					// Per-vertex surface normal vector
	float3 camW = normalize(frg.camW);		// Unit viewing ray
	float3 vVrt = vCameraPos - frg.camW;	// Geo-centric vertex position
	float3 vPlN = normalize(vVrt);			// Planet mean normal	
	float  fRad = dot(vVrt, vPlN);			// Pixel Geo-distance

	// Specular Water reflection
	//
	if (bSpecular) {

		#if defined(_SURFACERIPPLES)
			float Fct = min(2.0f, 10000.0f / fCameraAlt);
			float3 cNrm = (tex2D(OceaTexS, frg.texUV.zw).xyz - 0.5f) * Fct;
			cNrm.z = cos(cNrm.x * cNrm.y * 1.570796); 
			// Compute world space normal 
			nrmW = (vTangent * cNrm.r) + (vBiTangent * cNrm.g) + (vPlN * cNrm.b);
		#endif

		float f = 1.0-saturate(dot(camW, nrmW));
		float s = dot(reflect(-vSunDir, nrmW), camW);
		float m = (1.0 - cMsk.a) * saturate(0.5f-frg.aux[AUX_NIGHT]*2.0f);

		cSpe = m * pow(saturate(s), 200.0f) * vWater.rgb * 2.0f;

		float3 cSky = float3(1.2, 1.4, 3.5) * 0.7;
		cTex.rgb = lerp(cTex.rgb, cSky, m * (f*f*f*f));
	}

	else {

		if (bMicro) {

			float dist = frg.aux[AUX_DIST];

			float2 UV  = frg.texUV.xy;
		
			float3 cFar = tex2D(MicroCS, UV*8.0f).rgb;			// High altitude micro texture C
			float3 cMed = tex2D(MicroBS, UV*64.0f).rgb;			// Medimum altitude micro texture B
			float3 cLow = tex2D(MicroAS, UV*512.0f).rgb;		// Low altitude micro texture A
			
			float step1 = smoothstep(25000, 12000, dist);
			
			float3 cFnl = (cFar+0.5f) * (cMed+0.5f) * (cLow+0.5f);
			
			// Create normals
			float3 cNrm  = float3((cFnl.xy - 1.0f) * 2.0f, 0) * step1;
			cNrm.z = cos(cNrm.x * cNrm.y * 1.57); 

			// Approximate world space normal
			//nrmW = normalize(-(vTangent * cNrm.x) + (vBiTangent * cNrm.y) + (nrmW * cNrm.z));
		
			// Apply luminance
			cTex.rgb *= lerp(1.0f, cFnl.b, step1);
		}
	}

	// Do we have an atmosphere ?
	//
	if (!bOnOff) { // No

		float fDRP = dot(vPlN, vSunDir);

		nrmW = lerp(nrmW, vPlN, pow(0.05, saturate(fDRP)));

		float fDNS = dot(nrmW, vSunDir);
		float fDNR = dot(nrmW, camW);
		float fDRS = dot(camW, vSunDir);

		float fX = saturate(fDNS);						// Lambertian
		float fY = 0.05 + saturate(fDNR)*0.4;			// Lommel-Seeliger compensation
		float fZ = pow(abs(fDRS),10.0f) * 0.3f;			// Shadow compensation
		float fLvl = fX * (fZ+rcp(fX+fY)) * fExposure;	// Bake all together

		float3 color = cTex.rgb * max(fLvl, 0);			// Apply sunlight

		return float4(pow(abs(color), fAux4), 1.0f);	// Gamma corrention
	}

	else { // Yes, atmosphere is present

		if (bLights) frg.atten.rgb += 3.0f * cMsk.rgb * (saturate(frg.aux[AUX_NIGHT]) * fNight);
		float3 color = cTex.rgb * frg.atten.rgb + cSpe + frg.insca.rgb + cNgt;
		color = (1.0f - exp2(-color*fExposure)) * vWhiteBalance;  
		return float4(pow(abs(color), fAux4), 1.0f);  
	}
}

---------- Post added at 01:02 ---------- Previous post was at 00:58 ----------

What do you think about this alternative instead?
cFnl = 2/3 * (cFar + cMed + cLow)

Ok, I'll run some test with it.

 

[martins]

What do you think about this alternative instead?

cFnl = 2/3 * (cFar + cMed + cLow)

I suspect that this additive blending equation might result in low contrast of the microtexture features. A multiplicative blender could create higher contrast:

cFnl = 2 * (cFar*cMed*cLow)^1/3

but I guess that might be too computationally expensive.

 

[jarmonik]

What do you think about this alternative instead?

cFnl = 2/3 * (cFar + cMed + cLow)

Ok, here are some test results. Like you said below the contrast is reduced a bit but it's also a little brighter.

Darker stripes are from the original blend equation and lighter stripes are from the one in quote.

I'll make an other test run for this: cFnl = 2 * (cFar*cMed*cLow)^1/3

BTW, just noticed that base shadows go randomly on and off.:lol:

 

[jarmonik]

-double post forum making tricks-

 

[martins]

The reason the new version is brighter might be because the old algorithm actually biased towards lower intensities when averaging over the 3 microtextures:

cFar = cMed = cLow = 0.5 => cFnl = 1

but

cFar = 0, cMed = 0.5, cLow = 1 => cFnl = 0.75

Actually, this probably means that my second suggestion will turn out much too dark.

 

[jarmonik]

Actually, this probably means that my second suggestion will turn out much too dark.

Actually, No. Did I make mistake somewhere ??

Had to make some change to get it compile: float3 cFnl = pow(abs(cFar*cMed*cLow), 0.33333f) * 2.0f;

Original is the darker one.

EDIT: I would say that contrast is a little better in this one than cFnl = 2/3 * (cFar + cMed + cLow)