The shader thread?

[Mr Whippy]

I'm messing with shaders a bit right now.

Right now I am looking specifically at adding control for specular to the existing reflect shader... on Deyan's Murcielago the unified wheel/tyre texturing poses some problems.

I have to control reflection strength with an alpha map, so use the old trick of adding a syntax in the reflection shader so if we define reflection = -1, the baseCol.a is used to control reflection strength.

ambientColor=FresnelMix(ambientColor,envColor.rgb,(Kr>0?Kr:-Kr*baseCol.a),fresnel);


The problem now is that I have no spare texture slot for specular/shininess control.

So I have added this little fragment of code (from the bump speca shader)

Ks=baseCol*Ks;

This now allows the colouring of the outside of the tyre in the diffuse (rgb) channels, to do some work.



However, because the tyre is still dark in the diffuse texture (near black as it is a tyre), it doesn't have much effect on it's own.

I need to somehow increase the brightness of the texture so Racer pumps up the specular we see.

So I added a *5 multiplier to the Ks as you can see in the image attached.

Now what I don't understand is why the two images look different.

First case:
Ks in car.shd is 8, while Ks = baseCol * Ks * 5 essentially gives us Ks = baseCol * 40

Second case:
Ks in car.shd is 40, while Ks = baseCol * Ks essentially gives us Ks = baseCol * 40


Now, I don't get why they look different. Ks is meant to control the size of the specular too, but in either case they both calculate out to a peak shininess of 40, because the baseCol lookup isn't changing.
The specular settings remain the same.

It appears that multiplying by 5 in the shader means the specular level is pumped up 5x, while the shininess (size of the specular) is retained at 8 in the car.shd...

That is the effect I wanted to achieve, pumping up the actual output of the specular blob, while keeping it soft at a value of 8, but it seems un-intuitive/wrong because the value that really needs bumping up is the specular value, not shininess? (though the specular being multiplied by 5x (2.5 2.5 2.5) looked awful)

Am I missing something obvious here?




Ideally I'd like to control reflection amount, specular AND shininess using one shader here...

Probably using rgb.a for diffuse.reflection strength, and then another 3 channel texture for spec/shine and leaving another slot free for something else (perhaps transparrency?)

Specular/shininess maps add so much to these kinds of materials, so it is a shame that they haven't really been considered in the default shaders very much. I think they should almost be defaults in all materials that err away from being generally reflective (ie, using the envmap)


Bodging in the code I have for the Lambo works quite nicely in this case, but it's obviously not ideal to use the diffuse channel to control specular in this way.



Hmmmm

Dave

 

[Ruud]

Yep, I'll do lots of testing over the next few days. I want to see the impact of this huge vista I am rendering. Is there any way I can set the normal draw distance to 5000m, but have these long-distance polygons always drawn like the sky dome...? Ie, a flag I can use to extend the draw distance for said objects?

We had something similar recently, an internal rFactor conversion where we had a skydome, and some panorama objects far away. Note that 'sky=1' doesn't set the far plane to 10000m anymore (which page was that? Couldn't find it on racer.nl). Rather, the sky*.cg shaders clamp the Z coordinate to whatever the far plane distance is (by setting z=1 in the vertex shader, a neat trick, since in clip coordinates, Z=1 is the far plane, the value that is actually in your depthbuffer, which is not a distance in meters, but a non-linear value from near->far, normalized to 0..1).
The good thing is we don't need to swap the projection matrix halfway during rendering for all the sky objects, but render everything in the same projection matrix (same near/far planes).

Here's what we did:
-use a specific shader that is similar to a sky object:

vf_panorama
{
  vertex_shader
  {
    file=panorama_v.cg
  }
  fragment_shader
  {
    file=panorama_f.cg
  }
  diffuse=1 1 1
  ambient=1 1 1
}
shader_misct068~vf_panorama
{
  cull=none
  sky=1
  sort_offset=1
  layer0
  {
    map=mountain.dds
    ; Map has transparency so enable blending
    alpha_to_coverage=1
    alphafunc=gequal 50
  }
}

- you see we had to create a specific panorama shader for this. I will include them in v0.8.22 (out somewhere this week). Ah, I see they should already be there. I've attached them just to make sure. The panorama_v.cg is just like sky_v.cg, sticking the Z to the rear. However, it adds a small offset (right now 0.1, not really small) to make the Z slightly smaller than the sky, so the panorama objects will stick just in front of the sky. We did have some problems with overlapping panorama objects, hence the 'sort_offset' in the shader definition.

The problem here that remained is that the panorama objects should be moved within the viewing distance, so they won't be culled. There seemed 2 methods for that: method 1: create 1 object all around your world, so its bounding box is so large that it never culls (like your 5000 polygon environment). (the cull=none in the shader only means backfacing polygons, there should really be a different name for viewfrustum culling). Method 2: move panorama objects inside the viewing range. We used method #3 . In geometry.ini, give the panorama object 'flags=1', which is 'sky geometry', which effectively means: don't cull with the view frustum:

misct069_157_horizon_00
  {
  file=misct069_157_horizon_00.dof
       flags=1
  }

This now didn't cull the object for its far-away distance, and the panorama Cg shader keeps the Z pegged at 1.0-0.1 = 0.9, so it stays visible while rendering, even though it really started out being further than the render 'far' plane.

Now, the atmosphere is applied to the object, and we found it a bit too misty, so we hacked in (ofcourse this was a quick project) an extinction factor, which you find in panorama_f.cg:

outColor.rgb=lerp(skyColor,dayCol,IN.extinction*5);

The '*5' is the big hack to reduce the amount of extinction, so it 'fit' more with the image. Slapped it on a sandwich and off it went and all was forgotten. Now if you'd add a scale to track.shd:

shader_misct068~vf_panorama
 {
   cull=none
   sky=1
  scale=5.0
   sort_offset=1
   layer0
   {
     map=mountain.dds
     ; Map has transparency so enable blending
     alpha_to_coverage=1
     alphafunc=gequal 50
   }
 }

that would be passed to each shader, so you'd add the scale parameter to panorama_f.cg to allow fine-tuning of the extinction hack:

- add the scale parameter near the other params in panorama_f.cg:

uniform float scale

- change:
outColor.rgb=lerp(skyColor,dayCol,IN.extinction*5);
into:
outColor.rgb=lerp(skyColor,dayCol,IN.extinction*scale);

Long story, hope it's clear.

 

[Mr Whippy]

Sounds sensible enough for now. I'll have to see how it works in practice soon.

How does flags=1 work... if an object like a big torus is drawn as one geob around the car start position, is it always ALL rendered in the pipeline, even behind you...
OR...
...does Racer only do view frustrum culling per geob? Ie, if it's one large object is it essentially running flags=1 already?



I'm guessing that the fudge factor scale is basically forcing a distance upon an object with regards to the extinction factor, so giving it back it's depth relative to other objects using this shader. So say object X is 5000m away, and object Y is 10,000m away, then you may use a fudge factor of 2 for the one that is twice as far away to make it look twice as faded!


All good, I'll have a play. I'll send you the track hopefully in the next week just so you can see what I am trying to achieve here with regards the view range etc... I'm just wanting to get it really optimised as there is so much stuff to go in there... I want it to be fast AND flash

Dave

 

[Stereo]

Note that 'sky=1' doesn't set the far plane to 10000m anymore (which page was that? Couldn't find it on racer.nl).

http://www.racer.nl/tracked.htm#addsky

These days, you also need a shader for all the sky materials and set 'sky=1' in the shader as well. This makes the render engine use a further viewing distance for those objects (9999 meters, so avoid skies bigger than this).

Now, the atmosphere is applied to the object, and we found it a bit too misty, so we hacked in (ofcourse this was a quick project) an extinction factor, which you find in panorama_f.cg:

outColor.rgb=lerp(skyColor,dayCol,IN.extinction*5) ;

The '*5' is the big hack to reduce the amount of extinction, so it 'fit' more with the image. Slapped it on a sandwich and off it went and all was forgotten. Now if you'd add a scale to track.shd:

I guess this is compensating for the extinction already visible in the textures? It's hard to find panorama shots without at least some fog due to distance.

 

[Ruud]

flags=1 is more a note to the renderer that it should not cull. So it's useful for smaller objects that would be culled against the camera's far plane (such as a single mountain polygon far away).
A big torus would indeed have a bounding box where the car never escapes from, so it's rendered. Racer's smallest static geometry chunk is a geob, and culling takes place also per geob.
So if you have a big torus of 5000 polys all around you, it's all rendered. However, the gfxcard quickly avoids drawing the faces behind you (out of the clipspace, which is created from the camera view frustum). And with VBO on everything is on the card anyway, so not much harm. It seems setting up shaders, arrays and such and passing that to the card is more expensive these days than throwing large amounts of polys at once.

 

[Mr Whippy]

Hmmm...

I wonder if there is any harm generally in having a view distance of say 15,000m, and having some objects visible at this distance still.

I may also have a play with just using LOD's for most objects so I can turn them off at varying distances when they become not so significant... ie, big trees might be quite obvious at 5000m away still, on top of the near horizon, but at 7500m they are not, while the terrain still is important.


Hmmm, I guess the problem is not so poweful computers might use a 1500m view distance, which then means everything will disappear at 1500m away, except my sky and terrain... it'll look a bit odd maybe

Hmmm

Testing to do

Dave

 

[Ruud]

Now I just have to figure out how to do lightning flashes..

Just use 'thunder 1' or in special.ini specify in near rain=1.

 

[Ruud]

Hmmm...

I wonder if there is any harm generally in having a view distance of say 15,000m, and having some objects visible at this distance still.

I may also have a play with just using LOD's for most objects so I can turn them off at varying distances when they become not so significant... ie, big trees might be quite obvious at 5000m away still, on top of the near horizon, but at 7500m they are not, while the terrain still is important.

Yes, GPL had for every spline entry 2 variables that dictated how far you could look ahead (and back). For Racer, a high visibility might be ok, then constrained using LOD. There isn't an LOD factor for nothing; it might suit this type of rendering.
Always wondered how GTA does this, with its big skyline objects (such as the Liberty model which you can see from far away). Must be LOD all over the place and almost limitless far planes.

 

[Mr Whippy]

Yep, this is where I'm glad Cam made a nice geometry.ini script generator from Max10, which means I can just select a load of objects, add a comment for the LOD properties for that selection, and then they all get the right data added at export
Even easier because all objects will be separated out into layers relative to what they are. Ie, grass scatters, trees, walls, etc etc... hmmmm...

I think it's probably going to be the way to go thinking more about it, and what you just mentioned about GTA!


Will do some testing on FPS and chopping up my terrain tonight. Will be interesting to see the FPS hits/changes with draw distance and splitting and so on! Results will go up in a nice post entry. Will be interesting to see.

Thanks

Dave

 

[Some1]

Yes, GPL had for every spline entry 2 variables that dictated how far you could look ahead (and back). For Racer, a high visibility might be ok, then constrained using LOD. There isn't an LOD factor for nothing; it might suit this type of rendering.
Always wondered how GTA does this, with its big skyline objects (such as the Liberty model which you can see from far away). Must be LOD all over the place and almost limitless far planes.

I would guess GTA used impostors. I know I Crysis did...
In case you don't know, impostors are runtime generated billboard objects containing the image of the object they are... imposing?

 

[Mr Whippy]

That sounds pretty clever...

So that might mean a house which is made of polys and textures is billboarded as a flat textured object for when you are a long distance away... I wonder if it is then lit/shaded appropriately for varying time of day, or fixed at a given TOD.


Lots of interesting techniques out there, but I guess Racer with it's huge potential with TOD might make some of them less useful.

Ie, I was watching GT5 videos and I could see the envmap swapping as they drove around the track at certain points. Unless they are done on the fly from many track locations as the TOD changes, then they are probably for a fixed TOD... hmmm... interesting

Dave

 

[luthobu]

.. I could see the envmap swapping as they drove around the track at certain points.

Now THAT sounds like a good idea. Instead of a dynamic vs. one-time static env render,
it could be updated at intervals, maybe set in either racer.ini or in the track.ini..

env_update=500 (update for each 500m)
-or-
env_update=60 (update every 60second)

.. just an idea.. heh.

 

[Mr Whippy]

I think the ideal would be to choose locations where you, as an author, think the change is dynamic enough.

The problem is generating them at all the TOD's you want to cover though, if you want dynamic time, and perhaps in the future dynamic weather, and cloud movement etc...


Given the onward march of GPU power, it would seem sensible to just keep using live env-mapping really. It has lots of benefits.

Currently I use it at 1 side update per frame and it 'feels' seamless enough. Yes you can sort of sense it jerking if you watch it, but again in motion when playing and absorbed by that, then it is good enough imo!

The presence of live envmapping at a lower rate is better to me than stepping in the envmap, or limiting flexibility by pre-rendering elements of it.


Dave

 

[camsinny]

Now THAT sounds like a good idea. Instead of a dynamic vs. one-time static env render,
it could be updated at intervals, maybe set in either racer.ini or in the track.ini..

env_update=500 (update for each 500m)
-or-
env_update=60 (update every 60second)

.. just an idea.. heh.

Hmmm - I thought this was already in there but apparently we can only set it a side per frame and not less than that. Would be sorta nice I guess to skip X frames between updates...Then people with slower machines could have the live env. map at not so much of a cost. (I remember NFS MW did this. That was a few years ago so it's not exactly cutting edge)

 

[Mr Whippy]

OK, I'm looking at trees here a bit again.

Racer has a tree shader which basically forces the normal upwards. This works ok most of the time. However, the one weakness is that with an X type tree, the tree casts shadows onto itself, and that can end up making it look a bit ugly.

Does anyone know of an elegant way to make trees cast shadows onto other things, but not onto themselves?

That said, it'd be nice to have a tree receive shadows from other objects, just not from itself... is that possible at all?


I've also been tinkering with normal maps and planar trees, using a normal map to define the shape of the tree which has worked well. I'll post some examples of them working later!


Thanks

Dave

 

[Some1]

For decals (logos and stuff) I use the following shader:

vf_standard_transparent
{
  vertex_shader
  {
    file=dyn_standard_v.cg
  }
  fragment_shader
  {
    file=standard_transparent_f.cg
  }
}

shader_decals~vf_standard_transparent
{
  cast_shadow=0
  layer0
  {
    map=decals.tga
    blendfunc=blend
    alpha_to_coverage=1
  }
}

Of course, the decals won't have envmapping, but that's usually not a big deal, as the decals are small generally.
The 'blendfunc' in car.shd is generally still needed for transparent stuff.

For example, this is my glass shader:

vf_reflect_window
{
  vertex_shader
  {
    file=dyn_standard_reflect_window_v.cg
  }
  fragment_shader
  {
    file=dyn_standard_reflect_window_f.cg
  }
}

shader_glass~vf_reflect_window
{
  reflect=0.5
  sort_offset=99
  fresnel
  {
    bias=0.1
    scale=0.9
    power=2.0
  }
  layer0
  {
    map=glass.tga
    blendfunc=one one_minus_src_alpha
  }
  layer1
  {
    map=$trackenvmap
  }
}

- 'alphafunc' is probably deprecated and replaced with 'alpha_to_coverage'.
- 'sort_offset' is still needed to sort out any possible z-sorting issues.

 

[Mr Whippy]

Hmmm, alphafunc seems to be needed on the trees, so that their reflection is of the object with the masking applied, otherwise they just appear as big squares

That then means you need alpha to coverage to give them a softer fuzzier edge again, otherwise the alphafunc masking looks really horrible... not sure on why, but that appears to be what I have to do.


This is my shader for my X type trees so far:

vf_tree_up
{
vertex_shader
{
file=standard_tree_v.cg
}
fragment_shader
{
file=standard_tree_f.cg
}
}

shader_tree~vf_tree_up
{
shininess=16
specular=0.1 0.1 0.1 1
cull=none
cast_shadow=1
layer0
{
map=texture\ash_01.tga
depthwrite=1
alpha_to_coverage=1
alphafunc=gequal 64
}
}


As for your glass shader, is there any reason you use reflect = 0.5? Surely that is wrong right away as glass is a mirror at 90deg isn't it? You won't be able to achieve a mirror reflection with 0.5 unless you use fresnel to compensate?

In unbiased renderers at least, from the likes of Maxwell and Fry Render, we always use near 100% reflectivity at 90deg angles for a smooth glass type material... otherwise it responds incorrectly. Indeed, I'm not sure, but I think all materials will reflect near 100% at 90deg, but the falloff of that is what matters.
Get a pad of matte white writing paper and hold it up to the sky and look at it at a steep angle, at the very shallowest angle it's quite blue (not sure how close to 100% it gets, maybe only 80%?! but it's still quite reflective )

But then we are mixing diffuse reflection with Racers envmap reflection... hehe... but I think a smooth shiny glass should be reflectivity = 1 anyway, and then if it looks wrong, it's likely down to fresnel coefficients and mapping values!?

Dave

 

[Some1]

As for your glass shader, is there any reason you use reflect = 0.5? Surely that is wrong right away as glass is a mirror at 90deg isn't it? You won't be able to achieve a mirror reflection with 0.5 unless you use fresnel to compensate?

Yeah, no reason actually. It was 0.5 because I was experimenting. I noticed that the glass didn't get any fresnel, but I think it was my eyes playing tricks on me.

Actually, you're right about the alphafunc, it is needed.
This is my tree shader:

shader_tree01~vf_standard_transparent
{
  cull=none
  layer0
  {
    map=tree01.tga
    alphafunc=gequal 128
    alpha_to_coverage=1
    wrap_s=clamp_to_edge
  }
}

 

[Alexander Knoll]

some1...the
wrap_s=clamp_to_edge

is it not only for sky? I never tried it for trees...hmm....test it^^

EDIT:
THX, looks nicer...i think..

 

[Ruud]

For decals, we also add depthwrite=0 to avoid getting shimmering shadows near the logo edges (we actually use it on skidmarks on top of tracks, to avoid it throwing funny shadows onto the track).

@Dream Factory: nice images. The grass might use a detail texture added to be in the same ballpark as the detail in the foliage. Grass really needs its own devoted stretch of development time, as do trees and roads... (since those are the things you see most)

 

[camsinny]

Indeed, Dave mentioned using the viewing angle compared to the surface normal to affect the alpha of the surface. Might be nice for grass and such that use X or * shaped arrays - you'd only see what's facing you and therefore wouldn't get the dodgy look you inherently get from X or * objects.