Many people take for granted the details involved in getting good anti-aliasing. Sometimes, people don't know exactly which settings should be changed to get the AA quality up without blowing out the render speed for no reason.
Here is the truck rendering at the beginning. Notice how bad the anti-aliasing is overall.
Here's a close up that shows the worst parts of this rendering. Look at the first image. The grill is so bad it looks like tin foil! The other image is so bad, that the highlight line is broken in 3 places! This is awful. Then, wait till you get this animating. These spot will CRAWL making your CG look even faker.
The first thing I want to talk about is understanding the display of the anti-aliasing samples. This next image shows what happens when you render with the "Show Samples" checkbox turned on. Alot of these concepts work with metal ray also, but I will talking specifically about Vray. Click the image below for a larger one.
Vray is a raytracer. I does things by throwing rays at the image. When you throw more rays at a pixel in an image, that pixel begins to smooth with the pixels around it and looks anti-aliased. (Some programmer is rolling his eyes at that explanation.) The colors in the sample image represent where the renderer is putting it's rays. Dark blue is less rays, and light blue is more. Broad areas of the image that have similar color should be darker blue, and areas of high detail should be lighter.
The balance of this is all about the Color Threshold value in the AA rollout. This spinner's range is from 0 -1. 0 will push your sampler to light blue everywhere (bad cause it's slow) and 1 will push it to dark blue everywhere (Bad cause it will look like shit.) Here are 2 settings, .001 and 1.0 So, with this in mind, you can now balance the threshold to your needs.
Now, experience tells me we can solve a lot of this with a different anti-aliasing method. The image above is Adaptive subdivision. This type of AA does under-sampling. (Notice the -1 set as the min rate) And while this really helps with the speed of an image, sometimes, it just doesn't do the trick. Under-sampling can lead to small details being lost. So lets switch our AA over to Adaptive DMC (Deterministic Monte Carlo if you're wondering) Now with Adaptive DMC chosen, turn off the setting "Use DMC sampler thresh" This will link your adaptive threshold with the renderer's general DMC noise setting, and for now, we want to just work with the clr threshold in the AA rollout. The default settings are 1 min and 4 max, sampler threshold is .01.
Notice how the DMC sampler is different than the Subdivison one? To me, it's like the image has greater contrast and seems to overlap the problem areas with more light blue coloring. This is much better now, but I need more samples at the high end. I'll change the upper sample from 4 to 6.
Ok, a little better. It's hard to see in the RGB image, but you can see that the grill is looking a bit better. I'll reduce threshold from .01 down to something smaller like .005. Lets see where that gets us.
Pretty damn good now. Probably a lot slower, but much better. I think in the end, I didn't go this high. I left it at .01 because of what I did next. Can you see that the edges of the grill are still a little noisy? Do you see those little dots around where the hood and grill meet? At this point, the AA isn't the problem, it's really the chrome itself is. The chrome is trying to reflect an HDRI image. Those glitches are due to the default chrome not having enough reflection samples.
I'm gonna double the samples to 16, which will help when reflecting the corners. Now, while I'm here, lets talk about the glossiness. When glossiness is at 1, I believe Vray only throws a single ray at it. (Since it's like a mirror) When glossiness gets below 1, Vray then considers it a glossy reflection and uses the subdivision setting below. So, for this image I want to get a smoother chrome, so I will also change the glossiness to .9 to ensure that more rays are throw at the chrome itself.
Notice how the small highlight glitches near the hood are gone! The edges around the grill are a cleaner, and I can move forward with confidence on rendering the animation out later. However, there is still one more thing that can soften this image. That's changing the filtering type. "Area" is a filtering type that was created by the max scanline rendering back in the day. I think release 4 implemented many known anti aliasing filters that are in max today. Back 10 years ago, using these filters with a SD frame (720x540) would give blurry results. But today, with everything being in HD (1920x1080) these filters are now worth even more. I like the Soften filter. I did some tests with all the types, and Soften seems to smooth the results without looking very "blurry".