[tutorial] Modeling Tips and Tricks

[TriggerGod]

NOTE: THIS IS THE THREAD FOR DISCUSSION AND FOR ANY POSSIBLE ADDITIONS TO THE TIPS AND TRICKS BY OTHER RESIDENT MODELERS.

 

Well, this has been something I've [sort of] waited a long time to do, mainly to hone my own modeling skill into something better than what I left you with. And I believe that at this point in time, I can bestow some new knowledge that not everyone may know. I'd like to make note that my section relies entirely on 3ds Max 2011. Some people may have a newer or older version, or using GMax or, in the case of the rare few, using some other programs. But the theory and in some cases the methods are essentially the same.

 

First off is the basics. My sections assumes you know how to basically model something, I'm here to optimize this:

 

1. Model with a reference picture. The human mind is surprisingly bad at visualizing something. Let someone else do that visualization for you, pick out a reference picture. If you look around at websites, I'm sure you'll find a design that piques your interest. But don't pick something that is a bit more than you can chew.
   
2. Maintain a budget. For TSL and KOTOR, I started out with budgets of 1000 polys, which translates to ~2000 triangles. Its OK to go over, but don't go overboard on your game model.
   
3. Choose your sides properly while using cylinders. 8 sides is a good, decent-looking bare-minimum, while 12 sides is good and decent looking while sacrificing quite a bit more polys in your budget. Try to keep the sides to a nice, even number. Odd numbers can be a problem later on in my guide here.
   
4. Don't go overboard with your modeling, add the bare minimum you need; a basic shape. After that, you can add the detail that you can't/don't texture on. We'll discuss faking details in a later section.
   
5. Maintain a good work flow. If your model's wireframe looks somewhat clean and you can safely adjust it, you have a good poly flow.
   
6. Optimize. If you can weld 2 vertices together without taking away from needed detail, then do it. That removes 2 verts and some triangles. In older engines like these, every poly and triangle count.
   

 

If you intend to follow my tips to the letter, I suggest that after you model your game model, you make a copy of that and hide one of them. You will be making something for that game model. For sake of convenience, give one of your models the suffix _L or _LP for Low Poly, and the other _H or _HP for High Poly.

 

For the rest of this tutorial, I will be using a lightsaber model I quickly whipped up, for this purpose. It's called Hydra. I deliberately made some modeling errors I would have made long ago, to show how to fix for what I'm to show you. Also, if you modeled in "Editable Mesh", once your done, convert it to "Editable Poly" (just right click like you did when you had just your standard primitive, go to "Convert To" and then select "Editable Poly"). Some parts can only be done in Editable Poly (to my knowledge).

 

N-Gons

I'm sure you are asking yourself what an N-Gon is. If you modeled using T7's lightsaber modeling tutorial, you should have a faint idea. N-Gon is a term referring to a shape with a specified amount of sides, typically above 4, where n is the number of sides. Why does this matter, you ask? Normally, it doesn't matter and you can simply export your model right after you finish everything. But, as I've quickly learned, N-Gons make smoothing out a high-poly model a pain. My tutorial will dip into a simple-yet-effective high-poly modeling technique, in order to provide a useful starting point for a texture with shadows and such mapped already for you. To start you off on what an N-Gon looks like before and after (render) the TurboSmooth modifier, without editing the n-gon.

 

This just looks ugly. I'm sorry I subjected you all to that. How would one fix that? Heres how; you use the cut function. The easiest way to do this without much guesswork is to use the cut function while under the vertex subgrouping. You'll want to match up the vertices so that each line between them looks like it cuts the n-gon in half. Have a visual reference. It's not the perfect visual reference, but that's the idea. All you do is you press the Cut button (as indicated earlier), select the vertex, then select the vertex on it's opposite side, where it would make the half.

 

Now, how would would know if there was an N-Gon somewhere in their model? Well, one good way to know is if you have vertices, but no completely looping edge to go with it! The red circle shows the "floating" vertices, while the blue line shows the cut that should be made. Yes, this can be a tedious process but it's worth it in the end.

 

Some engines, even today, do not render or shade n-gon's properly. IIRC, TSL and KOTOR automatically cuts everything up into triangles for you, regardless of the shape or amount of sides. And believe it or not, these extra few lines will help us a bit in our next section..

High Poly Modeling!

There are multiple ways to model in the high poly, ranging from sculpting with a program (Autodesk Mudbox and ZBrush are 2 that come to mind) to manually making the model high poly. Some methods take longer than other methods, and I will demonstrate a somewhat shorter method perfect for making the details you need for a high poly model and the bake for the low poly game model.

Now, you may have noticed that I mentioned Turbosmooth in my above section on N-Gons. Turbosmooth is a modifier that subdivides polys into nice, smoother polys. It does not deal with N-Gons well, which is another reason why dividing up those n-gons is a good idea. Turbosmooth divides up based on edges. Which is why you'll want Supporting edges! Not everything will need a supporting edge, but if you want a fairly crisp edge in your high-poly model, you'll want them. For example, lets look at this part of Hydra. Say I wanted the groove areas to be smoothed and rounded, but I want the edges prior to the grooves to be crisp. I would put connecting edges close to the edge of that area. If I'm not explaining this correctly, I apologize, but pictures speak a million words. (Smoothed + Supporting Edges | Smoothed + No Supporting Edges | Not Smoothed + Supporting Edges)

 

While connecting, consider that the Pinch and Slide values are proportional to the length, and are based on a percentage (100% to -100%. Anything more or at that value is just placing that edge on top of it's parent or, in the case of adding 2+ edges, on top of it's sibling)

 

After all my NGons had been taken care of, and after all the supporting edges have been added, its safe to add your turbosmooth modifier. Head over to your modification toolbar off to the right, open up your modification list, and select turbosmooth (the list is in alphabetical order). The number of iterations you choose is mainly up to the speed of your computer. My computer can handle a model of up to about 240,000 polys before it start to lag and stutter; in my model's case, it is at the third iteration. If you notice any strange errors, such as "folding" in your model, add some supporting edges. This is a high poly model, go crazy on your polycount! Here is the final result for Hydra.

 

UV-Mapping

You thought you were done with your low poly?

This tutorial also assumes you know how to do your own UV Map. Theres just two little things that I did until I learned I shouldn't do them. For the next section, on a cool lighting technique that will introduce shading onto your texture, you will need to have absolutely no overlapping faces in the UVW Map. An easy way to check this is with the "Select Overlapping Faces" feature. Another thing you should make sure to do is make sure objects with separate smoothing groups are on their own "UV Island", and make sure they are detached. All the green lines in my previous picture shows all the areas that are separated from the others. On the areas such as the lower grip and the lower boxes on the pommel, where all the faces are on different smoothing groups, in the UV Map they all detached from one another. First you must select the face on the Map you wish to detach. Then you can either go to Tools->Detach Edge Verts in the UV Editor window. Or you can use the default hotkey, where you first press D, and then use Ctrl+D.

 

Ambient Occlusion Pass (AO Bake)

Ambient Occlusion, also known as "a really cool and interesting technique i will use forever", is a simple process to accomplish, with a few steps. The first step is to open up your Material Editor window (Default hotkey M), and create a new standard material. In 3ds Max 2011, the editor window is different than it has been in years past, and I'm personally not used to it. Definitely unique, however. When you open up a new Standard material, open it up (double click the top blue bar in 2011), and set the colors to pure white. Apply the material to your high poly model. This won't have any affect on the actual AO bake, but its to get an idea on where the shadows will be. The next step is to drop a skylight into the scene; anywhere will do, as skylight will simulate light coming from all directions and cast shadows depending on your geometry. Next, make sure your skylight is casting pure white light. Select your skylight and make sure the color is pure white. Unless the RGB and Value values are at 255 and the Sat value is at 0, its not pure white. However, there is a problem; skylight makes a solid color using Max's standard scanline renderer! What do we do now? everything is ruined!

 

Stop crying now and open up Render Setup. Scroll all the way to the bottom of the window that pops up and expand the "Assign Renderer" section in the "Common" tab. Some new lines appear. What you want to do is press the "..." button next to the "Production" line. By default, Max wants to use it's standard scanline renderer. We will want to use mentalray. If it wasn't obvious, select that. If you dare, press render again, and you will get this. Much better and it shows the shadows we want. But we have some tweaking to do. Much of this next part of this section is reliant on how much time you want to spend rendering, or how fast/slow your computer is.

 

Go back to the Render Setup window (F10), and enter the "Renderer" tab. Look to the "Sampling Quality" section, and increase the Minimum and Maximum sampling quality. The higher the number, the better the quality will be, but the rendering time will be increased. As you can see, I set my min/max to 256, and the rendering time escalated from a few seconds to a few minutes. The result is here, the difference is minor but there.

 

When the render window opens up, there will be an added window below it, for extra settings. You will want to play with these settings in order to find a way to get AO the way you want. You will also want to play with the Final Gather points slider in order to get a better render. My ideal settings are a Final Gather precision of "High" and with sampling min of 16 and max of 64.

 

However, this technique is completely useless for the game model bake. Yes, I deceived you into learning something. This is a technique artists use to add definition to their models in post production. For example, creating your scene of a dragon in its textured state, then creating the AO Pass, placing the AO Render on top of the textured layer adds some definition. Some artists use the AO pass with cartoon-style renders (cel-shading) to define mass. If the hit hat-based simulator Team Fortress 2 didn't use AO passes in it's textures for many things (and Specular and Normal maps), it would just be a cartoon-y game with the most basic definition. Now its a cartoon-y game with good definition! In textures, the AO pass allows you to fake a shadow in the texture. Don't worry, I'll show you.

 

(The Real) AO Bake

It is important that you have your UVW Map done for your low poly model. Unhide everything you plan to use now. Lowpoly and Highpoly need to be here. Select your low poly model and Collapse All. Don't worry, your Map is still there. If you don't have your lowpoly selected anymore, select it again. Press 0 (or alternatively, open the "Rendering" dropdown menu from the toolbar and select "Render to Texture").

 

This will open an overwhelming window for Rendering to Texture. Don't worry, most of this stuff is useless for our needs. Change your path at the top if needed, and we will just dive right in to changing the options. Look for Padding options and change it to a 4 (default is 2). Next, enable Projection Mapping. Next, press the "Pick.." button on that same line, and select your high poly model. This will automatically create the Projection modifier on your model, and it will automatically f*** it up. Here is Hydra's f'ed up Projection. The blue lines show the Projection, while the yellow half-circle represents that Skylight we put in earlier (if you don't have this, put one in). Change your projection mapping to Vertex mode and press the Reset button to make your projection tight alongside your model. What you want to do with the projection cage is that you want to push and tweak it so that the cage covers the entirety of your high poly model. This will require quite a lot of tweaking. After you tweak the cage to cover your high poly, return to the Render to Texture window. Make sure to have the option for "Use Existing Channel" and then we are ready for the fun part.

Press the Add button and add the "Complete Map" option. Complete Map is meant to add all your silly little maps (Normal, Spec, Diffuse, the whole 9 yards) into one nifty little texture, but for our use, we will be using it to generate our AO bake. Change the texture size and name in the same section if you desire (I dump all my 3ds max generated textures into one folder, so I name each one specially. I also use 1024x1024 for everything, and if I need to, I will resize it down. I never use 2048x2048 for anything commercially anymore). Grab a drink, depending on if you wanted extra maps (a normals map, which you could combine with your texture into the complete map later if you so desired; I'm not going into detail on that, can make the process take twice as long as normal) or not, this will take a while. When its done, you will find a preview of your finished product and you can access your AO Bake in the folder you dumped it in. Open it up and you may have some issues. Thats alright, you can touch it up in photoshop or GIMP or whatever you use.

 

Now, what exactly is the AO Bake? It transfers some type of faked "shading" onto your texture map. After tweaking, my AO Bake looked like this. Its still a mess, and I'm still personally figuring out how to perfect the whole render to texture thing. Now, what can you do with this?

 

Using the AO Bake

Open up your AO Bake file in whatever program you use. It helps to have your UV Map open in another file as well (multitude of methods for this, not going into details about it here. Use whatever you think is best). Open a third file at the same size/size you want. Put your UV map on top and make it so that you only see the lines in the map (with my method, all I have to do is apply the "Screen" overlay type in Photoshop). Under that layer, put your AO Bake. Make sure you have it lined up perfectly, especially if you saved it initially as a PNG as that saves transparency and doesn't keep the same size as it does in it's parent file. Give that layer with the AO Bake the "Multiply" overlay type. This will transfer the shadows and anything that isn't pure white onto the layer(s) beneath it. That means you can give shading and some definition to a solid color, if you so wish. For example, a light gray and a dark gray for metal and grip underneath the AO layer gives this effect

 

This entire post just described a fancy texturing method, and taught you a little bit about some modeling techniques that are in use by the community for new-ish engines. Hope this helped somebody.

 

Do you have a tip or trick you'd like to share with the community? Post your findings here and I will update the tutorial thread with a new post, giving you the full credits!