new breakthroughs in modding!

[Darth Sapiens]

hey guys, loooooong time no see Well, I'm back, though i usually hang out more at deadlystream, I didn't want leave you guys out of the loop with the advancements I have made with:

• bumpmapping
  
• environment mapping
  
• vfx
  

so here is a quick run through and a link to the relevent DS threads.

 

i have made custom bump maps work in kotor/tsl and provided the guide lines for getting them there,

Show spoiler

(hidden content - requires Javascript to show)

 

here is the thread with experiment.

 

i have made it possible to replace the cubemap textures (CM_bright, CM_baremetal etc.)

Show spoiler

(hidden content - requires Javascript to show)

 

here is the thread with experiment

 

i have discovered new, (actually leftover from nwn, so old i guess) functionality in visualeffects.2da, which i'm still investigating lol.

Show spoiler

(hidden content - requires Javascript to show)

 

comments & ideas are always appreciated

[90SK]

Oooh that last one looks awesome. Great job, can't wait to see more!

[DarthParametric]

i have made custom bump maps work in kotor/tsl

Greyscale bump maps, or normal maps like in TSL?

[Darth Sapiens]

Greyscale bump maps, or normal maps like in TSL?

 

http://deadlystream.com/forum/topic/1726-bump-mapping-research/?p=29875

 

they both use the same two kinds of bumpmaps, pre-rendered tangent normals, which we really cant recreate ATM, and grayscale bump maps, which the game then turns into a normal map on the fly, which is something we can create. apparently as far as the gpu is concerned it gets a normal map either way.

[DarthParametric]

Presumably they are using the same method pretty much every game engine does. Store the X and Y and recreate the Z via swizzling. In that case it's not a greyscale bumpmap, it's just a tangent normal with only 2 channels.

[Darth Sapiens]

Presumably they are using the same method pretty much every game engine does. Store the X and Y and recreate the Z via swizzling. In that case it's not a greyscale bumpmap, it's just a tangent normal with only 2 channels.

well there is only 1 channel, no alpha for what we have working,

the 3 channeled ones are tangent normals with xy switched.

[DarthParametric]

Well there are 2 channel normal maps in there. Check out N_Selkath01b.tga in K1 for instance. Looks like X in G and Y in B.

 

Put those in R and G, respectively, fill B with white, run nVidia NormalMapFilter PS plugin in Normalize mode (or use alternative swizzle tool) and bingo, standard RGB tangent map.

[redrob41]

Alright, I've done some experimenting, and here is what I've come up with so far. I decided to use G0T0 because he has his own bump map built in, and it is mostly straight edges, and I like to use basic grey (R=128, G=128, B=128)so things are easy to see in-game. Here is the setup:

 

appearance.2da

Row Label |label |race |racetex|envmap |

453 |Party_NPC_G0T0 |P_g0t0 |**** |DEFAULT|

 

Test 1 through 3 had no txi files in the override folder, but I did replace the P_Gt.tga with versions that had neutral grey for the texture, and alpha channel from 1% to 50% to 100%, and I replaced and P_GtB.tga with a greyscale mode image without an alpha channel. In the following setup images, the base texture is in the upper left corner, the basic alpha channel is the lower left corner, and the greyscale bumpmap is on the right side.

 

Test 1 Setup

Show spoiler

(hidden content - requires Javascript to show)

Test 1 Results

Show spoiler

(hidden content - requires Javascript to show)

 

Test 2 Setup

Show spoiler

(hidden content - requires Javascript to show)

Test 2 Results

Show spoiler

(hidden content - requires Javascript to show)

 

Test 3 Setup

Show spoiler

(hidden content - requires Javascript to show)

Test 3 Results

Show spoiler

(hidden content - requires Javascript to show)

 

What this shows, is that the amount of black in the alpha channel has an impact on how the bumpmap is applied; basically, the darker it is, the more bump gets applied. Also, it shows that just replacing the bumpmap.tga will work, without needing to add txi files. It will just default to settings that are embedded somewhere (I'm not sure if it is in the mdx, mdl, or original tpc).

[redrob41]

Test 4

 

appearance.2da

Row Label	|label		|race	|racetex|envmap	|
453		|Party_NPC_G0T0	|P_g0t0	|****	|DEFAULT|

 

The mdl & mdx were extracted using KotOR Tool and copied into the override folder (without making any changes to either).

 

No P_Gt.txi was used

 

P_GtB.txi was used with these contents:

isbumpmap 1

isdiffusebumpmap 0

isspecularbumpmap 1

bumpmapscaling 10

 

Test 4 image setup was the same as Test 1

Show spoiler

(hidden content - requires Javascript to show)

Test 4 Results

Show spoiler

(hidden content - requires Javascript to show)

 

This shows that if there are unchanged mdl & mdx in the override, the bumpmap connection is not lost. The embedded link still points to P_GtB.tga and txi (it applied a deeper bumpmap scale)

 

Test 5, same as above, except

 

P_Gt.txi

bumpyshinytexture CM_baremetal

bumpmaptexture P_GtB-Copy

 

P_GtB-Copy.txi

isbumpmap 1

isdiffusebumpmap 0

isspecularbumpmap 1

bumpmapscaling 10

 

Test 5 Setup image

Show spoiler

(hidden content - requires Javascript to show)

Test 5 Results

Show spoiler

(hidden content - requires Javascript to show)

 

This shows that the contents of P_Gt.txi do work, since it points to P_GtB-Copy.tga instead of defaulting to P_GtB.tga (which was still in the override folder at the time).

 

Test 6

 

appearance.2da changed to reference a different model name

Row Label	|label		|race	|racetex|envmap	|
453		|Party_NPC_G0T0	|R_G020	|****	|DEFAULT|

 

Renamed the model files to

R_G020.mdl

R_G020.mdx

but did not edit the contents of the files.

 

everything else was the same as Test 5, and it had the same results as Test 5. This tells me that renaming the model files doesn't remove whatever it is that allows a model to have a bumpmap.

[redrob41]

Test 7

 

appearance.2da same as Test 6

Row Label	|label		|race	|racetex|envmap	|
453		|Party_NPC_G0T0	|R_G020	|****	|DEFAULT|

 

I Hex edited R_G020.mdl so that all 13 instances (bones?) of P_Gt were replaced with R_41 and added R_41.tga to the override folder. The images from Tests 4 & 5 were still in there as well.

 

Test 7 Setup image

Show spoiler

(hidden content - requires Javascript to show)

Test 7 Results

Show spoiler

(hidden content - requires Javascript to show)

 

Test 8

 

Same as above, but added

R_41.txi

bumpyshinytexture CM_baremetal

bumpmaptexture P_GtB-Copy

 

Test 8 Results

Show spoiler

(hidden content - requires Javascript to show)

 

This tells me that if whatever file name is changed inside the mdl file, it doesn't automatically update the embedded info for the bumpmap. In Test 7 case, it wound up looking for R_41.txi and didn't find one, so it defaulted to transparent effect, and the 100% black alpha channel made the whole model invisible.

 

Once I added the R_41.txi to the override, it was able to match a texture to a bumpmap tga, but it didn't automatically look for a file with the same name plus B tacked on the end (R_41.tga vs R_41B.tga), but definitely looked for the file named within the txi.

 

Test 9 same as above except

 

R_41.txi

bumpyshinytexture CM_baremetal

bumpmaptexture R_41B

 

R_41B.txi

isbumpmap 1

isdiffusebumpmap 0

isspecularbumpmap 1

bumpmapscaling 5

 

Test 9 Results

Show spoiler

(hidden content - requires Javascript to show)

 

Just further confirmation that the txi files work.

 

Test 10

 

Hex edit R_G020.mdl so that the first 6 instances of P_G0T0 stayed the same, but the last 76 instances were changed to R_G020

 

Test 10 Results

Show spoiler

(hidden content - requires Javascript to show)

 

Test 11

 

Hex edit R_G020.mdl so that the first 6 instances of P_G0T0 were also changed to R_G020

 

Test 11 Results

Show spoiler

(hidden content - requires Javascript to show)

 

These tell me that Hex editing the mdl references to the model itself doesn't cause the loss of bumpmap connection. I'm guessing that using the Renamer function of KotOR Tool would be safe, but haven't tested it.

[redrob41]

I did try some tests where I actually ran the mdl through MDLOps 6 and then loaded it into 3DS Max 2011 using the NWN plugin. After making changes to both the xyz & uv coordinates of two bones, I exported the model. The hard part was trying to run it through Taina's replacer. I kept getting either a Read error or a Write error (it seemed random), and it always referenced a different Hex address. I have no idea if it is an address of Taina's program or of the mdl itself.

 

 

Test 12 reverted everything back to Test 1 settings

 

Row Label |label |race |racetex|envmap |

453 |Party_NPC_G0T0 |P_g0t0 |**** |DEFAULT|

 

I eventually just ran the ascii.mdl back through MDLOps again to get the binary mdl & mdx again. While it did load in-game, It lost all connection to the bumpmap and edge smoothing.

 

Test 12 Results

Show spoiler

(hidden content - requires Javascript to show)

 

I'm still not sure which of the community built mod tools is causing it, but it seems pretty clear to me that we can replace bumpmap textures quite easily IF they already exist for a model, but anytime we try to modify the model itself, we are going to lose the bump and edge smoothing. I'm not sure how to test the programs, or decompile/translate the mdx to find whatever it is that enables the bumpmap to work. I don't think that it is simply a matter of tpc vs tga, since the txi files work just fine. Besides, Taina's won't work on a model that has a bumpmap (I've only tried it on the Twi'lek male head and G0T0), so that seems to me that some line in the original mdl isn't lining up with the modified one (Taina's just isn't built to compensate for "whatever" extra code allows for bumps). I'm guessing that it is a variable (0/1) but I'm not sure.

 

Hopefully all of my blunders into madness will provide some spark of insight to anyone who knows more about programing than I ever will. I know it's a longshot, but at least these tests are saved for future reference.

 

Happy Modding

[Fair Strides 2]

These tell me that Hex editing the mdl references to the model itself doesn't cause the loss of bumpmap connection. I'm guessing that using the Renamer function of KotOR Tool would be safe, but haven't tested it.

 

Hey, RedRob, I'd just like to say:

 

THANK YOU SO MUCH!!!:D:D

 

I've taken over development of KAurora and MDLOps, from permission and official announcement by MagnusII on KA and by default on MDLOps.

 

One of the first things I'll be looking at is either phasing out MDLOps or combining the two. You've just helped a great deal on it.

 

See, KAurora is preferable since it has the most up-to-date MDL format, but MDLOps has the replacer and actually supports animations(as far as I know, KAurora doesn't like those). By merging them, we'll have a good tool.

 

And one of the discoveries in the mdl format is that it actually has slots for four textures in the mdl format for each mesh. The first is the native texture, the second is assumed to be the lightmap, the third is assumed to be the envmap/bumpmap, and the fourth is assumed to be the cubemap...

 

If I can successfully cull the mdl format from MDLOps and KAurora into one cohesive table, and drag the code from AniCam into it as well, we'll have the best format we can expect. At that time, I can look into sorting out Aurora Lights, AniMeshes, and DanglyMeshes. But before I do that, I'll be making/modifying a Model tool that will allow you to add a bumpmap to a model, since currently not all of the game's models have one coded into them...

[redrob41]

I've taken over development of KAurora and MDLOps, from permission and official announcement by MagnusII on KA and by default on MDLOps.

 

One of the first things I'll be looking at is either phasing out MDLOps or combining the two. You've just helped a great deal on it.

Ah, good to know that someone is on the job Much good luck to you Sir!

 

And one of the discoveries in the mdl format is that it actually has slots for four textures in the mdl format for each mesh. The first is the native texture, the second is assumed to be the lightmap, the third is assumed to be the envmap/bumpmap, and the fourth is assumed to be the cubemap...

 

I thought about that, and I'm not sure how to get 3DS/gMax to export a model with more than one bitmap attached to it. So far, I've been able to load a bumpmap material into max, but it's only worked in the renders. I haven't had exporting success yet.

 

And now, off to bed. My Christmas holidays haven't started yet, and I've still got to go to work in the morning

[DarthParametric]

I've taken over development of KAurora and MDLOps

Well now, that's some interesting news. I'm keen to see what eventuates.

 

You may want to look at NWMax Plus, the extension/update of the original script. I've briefly tested a few KOTOR imports and it seems to work OK, but nothing beyond that. Looks like development ceased back in 2012, but there may be improvements that are worth auditing to see if this can become the new de-facto import/export script.

 

http://www.tbotr.net

 

I'm not sure how to get 3DS/gMax to export a model with more than one bitmap attached to it.

The problem isn't Max/GMax, it's NWMax presumably only having provision for a diffuse map. May require either modifying the script, or editing the ASCII file post-export.

[Darth Sapiens]

So i have done some thinking and chatting with nwn modders, cube-mapping and bumpmapping are definitely related, envmaps and specmaps (generated dynamically, see the deady stream post here) are applied to each smoothing group, that's why the reflection is seamless on the round part of g0t0, when a model loses the smoothing group data because of mdl converting etc. we also lose bump mapping, though i once thought the four textures in the model file were as FS related. for some reason i doubt it now, you can find models with adjoining lightmaps, you dont find the same kind of thing for bump maps. but nwn does have a provision for bump mapping in it's binary model format. and kotor has a few undeciphered structs in the mdx, magnusII theorized there might be a place to look, and my instinct says it's tied up with smoothing group information.

[sELFiNDUCEDcOMA]

Hey, RedRob, I'd just like to say:

 

THANK YOU SO MUCH!!!:D:D

 

I've taken over development of KAurora and MDLOps, from permission and official announcement by MagnusII on KA and by default on MDLOps.

 

One of the first things I'll be looking at is either phasing out MDLOps or combining the two. You've just helped a great deal on it.

 

See, KAurora is preferable since it has the most up-to-date MDL format, but MDLOps has the replacer and actually supports animations(as far as I know, KAurora doesn't like those). By merging them, we'll have a good tool.

 

And one of the discoveries in the mdl format is that it actually has slots for four textures in the mdl format for each mesh. The first is the native texture, the second is assumed to be the lightmap, the third is assumed to be the envmap/bumpmap, and the fourth is assumed to be the cubemap...

 

If I can successfully cull the mdl format from MDLOps and KAurora into one cohesive table, and drag the code from AniCam into it as well, we'll have the best format we can expect. At that time, I can look into sorting out Aurora Lights, AniMeshes, and DanglyMeshes. But before I do that, I'll be making/modifying a Model tool that will allow you to add a bumpmap to a model, since currently not all of the game's models have one coded into them...

 

I would use the KAurora source as a basis for a new tool incorporating new features that other tools may have, and leave MDLOps as is. It is written in Perl and there is only so much you can do with it, whereas KAurora, is written in C#.net (if I am not mistaken) and therefore you can do so much more with it -- like have an actual model viewer among other nice features.

 

You may want to consider replacing the DirectX(3D) with a managed solution like SharpDX -- it may require a bit more effort but should result in less pain dealing with DirectX.

 

Something else worth considering, is circumventing the need for NVmax altogether by adding in support to export models (or at least meshes) out via the tool in a common 3D model format -- .OBJ, .FBX or .DAE. What this means is that there is less onus on NVmax correctly exporting stuff, and less issues of what modeling program people have to use -- as essentially they should then be able to use any modeling program as long as it exports out into the correct file format for the tool.

 

There should be libraries that will handle this for you like Assimp that though the docs and site won't tell you, other sources -- source 1, source 2 -- indicate that it does now support the popular (for games) .FBX model format which should also allow future support of any animation tools you want to implement. Though, it should already handle the Collada (.DAE) format which I understand also can deal with animation data, however, it seems to be less well documented and used for this reason by devs.

 

Anyway, good to hear that someone is prepared to update the tools so that future modding becomes less of a hassle.

[magnusll]

Small comment here.

 

As far as I can remember, the 4 textures the Kotor MDL format handles are not in any way "fixed" to be "normal texture - lightmap - bumpmap - cubemap". They are not linked to their position. E.G. you could have the lightmap be the first texture instead of the second and it would work perfectly fine as long as the rest of the info in the .mdx and the .txi matches.

What I did find is that there was no model at all shipped in either Kotor I or II resource files having more than 2 textures applied to it.

 

And yes, there still are undecoded fields in the .mdl node structures which could indeed be used as indications for possible bumpmap application.

[InsanitySorrow]

This is a brilliant find