Module Questions

[Miltiades]

As most of you guys already know, we can only use the modules we have in the games to create new modules. We can't create modules from scratch. My question is: Why? Why can't we create new modules from scratch? Maybe this asks for a difficult to understand answer, but don't hold back on that.

 

Furthermore, have people tried to make modules from scratch, and if people have, what problems did they encounter and how far did they progress?

 

And what are the possibilities we have to edit modules so they become unique?

[Elven6]

Because we don't have access to the proper technology or the engine.

[stoffe]

Cannot is perhaps too strong a word, more like very difficult. The main problems as I understand it is that the modding tools currently available don't fully support modifying all the forms of data required to form a new area from scratch. Such as walkmeshes to determine what parts of the area can be walked on, lightmaps to give the area proper lighting and other model related issues such as animations and emitters. Not all file formats are fully understood, the MDL/MDX and WOK formats among them.

 

When re-using existing areas you can use those types of data already present in the game. If you create new areas from scratch the player wouldn't be able to move around if you can't create a walkmesh, and the place would be very dark without lightmaps etc.

 

Now, this is just as far as I understand it, which shouldn't be taken as the truth. Modelling is not one of my strong points.

 

Regardless, if you are interested in making new areas from scratch I wouldn't let the above deter you. If nobody ever is looking into this it will never be figured out. Sometimes you have to go off the beaten path to make new discoveries.

[Miltiades]

Thanks for the info. Yes, I'm interested, because it would open many opportunities for modders and we would see more of the creative minds of the modders. I'm not saying that I'll be hurting my brains day and night at how to create walkmeshes, lightmaps etc., but I may be looking into it later on. And maybe other people can try solving the problems you pointed out.

 

Anyway, thanks.

 

Edit: Can't we look at Neverwinter Nights for these kinds of problems? I haven't played NWN, but I know Kotor is based on it.

[Darth Maleval]

how does fred tetra make the maps? what programs does he use?

[Pavlos]

Well to be honest... I'm wondering what the advent of NWN2's heightmapped terrain will teach us about KotOR modules. Probably very little (KotOR runs on a "room" system rather than rolling terrain doesn't it?) but we may be able to adapt some of the tools being engineered by the NWN2 community to give new life to KotOR modding.

[stoffe]

Edit: Can't we look at Neverwinter Nights for these kinds of problems? I haven't played NWN, but I know Kotor is based on it.

 

The way areas are created is very different between Neverwinter Nights and the KotOR games, so I doubt much can be taken from there, unfortunately. The 3D model format is different. NWN uses Tile-based areas (a bit like with pre-made 10 by 10 meter LEGO blocks that can be attached to eachother in a variety of ways in a grid pattern, where every tile has its own walkmesh and pathfinding data) with "dynamic" lighting, while KotOR uses 3D models that are joined together to form an area (where the whole area has separate pathfinding data), where the lighting is static, AFAIK. There are probably other differences as well.

 

NWN was made to be modded, while the KotOR games were not.

 

Well to be honest... I'm wondering what the advent of NWN2's heightmapped terrain will teach us about KotOR modules. Probably very little

 

If what Obsidian has said before about NWN2 is true they have started out with the NWN code base and then ripped out the part dealing with graphics and rewritten their own from scratch. If that's the case the model format will probably have very little in common with the KotOR games, which uses a graphics system written by Bioware.

[Miltiades]

Why the difference? Why change the way maps are being made? Is there an advantage with the way Kotor maps are made?

 

Another remark. The way NWN and Kotor maps are being made is different, but they use the same files, like MDL/MDX files, WOK files and others. Can't we learn anything from this?

 

And if we can't, the only we we can is to encode them somehow, isn't it? There are some programs to do that, but they are not sold for public. This means we can only make our own encoders. Am I right, or totally wrong? And if I'm right, is it possible to make such decoders?

[stoffe]

Why the difference? Why change the way maps are being made? Is there an advantage with the way Kotor maps are made?

 

It's a completely different game, with a new graphics rendering engine. Data compatibility with one of their older games was probably the least of their priorities.

 

NWNs tileset technique was used to make creating areas very easy without the need for 3d modelling tools, but it was also by nature rather inflexible and resulted in fairly generic areas without much unique character to them.

 

Since KotOR was not made to be modded that inflexibility was probably more trouble than it was worth, and it was likely easier to allow the modellers to just build the area models in 3dsmax directly just as they wanted them, rather than have to chop things up and build generic tilesets. To my knowledge making NWN tilesets were a tremendous grunt job with lots of work involved. This is probably why brand new tilesets are among the rarest releases by NWN modding community, with only a handful having been made.

 

Another remark. The way NWN and Kotor maps are being made is different, but they use the same files, like MDL/MDX files, WOK files and others. Can't we learn anything from this?

 

The file extensions are the same, but the data formats inside are not the same.

 

For example, NWN only uses MDL files, while KotOR uses MDL files combined with MDX files. And even then the format of the MDL files of NWN and KotOR are not the same. KotOR does not have the same graphics rendering system as NWN does. the MDL/MDX format changed slightly even between KotOR1 and KotOR2, if you take a TSL model and put it in KotOR1 it won't work in most cases without modifications.

 

 

And if we can't, the only we we can is to encode them somehow, isn't it? There are some programs to do that, but they are not sold for public. This means we can only make our own encoders. Am I right, or totally wrong? And if I'm right, is it possible to make such decoders?

 

The problem is that these data formats are not documented publically by Bioware or Obsidian, to my knowledge. Making any tools or converters require you to know the format of the data files to work with. The MDL/MDX formats are not fully understood, this is why you can't create a new model from scratch with MdlOps without using an existing model as blueprint. Since it doesn't know what all the data is it will need to re-use the mystery data from somewhere else.

 

The modding tool makers have done an amazing job to figure out as much data formats as they have already, but that knowledge does not cover all the data formats used by the game to 100%. Only so much can be done in people's spare time without any official support or aid from those who know how those formats are made.

 

This is another core difference between NWN and KotOR modding. Bioware actively encourages NWN modding, they released a toolset and complete technical specifications of most of the data files used by the game, and have been helping toolmakers and modding teams. KotOR modders get no aid from Bioware or Obsidian at all.

[Miltiades]

I see. But can't we look into an existing model, for example, to see and comprehend all data?

 

You also mentioned "only so much can be done in people's spare time". Does this mean that it's possible, that it just requires a lot of time? The fact that not many understand all this would explain why there isn't much progress.

 

And about using existing models and such. It struck me that a lot of things that involve modding Kotor and Kotor2 need existing things to work. This means that not only module editing is limited. One of the exceptions, if I'm not mistaken, is the dialog file. With the dialog editors, we can make dialogs from scratch. Am I right? If I'm right, does this mean the Dialog format is fully known then?

[stoffe]

I see. But can't we look into an existing model, for example, to see and comprehend all data?

 

Sure, you can open a MDL and MDX file with a hex editor and have a look at the data. Good luck comprehending it though, you'll likely need it. The model format is probably one of the more complex data structures used by the game. Significant parts of the MDL/MDX format has been figured out by cchargin and others and incorporated into MdlOps, but there is still much unknown (animations, emitters, smoothing, various data etc). If you're willing to contribute to uncovering more of that format I'm sure your efforts would be appreciated by many.

 

You also mentioned "only so much can be done in people's spare time". Does this mean that it's possible, that it just requires a lot of time?

 

Lots of time, patience, solid knowledge of mathematics and the workings of 3d graphics, I'd wager. Fairly complex stuff, I'm very impressed by the people (cchargin, Fred Tetra, JdNoa comes to mind) who have been able to figure out what's known so far. I certainly couldn't have done it.

 

And about using existing models and such. It struck me that a lot of things that involve modding Kotor and Kotor2 need existing things to work. This means that not only module editing is limited. One of the exceptions, if I'm not mistaken, is the dialog file.

 

Are there? I can't think of any formats other than the models that require you to work from already existing data rather than be able to create something from scratch.

 

GFF format files (files with extensions UT*, DLG, ARE, GIT, IFO, PTH, RES, FAC, JRL, BIC, BTC, ITP, GUI...) are more or less all known, since the GFF format has undergone relatively small changes from NWN, and those few changes have been found. Once the base GFF format is discovered you can check the fields in an existing file of that type to see what their labels, datatypes and general structure should be, and be able to figure out reasonably well what the data is from their labels. Then it's a relatively simple matter to create new such files from scratch.

 

The same goes for other formats that are more or less the same as in NWN and thus have bioware documentation available, such as TLK (string tables), NCS (scripts), KEY/BIF and ERF (with extensions ERF, MOD, SAV) files. And other formats that are not the same, but still fairly similar to old formats, such as 2DA, SSF and RIM files. The few plaintext data formats (such as LYT files) used are fairly easy to read, but still requires interpretation to figure out what the data means.

 

The DXT compressed TPC texture format was probably tougher to discover, since it was not used in earlier games, but that happened before I started modding so I don't know much of that.

 

NCS files (scripts) and MDL/MDX/WOK (3D models) are probably the most complex file formats used by the game if I may guess, and thus the hardest to figure out. Luckily the script format is the same as in NWN, so that didn't require anything special.

 

Hopefully someone more knowledgeable can elaborate on where the difficulties with decoding the MDL/MDX format lie, since this kind of thing is way out of my league.

[Miltiades]

Sure, you can open a MDL and MDX file with a hex editor and have a look at the data. Good luck comprehending it though, you'll likely need it. The model format is probably one of the more complex data structures used by the game. Significant parts of the MDL/MDX format has been figured out by cchargin and others and incorporated into MdlOps, but there is still much unknown (animations, emitters, smoothing, various data etc). If you're willing to contribute to uncovering more of that format I'm sure your efforts would be appreciated by many.

 

So all data that is in such a file you can actually see with a hex editor. I'm not taking this too lightly, mind you. That would dishonor the people who've done the incredible work of decoding what's known today.

 

But does this also includes walkmeshes and lightmaps. To figure these out, do we need to use the same method as the MDL/MDX? Because MDL/MDX files don't really need to be decoded and fully known to create a module from scratch.

 

And as I see it, a large part of the files that are needed to have a module are known. Am I right, or are there still a lot of files unknown?

[stoffe]

But does this also includes walkmeshes and lightmaps. To figure these out, do we need to use the same method as the MDL/MDX? Because MDL/MDX files don't really need to be decoded and fully known to create a module from scratch.

 

As far as I understand it, walkmeshes are roughly speaking an invisible 3D "model" that is overlaid on top of the normal area model. It acts like a pavement to determine where characters can walk in the area, and what parts are impassable. This works in all three dimensions, since characters are always stuck on the walkmesh on the Z axis (height). Instances in the game where you see a character's feet disappearing through the floor are caused by the walkmesh having been placed too low in that area (like the Ebon Hawk engine room and parts of the main hold), just like it has been placed too high in areas where characters seemingly hover a bit above the ground (like the landing pad on Dantooine).

 

As for what parts are required/missing to create brand new areas, why not experiment and see what you find out? Experimentation leads to progress in modding. Start out with a simple box as a single room area, and see if you can get it to work in the game.

 

As far as I know (which isn't very far) you'll need to create these files at the very least to create an empty but working area and module:

• mymodule.mod
  • myarea.are - (GFF) Static area info file. Specified things such as area event scripts, weather, ambient light levels, stealth xp, ability to change party members or fast travel, room-specific settings. Name this the same as the LYT file (see below).
    
  • myarea.git - (GFF) Dynamic area info, positions objects inside the area (creatures, triggers, placeables, items, sounds etc...). Name this the same as the LYT file.
    
  • myarea.pth - (GFF) Pathfinding definitions, tells AI controlled characters where to walk in order to be able to move between two points in the area. Name this the same as the LYT file.
    
  • module.ifo - (GFF) Module definition, sets the default starting location/area when the module is loaded, module-specific event scripts and XP awarding scale when killing things. This file is always named exactly "module.ifo".
    

  [*]override folder

  • myarea.lyt - (Text) Determines the locations/coordinates of the different "rooms" (3d model pieces) that make up an area, as well as "door frame" locations where doors could be placed. This tells what 3D models are used to form the area, as I understand it.
    
  • myarea*.mdl - (MDL) 3d model to use for a "room" in the area. You'll have one model for each "room", where the * is an enumeration for the different rooms. Doesn't really matter how you name your room models though (as long as they're valid ResRefs) since you specify the names in the LYT file. Obsidian/Bioware just use enumerations to make it easier to keep track of which model pieces belong to the same area, I'd guess.
    
  • myarea*.mdx - (MDX) Extra model info, see the MDL above. Name it the same as the MDL file.
    
  • myarea*.wok - (BWM) Walkmesh for a "room", determines where in the area creatures and characters can move. Name it the same as the MDL file.
    
  • myarea*_lm*.tpc - (TPC) Image files used to specify the lightmaps for (part of) a "room". Not sure how these work.
    
  • myarea.vis - (Text) Not sure what this file does, but should be named the same as the LYT file.
    
  • whatever.tga/txi/tpc - (TGA/TXI/TPC) Whatever textures are needed to make your area look like something other than a collection of flat 3d surfaces.
    

[Miltiades]

So all these files need to be fully understand to be able to make a new module? So mdl/mdx files are also needed for modules, not only for weapons, animations etc.?

 

And of the files you named, how much are not fully understand?

 

Edit: Something else about the MDL files. You said that, and not only for these files, there was a difference between NWN and Kotor. Does this mean they used different programs in the development of these two games?

[Jacob fett]

Walkmeshes wouldnt be too hard if Bioware and obsidion would help us out but they wont help us out and im Just noting that if someone studied Walkmesh and lightmap code then someone could perhaps make a tool to help people with this problem witch could lead to new modules and perhaps new possibilitys with the game engine

 

 

 

 

EDIT: i beleive ive found a way to edit the coloring of a module but i have yet to test it but it came to me a month ago and i have a idea:

Get A .mod file and rename it to a suitable Paint, or any other color program and Color it to your pleasure save it and then rename it as a .mod file again and use The Kotor Tool to see if its possible to use the module it might not and then again i might jus have found out how to edit Modules in a certain way

[Darth Maleval]

thanks stoffe -mkb-. how would we make a .map file out of that?

[goldberry]

You mean .mod, right?

[Agent Xim]

You mean .mod, right?

 

I think .map might be for the KT Module Editor... I'd love to be able to create those!

[goldberry]

Oh those... silly me. Yes it would be most beneficial if we could, but i'm sure FT will return someday and pass on his teachings. Until then, the wanton slaughter of gff edited modules will continue...

[Darth Maleval]

yes thats what i mean agent xim

[Darth InSidious]

@stoffe: 'Difficult' and 'impossible'

 

It is impossible in that: Without walkmesh, you would be stuck on the spot. Current walkmesh editors just give you long strings of numbers which current knowledge cannot interpret. I've tried editing walkmesh and I got nowhere very fast. I ended up walking under the floor of the Valley of the Dark Lords.

 

Secondly, we cannot create lightmaps, and as I understand it, we have very little clue as to how they are put together. They can be edited, in that you can export them in .tga format, and recolour etc them using a picture editor. However, when making this edited set unique, you MUST also edit the accompanying .vis file. I have tried more than simply re-colouring, but it was painstaking and didn't get me very far.

 

Third, we cannot create new .lyt files. Entirely new ones, I mean. I draw this judgment from a simple test. Export from KT one .lyt file. Choose any one you like. Now remove a room, and change the "roomcount" at the top. Now put this lyt into your Override, start the game, load the module, and try to move outside that room. In most instances, you won't be able to, or at least you will have a problem with moving between them. Some rooms will be blocked off. The precise nature of this phenomenon changes between modules. In some it is more severe than in others.

 

Fourth, it would be more trouble than it would be worth to do. First of all, there is not a single model, but an area is broken into several little models. This means that you would have to save and export every room as separate from the rest. This would require building the overall area in Max (a task in itself), then cutting it up, skinning it, aligning it, attaching it and exporting it, and doing this maybe ten to twenty times.

 

Then, you would need to fabricate lightmaps, walkmeshes, and a .lyt, and also a .GIT, .ARE and .IFO, which is time-consuming and difficult.

 

It's enough of a pain doing some of this stuff for a unique re-skin of a module as it is. If you were to add in the modelling phase too...

 

In short, it is impossible, impractical and more effort than gain.

 

As for reskinning/naming modules, aside from the unique module info, by which I mean the .ifo, .are and .git files, you also need to extract the models from the models.bif, of the module you intend to edit, say, 711kor (all of its models are named 711kora etc...), extract the textures you intend to use, HEX edit the models to use the texture names, and/or the names of your new lightmaps, rename the models and add them to the Override folder.

 

Then you need to rename the .ARE and .GIT files (say, changing 711KOR to 712KOR, if this is the model name change you chose...), change all the relevant data within (the tag, the room names in the .are or .ifo, I can't remember which ), do all the editing you intend, and repack as a .mod file.

 

re: KT maps, I'm afraid only Fred holds the knowledge of those. As I understand it, in part it involves extracting the models and textures of the desired area, exporting all the models into 3dsMax, changing the view to right overhead, removing the roofs, moving the different room models into the right order, and then saving the file and doing some jiggery-pokery to make it into the .map file, which takes a very long time with K1 maps for an unspecifed reason (it's to do with compiling, though.). These steps may in reality be in a different order.

 

Hope this helps

[Agent Xim]

re: KT maps, I'm afraid only Fred holds the knowledge of those. As I understand it, in part it involves extracting the models and textures of the desired area, exporting all the models into 3dsMax, changing the view to right overhead, removing the roofs, moving the different room models into the right order, and then saving the file and doing some jiggery-pokery to make it into the .map file, which takes a very long time with K1 maps for an unspecifed reason (it's to do with compiling, though.). These steps may in reality be in a different order.

 

Hope this helps

 

That does help quite a bit actually I haven't done any 3D modeling since the Amiga 1000 (in high school, ahem), but I'm learning (again ). I've read FT's tutorial on how to prepare modules in advance for KT... but wasn't quite sure where to go next. Hmmm... so mdlops, 3dsMax, and maybe some hex editing?

 

After all, the more maps KT can use in it's module editor, perhaps the more modders will hop into module making? Although, I wonder if that's actually a good thing...

 

Anyway, thanks Darth InSidious!

[Darth InSidious]

No probs

 

I have a feeling, though, that FT may use some specialist tools to create the maps, and of course you'd need to edit a MapInfo.bfd .

 

Also, while I can't speak for FT, I'm not sure how happy he'd be about people editing/cracking his formats...

[Master Zionosis]

I see this post is a little old. Anyway, i don't really know how to post what i am about to say without typing pages, so ill try to cut it short, i see that it would be quite hard to create and edit the files like light maps and .lyt and walk mesh files, but if that can be done what about the actual module, now i have allot of area models created in 3dsmax one of which is padme's veranda on corucant, along with other's so if the model for the place is created isn't that a start?

[glovemaster]

Actually, i belive Team Gizka is creating the Droid Planet for KotOR II, and as in every post i mention this in, good luck to them. But they have made a trailer and their attempts seem successful, so maybe you should ask Team Gizka how they do it?

 

http://www.team-gizka.org/

 

btw, if anyone does find out, please tell us all... lol

 

I have contacted FT asking about making MAP files, i have no reply *yet*, lol