Phelon Posted January 4, 2004 Share Posted January 4, 2004 Hello there people! You can call me an idiot, as I'm sure I am. After a week now, I've been trying something out this night and was shocked... the idea worked, and was so simple I didn't even think of it! Take a look at this pic... I bet Bastila looks a bit weird now. This thing has helped me a step torward understanding the format BW has been using for their compression. But the texture used on Bastila is nothing else than a standard compressed 24 Bit TGA. Necessary steps for something like doing this : 1. Find out what file in the TexturePacks you wanna change. (in my case "P_BastilaBA01") 2. Open the file with a Hex-Editor and look at bytes 8 - 12. There you'll find the texture X & Y sizes, just calculate them to decimal system and you can fire up the graphic program of your choice. The right sizes are very important, otherwise the TGA won't show up! 3. Make your Texture and save it as 24 bit TGA. Name it exactly as the texture you want to exchange, with the ".tga"-ending. 4. Put that file into your OVERRIDE-folder 5. Fire up KotOR and look at what you've done. Current downside is that I'm still unable to extract the needed image data for any true skinning, so you'll either have to play around a lot or wait until I figure that out as well. But I'm at it people. This discovery makes some things somewhat easier and more transparent. Just wanted to let you know. PS. : I told ya, Lil' Jawa! Link to comment Share on other sites More sharing options...
Devanstator Posted January 4, 2004 Share Posted January 4, 2004 Phelon, I have been working on extracting textures directly from the erf files with limited sucess. Now, when you say to look at bytes 8-12, you mean that P_BastilaBA01 has already been extracted, but is not in a tga file, right? I haven't been keeping up with you skinners on things so I don't know where that file came from. Did you extract it from the erf files, or the bif file? Sorry if I'm uninformed, but I'm a programmer and have been spending most of my time modding the save files. Link to comment Share on other sites More sharing options...
eidospsogos Posted January 4, 2004 Share Posted January 4, 2004 oh yeah. very good news. as soon as we get this out of the realm of programming, and more into the realm of art, i'll be a little happier though. i am however very happy that someone who would not normally be doing any skinning is taking the time to figure it out for those of us who are, for lack of a better word, programming impaired. Link to comment Share on other sites More sharing options...
Lil' Jawa Posted January 4, 2004 Share Posted January 4, 2004 I knew you could do it! Though to say the truth, I think Bastilas new outfit looks kinda.... creepy Link to comment Share on other sites More sharing options...
Phelon Posted January 4, 2004 Author Share Posted January 4, 2004 Good Morning everybody! (saying this around 18:00 makes me feel a bit weird though...) Devanstator : Textures always come from the ERF-Files, they are in the TexturePack directories. For extraction purposes you can use the ERF-Tool for NWN, reinserting does not work right with it though. The original Textures are not, I repeat, are not TGAs! Do not try to insert a TGA instead of another original texture into an ERF, it will not work! (in case of the "swpc_tex_gui.erf" KotOR will crash on you!) I'm a (bad ?) programmer as well, I don't really have hand for art. eidospsogos : Maybe I'll throw out a quick utility for getting the sizes from the ERFs directly, so you can work with it a little easier. That wouldn't be a bad idea. Okay, I'll see what I can do. Lil' Jawa : I bet you wanna see the texture I used on her. Might be good for a grin. Link to comment Share on other sites More sharing options...
Devanstator Posted January 5, 2004 Share Posted January 5, 2004 Good greif, and all this time I was trying to make an erf extractor. I knew there were some tools you could use from NWN, but I thought they were for something else. How do I get ahold of this NWN erf extractor? Link to comment Share on other sites More sharing options...
Devanstator Posted January 5, 2004 Share Posted January 5, 2004 Ok, I got it. These files that have been extracted are directly from the erf files. The greatest sucess I have had before this was to do this same thing. I was able to guestimate where a file started and ended and I took it from the erf file. You're right, these aren't tga's. They have a 128-byte header and don't match any filetype I can find specification for. I almost thought they were .pcx but that doesn't work either. There must be a way to change these files from their present state to a tga that can be worked with. Then the skinners will be unleashed. ... must figure it out. Link to comment Share on other sites More sharing options...
Phelon Posted January 5, 2004 Author Share Posted January 5, 2004 Devanstator : Actually the ERF-File is so plain simple, I've never used the ERF-Tool from NWN. I simply wrote myself a utility to extract and rebuild it. About the textures : I could give you a description of what I found out about the texture format, and you could give me a hand at figuring it out. How about teamin' up on that, Devanstator? Link to comment Share on other sites More sharing options...
Devanstator Posted January 5, 2004 Share Posted January 5, 2004 That's great, Phelon, here's what I've learned. Once I had a clear look at these files, it took very little time to derive the following: ============================================== -Files extracted from an erf have a 128-byte header. -At offset 8 is an unsigned short for width -At offset 10 is an unsigned short for height -Immediately following the header is the image data -Byte order is left to right, bottom to top -32 bits for a pixel (Red,Green,Blue,Intensity) -Image data ends at offset = (x*y*4)+128 -The file footer is some number of strings like "mipmap 0", which must be discarded ============================================== -The Targa files have an 18-byte header. -At offset 12 is a short for x-dimension -At offset 14 is the short for y-dimension -Immediately following the header is the image data -Byte order is left to right, bottom to top -32 bits for a pixel (Blue,Green,Red,Intensity) -Image data ends at offset = (x*y*4)+18 -The file footer is unimportant info followed by "TRUEVISION-XFILE" =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- HOW TO CONVERT FROM ERF-EXTRACTED FILES TO TARGA -Create new targa file and place first bytes as "000002000000000000000000" -Get 4 bytes from other file at offsets 12-16 and put in tga file -Place "2008" to complete the header -Read each set of 4 bytes from file and switch byte 1 with byte 3 then place it in the tga -Keep doing this until your read point is at offset = (x*y*4)+128 -Add "000000000000000054525545564953494F4E2D5846494C452E00" for the footer -Save file and you are done. ------------------------------------------------------------------------------------ I have just started writing a program to do this and since I still have all my low-level c functions handy I can borrow them from Kotor Mod Utility. Within 48 (I hope) hours skinning in Kotor will be no harder than any other game. Link to comment Share on other sites More sharing options...
eidospsogos Posted January 5, 2004 Share Posted January 5, 2004 wow! you have no idea how happy it makes me to hear you say that. and maybe that is a bit sad. Link to comment Share on other sites More sharing options...
Phelon Posted January 5, 2004 Author Share Posted January 5, 2004 Something about your files extracted from the ERFs, does not totally match up with my finding. But I've been studying the GUI-ERF more closly, they are somewhat diffrent. As I found this would be the easiest, I have started of with "800x600back", the smallest Background GFX I could find. Here are my observations : ---------------------------------------------------------------------- The first 128 bytes of an image always consist of the image header, of which only the first 16 bytes are actually used. --------------------------------------------------------------------- Part 1 : Possible Header (Hexadecimal Listing) --------------------------------------------------------------------- 00 00 04 00 | 00 00 80 3F | 00 02 00 02 | 04 01 00 00 --------------------------------------------------------------------- 1. (4 bytes) These bytes seem either to have something to say about the exact filesize, or are used as an image identifier, I don't know for sure. But they often (not always) match up with the exact data size used by the Image. 2. (4 bytes) Unknown, haven't found out anyhting about these 4 bytes, except that the example values are used in most GUI-Images and also in a few TP-Images. 3. (4 bytes) Looks like X & Y Dimensions. 4. (4 bytes) I don't really know, but it is important. IMPORTANT NOTE : An incorrect header within a GUI-Image results in a crash to the desktop, and might have a complete hangup following up. ---------------------------------------------------------------------- Part 2 : Possible Image Data (Hexadecimal Listing) ---------------------------------------------------------------------- 00 FF FF FF | FF FF FF FF | FF FF 00 00 | FF FF FF FF ---------------------------------------------------------------------- 1. (1 byte) This is a Color Saturation or Alpha Value for the Background Color ranging from 0 to 255. 2. (7 bytes) These 7 bytes are a mystery I haven't figured out yet. But some of them have an influence on Colors as well. I can't say for sure though. 3. (2 bytes) Those 2 Bytes are the Value of the Background Color, encoded in 565-format, the latter is the HIGH BYTE. 4. (2 bytes) Same as 4, only Value of Foreground Color. 5. (4 bytes) This model might vary, but that's some info that might be read from the header. With this header it looks as follows : ------------ |xxxx| 4. Byte |xxxx| 3. Byte |xxxx| 2. Byte |xxxx| 1. Byte ------------ The x's are pixels set in those bytes. Every pixel consists of 2 bits, that seem to change the color adjustment. bitwise 00 = Don't draw Foreground Color bitwise 01 = Draw FGC Method 1 bitwise 10 = Draw FGC Method 2 bitwise 11 = Draw FGC Method 3 I'm not sure how exactly these methods final values are calculated. NOTE : The entire image is built up in rectangle-shaped parts, starting from the bottom left to the top right. Reminds me of the good old BMP-format... --------------------------------------------------------------------- Part 3 : Possible Optional Data --------------------------------------------------------------------- This varies in size, most of time you find a "mipmap 0 " followed by Hex Values 0D & 0A, but also some even weird constellations. Some of the other GUIs GFXs are simply RGBA encoded, nothing to worry about, as this is more than easy. "Redfill" or "Blackfill" for example. Link to comment Share on other sites More sharing options...
Devanstator Posted January 5, 2004 Share Posted January 5, 2004 Good point, I guess I should have tried my method on other files too. I was playing around with blackfill, redfill and greenfill because they are only 155 bytes. I think the tricky part is going to be that there must be several different file types represented here. I think some of them were bitmaps before being archived, but there's not much to tell which is which. The program was coming along great, but I think I got carried away before figuring out how to convert all of the files. hmm... Perhaps I should continue and see what it does to the files that don't work. That would give a clue as to what to do with those ones. Link to comment Share on other sites More sharing options...
eidospsogos Posted January 5, 2004 Share Posted January 5, 2004 is there a better process than that of elimination through trial and error based upon educated quessing? there may well be, but i wouldn't know. Link to comment Share on other sites More sharing options...
Phelon Posted January 5, 2004 Author Share Posted January 5, 2004 eidospsogos Yah, beating the info out of someone who created the format. Sorry to eidospsogos, but even the possibilities of the best programmers are limited to this, if he has a type of file that is not similar to any other graphic format available. He would as much have to figure out by trial an error as we do. Devanstator : BTW Devanstator, about your guess about how the file is encoded. The info must be in the header, speaking logically, but can only be at positions 5-8 or 13-16. These got nothing to do with the filesize, so it's the only logical explanation. You got a chat or something where we can meet up, and discuss all the details without having create bandwith overflow on this board? Link to comment Share on other sites More sharing options...
Devanstator Posted January 5, 2004 Share Posted January 5, 2004 I have msn messenger, but not ICQ or AIM. I am devanengland@hotmail.com (I like to keep it simple). I am researching RLE compression now. Link to comment Share on other sites More sharing options...
eidospsogos Posted January 5, 2004 Share Posted January 5, 2004 uhhh....it was just a joke. i wasn't making a crack at anyone. trust me i'm much more in the dark than either of you. i managed to get in there, but as the only skinning i have ever done has been for quake type games, i was a bit confused by what i found. so both of you are better off than i am. i was attempting levity. Link to comment Share on other sites More sharing options...
Phelon Posted January 5, 2004 Author Share Posted January 5, 2004 eidospsogos: I knew, but I was bored... and only the first comment would've looked so alone. Link to comment Share on other sites More sharing options...
eidospsogos Posted January 5, 2004 Share Posted January 5, 2004 well trust me for my sake i wish you guys the best of luck. skinning is really the only thing i've ever done as far as game modding is concerned. it took a little bit of figuring things out to skin for single player in JA, what with the exta model components and all. and heck i was proud of myself for figuring out how to use those multi-model .skin files and the alpha transparency levels in their .pngs(which actually turned out to be more of a problem with photoshop handling the .png format incorrectly....took forever to find a plugin that would work) but this is way out of my league. so good luck on your progress. i am with you in spirit. Link to comment Share on other sites More sharing options...
Phelon Posted January 5, 2004 Author Share Posted January 5, 2004 Devanstator: That damn MSN doesn't really work at my end... oh well. But that's not why I'm here. I finally found out where we can determine the encoding, and an estimate of how the encoding variants look like. --------------------------------------------------------------------- Possible Header (Hexadecimal Listing) --------------------------------------------------------------------- 00 00 04 00 | 00 00 80 3F | 00 02 00 02 | 04 01 00 00 --------------------------------------------------------------------- 1. (4 Bytes) These bytes are the exact Image-Size in bytes (scrap the rest of the file). If zeroed out, it might be that the encoding variant is RGBA. I have found this true for Redfill, etc. 2. (4 Bytes) Still Unknown, but defenitley not so important 3. (4 Bytes) X & Y Sizes, as we both know 4. (1 Byte ) 16-Bit Encoding Method I have found 2 types of encoding methods by now. All create a 4x4 rectangle that is blitted on screen. Variant 1 [04] = 16 Bytes - That's the variant I explained in my post above Variant 2 [02] = 8 Bytes Built up like this : 2 Bytes .... 16-Bit Background Color (565-Encoding) 2 Bytes .... 16-Bit Foreground Color (565-Encoding) 4 Bytes .... Pixel settings in the 4x4 rectangle using exactly the same method as mentioned in Variant 1. 5. (1 Byte) Unknown, a resource number perhaps 6. (2 Bytes) Always 00 00, therefore unknown -------------------------------------------------------------------------------- Summary : Once we can figure out how the color set is calculated by the Drawing Methods I outlined, we KNOW how to deal with all of KotORs textures. Link to comment Share on other sites More sharing options...
Prime Posted January 6, 2004 Share Posted January 6, 2004 Originally posted by The Devanstator Then the skinners will be unleashed. Oh yes...the skinners will be unleashed <evil grin> Excellent work guys. As a software designer I know how things like this can be a pain in the ass Link to comment Share on other sites More sharing options...
Devanstator Posted January 6, 2004 Share Posted January 6, 2004 Perfect, and I have a program shell ready for when we do. I've been trying to get a screen capture that is identical to a texture (which might be pretty hard) but by comparing them, I'd be able to figure out which colour a pixel is supposed to be and maybe how they encoded it. A neat little thing that I found was that textures are "hot swappable" or they can be changed or switched around while kotor is still running. Also if you have a texture that is in the override folder and you remove it while it's running, then kotor will not load the original to replace it. It will just be white. Also, a while back I tried modifying the erf file directly and it didn't seem to make any difference. Have you tried that and did it work for you? If we can start changing some values to see what difference they make to the textures, then we'd have a better idea what each byte does. Link to comment Share on other sites More sharing options...
Devanstator Posted January 6, 2004 Share Posted January 6, 2004 Alright, here's another hurdle. I took a screenshot of my feats menu ingame and brought it directly into photoshop. Then I took i_empathy03 (the feat graphic for master empathy) in the gui erf and replaced it with one that was solid red. The texture's header said it was 32X32 pixels so I made mine that size. It worked fine and when I looked at my feats there was a small red box where the picture used to be so I screen captured that too. Then I selected the box texture which was clearly defined and then dragged it into the first screen so I would have a selection of the same size and shape. I positioned it over the empathy graphic and copied that into a new texture. The only problem was that this selected area was only 27x27 pixels. I played around with it more, making sure there were no resizing errors and found that i_empathy03 actually is 27 pixels but was then blown up. The texture had to be resized to 32 (the nearest higher power of 2?) for Kotor to display it properly. This means that probably many of the textures, once we sucessfully convert them, will be blown up by an untold number of pixels. I tried to find header data that could represent a 27 or 5, but I don't think it's stored in the file. Just another thing to keep in mind. Link to comment Share on other sites More sharing options...
Phelon Posted January 6, 2004 Author Share Posted January 6, 2004 Devanstator: The hurdle is KotOR itself, as almost no graphic is displayed in the designed size. Images are all a multiple of 2 in size, and stretched or shrinked acccordingly. Images must in all cases be a multiple of 2 in size. I've examined the i_empathy3 Image you named, and I find it sustains my guesses up to now. Size of it is 4235 Bytes. In the Header : Data Size = 0, encoding therefore is RGBA. Image size is 32x32 according to the header. Calculation: Header + RGBA +Optional = Final Size 128 + (32 * 32 * 4) + 11 = 4235 Summary : The Image is 32x32, regardless of what it is displayed in the engine. "800x600back" is also 512x512 in size, and then stretched to 800x600. Go make a 32x32 Tga, name it "800x600back.tga" and put it in the override folder, fire up KotOR and you'll see I speak true. It alway must be a multiple of 2, how it is stretched is somewhere else to find, but not in the Textures. ----------------------------------------------------------------------------- Color Calculation : I did change the colors directly in the files, and reinserted them in the ERF so I could find out what they are doing. How did you think I came up with all the info I extracted? BTW Color Calculation Mehtods seem to be something like this: 00 = BGC 01 = ( BGC / 4 ) + ((FGC / 4) * 3) 11 = ( BGC / 2 ) + (FGC / 2) 10 = (( BGC / 4 ) * 3) + (FGC / 4) Assumption taken on fully maximum color values, make up something like (255, 255, 255 | 62, 62, 62 | 126, 126, 126 | 190, 190, 190). I haven't tried this out with other colors as of yet, but it's a guess. Link to comment Share on other sites More sharing options...
Phelon Posted January 6, 2004 Author Share Posted January 6, 2004 Update - 8 Byte Method verified I've written a quick tool to the job for me... unuseable by anyone else, but it has helped me figure out that my methods for the exact color calculation are wrong. I still do not know how the color is calculated right. But I've made a cross-examination. The colors do not match at any rate, however the shapes of both images (original and converted) match up with each other, so the decoding process seems to be correct. It seem I was right, when I said all we had left was to find out the color calculation process. Oh, I love it when I'm right. Link to comment Share on other sites More sharing options...
Devanstator Posted January 7, 2004 Share Posted January 7, 2004 Excellent. So this app you made, it was able to convert into a readable targa, but the colours were wrong? Even so, the colour might be hard to absolutely verify. I'm afraid that most of what we're doing is new territory for me, but I'm learning as I go and the whole technology is fascinating, but sensical. Also, all that I meant above was that final converted textures will appear pixelated because of their size. Skinners may have to scale a texture down before working on it and then blow it up again before placing it in the Override folder. Link to comment Share on other sites More sharing options...
Recommended Posts
Archived
This topic is now archived and is closed to further replies.