Info on *.LVL files

[Riley75]

Hello everyone in the editing forum.

 

First off, I should say that, for the time being, I am going under the assumption that Pandemic / LucasArts will *not* be releasing any editing tools. Certainly not in the near future. Until I hear a specific confirmation along those lines, I don't think it's wise to assume they ever will.

 

Last night I started looking through a bunch of the *.lvl files. Here's what I've been able to figure out so far.

 

Each *.lvl file contains a series of what I'll call "chunks". In general, it doesn't seem to matter what order the chunks are in; although with some specific files (such as core.lvl or mission.lvl), it's definitely possible that order matters.

 

Integer values are stored with the least-significant byte first.

 

At the beginning of every *.lvl file is the 4-byte character string "ucfb". This is followed by a 4-byte integer indicating the size of the file (actually, the size minus 8; since it doesn't include this 8 bytes worth of header data).

 

Each chunk stores a particular kind of data, and has 8 bytes worth of header information. The first 4 bytes is a character string indicating the chunk type. The next 4 bytes is a 32-bit integer indicating the size of the chunk. In most chunks, there appear to be sub-chunks that I'll refer to as "sections". Each type of chunk seems to expect specific types of sections, often in a specific order.

 

Some of the chunk types I know of so far are as follows:

 

tex_ = Textures (can contain many)

skel = Unknown

modl = Model information

gmod = More model information?

coll = Collision information?

expc = Explosion information?

scr_ = Script?

 

I put together a really simple program that extracts some basic information from *.lvl files. You can see a (very preliminary) sample here:

Load\bes1.lvl

The above is the Load/bes1.lvl file, which contains the big bitmap image you see when the Bespin Platforms map loads. (ie. when it zooms from the galaxy, to the system, to the terrain).

 

Since textures/bitmaps are typically the easiest thing to decipher, I'm starting there. I know a little about how they're stored, such as pixel dimensions. My plan is to extract the pixel data and throw it into individual BMP files. At first, they'll probably just be grayscale. Hopefully I'll be able to determine color information sometime soon. Most of the textures seem to be 16-bit color, so I don't know if the R-G-B bits are 5-6-5 or 6-4-6 or something else.

 

What I do know about the tex_ chunk is they expect a NAME, INFO, and one or more FMT_ sections. From there, it's organized rather strange. Each FMT_ section contains an INFO (yes, another one) and one or more FACE sections. Each FACE section seems to contain one or more LVL_ sections. And each LVL_section contains an INFO (yet again!) and a BODY section. The BODY section is where the pixel data is actually stored.

 

What I hope to come of this is that, assuming editing tools never come, we can eventually build new maps and insert them into the game. If editing tools do come, then what the hey -- it's always good to know the file structure!

[kingguru]

This could be a nice step torward image changes , skybox and floor tile modding and mainlt repositioning objects will move along alot faster if we can crack the model position part. Please keep us informed of anything new you find.

[subs0nic]

Good work! A tool to extract even that information would be a big help - any chance of releasing it?

[Visceral]

Are the sounds in the sound .lvls divided into chunks? If so, this tool could tell us where they all are so it might be a lot easier to make a custom sound set.

[MarcusLeCoy]

whoa thats good stuff

[Riley75]

Thanks guys.

Feel free to join in digging through the hex code to figure out how more of the various chunks / sections work.

 

Yes, I'll be releasing the extracting tool once I've figuring out some more things. For now it's only written in Java. I'll give out the source code so that people can rewrite it in their preferred language. I did it in Java only because I work in Java all day every day.

 

I ran it against Sound/geo.lvl. It didn't produce any errors, but the chunk names are strange. So to answer that question, I *think* they are. Heh. It looks like the important stuff to look at in the sound files is near the end of each file.

 

I figured out a bit more about textures. The pixel data is stored in "DXT1" or "DXT3" format; the DXT stands for DirectX Texture. Find out more here (you might need to use Internet Explorer for this link):

MSDN DirectX Texture (DXT1)

 

It's a very ... interesting format.

[subs0nic]

Using this info and working through Sound/hot.lvl, I discovered it has three chunks. There are obviously more sounds than this on hoth which makes me think that the middle chunk is some kind of compressed file (it is considerably larger than any of the other chunks). The last chunk, which is tiny in comparison, looks to me to be a form of reference to the compressed file. It repeatedly has the marker DATA, followed by a byte 0D and 3 empty bytes, followed by up to 12 bytes of data, then another 4 empty bytes before the next DATA marker. There are also SCOP markers at various points throughtout this chunk whose function I can't determine.

 

One difference I noticed that may account for your odd chunk names when you ran the extract program on a sound file is that there are an extra 8 bytes at the start of the file before the first chunk. The first 4 bytes I can only guess are a marker of some kind (since they are repeated at the next chunk), and the last 4 bytes are the size of the chunk, including the normal marker headers (emo_ and the following 4 byte chunk size). Possibly some kind of 'chunk within a chunk'?

[Riley75]

This is what it produces for Sound\hot.lvl:

Sound/hot.lvl

 

Looking at it in UltraEdit, it does look like they're a "chunk within a chunk". The funny "!¹[" marker always has a "emo_" directly inside of it.

 

Re: The DATA markers you're seeing there... The 0D 00 00 00 is indicating that 13 bytes of data follow. It's just that each chunk / section must be aligned to a 4-byte boundary, so you get three zeros at the end that act as padding.

 

eg. in Sound\hot.lvl, at 04D2F850h, you get:

44 41 54 41 0D 00 00 00 <-- DATA marker, with 13 bytes to follow:

B1 4A 32 BD 01 00 00 80 3F 00 00 00 00 00 00 00 <-- first 13 bytes are the actual data, the last 3 are just for padding

 

My thinking right now is that the planet-named files in \Sound\ might just contain the music selections. The cw.lvl and gcw.lvl might contain weapon sounds, vehicle sounds, etc. There's also a common.bnk file that seems to have the same chunk-style format -- why it has a different extension (bnk? Is that like the Bink from RAD Game Tools?) I don't know.

 

 

I was only partially right about textures in my above post. It actually stores any Direct3D surface texture, as stored in the DDS file format. I've been out of touch with DirectX since version 5, so I hadn't heard of DDS. They store all kinds of different textures, including mip-maps and bump-maps. It also seems like it can store stuff like alpha and luminance information.

 

I dug around the 'net for info on this. Microsoft has a tool called DXTex (that comes with the newest DirectX SDK) that can be used to edit DDS files. Their MSDN web site has technical information on the file format as well:

DDS File Format

I guess today's video cards have hardware that accepts and works with the various DDS-style textures directly.

 

It's starting to look more and more that, while the way the *.lvl files are glued together is probably proprietary, these files actually contain alot of standard format data at their core. Who knows... maybe we could even use some of the tools already existing for Battlefield 1942, and then have some simple Battlefront-specific tools that extract the data from those and put it into the *.lvl chunk format.

[Skellington]

I have a thought about the skel heading. My guess is that it tells the engine what joint sets, or skeletons, to load for each character model, to apply animation to. It seems pretty clear Pandemic used only a couple of generic motion capture files for the character moverment, and that generic motion has to be applied to a particularly sized and spaced set of joints for each character.

Also, what you may be looking at in the massively subdivided texture sections is not just the bitmap texture, but also the bitmaps which define specular shading, bump mapping, shaders, and the like. Most often, the specular map and bump maps are separate texture files, and need to be combined and layered with the basic colour texture.

[Riley75]

Thanks for the thoughts skellington. It somehow seems appropriate that someone with your handle should comment on the skel chunk.

 

I'd already started to figure out all that about textures, once I looked over the Direct3D surface texture information on MSDN.

 

I put up a complete description of the tex_ chunk here:

tex_ Chunk Information

I'm planning on putting up a page like this for each chunk type that I figure out.

 

Things are a bit slow going, as I have limited time to work on the LVL file extractor program I started writing. The program currently searches the LVL files thoroughly for chunks and sub-chunks. Often a little too thoroughly, as it occassionally "finds" sub-chunks that aren't actually sub-chunks. But I'm doing it this way so that we can analyze the output fully.

 

I don't know if I should release the program just yet, until we know more about -- at the very least -- the most common chunk types. Instead, I've posted the HTML output for a small selection of files.

 

mission.lvl

 

core.lvl (Zipped)

 

BES/bes1.lvl (Zipped)

 

FPM/ALL/allxwing.lvl

 

FPM/IMP/impatat.lvl

 

Load/load.lvl

 

Load/bes1.lvl

 

Load/bes2.lvl

[sb_gat]

Isn't that DDS texture format that was also used to store Max Payne bitmaps?

I used to have a DDS plugin for Adobe Photoshop to view and edit DDS bitmaps, if that helps...

[Riley75]

I don't have Max Payne, but I wouldn't be the least bit surprised if it uses the DDS format. I came across the plug-in you mentioned when I was looking over the modding tools people use for Battlefield 1942.

[Niteshift]

Hey there Riley, Do u want a Gmail account

[Nimlot]

Well, I dont know if Im on the right track, but I have been able to produce semi-results with this great info you have posted. Going off of the lvl for textures, i have manages to create a very simple program(i still enter most locs) which extracts the basic dds. Using load/bes1.lvl, i am able to create this image:

 

 

I honestly dont know if this is correct, but it seems pretty straight forward and may be on the right track.

 

ps. the image is zoomed in a bit and compressed, origonally 256 by 128 pxls.

[Niteshift]

Originally posted by Nimlot

Well, I dont know if Im on the right track, but I have been able to produce semi-results with this great info you have posted. Going off of the lvl for textures, i have manages to create a very simple program(i still enter most locs) which extracts the basic dds. Using load/bes1.lvl, i am able to create this image:

 

 

I honestly dont know if this is correct, but it seems pretty straight forward and may be on the right track.

 

ps. the image is zoomed in a bit and compressed, origonally 256 by 128 pxls.

 

I could say u are going in the right direction to have a semi-result. its a start, great job,

[kingguru]

Please show more results as you find them...this may be the first step in a real mod of the models/and skybox etc... I posted the cap of the load screen from your app over the load screen. These are a match... I would like to see what you using to show these...do you have an app ready?

 

[Riley75]

Good stuff, Nimlot! I was hoping someone would start working on a texture extraction program.

 

I didn't post it here yet (I posted on this the gametoast.com forums), but I've been building a more complete editing reference web site here: SW:BF LVL Reference

 

Have a look over the tex_ page and see if there's anything new there that might help. I've marked a number of values in the INFO sections as "unknown". Any idea what some of those unknowns are?

 

Edit: Niteshift, thanks but I think I have all the e-mail accounts I can handle.

[TK-8252]

Originally posted by Niteshift

Hey there Riley, Do u want a Gmail account

 

Ever heard of ?

[Niteshift]

Originally posted by TK-8252

Ever heard of ?

 

TK, i was going to give him a GMail account for all the work he is doing in the modding community, not so i could pm him

[Cpt. Bannon]

has anyone tested if increasing both the filesize and chunk size info to add (or decrease) information, so we wont have to keep everything the same size, and we dont have to do anthing but check the number of bytes we've added (or detracted)?

[Cpt. Bannon]

this should be merged and/or stickied...

[Riley75]

LVL files definitely don't have to remain the exact size that they currently are. It would be possible, for example, to swap in a new texture that's bigger or smaller. The file would just have to be reconstructed, and the file size marker would have to be changed.

 

I've updated my LVL File Reference site, and will continue to do so in the days and weeks ahead. See my signature.

[Riley75]

I've posted a big update on my reference site regarding models. I have a fair bit figured out, but there's still a ways to go. Anyone here who is knowledgable about the Direct3D *.x file format, or 3D models in general, might be able to make more sense of certain things than I can.

 

modl chunk reference information

[Cpt. Bannon]

i think MTRL (material?) could be the reference to the texture for the model

 

SPHR (sphere?)...what models did it appear on? if it only appeared on one model, thin my guess would be the droideka's shield

[Riley75]

Take another look. There's a section called TNAM that references the texture name. Each mesh segment within a model can apparently have its own texture.

 

MTRL may indeed mean "material", but if it does, it isn't a reference to a particular texture. It almost always seems to just store a single 32-bit integer (usually a value of 1).

 

All the models have a SPHR section. At least all the ones that I've looked at. This is just an educated guess, but it might just be a vector indicating the bounding box of the model, used as an early-out for collision detection.