Bone "data.ttarch" file format

[counting_pine]

I was bored, so I did a litle investigating.

data.ttarch is an uncompressed archive of thousands of smaller files, ranging in size from 48 bytes to just over 4 megabytes. I don't know how to make sense of any of the files inside, but I know how to extract them.

 

The format inside is:

 

A DWORD giving the number of "paths"
(they seem to be subfolders of some sort, eg ".\textures" or ".\animations",  but don't seem to correspond
to anything in the Boneville folder. I think they might correspond to different types of file in the archive)

The paths follow straight after.  They consist of a DWORD for the path length, followed by a STRING
of that length containing the path.

Then there is a DWORD giving the number of files in the archive.
The file info follows straight after.  I think they're listed in alphabetical order.

It consists of:
A DWORD for the filename length, followed by a STRING of that length containing the filename
A DWORD which is always zero
A DWORD containing the file's offset in the archive data
A DWORD containing the file's length

Then there is a DWORD for the offset of the beginning of the archive data in the file
Then there is a DWORD for the length of the archive data

The archive data follows straight after.

Sorry if that's hard to follow

 

This FreeBASIC program will extract the files into c:\bonedata.

I've preset the archive data offset to 0x3111D in the program, but you might want to run it through once without extracting the files, to make sure that your copy of data.ttarch has the same value.

 

option explicit

dim numpaths as long

dim pathlength as long, path as string

dim numfiles as long

dim filenamelength as long, filename as string
dim zero as long, filelength as long, fileoffset as long

dim filedata() as byte

dim dataoffset as long
dim datalength as long

dim seek1 as long
dim i as long
dim extractfiles as string * 1

'presetting dataoffset:
dataoffset = &h0003111d

input "Extract files? ", extractfiles
extractfiles = lcase$(extractfiles)

if extractfiles = "y" then shell "md c:\bonedata"

open "C:\Program Files\Out from Boneville\Pack\data.ttarch" for binary access read as #1

   get #1, 1, numpaths
   print "Number of paths:"; numpaths

   for i = 1 to numpaths
       get #1, , pathlength
       path = space$(pathlength)
       get #1, , path        
       print path
   next i

   get #1, , numfiles
   print
   print "Number of files:"; numfiles

   for i = 1 to numfiles

       get #1, , filenamelength
       filename = space$(filenamelength)
       get #1, , filename: print filename

       get #1, , zero:        print right$("00000000" + hex$(zero), 8),
       get #1, , fileoffset: print right$("00000000" + hex$(fileoffset), 8),
       get #1, , filelength: print right$("00000000" + hex$(filelength), 8)

       seek1 = seek(1)

       if extractfiles = "y" then
           open "c:\bonedata\" + filename for binary access write as #2
               redim filedata(0 to filelength - 1)
               get #1, dataoffset + fileoffset + 1, filedata()
               put #2, 1, filedata()
           close #2
       end if

       seek #1, seek1            

   next i

   get #1, , dataoffset
   get #1, , datalength
   print
   print "data offset: "; right$("00000000" + hex$(dataoffset), 8)
   print "data length: "; right$("00000000" + hex$(datalength), 8)

close
sleep

[john_doe]

Cool.

I had a look at the extracted files, of course

Most of the files seem to start with "SEBM", the following data is xor'ed with 0xFF. The first bytes of each file seem to use another value but it works for the rest of each file so you can read text.

[Alien426]

Could somebody write a program that cuts the first 126 bytes from the *.aud files so that one can listen to the OGGs in the "StreamedData" folder?

[counting_pine]

I will, if you buy the game

[Metallus]

Nice.

[Benny]

Could somebody write a program that cuts the first 126 bytes from the *.aud files so that one can listen to the OGGs in the "StreamedData" folder?

 

Here you go: Bone Music Extractor

 

Nice one CountingPine, that was quick work

[Benny]

For those who are lazy like me, here's a gui version of counting_pine's dumper - Bone ttarch Dumper (hope you dont mind counting_pine ).

 

One odd thing I noticed about the archive is that theres 709 bytes of (junk?) data at the end of the file that never gets used.

[counting_pine]

No, I don't mind. And thanks for crediting me

 

Looking at the data at the end of the file, my theory about the 709 bytes would be that at one point the file used to be longer, and when they wrote the final archive, they didn't clear the data off the end.

[Alien426]

Here you go: Bone Music Extractor

Thanks, Benny, that program is neat! Mind if I put this on PC gameScore?

[Benny]

Not at all, go ahead.

[Benny]

Also did anyone else notice that some of those files were .chore files? I dont for a second think there'll be any similarities, but both scumm and grime used 'chores', I think they were used for actor animation or similar, I cant really remember. Also one of the developer mode command lines from Grim Fandango allows you to invoke the 'chore' tool, but only if you have the (lucasarts only) choreface.dll. Anyway, there you go. A tenuous Lucasarts-Telltale link.

 

 

[Edit]

It looks like the .d3dtx files are just standard .dds files with a header and xor'ed by 0xFF as John_Doe said. I'll put together a quick viewer or dumper for them later

[Benny]

Weird, the images are .dds images but each image has an area of corruption on it:

Example 1

Example 2

 

I dont know why this happens, its very odd.

[counting_pine]

Different viewers seem to interpret the corupted areas in slightly different ways - I tried WTV and the Compressonator.

 

http://img91.imageshack.us/img91/6981/uimenulogowtv8fg.png

http://img91.imageshack.us/img91/8434/uimenulogocompressinator2rh.png

 

I guess that means there's a right way to interpret it...

 

Don't suppose there are any dds experts about?

[Benny]

I tried with WTV and the dds viewer in InfranView and got the same results. It does it with different levels of DXT compression too so its not just 1 thats wonky. I checked the size and that was correct too.

 

From memory the header before the dds image was:

4 bytes - DXT compression type

4 bytes - width

4 bytes - height

4 bytes - 0

1 byte - ?

4 bytes - size of dds image (without header)

[john_doe]

It could also be that not the whole file is xor'ed but only several bytes, then several bytes not (or with a different value) and so on. This could explain the pattern of the corruptions in the dds images.

[Benny]

The corruption is easiest to see in ui_options.d3dtx because the corruption is at the bottom on a black background.

 

The corrupt bit is this (before any xor'ing):

 

840D 180D 9050 A69B 840D 180D 9050 A69B 840D 180D 9050 A69B 840D 180D 9050 A69B 840D 180D 9050 A69B 840D 180D 9050 A69B 840D 180D 9050 A69B 840D 180D 9050 A69B

 

The thing to do I suppose, is to try and get those values to 00 like the surrounding bytes (or FF as they are when xor'ed).

 

Another weird thing that I've noticed is this: In most files, when xor'ed with FF there's a path eg. 'F\Telltale\ArtData\UI\Bone\ui_menu_logo.tga' with a dword before that, which specifies the length of the path string. However in ui_menu_options the string is incomplete at the beginning 'elltale/ArtData/UI/Bone/ui_options.tga' and there doesnt seem to be a dword before it to indicate path length. This might mean that the missing bits are xor'ed with a different value.

[counting_pine]

Hmm... A repeating QWORD, 64 bytes total. I notice it starts at 0x4100.

 

I've checked a couple of files, and the path ("C:/Telltale/ArtData/...") always seems to start being legible 68 bytes in. So maybe there's another 64 bytes of corruption straight after the SEBM.

 

Funny how sometimes it's back slashes, sometimes forward slashes.

[Benny]

Which file is that in? Also whats a QWORD? 8 bytes?

[counting_pine]

Yeah: Byte, Word, DWord, QWord. 8 Bytes.

 

I don't know if this is any help, but a couple of the files have more extractable data after byte 68, like this (without the spaces):

 

d3dtx [63][00][00][00] [5B][00][00][00] C:/Telltale/ArtData/Characters/Sk08/TedsBigBrother/Textures/sk08_tedsBigBrother_bandAid.tga

[jaap]

Here you go: Bone Music Extractor

 

Nice one CountingPine, that was quick work

 

 

Hey thanks for the music extractor!!

I tried to download the right plugin for winamp to play standard .aud files, but this didn't work of course.

However, thank to you guys I can listen to the music, which is very cool!

 

Ted Theme and Possum kids are my favorites.

 

Jaap

[Benny]

The font files also use dds images and they also have the same corruption.

[counting_pine]

It doesn't seem to be xor encryption at the beginning of the SEBM files. I tried xoring the first 64 bytes after the header with a QWord that would make some of the bytes legible, but the other ones still wouldn't make sense.

[john_doe]

The strange thing is that with some files, there's unencrypted data at the end (i.e. readable ASCII text) while the bytes before are 0xFF-xor'ed and the beginning of the file somehow else.

[Benny]

Perhaps it might help to see what the game makes of any changes.

 

Bone Ttarch Creator - select a folder and it'll compile the files inside to a new .ttarch archive. I've tried it with the game briefly and it seems to work fine.

[john_doe]

Hmm...the *.lenc files seem to be encrypted Lua scripts but they're not xor'd with 0xFF.

I just got an idea: Iirc there's a tool in the DirectX SDK for debugging DX programs. I think you can also dump all textures uploaded to the system. If one can get some "good" textures we can compare them to their encrypted versions. This would help to find out how the encryption works.